Ceph security

[CedricKid]

Hello, I have a quick question. I have 3 Proxmox VE nodes in a cluster with virtual machines, and I'm planning to set up a Ceph cluster. I was advised to use a second switch and put my Ceph cluster on a separate network, which I did, but I'm not sure I understand the security reasons behind this approach.

 

[Jinba Ittai]

I was advised to use a second switch and put my Ceph cluster on a separate network, which I did, but I'm not sure I understand the security reasons behind this approach.

The advice to run Ceph on a dedicated network isn't for security reasons.

It's to ensure no other traffic will interfere with the Ceph traffic, which is very sensitive to latency.

See Proxmox Wiki: Deploy Hyper-Converged Ceph Cluster

Network​

We recommend a network bandwidth of at least 10 Gbps, or more, to be used exclusively for Ceph traffic. A meshed network setup [4] is also an option for three to five node clusters, if there are no 10+ Gbps switches available.

The volume of traffic, especially during recovery, will interfere with other services on the same network, especially the latency sensitive Proxmox VE corosync cluster stack can be affected, resulting in possible loss of cluster quorum. Moving the Ceph traffic to dedicated and physical separated networks will avoid such interference, not only for corosync, but also for the networking services provided by any virtual guests.

 

[CedricKid]

The advice to run Ceph on a dedicated network isn't for security reasons.

It's to ensure no other traffic will interfere with the Ceph traffic, which is very sensitive to latency.

See Proxmox Wiki: Deploy Hyper-Converged Ceph Cluster

ah ok I see, thank you, but it can still be useful for security reasons, I suppose. If a VM gets infected, it would have a harder time spreading to Ceph if it's on a separate network.