Cluster management/Corosync network - does it need to be faster than 1GB?

[michaeljd]

Hello,

I'm running PVE8.0 at home. Multiple users working from home with Windows VMs. Many small Linux VMs doing lots of low-bandwidth stuff.

I'm in the process of setting up a 3-node cluster for HA. All 3 nodes have identical hardware. All 3 nodes are on a 10GB fiber network. All 3 nodes use shared storage (NFS share on TrueNAS) for VMs and backup traffic. TrueNAS server is on the 10GB network.

I've read in the admin guide that it's best practice to have a separate network for Corosync/the HA management network, especially if using shared storage, because Corosync is very sensitive to latency.

Does that separate management/Corosync network need to be fast, as in faster than 1GB? I have both 10GB SFP+ ports and 1GB copper ports available on my switch, but I would need to purchase another six 10GB SFPs if having to use 10GB fiber vs. 1GB ethernet. I know that generally speaking, faster is always better, but if I don't need the Corosync network to be on 10GB fiber, I can save some money and time.

Thank you.

 

[t.lamprecht]

Hi,

Does that separate management/Corosync network need to be fast, as in faster than 1GB?

No, for just 3 nodes even 100 Mbps would be normally fine to use as corosync cluster network.
The important thing is that the cluster network runs on its own, physically isolated, network.
As the recommendation for a separate network isn't due to bandwidth requirements, but rather for the fact that the cluster network is quite latency sensitive, and any bandwidth hungry traffic (like from IO, Backups, data streaming, ...) can cause big latency spikes even if the link isn't fully saturated yet.

 

[michaeljd]

That's good news. Thank you, t.lamprecht.