Cluster management/Corosync network - does it need to be faster than 1GB?

michaeljd

New Member
Aug 30, 2023
18
0
1
Hello,

I'm running PVE8.0 at home. Multiple users working from home with Windows VMs. Many small Linux VMs doing lots of low-bandwidth stuff.

I'm in the process of setting up a 3-node cluster for HA. All 3 nodes have identical hardware. All 3 nodes are on a 10GB fiber network. All 3 nodes use shared storage (NFS share on TrueNAS) for VMs and backup traffic. TrueNAS server is on the 10GB network.

I've read in the admin guide that it's best practice to have a separate network for Corosync/the HA management network, especially if using shared storage, because Corosync is very sensitive to latency.

Does that separate management/Corosync network need to be fast, as in faster than 1GB? I have both 10GB SFP+ ports and 1GB copper ports available on my switch, but I would need to purchase another six 10GB SFPs if having to use 10GB fiber vs. 1GB ethernet. I know that generally speaking, faster is always better, but if I don't need the Corosync network to be on 10GB fiber, I can save some money and time.

Thank you.
 
Last edited:
Hi,
Does that separate management/Corosync network need to be fast, as in faster than 1GB?
No, for just 3 nodes even 100 Mbps would be normally fine to use as corosync cluster network.
The important thing is that the cluster network runs on its own, physically isolated, network.
As the recommendation for a separate network isn't due to bandwidth requirements, but rather for the fact that the cluster network is quite latency sensitive, and any bandwidth hungry traffic (like from IO, Backups, data streaming, ...) can cause big latency spikes even if the link isn't fully saturated yet.
 
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!