Corosync Best Practice

Ashkaan Hassan

Active Member
Feb 16, 2019
13
1
43
38
I'm trying to figure out the best way to ensure corosync has its own network. Please let me know if any of these points weren't ideal and why:
  • I created a VLAN with no gateway (just a space on my switch with a unique subnet) 10.5.0.X and set all 8x of my nodes to addresses on that subnet.
1693712776178.png
  • I created a Linux Bridge for the VM + Management Network, but did NOT create a bridge for the cluster NIC
1693712755184.png
  • How can I tell if corosync and migration are really running though the 10.5.0.X network?
    • On my previous cluster, I configured the cluster IPs and Management IPs on the same subnet (different NICs) and one day I unplugged the cluster NIC and the cluster continued on like nothing happened. It was really disappointing to see that all traffic was going through the Management NIC.
Thank you!
 
How can I tell if corosync and migration are really running though the 10.5.0.X network?
For corosync you can verify the current setting on the cli:
Code:
~# corosync-cfgtool  -n
Local node ID 7, transport knet
nodeid: 2 reachable
   LINK: 0 udp (10.3.16.1->10.3.16.9) enabled connected mtu: 1397
   LINK: 1 udp (10.11.16.1->10.11.16.9) enabled connected mtu: 1397

It is important to have two independent connections for these links. If both links are just VLANs sharing the same cable you have not reached that goal.


The network to be used for migration is specified independently in Datacenter --> Options --> Migration Settings.

Best regards
 
  • Like
Reactions: Ashkaan Hassan
Wow! This is very helpful. Thank you!

Yep, they are 2x totally different NICs (10G Fiber for Management and VMs / 1G Copper for Cluster).

Last question: If you had my setup, would you push migrations through the 1G Cluster or 10G Management?
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!