Proper way to set up a Proxmox cluster on a dedicated VLAN (separate from Management)

[volkmen]

"Hi,

I am setting up a 3-node Proxmox cluster (WIK-1, WIK-2, WIK-3). I want to do it the right way from the start by separating the cluster traffic (Corosync) from the management/VM traffic.

My setup:

• Management Network: 192.168.120.0/24 (GUI access, Web, etc.)
• Cluster Network (Dedicated VLAN 140): 192.168.140.0/24

My goal:I want Corosync to communicate ONLY via VLAN 140.

My questions:

1. Should I point the hostname in /etc/hosts to the management IP (120.x) or the cluster IP (140.x)?
2. When creating the cluster via GUI, I select the 140.x address as Link 0. Is this enough to ensure the cluster is isolated?
3. I previously encountered 'Error 500' and 'pmxcfs lock' issues when trying this. What is the correct sequence of steps to bind the cluster to the dedicated VLAN so the Web GUI stays stable on the management IP?

I want to avoid 'split-brain' scenarios and ensuring that even if the management network is saturated, the cluster remains quorate.

Thanks for any advice!"

 

[VictorSTS]

Cluster Network (Dedicated VLAN 140): 192.168.140.0/24

For a fully redundant corosync, use at least two logically and physically independent links.

Should I point the hostname in /etc/hosts to the management IP (120.x) or the cluster IP (140.x)?

Management. That will be populated automatically by PVE on install using the IP address you configure during setup.

When creating the cluster via GUI, I select the 140.x address as Link 0. Is this enough to ensure the cluster is isolated?

Yes, but you should add at least a second link to "do it right"

I previously encountered 'Error 500' and 'pmxcfs lock' issues when trying this. What is the correct sequence of steps to bind the cluster to the dedicated VLAN so the Web GUI stays stable on the management IP?

As easy as making sure every host sees each other via all corosync links and that MTUs are correct on all of them. Then use them when creating the cluster as you mention above.

I want to avoid 'split-brain' scenarios and ensuring that even if the management network is saturated, the cluster remains quorate.

Make sure corosync uses a different nic for each link and those are different nics than Management/vmbr0. Also verify there's no bottleneck on the redundant switches' uplinks (cause you are using more than one switch, arent you? )

 

[volkmen]

Hello

I did it this way

link 0 192.168.140.0/24
Link 1 192.168.120.0/24

This caused the cluster to split

When I set it up as
Link 0 192.168.120.0/24

it works

As for the network cards, I don't have them separate—just on a different port.

I need to check if I can connect it to a different port on the card

Should I leave only link 0 and how do I add link 1 as 192.168.140.0/24

 

[alexskysilk]

different ports = different interfaces. discussion about proper nic redundancy notwithstanding.

for the context of conversation, interfaces are logical. While for corosync you want link seperation as well as logical, you might also want to consider link fault tolerance for your guest bridges, control plane traffic, etc.

see https://pve.proxmox.com/wiki/Network_Configuration#sysadmin_network_bond, https://pve.proxmox.com/wiki/Network_Configuration#sysadmin_network_vlan