I have a three-node Proxmox 8.4.1 cluster setup as I learn more about Proxmox ahead of a switch from VMWare. Each node has four NICs - the names vary depending on hardware so, for ease, we'll call then eno11, 12, 13 & 14 (first node), eno21, 22, 23 & 24 (second node) and eno31, 32, 33 & 34 (third node).
- Each node has its own vmbr0 with either eno11, 21 or 31 associated with it, used for managing Proxmox. These are "access" ports on the switch.
- Each node has its own vmbr1 with either eno12 & 13, 22 & 23 or 32 & 33 associated with it for Ceph. This is a dedicated switch just for storage traffic.
- Each node has other bridges (vmbr100, 101, 102, 103, ...) associated with Linux VLAN devices tied to the fourth NIC (eno14.100, eno34.103, etc.) and "trunk" ports on the switch.
- Do I just create a new vmbr2 on each node, associate it with the fourth NIC and then use vmbr2 when I create the SDN VLAN zone configuration? Is vmbr2 VLAN aware at the node level? SDN Zone level? Neither?
- I just add SDN VNets with the VLAN info then migrate the hosts from the old Linux VLAN bridges to the new SDN VNets and then delete the node-by-node Linux VLAN bridges and Linux VLAN interfaces?
- If I add more nodes in the future, do I have to plan ahead that the name vmbr2 has be used for the SDN VLAN bridge? I can't use vmbr2 as the "uplink" on some nodes but "vmbr3" on other nodes? (I have to break my VMware dvSwitch thinking where I can assign arbitrary uplinks to the configuration.)
- If I want to add multiple uplinks per node in the future, I remove the interface from vmbr2, add it to a new Linux bond interface and then add that bond interface to vmbr2 for the SDN VLAN zone to use?