Proxmox VLAN Design: VLAN-aware bridge vs. bridge-per-VLAN – what do you use and why?

S

Stril

New Member
Dec 10, 2024
2
0
1
Hi everyone,


I’m currently reworking the network setup on one of our Proxmox VE clusters and I’d like to hear what VLAN approach others are using in practice.


There seem to be two common design patterns:



1) Single VLAN-aware bridge​


Example:


  • bond0 → vmbr0 (vlan-aware)
  • VMs get their VLAN tags directly in the VM config

Pros:
  • Clean and short network config

Cons:
  • No option to monitor the "per VLAN-traffic"
  • VLAN traffic is mostly just forwarded inside the bridge
  • No


2) Bridge per VLAN using VLAN subinterfaces​


Example:


  • bond0.49 → vmbr49
  • VMs connect untagged to the corresponding bridge

Pros:
  • Easy to monitor per VLAN (SNMP counters on bond0.X)
  • Better visibility (VM shows VLAN description and not only its ID)

Cons:
  • More config overhead if you have many VLANs
  • More bridges/interfaces to manage


In my case, per-VLAN monitoring (e.g. via SNMP/PRTG) was a requirement, so the bridge-per-VLAN approach seemed more practical. Interestingly, Linux bridge interfaces (vmbrX) often don’t show forwarded traffic counters, while the VLAN subinterfaces (bond0.X) do.


So I’m curious:


What VLAN setup are you running in Proxmox, and what made you choose it?
Do you prefer VLAN-aware bridges for simplicity, or separate bridges per VLAN for operational clarity and monitoring?


Any real-world experiences, pros/cons, or gotchas are appreciated.


Thanks!
 
Ten (twelve!) years ago : 2).
For better visibility. But at that time, a pve reboot was needed after any vmbr / bond creation...

Nowadays : 1).
Easier to manage...

Christophe.
 
Very offen, like pvps1 says, due to the high grade of automation in our environments.

This also has the benefit, that "unfamiliar" cluster user with restricted priviledges, can not forget to set the correct vlan in the network configuration of the vVMs during the creation with the wizard. As far as I remember, from last time tested, there was also no meaningful error message. So it provides some convenience for the actual VM maintainers.

At the same time, if you manage your network in a manual way, you need to take a lot of care for changing the network configuration of the hosts.
It is easy to introduce inhomogeneous bridge configurations. So your VMs won't migrate/start in case of a HA caused recovery or traffic might be send into the wrong vlan.

To avoid this, depending on the clustersize and administrator knowledge, the new SDN features can be very helpful, at least for the VM connectivity.

BR, Lucas
 
I take it you are only considering VMs and by that I mean what their virtual NICs get plugged into.

When it comes to VLANs, I think that if a device only needs an access port (single VLAN) then the tagging should be done by the network port. If a device needs multiple networks/VLANs then it should be connected to a trunk (also often known as a hybrid, except when it isn't) port.

You'll note that so far, throughout the above waffle, I have not mentioned bridges or bonds. I have defined part of my policy for networking at layer two.

So, given my policy worked out above, I want most of my VMs to be plugged into bridge interfaces that already define a single untagged VLAN from the perspective of the VM. That's your option 2.

In the case of a VM needing multiple VLANs, there will be an agreed default, untagged VLAN and the rest are tagged. The default may actually be a discard jobbie ... allow me to explain 8) I generally define a VLAN that I call DUFF and give it a high number eg 4090. Some implementations eg CIsco have a discard VLAN which dumps all traffic. My DUFF VLAN allows me to attach packet capture (network probe) -> SIEM to look for traffic that is off-piste.

So if I need a port to be a trunk, I might set the default VLAN to the DUFF VLAN and tag everything else. For example a router might have multiple WANs on a single physical cable, eg a QSFP+ or slightly more normal, a SFP+. I set the default to DUFF and tag the rest. I also might monitor the DUFF VLAN for oddities (or not bother - its dumped anyway and monitoring depends on your resources and just how paranoid you are).

Anyway, I suggest you define your policy and make your systems work the way you want them to.
 
You must log in or register to reply here.
Top Bottom
Back