Where VLAN tagging of the Ethernet Frames take place?

[noop1989]

Hey,

Where does the VLAN tagging of the Ethernet frames take place?

When I set the VLAN tag on the VM, the VLAN tagging is performed on the VM's network interface. So outgoing packets from the VM receive the VLAN tag.

But what if I use “Linux VLAN”? Or just use SDN? At what level is VLAN tagging then performed?

If I do a mixed operation, i.e. tag VM-01 in the config with ID=100 and set VM-02 to the "Linux VLAN" ID=100, both VMs have no network.

Does tagging get in the way?

 

[gunterwa]

Can you explain your question in detail? Nice to have screen captures which can we see where the tagging been performed...

 

[louie1961]

In the trunked links https://www.firewall.cx/networking/vlan-networks/vlan-tagging.html

 

[noop1989]

##### @gunterwa #####

I had a case where the VLAN tag was set in the VM configuration. The Linux Bridge was also set to be VLAN aware.

After a "Linux VLAN" was created with the same VLAN ID, the node no longer had any connections. So the VLAN network was completely down for the node.

So I asked myself whether and how the VLAN tagging procedure affected the network connections. Hence this post.

##### @louie1961 #####

Thanks for the link. However, this only looks at part of it.

I would like to know how the different VLAN setting options in Proxmox affect the Ethernet frame and how we can influence the different configuration options.

1. VLAN tagging in the VM configuration

2. "Linux VLAN"

3. SDN
3.1 VLAN
3.2 VXLAN

In the logs I was able to determine that a VLAN config can only be set once on a physical interface. When duplication occurs, the VLAN appears to stop working, with no error messages.

That would be fatal if Proxmox admins had no network management scheme and mixed the configuration options.

 

[louie1961]

I don't really use many, if any, of the VLAN settings in Proxmox, so I can't really answer that question. My single Proxmox NIC is configured to be VLAN aware. My VLANs are defined inside of pfSense and my managed switch. I feed Proxmox with a trunked port with all VLANs tagged. And when I create my VMs or LXC containers, I specify the VLAN I want them to sit on, in the configuration screens. I also don't use the Proxmox firewall, either. I leave all of that for the pfSense box. In most cases I leave my networking inside of Proxmox set to DHCP, and I do DHCP reservations inside of pfSense using the MAC address of the NIC. The result is that I can control all of my networking in pfSense for the most part. Here is my /etc/network/interfaces config

auto lo
iface lo inet loopback
iface eno1 inet manual

auto vmbr0.99
iface vmbr0.99 inet static
address 10.10.99.2/24
gateway 10.10.99.1

auto vmbr0
iface vmbr0 inet static
bridge-ports eno1
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4092

In the picture below, I can set the VLAN tag to anything that is supported in my network

 

[noop1989]

Yes, exactly, that is method 1. VLAN tagging in the VM configuration. This is also an attitude that can be found everywhere on the Internet.

But what about method 2. If I create a “Linux VLAN” in the network config of the node and place a “Linux Bridge” on it. Then I no longer need VLAN tagging in the VM configuration. Then I just need to connect the VM to the Linux bridge “vmbr20” (VLAN-20) and the “Linux VLAN” takes over the tagging.



Then how does it work with SDN?

Why does the connection break when I use method 1 and 2 at the same time?

 

[noop1989]

I don`t understand this Part:

auto vmbr0.99
iface vmbr0.99 inet static
address 10.10.99.2/24
gateway 10.10.99.1

vmbr0 = Linux Bridge
.99 = Linux VLAN ???

 

[louie1961]

I don`t understand this Part:

auto vmbr0.99
iface vmbr0.99 inet static
address 10.10.99.2/24
gateway 10.10.99.1

vmbr0 = Linux Bridge
.99 = Linux VLAN ???

Yes, this is my /etc/network/interfaces file. I have defined a virtual bridge (kind of like a virtual NIC, which is also possible) assigned to VLAN 99 which is the VLAN I use as my management VLAN

 

[gunterwa]

##### @gunterwa #####

I had a case where the VLAN tag was set in the VM configuration. The Linux Bridge was also set to be VLAN aware.

After a "Linux VLAN" was created with the same VLAN ID, the node no longer had any connections. So the VLAN network was completely down for the node.

So I asked myself whether and how the VLAN tagging procedure affected the network connections. Hence this post.

##### @louie1961 #####

Thanks for the link. However, this only looks at part of it.

I would like to know how the different VLAN setting options in Proxmox affect the Ethernet frame and how we can influence the different configuration options.

1. VLAN tagging in the VM configuration

2. "Linux VLAN"

3. SDN
3.1 VLAN
3.2 VXLAN

In the logs I was able to determine that a VLAN config can only be set once on a physical interface. When duplication occurs, the VLAN appears to stop working, with no error messages.

That would be fatal if Proxmox admins had no network management scheme and mixed the configuration options.

Hi noop1989,

I try to reproduce your issue in my lab but the result is just norm.
My PVE is on v8.1.4, I created a Linux Bridge with VLAN aware set, and created two VMs running Ubuntu20.04. I configured VM1's and VM2's NIC both in Bridge10 ( Linux Bridge) with same vlan_tag 50 the two VMs can ping to each other, when I changed VM1 NIC's vlan_tag to 55 the ping echo failed..

 

[noop1989]

@gunterwa - you only use a Linux Bridge without "Bridge Ports".

My Szenario:

I have an external OPNsense on Interface ens32/vmbr0 to route VLANs. VLAN 100 has an DHCP 10.0.100.0/24 from 150 to 200.

I use method 1 - VLAN tagging in the VM configurations:


and i have a connection. All fine.
Now i switch to Method 2 - add "Linux VLAN" with same VLAN-ID 100 :


and i get an error.



CT has no connection now:



But, i try to reload per "ifreload -a -d"



all fine "exit status 0".

But my VM/CT still has no connection.

I reboot my CT and get Error:



If i change now the VM config:



and get new errors:



Also, i can't mix the Methods. If i want to add "Linux VLAN" or switch to SDN, i have to remove all network settings first. In a production-system not realy helpful.

I search for an solution to change a production-system to SDN without remove all Network Setting as first or create completly new Network-Infrastructure to migrate.

Also my question: Where does the VLAN tagging of the Ethernet frames take place? Is that the reason why mixed Methods crash network connections?

 

[Spiros Pap]

The ethernet frames from the VM come from the tap adapter. The VLAN tagging of these frames happen when the packets are processed by the bridge, You can see the vlan tag with :
bridge vlan show dev tap100i0
port vlan-id
tap100i0 444 PVID Egress Untagged

 

[jsterr]

Has someone actually tcpdumped and checked or knows:

• what happens if a vm with a tagged vm-hw-adapter gets its vlan tag overwritten when user sets them inside the vm?
  • will the vmbridge-adapter drop the "wrong" vlan-tag because it does not match?

 

[gunterwa]

@gunterwa - you only use a Linux Bridge without "Bridge Ports".

My Szenario:

I have an external OPNsense on Interface ens32/vmbr0 to route VLANs. VLAN 100 has an DHCP 10.0.100.0/24 from 150 to 200.

I use method 1 - VLAN tagging in the VM configurations:
View attachment 65289
View attachment 65290
and i have a connection. All fine.
Now i switch to Method 2 - add "Linux VLAN" with same VLAN-ID 100 :
View attachment 65291

and i get an error.

View attachment 65292

CT has no connection now:

View attachment 65293

But, i try to reload per "ifreload -a -d"

View attachment 65294

all fine "exit status 0".

But my VM/CT still has no connection.

I reboot my CT and get Error:

View attachment 65297

If i change now the VM config:

View attachment 65299

and get new errors:

View attachment 65301

Also, i can't mix the Methods. If i want to add "Linux VLAN" or switch to SDN, i have to remove all network settings first. In a production-system not realy helpful.

I search for an solution to change a production-system to SDN without remove all Network Setting as first or create completly new Network-Infrastructure to migrate.

Also my question: Where does the VLAN tagging of the Ethernet frames take place? Is that the reason why mixed Methods crash network connections?

Sorry for late response ( I know it's toooo late )...

I had not done packets-capturing and analysis yet, but I did try both vlan tagging on VM NIC level (PVE physical NIC encapsulated trunk to switch, all Vlan allowed) and "Linux VLAN". For your "method 2" - add "Linux VLAN" with same VLAN-ID 100... it's actrully adding a L3 interface to the VM just like configuration on Cisco switch:

interface vlan 100
ip add 10.1.1.1/24

this L3 interface belongs to VLAN 100, your can ping from any VM NIC with IP address inside 10.1.1.0/24 and VLAN 100.

So...
"Method 1" : Adding a L2 interface to the VM, just like cabling your VM ethernet port to a L2 switch, ports with same VLAN ID can ping to each other;
"Method 2" : Adding a L3 interface to your VM, your VM is working like a router now. Your VM (Router) can now communicate with other router via this L3 interface.

I am not sure if I'd clealy described what I experienced, hope it could be of any help to answer your question.

 

[shanreich]

what happens if a vm with a tagged vm-hw-adapter gets its vlan tag overwritten when user sets them inside the vm?

The packet should then be dropped. The tagging setting uses the bridge vlan feature to define allowed and native vlans (see [1]).
This means tagging takes place when the packet traverses the bridge port.

[1] https://man7.org/linux/man-pages/man8/bridge.8.html

 

[jsterr]

The packet should then be dropped. The tagging setting uses the bridge vlan feature to define allowed and native vlans (see [1]).
This means tagging takes place when the packet traverses the bridge port.

[1] https://man7.org/linux/man-pages/man8/bridge.8.html

Thanks! A side-question to that topic. If a user puts an ip for the web-ui/mgmt on the vmbr0 (access-vlan, no tagged-vlan-adapters) - is this still a bad idea in 2025? Any downside, because vms can identify the hypervisor or is this information outdated nowdays? Cant find the reasons for it anymore

Like this: vmbr0: IP-WEB-UI + vmbr0 (VLAN-Aware - bond0) where VM/CT work with Tagged-VLAN-Adapters? Can they identify the hypervisor or harm it in any way?