Confusion regarding trunking with 'VLAN aware'

[William Edwards]

Goal

Be able to specify vLANs in two places:

• Inside guest OS
• VM NIC in Proxmox

Actions taken

Enabled 'VLAN aware' on one node.

Expected results

On the node where 'VLAN aware' is enabled on the bridge, I am able to join the same vLAN regardless of whether it is configured inside the guest OS or on the VM NIC in Proxmox.

Actual results

After enabling 'VLAN aware' on one node, the 'VLAN aware' functionality (ability to join vLAN regardless of whether it is configured inside the guest OS or on the VM NIC in Proxmox) works on all nodes.

I expected for this to work only on the node where 'VLAN aware' has been enabled.

Question

Why does this work?

Environment details

Node with 'VLAN aware' enabled:

root@proxmox04:~# cat /sys/class/net/vmbr999/bridge/vlan_filtering
1

root@proxmox04:~# pveversion
pve-manager/6.2-6/ee1d7754 (running kernel: 5.4.44-2-pve)

Node without 'VLAN aware' enabled (while functionality is same as on `proxmox04`):

root@proxmox03:~#  cat /sys/class/net/vmbr999/bridge/vlan_filtering
0

root@proxmox03:~# pveversion
pve-manager/6.2-6/ee1d7754 (running kernel: 5.4.44-1-pve)

 

[spirit]

without vlanaware, when you define a tag on a vm nic, proxmox create new bridge like "vmbr0xY, with eth0.Y" where Y is the vlan tag.

 

[William Edwards]

without vlanaware, when you define a tag on a vm nic, proxmox create new bridge like "vmbr0xY, with eth0.Y" where Y is the vlan tag.

I know. Unless I don't understand your comment correctly, I'm not sure what this has to do with my post.

My point is that not tagging the VM NIC, and specifying a vLAN inside the guest OS, should only work when 'VLAN aware' is enabled. But this has started working on all nodes after enabling 'VLAN aware' on just one node.

 

[jlebherz]

you can also use vlans inside your VM if you didnt set vlan aware but you have trouble if you habe one vm with a vlan assigned via the interface and another vm which is using vlans inside, in this case you need the set vlan aware to get it working

 

[William Edwards]

you can also use vlans inside your VM if you didnt set vlan aware but you have trouble if you habe one vm with a vlan assigned via the interface and another vm which is using vlans inside, in this case you need the set vlan aware to get it working

That's what I expected, but with VLAN aware disabled, I can reach all VMs in the vLAN from all VMs in the vLAN; regardless of whether the tag is specified on the VM NIC or inside the guest OS.

 

[spirit]

That's what I expected, but with VLAN aware disabled, I can reach all VMs in the vLAN from all VMs in the vLAN; regardless of whether the tag is specified on the VM NIC or inside the guest OS.

if vlanaware is disable, vmbrX don't transport vlan tag. (so if you defined a tag inside the guest os, it'll be removed. But maybe your physical switch will add a default vlan tag)


if you defined tag at the vm nic level on proxmox, the vlan tag is added with the packet is going out the vmbrX port (with vlan aware), or when packet is going out ethX.Y with non vlan aware bridge. and the vlan tag is removed when is packet is going to the vm.
(it's like a mode access on cisco)

if you don't define any vlan tag at vm nic level, and you use bridge vlan aware, it'll work like a trunk port on cisco, the vlan will be transported to the guest os. (and you can tag inside the guest in this case)

My point is that not tagging the VM NIC, and specifying a vLAN inside the guest OS, should only work when 'VLAN aware' is enabled. But this has started working on all nodes after enabling 'VLAN aware' on just one node.

vlanaware should be enabled if you defined vlan inside the guest. I don't known here how is defined vlans on other nodes/vms. (if it's done on proxmox vm nic, it'll work with or without vlan aware).

 

[William Edwards]

I know all this. The functionality offered by VLAN aware (not removing vLAN tag when tagged inside guest) is active without VLAN aware being enabled. I don't understand why.

 

[spirit]

I know all this. The functionality offered by VLAN aware (not removing vLAN tag when tagged inside guest) is active without VLAN aware being enabled. I don't understand why.

mmm, this is really strange, if vlanfiltering is not enable on switch, you shouldn't be able to see vlan tag. do you have tried to tcpdump (tcpdump -e -i vmbrX) the vmbr directly, to see if the vlan tag exist?

 

[William Edwards]

mmm, this is really strange, if vlanfiltering is not enable on switch, you shouldn't be able to see vlan tag. do you have tried to tcpdump (tcpdump -e -i vmbrX) the vmbr directly, to see if the vlan tag exist?

Yep, I see a VLAN tag.

Here's the trace of an example ping:

From the source VM (VLAN configured in guest OS, VLAN not on NIC, VLAN aware not enabled on host):

Out 1e:5b:03:0a:30:26 ethertype 802.1Q (0x8100), length 124: vlan 951, p 0, ethertype IPv6, fc00:b6d:cfc:951::6 > fc00:b6d:cfc:951::12: ICMP6, echo request, seq 69, length 64

From the source VM's hypervisor:

Out 1e:5b:03:0a:30:26 ethertype 802.1Q (0x8100), length 124: vlan 951, p 0, ethertype IPv6, fc00:b6d:cfc:951::6 > fc00:b6d:cfc:951::12: ICMP6, echo request, seq 105, length 64

To the destination VM's hypervisor:

Out 1e:b9:4c:18:4e:c9 ethertype IPv6 (0x86dd), length 120: fc00:b6d:cfc:951::12 > fc00:b6d:cfc:951::6: ICMP6, echo reply, seq 240, length 64
Out 1e:b9:4c:18:4e:c9 ethertype 802.1Q (0x8100), length 124: vlan 951, p 0, ethertype IPv6, fc00:b6d:cfc:951::12 > fc00:b6d:cfc:951::6: ICMP6, echo reply, seq 240, length 64

To the destination VM (VLAN not configured in guest OS, VLAN on NIC, VLAN aware not enabled on host):

Out 1e:b9:4c:18:4e:c9 ethertype IPv6 (0x86dd), length 120: fc00:b6d:cfc:951::12 > fc00:b6d:cfc:951::6: ICMP6, echo reply, seq 52, length 64

 

[William Edwards]

To make things crystal clear, I can reach devices in these ways:

• Host with VLAN aware enabled -> VM with vLAN defined inside guest -> VM with vLAN defined on NIC
• Host without VLAN aware enabled -> VM with vLAN defined inside guest -> VM with vLAN defined on NIC
• Host with VLAN aware enabled -> VM with vLAN defined on NIC -> VM with vLAN defined inside guest
• Host without VLAN aware enabled -> VM with vLAN defined on NIC -> VM with vLAN defined inside guest
• Host with VLAN aware enabled -> VM with vLAN defined on NIC -> VM with vLAN defined on NIC
• Host without VLAN aware enabled -> VM with vLAN defined on NIC -> VM with vLAN defined on NIC
• Host with VLAN aware enabled -> VM with vLAN defined inside guest -> VM with vLAN defined inside guest
• Host without VLAN aware enabled -> VM with vLAN defined inside guest -> VM with vLAN defined inside guest

 

[William Edwards]

I figured it out. This was an error at my end.

Thanks for your replies, @spirit !