VM with 802.1q interface has wrong vid on proxmox bridge when multicast snooping

[chill]

Hi,

Running a very simple Debian test vm on a vlan aware bridge (vmbr0) with two network interfaces passed to it. One is tagged to vlan10 (net0 below) and the other is a trunk (net1 below). Problem - the proxmox bridge puts the wrong vid tag when performing multicast snooping (and thus breaks ipv6).

VM configuration, nothing exotic, note the net0 and net1 config.

root@proxmox:~# pveversion
pve-manager/8.0.4/d258a813cfa6b390 (running kernel: 6.2.16-19-pve)

root@proxmox:~# cat /etc/pve/nodes/proxmox1/qemu-server/101.conf
boot: order=scsi0;ide2;net0
cores: 1
cpu: x86-64-v2-AES
memory: 2048
meta: creation-qemu=8.1.2,ctime=1700156943
name: debtest
net0: virtio=12:62:A6:28:67:82,bridge=vmbr0,firewall=1,tag=10
net1: virtio=D6:16:52:00:12:D3,bridge=vmbr0,firewall=1
numa: 0
ostype: l26
scsi0: local-zfs:vm-101-disk-0,iothread=1,size=32G
scsihw: virtio-scsi-single
smbios1: uuid=ad41e289-a468-49e2-a4b4-9ba07db882a1
sockets: 1
vmgenid: 78c03117-40de-409e-8265-1f24548b6c89

In the Debian VM, the two network interfaces are setup as dhcp / auto, and we see the multicast addresses (grepped for the solicited node). Again, nothing exotic.

root@debian:~# cat /etc/network/interfaces
source /etc/network/interfaces.d/*

auto lo
iface lo inet loopback

auto ens18
iface ens18 inet dhcp
iface ens18 inet6 auto

auto ens19.10
iface ens19.10 inet dhcp
iface ens19.10 inet6 auto

root@debian:~# ip maddr show ens18 | grep ff02
    inet6 ff02::1:ff28:6782 users 2
    inet6 ff02::1
root@debian:~# ip maddr show ens19 | grep ff02
    inet6 ff02::1:ff00:12d3
    inet6 ff02::1
root@debian:~# ip maddr show ens19.10 | grep ff02
    inet6 ff02::1:ff00:12d3 users 2
    inet6 ff02::1

So far so good. Proxmox can ping them.

root@proxmox:~# ping -c 1 ff02::1:ff28:6782%vmbr0.10
PING ff02::1:ff28:6782%vmbr0.10(ff02::1:ff28:6782%vmbr0.10) 56 data bytes
64 bytes from fe80::1062:a6ff:fe28:6782%vmbr0.10: icmp_seq=1 ttl=64 time=0.376 ms

--- ff02::1:ff28:6782%vmbr0.10 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.376/0.376/0.376/0.000 ms

root@proxmox:~# ping -c 1 ff02::1:ff00:12d3%vmbr0
PING ff02::1:ff00:12d3%vmbr0(ff02::1:ff00:12d3%vmbr0) 56 data bytes
64 bytes from fe80::d416:52ff:fe00:12d3%vmbr0: icmp_seq=1 ttl=64 time=0.406 ms

--- ff02::1:ff00:12d3%vmbr0 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.406/0.406/0.406/0.000 ms

root@proxmox:~# ping -c 1 ff02::1:ff00:12d3%vmbr0.10
PING ff02::1:ff00:12d3%vmbr0.10(ff02::1:ff00:12d3%vmbr0.10) 56 data bytes
64 bytes from fe80::d416:52ff:fe00:12d3%vmbr0.10: icmp_seq=1 ttl=64 time=0.306 ms

--- ff02::1:ff00:12d3%vmbr0.10 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.306/0.306/0.306/0.000 ms

Now for the problem. Turning on multicast snooping and looking at the mdb table we see the address associated with net0 / ens18 has "vid 10" (good), whereas the address for net1 / ens19.10 is only against "vid 1" (bad).

root@proxmox:~# echo 1 > /sys/class/net/vmbr0/bridge/multicast_snooping
root@proxmox:~# bridge mdb | grep 101
dev vmbr0 port fwpr101p0 grp ff02::1:ff28:6782 temp vid 10
dev vmbr0 port fwpr101p1 grp ff02::1:ff00:12d3 temp vid 1

And now the final ping fails.

root@proxmox:~# ping -c 1 ff02::1:ff28:6782%vmbr0.10
PING ff02::1:ff28:6782%vmbr0.10(ff02::1:ff28:6782%vmbr0.10) 56 data bytes
64 bytes from fe80::1062:a6ff:fe28:6782%vmbr0.10: icmp_seq=1 ttl=64 time=0.376 ms

--- ff02::1:ff28:6782%vmbr0.10 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.376/0.376/0.376/0.000 ms

root@proxmox:~# ping -c 1 ff02::1:ff00:12d3%vmbr0
PING ff02::1:ff00:12d3%vmbr0(ff02::1:ff00:12d3%vmbr0) 56 data bytes
64 bytes from fe80::d416:52ff:fe00:12d3%vmbr0: icmp_seq=1 ttl=64 time=0.406 ms

--- ff02::1:ff00:12d3%vmbr0 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.406/0.406/0.406/0.000 ms

root@proxmox:~# ping -c 1 ff02::1:ff00:12d3%vmbr0.10
PING ff02::1:ff00:12d3%vmbr0.10(ff02::1:ff00:12d3%vmbr0.10) 56 data bytes

--- ff02::1:ff00:12d3%vmbr0.10 ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms

 

[gurubert]

I do not think it is a good idea to have two vNICs for one VM on the same bridge.
You get the same packet passed twice to your VM.
ARP or Neighbor Discovery replies will be sent out with the MAC of either of the two vNICs which confuses the network.

 

[chill]

I do not think it is a good idea to have two vNICs for one VM on the same bridge.

You are correct, however this example was just to illustrate that trunking vlans into a VM breaks multicast snooping on the proxmox bridge.

E.g. say I had 100 vlans, I would need 100 vNICs to make the multicast snooping work correctlly it seems.