[SOLVED] Proxmox Inconsistent with IPv6

[MeCJay12]

Hello! I recently spun up three Proxmox nodes (2 in HA, 1 standalone, all on the same switch stack) and everything was looking go so I got fully moved over but in the middle of moving my second DNS server, my entire network died. The primary DNS server was already migrated and wasn't responding regularly on IPv6. Most of my network uses IPv6 DNS server addresses exclusively so when I took the second DNS server offline to migrate it over, everything died. Proxmox lost the addresses of the storage servers so the VMs halted, which halted the firewall VMs. I was able to get everything back up by switch things over to manual IPs but this isn't ideal. Please help me understand what I misconfigured to cause this IPv6 issue.

On each node I have:

• An active-passive bond (bond0) to switch A and switch B with all VLANs allowed
• vmbr0, vlan aware, attached to bond0
• SDN zone "main" attached to vmbr0
• One Vnet for each VLAN
• Proxmox management IPs assigned to the "lab" Vnet

IPv4 is working perfectly fine as far as I can tell and the VMs were all restored from backup rather than rebuilt so the previously working networking configs are untouched on them. The DNS servers are also in the "lab" Vnet. Different VMs are connected different ways; some are connected to the lab Vnet, others are connected to vmbr0 with VLANs configured in them. The DNS servers specifically are Docker containers connected to a Macvlan, connected to a vlan interface in the VM, connected to vmbr0. I did notice that the Proxmox hosts themselves are not learning the link local default gateway from RAs but the VMs are. It seems to me like something is blocking icmpv6 as when I do tcpdumps, I'm not seeing neighbor solicitations come through. I have all firewalls turned off in the Proxmox UI. Thanks in advance for the help!

Spoiler: Networking config

$ sudo cat /etc/network/interfaces
# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

source /etc/network/interfaces.d/*

auto lo
iface lo inet loopback

auto enp4s0f0
iface enp4s0f0 inet manual

iface eno1 inet manual

auto enp4s0f1
iface enp4s0f1 inet manual

auto bond0
iface bond0 inet manual
        bond-slaves enp4s0f0 enp4s0f1
        bond-miimon 100
        bond-mode active-backup
        bond-primary enp4s0f0

auto vmbr0
iface vmbr0 inet manual
        bridge-ports bond0
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094
#trunk

auto lab
iface lab inet static
        address 192.168.2.52/24
        gateway 192.168.2.1

iface lab inet6 static
        address fc00:0:0:2::52/64
        gateway fc00:0:0:2::1
$ sudo cat /etc/network/interfaces.d/sdn
#version:9

auto home
iface home
        bridge_ports vmbr0.3
        bridge_stp off
        bridge_fd 0

auto lab
iface lab
        bridge_ports vmbr0.2
        bridge_stp off
        bridge_fd 0

auto noinet
iface noinet
        bridge_ports vmbr0.5
        bridge_stp off
        bridge_fd 0

auto papf
iface papf
        bridge_ports vmbr0.303
        bridge_stp off
        bridge_fd 0

auto sync
iface sync
        bridge_ports vmbr0.300
        bridge_stp off
        bridge_fd 0

auto vpn
iface vpn
        bridge_ports vmbr0.4
        bridge_stp off
        bridge_fd 0

auto wan1
iface wan1
        bridge_ports vmbr0.301
        bridge_stp off
        bridge_fd 0

auto wan2
iface wan2
        bridge_ports vmbr0.302
        bridge_stp off
        bridge_fd 0

Spoiler: IP -6 output

$ ip -6 a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 state UNKNOWN qlen 1000
    inet6 ::1/128 scope host noprefixroute
       valid_lft forever preferred_lft forever
6: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fe80::21b:21ff:fea3:e520/64 scope link
       valid_lft forever preferred_lft forever
8: home: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fe80::21b:21ff:fea3:e520/64 scope link
       valid_lft forever preferred_lft forever
10: lab: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fc00:0:0:2::52/64 scope global
       valid_lft forever preferred_lft forever
    inet6 fe80::21b:21ff:fea3:e520/64 scope link
       valid_lft forever preferred_lft forever
12: noinet: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fe80::21b:21ff:fea3:e520/64 scope link
       valid_lft forever preferred_lft forever
14: papf: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fe80::21b:21ff:fea3:e520/64 scope link
       valid_lft forever preferred_lft forever
16: sync: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fe80::21b:21ff:fea3:e520/64 scope link
       valid_lft forever preferred_lft forever
18: vpn: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fe80::21b:21ff:fea3:e520/64 scope link
       valid_lft forever preferred_lft forever
20: wan1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fe80::21b:21ff:fea3:e520/64 scope link
       valid_lft forever preferred_lft forever
22: wan2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fe80::21b:21ff:fea3:e520/64 scope link
       valid_lft forever preferred_lft forever
$ ip -6 r
fc00:0:0:2::/64 dev lab proto kernel metric 256 pref medium
fe80::/64 dev vmbr0 proto kernel metric 256 pref medium
fe80::/64 dev lab proto kernel metric 256 pref medium
fe80::/64 dev noinet proto kernel metric 256 pref medium
fe80::/64 dev papf proto kernel metric 256 pref medium
fe80::/64 dev sync proto kernel metric 256 pref medium
fe80::/64 dev vpn proto kernel metric 256 pref medium
fe80::/64 dev wan1 proto kernel metric 256 pref medium
fe80::/64 dev wan2 proto kernel metric 256 pref medium
fe80::/64 dev home proto kernel metric 256 pref medium
default via fc00:0:0:2::1 dev lab proto kernel metric 1024 onlink pref medium

 

[MeCJay12]

Found this post from a year ago that fixed it: https://forum.proxmox.com/threads/ipv6-configuration-problem-with-vm-bridge.121404/

You can test it with: echo 0 > /sys/devices/virtual/net/vmbr0/bridge/multicast_snooping

And make it permanent with "bridge-mcsnoop no" in /etc/networking/interfaces