[SOLVED] IPv6 connection unstable

[Aluveitie]

Running Proxmox with two 10GbE interfaces, one in the LAN for directly accessible VMs and one in a separate INT network for Proxmox (and some VM).
Over IPv4 everything works fine, but over IPv6 I'm frequently disconnected from the server (SSH with root@server:~# client_loop: send disconnect: Broken pipe for example). It isn't a general problem, other IPv6 connections both same network and through the firewall run perfectly fine.

My Proxmox network setup:

auto lo
iface lo inet loopback

auto eno1
iface eno1 inet manual
    ovs_type OVSPort
    ovs_bridge vmbr01
    ovs_mtu 9000
#INT

auto eno2
iface eno2 inet manual
    ovs_type OVSPort
    ovs_bridge vmbr02
    ovs_mtu 9000
#LAN

iface enp5s0 inet manual

auto veno1
iface veno1 inet dhcp
    ovs_type OVSIntPort
    ovs_bridge vmbr01
    ovs_mtu 9000
#INT proxmox

iface veno1 inet6 dhcp

auto vmbr01
iface vmbr01 inet manual
    ovs_type OVSBridge
    ovs_ports eno1 veno1
    ovs_mtu 9001
#INT bridge

auto vmbr02
iface vmbr02 inet manual
    ovs_type OVSBridge
    ovs_ports eno2
    ovs_mtu 9000
#LAN bridge

auto vmbr10
iface vmbr10 inet manual
    ovs_type OVSBridge

auto vmbr11
iface vmbr11 inet manual
    ovs_type OVSBridge

post-up echo 1 > /proc/sys/net/ipv6/conf/all/accept_ra

(vmbr10/11 are bridges between VMs)

I think the issue is somewhere with `eno2`, when using a dedicated PCIe NIC passed through to the VM and eno2 disconnected I didn't experience those issues.

 

[Aluveitie]

I redid the networking setup with plain linux bridges, but still the same issue:

auto lo
iface lo inet loopback

iface enp5s0 inet manual

auto eno1
iface eno1 inet manual
   mtu 9000
#INT

auto eno2
iface eno2 inet manual
   mtu 9000
#LAN

auto vmbr01
iface vmbr01 inet dhcp
   bridge-ports eno1
   bridge-stp off
   bridge-fd 0
   mtu 9000
   up sleep 5
   up dhclient -1 -6 -cf /etc/dhcp/dhclient6.conf -lf /var/lib/dhcp/dhclient6.vmbr01.leases -v vmbr01 || true
#INT bridge

auto vmbr02
iface vmbr02 inet manual
   bridge-ports eno2
   bridge-stp off
   bridge-fd 0
   mtu 9000
#LAN bridge

auto vmbr10
iface vmbr10 inet manual
   bridge-ports none
   bridge-stp off
   bridge-fd 0
   mtu 9000
#truenas to truenas-backup

auto vmbr11
iface vmbr11 inet manual
   bridge-ports none
   bridge-stp off
   bridge-fd 0
   mtu 9000
#transmission to truenas

post-up echo 1 > /proc/sys/net/ipv6/conf/all/accept_ra

 

[Aluveitie]

Two interesting things I noticed:
vmbr01 gets the correct IPv6 from the DHCP server, but the wrong netmask... /128 instead of /64

4: vmbr01: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue state UP group default qlen 1000
    link/ether xx:xx:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff
    inet x.x.x.2/16 brd 10.1.255.255 scope global dynamic vmbr01
       valid_lft 6570sec preferred_lft 6570sec
    inet6 xxxx:xxxx:xxxx:1::2/128 scope global
       valid_lft forever preferred_lft forever

And more importantly, for IPv6 it vmbr01 uses eno2 instead of eno1. I see the IPv6 traffic being denied by the Firewall on the LAN interface (which eno2 is connect to) instead of the INT interface (which eno1 is connected to).

But the bridge configuration looks fine...

root@server:~# brctl show
bridge name    bridge id        STP enabled    interfaces
vmbr01        8000.18c04de3e67f    no        eno1
vmbr02        8000.18c04de3e681    no        eno2
vmbr10        8000.72ae80e906f4    no      
vmbr11        8000.f21beffddd54    no