No VLAN connection after reboot

[ioe]

Greetings.

I'm experience some weird behavior with my Proxmox (8.1.3) VLAN Setup.
When I change the network configuration from

auto vmbr0
iface vmbr0 inet static
        bridge-ports enp1s0f0np0
        bridge-stp off
        bridge-fd 0
        address 192.168.1.12/24
        gateway 192.168.1.1

to

auto vmbr0
iface vmbr0 inet manual
        bridge-ports enp1s0f0np0
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-100

auto vmbr0.20
iface vmbr0.20 inet static
        address 192.168.1.12/24
        gateway 192.168.1.1

it works fine until i reboot proxmox.
After the reboot i cant ping my proxmox server.
The configuration displayed with "ip a" is identical after the reboot.

3: enp1s0f0np0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr0 state UP group default qlen 1000
    link/ether brd ff:ff:ff:ff:ff:ff
5: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether brd ff:ff:ff:ff:ff:ff
    inet6 fe80::e42:a1ff:fe98:ff6c/64 scope link
       valid_lft forever preferred_lft forever
6: vmbr0.20@vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether  brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.12/24 scope global vmbr0.20
       valid_lft forever preferred_lft forever
    inet6 fe80::e42:a1ff:fe98:ff6c/64 scope link
       valid_lft forever preferred_lft forever

I've also tried to restart my network service and the interfaces.. but it still wont work.
To make it work again, I have to reset it to the none VLAN config, reboot, and then enable VLAN again.

Does anybody have any idea why this doesn't work properly?

regards

 

[ioe]

I narrowed to problem down to the used NIC (Mellanox CX4121A ConnectX-4). It seems there is an issue with VLAN hardware acceleration.

 

[jsterr]

I narrowed to problem down to the used NIC (Mellanox CX4121A ConnectX-4). It seems there is an issue with VLAN hardware acceleration.

Did you check your syslog? Are there errors regarding VLAN Offloading? Or wheres your conclusion coming from?

 

[ioe]

The syslog was quite normal.
The only thing that popped up were messages from the mellanox driver that the VLAN-List is full and some VLANs were trunked (when binding VLAN 2-4092 to vmbr0. For this reason i limited the VLAN binding to 2-100). Turns out the Connex-4 NIC only supports 512 Vlans.

When I swap the Mellanox with an older NIC, this setup works flawlessly.
So, it seems quite likely to me, that the VLAN offloading is not working correctly on Proxmox 8.1.3 and Mellanox.

 

[deanjarnold]

I also had these issues and was able to resolve with my ConnectX 4 LX following these two forum post's suggestions:

VLAN acceleration/512 issue: https://forum.proxmox.com/threads/proxmox-7-and-mellanox-connectx4-and-vlan-aware-bridge.104926/
Linux Bridge no ping/traffic issue: https://forum.proxmox.com/threads/m...nd-brigde-vlan-aware-on-proxmox-8-0-1.130902/

My resulting /etc/network/interfaces file looks like this:

auto lo
iface lo inet loopback

iface ens6f0np0 inet manual
        pre-up ethtool -K $IFACE rx-vlan-filter off

iface ens6f1np1 inet manual

auto vmbr0
iface vmbr0 inet static
        address 10.5.0.45/24
        gateway 10.5.0.1
        bridge-ports ens6f0np0
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094
        # 80:00.0 is the PCI address for ens6f0np0
        post-up devlink dev eswitch set pci/0000:81:00.0 mode switchdev

source /etc/network/interfaces.d/*