[SOLVED] VLAN aware bridge not working

[dustojnikhummer]

Hi, I could use some help with configuring a VLAN aware network bridge.
Traditional VLAN interface works. And by works I mean "it can see other devices on either VLAN as well as internet and other devices can see the node and the WebUI". PVE firewall is turned off (both on Datacenter and Node).
I'm using a USB network card to access the WebUI, that's the enxc0eac36421a9.
eno1 is connected to a Trunk port on my Mikrotik RB260G.
Node is an HP Prodesk 600 G4 with Intel I219-LM NIC.
Goal is a vm bridge with an IP address in VLAN 40 (for WebUI) and ability to set VLAN tags on vmbr0 for each VM

auto lo
iface lo inet loopback

auto eno1
iface eno1 inet manual
        post-up /sbin/ethtool -K $IFACE tso off gso off gro off

auto enxc0eac36421a9
iface enxc0eac36421a9 inet static
        address 10.200.40.11/24

auto eno1.10
iface eno1.10 inet static
        address 10.200.10.10/24

auto eno1.40
iface eno1.40 inet static
        address 10.200.40.10/24
        gateway 10.200.40.254

This manual VLAN setup works with other Linux devices using the classic dot1q setup too, so I don't think the switch or the router is the issue here.
The following (using vlan aware bridge) doesn't work

auto lo
iface lo inet loopback

auto eno1
iface eno1 inet manual
        post-up /sbin/ethtool -K $IFACE tso off gso off gro off

auto enxc0eac36421a9
iface enxc0eac36421a9 inet static
        address 10.200.40.11/24

auto vmbr0
iface vmbr0 inet static
        address 10.200.40.10/24
        gateway 10.200.40.254
        bridge-ports eno1
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 10 40

Or am I totally misunderstanding how the VLAN aware bridge is supposed to function?

 

[dustojnikhummer]

Well, after a few hours of troubleshooting I have a solution. In retrospect it is very easy and makes a lot of sense.

TLDR: Use a second NIC for setting it up (or a USB NIC), create a VLAN aware bridge with no gateway and no IP address and connect that to your switch's trunk port, create a bridge VLAN port on your preferred management VLAN for your host that will have IP and gateway. I only realized this after I spun up a container and saw that while my node couldn't see the internet the container could.

auto lo
iface lo inet loopback

auto eno1
iface eno1 inet manual
        post-up /sbin/ethtool -K $IFACE tso off gso off gro off

auto vmbr0
iface vmbr0 inet manual
        bridge-ports eno1
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 10 40

auto vmbr0.40
iface vmbr0.40 inet static
        address 10.200.40.10/24
        gateway 10.200.40.254