VLAN 64 with bond0 for the Proxmox VE management IP with VLAN aware Linux bridge

[tessus]

I've put my Proxmox machine in VLAN64 and the initial installation of Proxmox with just one interface was fine. I could reach the machine and VMs and also from a VM to my network worked without issues.

Then I wanted to create a bond and a VLAN aware Linux bridge. Thus I activated LACP on my switch for the 2 ports (Active, Long) and changed the network config to:

auto lo
iface lo inet loopback

iface enp4s0 inet manual

iface enp6s0 inet manual

auto bond0
iface bond0 inet manual
    bond-slaves enp4s0 enp6s0
    bond-miimon 100
    bond-mode 802.3ad
    bond-xmit-hash-policy layer2

auto vmbr0.64
iface vmbr0.64 inet static
    address  192.168.64.10
    netmask  255.255.240.0
    gateway  192.168.64.1

auto vmbr0
iface vmbr0 inet manual
    bridge-ports bond0
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes

But.... no more network connection to/fro that box. I must have screwed up the config, but I can't find the culprit. I can't see the forest for the trees.

These are the relevant messages in my dmesg output:

[    8.136716] Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)
[    8.138494] bonding: bond0 is being created...
[    8.148980] new mount options do not match the existing superblock, will be ignored
[    8.165182] softdog: initialized. soft_noboot=0 soft_margin=60 sec soft_panic=0 (nowayout=0)
[    8.169166] bond0: (slave enp4s0): Enslaving as a backup interface with a down link
[    8.197430] bond0: (slave enp6s0): Enslaving as a backup interface with a down link
[    8.218480] vmbr0: port 1(bond0) entered blocking state
[    8.219231] vmbr0: port 1(bond0) entered disabled state
[    8.220301] device bond0 entered promiscuous mode
[    8.221036] device enp4s0 entered promiscuous mode
[    8.221819] device enp6s0 entered promiscuous mode
[    8.332749] device bond0 left promiscuous mode
[    8.333682] device enp4s0 left promiscuous mode
[    8.334475] device enp6s0 left promiscuous mode
[    8.499720] 8021q: 802.1Q VLAN Support v1.8
[    8.500487] 8021q: adding VLAN 0 to HW filter on device enp4s0
[    8.501353] 8021q: adding VLAN 0 to HW filter on device enp6s0
[    8.502143] 8021q: adding VLAN 0 to HW filter on device bond0
[    8.550702] bpfilter: Loaded bpfilter_umh pid 1984
[    8.550910] Started bpfilter
[   11.148569] igb 0000:06:00.0 enp6s0: igb: enp6s0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
[   11.160563] igb 0000:04:00.0 enp4s0: igb: enp4s0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
[   11.436270] bond0: (slave enp4s0): link status definitely up, 1000 Mbps full duplex
[   11.437048] bond0: Warning: No 802.3ad response from the link partner for any adapters in the bond
[   11.437659] bond0: active interface up!
[   11.438442] bond0: (slave enp6s0): link status definitely up, 1000 Mbps full duplex
[   11.439076] vmbr0: port 1(bond0) entered blocking state
[   11.439704] vmbr0: port 1(bond0) entered forwarding state
[   11.440941] IPv6: ADDRCONF(NETDEV_CHANGE): vmbr0: link becomes ready
[   11.441693] IPv6: ADDRCONF(NETDEV_CHANGE): vmbr0.64: link becomes ready

Shouldn't there be at least one message about VLAN64?

As reference, the config below was the initial, out-of-the-box network config that is working. The ports on the switch (to which the Proxmox machine are connected) are using PVID 64.

auto lo
iface lo inet loopback

iface enp4s0 inet manual

auto vmbr0
iface vmbr0 inet static
    address 192.168.64.10/20
    gateway 192.168.64.1
    bridge_ports enp4s0
    bridge_stp off
    bridge_fd 0

iface enp6s0 inet manual

 

[spirit]

it's missing "bridge-vids 2-4094" in vmbr0 to defined allowed vlans

also, what is your swtich model ? do you have configured your switch to allow vlan64 instead forcing it? (for example, in cisco, this a trunk mode vs access mode)

 

[tessus]

Thanks a lot for the reply.

My switch is not as fancy as a Cisco one. I'm using a D-Link DGS-1210-10. And my VLAN requirements are really simple. All VMs and CTs on my Proxmox machine should be in VLAN 64. It makes not much sense to use different VLANs for different guest systems. (In my setup that is.) Although maybe someone can convince me otherwise.

With the initial network setup (from the install), my switch config looked like this:





Port 8 is the Trunk that goes to my router.

it's missing "bridge-vids 2-4094" in vmbr0 to defined allowed vlans

Thanks, this did it. However, I had to change my VLAN config on the switch to:



And it also has another slight caveat. It requires me to set VLAN tag 64 on every VM/CT, otherwise they can't be reached from my other VLAN. I allow inter-VLAN routing (only between 1 and 64) on my router, since I have a requirement to allow a few machines from each of these 2 VLANs to communicate with each other.

I would have thought that without setting a tag they would be automatically in VLAN 64, since the PVID is set to 64 for port 5 and 6.

I think in my initial setup without the VLAN aware bridge I did not have to set the VLAN tag explicitly to 64.

I also seem to have a few issues with the firewall settings in Proxmox, but that's a topic for another time. I do have quite some experience in that area, but it just doesn't seem to work properly. Also I think it is frowned upon in most forums to discuss different topics in one thread.

Maybe my 2 questions are out of scope, but here they are:

What would be the most straight forward setup, if I wanted

• all guests to be in VLAN 64?
• to use X different VLANs?