Kernel 4.2.3-2-pve - Kernel Bug at net/8021q/vlan.c:89

[methil]

Hi, got a nasty crash this morning while testing PVE 4.x



How-to reproduce
1 : install pve with latest iso and patch/upgrade to latest 4.x version from pve-no-subscription
2 : Network switch port is configured with 2 tagged vlan. No untag vlan assignement.
3 : Move pve management to a separate vlan and add a bridge with the same eth with vlan awareness active.
4 : Reboot once to apply and reboot one more time to get the crash.
As long as you have a bridge bind to that interface and another one with the same interface with a vlan you will get that crash.
See image below for network configuration on the pve host.



With pve 3.x no problem all is working fine. We have a multinodes cluster up and running with 100+ kvm guests.

 

[spirit]

>>Move pve management to a separate vlan and add a bridge with the same eth with vlan awareness active.

Can you post your /etc/network/interfaces file ?

 

[valshare]

-- delete --

 

[methil]

auto lo
iface lo inet loopback

iface eth0 inet manual

iface eth1 inet manual

auto vmbr1997
iface vmbr1997 inet static
address 10.10.253.115
netmask 255.255.255.192
bridge_ports eth1.1997
bridge_stp off
bridge_fd 0

auto vmbr2049
iface vmbr2049 inet static
address 192.168.129.105
netmask 255.255.255.0
gateway 192.168.129.1
bridge_ports eth1.2049
bridge_stp off
bridge_fd 0

auto vmbr0
iface vmbr0 inet manual
bridge_ports none
bridge_stp off
bridge_fd 0

auto vmbr1
iface vmbr1 inet manual
bridge_ports eth1
bridge_stp off
bridge_fd 0
bridge_vlan_aware yes

 

[spirit]

You don't need bridge_vlan_aware, if you use eth1.x iinterfaces.

vlan aware bridge are for tagging directly inside the vmbr1. (1 vmbr with multiple vlans).
Proxmox use it when you use vlan tag option in vms network config.

 

[methil]

You don't need bridge_vlan_aware, if you use eth1.x iinterfaces.

vlan aware bridge are for tagging directly inside the vmbr1. (1 vmbr with multiple vlans).
Proxmox use it when you use vlan tag option in vms network config.

I do understand what you say BUT, how can i bridge the datastore on vlan XXXX or the management then?
As far as i know there is no field available to specify ethX.vlan into the storage tab. Same thing for the management, we can not specify anything actually because there is not tab for it.
Clearly there is something wrong in the way vlans are handle in the bridge right now.
That what we used to do with PVE 3.x and it work perfectly but not in PVE 4.x

My current configuration on PVE 3.x is like this

vmbr0 is not bind to any interface we call it the dummy interface
vmbr1 is bind to eth0 and used with every kvm guest. Each guest specify it vlan appartenance in the Vlan Tag field of the network device.
vmbr4000 bind to eth0.4000 for the management network
vmbr4001 bind to eth0.4001 for the storage network
vmbr4002 bind to eth0.4002 as a temp vmotion storage
vmbr5000 bind to eth1 (same as vmbr1 but link to another physicaly separated network) and used in some guests
vmbr5001 bind to eth1.5001 for a remote storage network (crash desaster replication link)
I could add more vlans but we have enough here to understand the topology.
This is a very simple setup and vlan usage is minimal.

If you do bond interfaces, they will be regroup under a single bond/bridge and you are back to square one.
We do have such a cluster setup with multi switches lacp bonding bridges with a lot more vlans and networks
Dedicate a whole interface just for ethX.vlan assignement is not even an option on that setup.

 

[spirit]

I do understand what you say BUT, how can i bridge the datastore on vlan XXXX or the management then?
As far as i know there is no field available to specify ethX.vlan into the storage tab. Same thing for the management, we can not specify anything actually because there is not tab for it.
Clearly there is something wrong in the way vlans are handle in the bridge right now.
That what we used to do with PVE 3.x and it work perfectly but not in PVE 4.x

My current configuration on PVE 3.x is like this

vmbr0 is not bind to any interface we call it the dummy interface
vmbr1 is bind to eth0 and used with every kvm guest. Each guest specify it vlan appartenance in the Vlan Tag field of the network device.
vmbr4000 bind to eth0.4000 for the management network
vmbr4001 bind to eth0.4001 for the storage network
vmbr4002 bind to eth0.4002 as a temp vmotion storage
vmbr5000 bind to eth1 (same as vmbr1 but link to another physicaly separated network) and used in some guests
vmbr5001 bind to eth1.5001 for a remote storage network (crash desaster replication link)
I could add more vlans but we have enough here to understand the topology.
This is a very simple setup and vlan usage is minimal.

If you do bond interfaces, they will be regroup under a single bond/bridge and you are back to square one.
We do have such a cluster setup with multi switches lacp bonding bridges with a lot more vlans and networks
Dedicate a whole interface just for ethX.vlan assignement is not even an option on that setup.

In proxmox 3.X, you don't have the "bridge_vlan_aware yes" option.
If you want the same config in proxmox 4.0, don't enable it, keep exactly the same config.



If you want to have management ip, on a vlan aware bridge, you need to defined
bridge vlan management interface like this:

[COLOR=#333333]auto vmbr1.1997[/COLOR]
[COLOR=#333333]iface vmbr1.1997 inet static[/COLOR]
[COLOR=#333333]address 10.10.253.115[/COLOR]
[COLOR=#333333]netmask 255.255.255.192[/COLOR]

[COLOR=#333333]auto vmbr1.2049[/COLOR]
[COLOR=#333333]iface vmbr1.2049 inet static[/COLOR]
[COLOR=#333333]address 192.168.129.105[/COLOR]
[COLOR=#333333]netmask 255.255.255.0[/COLOR]
[COLOR=#333333]gateway 192.168.129.1[/COLOR]

 

[methil]

I see, the name mislead me and tough i should activate it to get the 802.1Q (vlan support) enable on that bridge.
That thing should be name "Enable 802.1ad (Q-in-Q)" not "Vlan Aware". Poor choice of words imo.

Thank you for your help.

 

[spirit]

I see, the name mislead me and tough i should activate it to get the 802.1Q (vlan support) enable on that bridge.
That thing should be name "Enable 802.1ad (Q-in-Q)" not "Vlan Aware". Poor choice of words imo.

Thank you for your help.

No,
It's Q-in-Q only with you plug an eth0.x<---vmbr(vlanaware)<----vm (tagged).
This will be double tagged.


vlan aware option only enable vlan support inside the bridge.
That's why I tell you to do:

eth0<------vmbr(vlanaware)<----vm (tagged)
-vmbr.X
-vmbr.x

Like this It's tagged only once, like a real switch.