Bonding, Bridges and VLANS

[dalbano]

Hi,

We have 3 Proxmox hosts in a cluster which attach to 2 separate switches. Each host has 2 LACP bonds, and both of these bonds are bridged through vmbr0. We also have 3 VLANs we need to access, vlan 200 (management), vlan 20 (production) and vlan 150 (san). vlan 200 is untagged on vmbr0, vlan 20 and 150 are tagged. VLAN 200 is working fine, but we can't bring up the vlans. /etc/network/interfaces follows:

# network interface settings
auto lo
iface lo inet loopback


iface eth0 inet manual
iface eth1 inet manual
iface eth2 inet manual
iface eth3 inet manual


auto vlan20
iface vlan20 inet manual
vlan_raw_device bond0.20
auto vlan150
iface vlan150 inet manual
vlan_raw_device bond1.150

auto bond0
iface bond0 inet manual
slaves eth0 eth1
bond_miimon 100
bond_mode 802.3ad


auto bond1
iface bond1 inet manual
slaves eth2 eth3
bond_miimon 100
bond_mode 802.3ad




auto vmbr0
iface vmbr0 inet static
address 10.200.1.61
netmask 255.255.255.0
gateway 10.200.1.250
bridge_ports bond0 bond1
bridge_stp off
bridge_fd 0


auto vmbr20
iface vmbr20 inet static
address 10.20.1.61
netmask 255.255.255.0
bridge_ports vlan20
bridge_stp off
bridge_fd 0


auto vmbr150
iface vmbr150 inet static
address 10.150.1.61
netmask 255.255.255.0
bridge_ports vlan150
bridge_stp off
bridge_fd 0

The server is version 2. Any help bringing up vlans 20 and 150 would be appreciated.

 

[dietmar]

You do not need any vlan configuration in /etc/network/interfaces. Simply select the VLAN ID when you create the VM. Well, I would create on bridge for each bond.

 

[dalbano]

VMs will reside on vlan20, but the host needs to access vlan 150 (san vlan) for iSCSI targets for each VM. At this point, with ip addressing defined in /etc/interfaces, I can't ping anything on the other vlans.

Dean

 

[webstaff]

I'm been using eth1.X x being vlan or bond0.x where that is the vlan that seems to work ok.

so I usally bond the two eths together then refere to them on bond0.20 bond0.150 bond0.200 and point vmbr at the bond on that vlan if you need to for any reason.
You can also do eth0.20 eth1.20 and bond those to bond0 no tag and that then shows properly in the GUI as PM VE doesnt like .x bonds in the web gui but works for me so hey. A word of caution though i'm having a few issues with HA ATM i don't think its related to how i'm using vlan tagging but if it is i'll report back when we figure out whats happening.

Regards Dave

 

[dalbano]

I tried doing exactly that. The server could not parse /etc/interfaces properly. I'm guessing that was due to too many interfaces defined. In short, I tried to define additional bonds for the vlans to use, then attach the bonds to the vlan bridges. Remember that here I have bond0 AND bond1 (both being LACP pairs going to separate switches, which are both bridged together and managed by spanning-tree).

 

[webstaff]

I tried doing exactly that. The server could not parse /etc/interfaces properly. I'm guessing that was due to too many interfaces defined. In short, I tried to define additional bonds for the vlans to use, then attach the bonds to the vlan bridges. Remember that here I have bond0 AND bond1 (both being LACP pairs going to separate switches, which are both bridged together and managed by spanning-tree).

Not sure about spanning-tree side of things not got that far yet but taking that out of the conversation.

But i think it should something more like below might need a little tweak of the bond 0/.20/.150 to bond0 bond1 bond 2 as calling the bond .20 and having the ethx.20 might look for a second vlan tag that doesn't exist as the lan took the packet, still not 100% with linux networking only started couple of weeks ago lol just about getting it now.

You'll see extra interfaces that don't need to be active but useful if you populate them with ip address to test and see where the problem is or hopefully isn't
auto lo
iface lo inet loopback

iface eth0 inet manual
iface eth1 inet manual
iface eth2 inet manual
iface eth3 inet manual
#define vlaned eths
auto eth0.20
iface eth0.20 inet manual
auto eth1.20
iface eth1.20 inet manual
auto eth2.150
iface eth2.150 inet manual
auto eth3.150
iface eth3.150 inet manual
#call all eths to bond0 narative vlan
auto bond0
iface bond0 inet manual
slaves eth0 eth1 eth2 eth3
bond_miimon 100
bond_mode 802.3ad
#bond on vlan 20
auto bond0.20
iface bond0.20 inet manual
slaves eth0.20 eth1.20
bond_miimon 100
bond_mode 802.3ad
#bond on vlan 150
auto bond0.150
iface bond1 inet manual
slaves eth2.150 eth3.150
bond_miimon 100
bond_mode 802.3ad

#vm lan / vlan respectively just as before but calling the respective vlans on the bond.
auto vmbr0
iface vmbr0 inet static
address 10.200.1.61
netmask 255.255.255.0
gateway 10.200.1.250
bridge_ports bond0
bridge_stp off
bridge_fd 0

auto vmbr20
iface vmbr20 inet static
address 10.20.1.61
netmask 255.255.255.0
bridge_ports bond0.20
bridge_stp off
bridge_fd 0

auto vmbr150
iface vmbr150 inet static
address 10.150.1.61
netmask 255.255.255.0
bridge_ports bond0.150
bridge_stp off
bridge_fd 0