Bonding, Bridging, and vLANS

[bitbud]

I have 2 NICs, eth0 and eth1. I want to using bonding to create a trunk port with my Cisco switch using 802.3ad. I have multiple vLANs on that switch, and would like the create separate bridge interfaces for each vLAN.

In Debian, I can accomplish this on the command line several ways. I would usually create a bond with the interfaces as eth0.5 and eth1.5, which would create a bond on the 5 vLAN. And then create a bridge on that bond. Now all traffic on that bridge will be on that specific vLAN.

How would I do this in the Proxmox web interface? Is it possible?

In the web interface, bonding is only allowed connections to the network interface eth0 or eth1, not eth0.xxx. When adding a slave as eth1.xxx an error message appears 'Unable to add slave device eth1.xxx'

Is there another method?

The ideal end result would be having multiple bridges available, each on a different vLAN. Those bridges would be connected over a bonded trunk of multiple physical interfaces.

I am aware that I can create a bridge directly over a single interface and assign a vLAN (bridge with bridge ports eth1.xxx, for example). The goal I am after is to support the bridge over the bonded interface, with vLANs.

Anyone?

 

[dietmar]

As a workaround, directly edit /etc/networ/interfaces:

auto bond0
iface bond0 inet manual
slaves eth2.23
...


Does that work?

 

[bitbud]

Doesn't work

That doesn't work.

This is my complete interfaces file:

auto lo
iface lo inet loopback

auto bond0
iface bond0 inet manual
slaves eth0.56 eth1.56
bond_miimon 100
bond_mode active-backup

auto vmbr0
iface vmbr0 inet static
address 10.5.56.72
netmask 255.255.254.0
gateway 10.5.57.4
bridge_ports bond0
bridge_stp off
bridge_fd 0


Cannot ping any addresses on the 56 vLAN (of which vmbr0 is assigned).


I supposed a WORK AROUND would be to skip bonding, and assigned bridges to a vLAN, and bond within the virtual - which would would reduce performance. VMWare's ESX allows the creation of a single trunk, and then the ability to divide it up the bridges by vLAN.

So a request (aside from making this just work) ... the web interface currently limits the number of bridges to 10 (vmbr0-9) can that be increased?

As you can see, I can get what I want via the command line, but would prefer to stick with the features Proxmox makes available via the web interface.

Thanks.

 

[dietmar]

Cannot ping any addresses on the 56 vLAN (of which vmbr0 is assigned).

What config do you use usually for that?

I tried the following:

auto eth1.56
iface eth1.56 inet manual
 
auto eth2.56
iface eth2.56 inet manual
 
auto bond0
iface bond0 inet manual
        slaves eth1.56 eth2.56
        bond_miimon 100
        bond_mode active-backup
        pre-up ifup eth1.56 eth2.56
        post-down ifdown eth1.56 eth2.56

That create all devices, but i cant test if it really works, because my switch does not like vlans ;-)

I supposed a WORK AROUND would be to skip bonding, and assigned bridges to a vLAN, and bond within the virtual - which would would reduce performance. VMWare's ESX allows the creation of a single trunk, and then the ability to divide it up the bridges by vLAN.

What if you use something like:

auto bond0
iface bond0 inet manual
        slaves eth0 eth1
        bond_miimon 100
        bond_mode active-backup
 
auto bond0.75
iface bond0.75 inet manual
        vlan-raw-device bond0

And then use bond0.XXX for the bridges? Does that work?

So a request (aside from making this just work) ... the web interface currently limits the number of bridges to 10 (vmbr0-9) can that be increased?

No, currently not. Whats the usage scenario for that?

- Dietmar

 

[bitbud]

That works! Thanks Dietmar!

Dietmar, your network-fu is better than mine. You nailed it! That's the correct config for the interfaces file to make it work properly. I'll post a completed one below for others to benefit from. THANK YOU!

Two things to note
1 - the web interface won't allow bond creation on interface.vlan, only interface (eth0 is OK but eth0.56 is not). However, if it is manually put into the interfaces file, it works, and shows up properly in the web interface.
2 - you asked for a usage case scenario for more than 10 bridges? I have 11 vlans on my network, so I need to have 11 bridges. Actually, I also need to have 11 bonds as well. Apparently, I could configure this manually and be OK - I was just making a suggestion for getting the web interface more usable. I think 0-99 of both bonds and bridges would cover most everyone.

I would like to convert all my production KVM servers (running Debian/ Ubuntu + KVM) over to Proxmox. One of those systems has 18 Guests, with 30+ bridges. I don't want to move to Proxmox on that system until the web interfaces fully supports its configuration (as others in my organization are wary of the command line - go figure).

BTW - GREAT work with Proxmox! This is a real break through in getting KVM into the mainstream. Would love to see the community get more involved in the development cycle (SCM access), but your quality control has been great, V1 is quite stable in a variety of work loads, so its hard to argue any changes there - it just might help in getting more features added quicker.

Anyway, my example interfaces file with two working bonds of different vLANS on two separate bridges:

# network interface settings
auto lo
iface lo inet loopback

auto eth0.56
iface eth0.56 inet manual

auto eth1.56
iface eth1.56 inet manual

auto eth0.11
iface eth0.11 inet manual

auto eth1.11
iface eth1.11 inet manual

auto bond0
iface bond0 inet manual
slaves eth0.56 eth1.56
bond_miimon 100
bond_mode active-backup
pre-up ifup eth0.56 eth1.56
post-down ifdown eth0.56 eth1.56

auto bond1
iface bond1 inet manual
slaves eth0.11 eth1.11
bond_miimon 100
bond_mode active-backup
pre-up ifup eth0.11 eth1.11
post-down ifdown eth0.11 eth1.11

auto vmbr0
iface vmbr0 inet static
address 10.5.56.72
netmask 255.255.254.0
gateway 10.5.57.4
bridge_ports bond0
bridge_stp off
bridge_fd 0

auto vmbr1
iface vmbr1 inet static
address 10.11.0.48
netmask 255.255.252.0
gateway 10.11.0.1
bridge_ports bond1
bridge_stp off
bridge_fd 0

 

[dietmar]

Two things to note
1 - the web interface won't allow bond creation on interface.vlan, only interface (eth0 is OK but eth0.56 is not). However, if it is manually put into the interfaces file, it works, and shows up properly in the web interface.
2 - you asked for a usage case scenario for more than 10 bridges? I have 11 vlans on my network, so I need to have 11 bridges. Actually, I also need to have 11 bonds as well. Apparently, I could configure this manually and be OK - I was just making a suggestion for getting the web interface more usable. I think 0-99 of both bonds and bridges would cover most everyone.

Ok, I will add this to our todo list.

BTW - GREAT work with Proxmox! This is a real break through in getting KVM into the mainstream. Would love to see the community get more involved in the development cycle (SCM access), but your quality control has been great, V1 is quite stable in a variety of work loads, so its hard to argue any changes there - it just might help in getting more features added quicker.

The source code is already available. So anyone can start adding features ;-)

Anyway, my example interfaces file with two working bonds of different vLANS on two separate bridges:

Did you also try my other suggestion ((bond0.XXX) which looks much easier)?

- Dietmar