bridge on a bond

[thefool808]

I have 4 NICs. I've left the first one as the installer setup (eth0 on vmbr0) and I've setup the other 3 in a bond (bond0 balance-rr). I've also created a vmbr1 which uses bond0. I've assigned an IP Address to vmbr1 but not to bond0 (I've also tried the reverse, and I've tried no IP address on either vmbr1 & bond0).

If I use vmbr0 in a windows 2000 KVM install everything works fine. If I use vmbr1 instead though, the same KVM install does succesfully retrieve an IP address from my (W2K3) DHCP server and can ping that server. However it cannot ping most other machines on the network, and I cannot go out on the internets beyond my gateway or even to the gateway itself.

Here's /etc/network/interfaces

# network interface settings
auto lo
iface lo inet loopback

iface eth0 inet manual

iface eth1 inet manual

iface eth2 inet manual

iface eth3 inet manual

auto bond0
iface bond0 inet manual
        slaves eth1 eth2 eth3
        bond_miimon 100
        bond_mode balance-rr

auto vmbr0
iface vmbr0 inet static
        address  192.168.1.71
        netmask  255.255.255.0
        gateway  192.168.1.1
        bridge_ports eth0
        bridge_stp off
        bridge_fd 0

auto vmbr1
iface vmbr1 inet manual
        bridge_ports bond0
        bridge_stp off
        bridge_fd 0

 

[Onyx]

Not sure, but don't you have to bond the 3 ports on your switch to? Try if it works with only one ethernet port in bond0

 

[dietmar]

Please can you test that configuration without a bound to see if it is a bondind or routing problem?

Adding 3 devices to a rr bond is not sugessted (performance will not increase much)

- Dietmar

 

[thefool808]

It works fine without the bond (e.g. vmbr1 on eth1).

When I was using vmware-server I would always overload the NICs before anything else. I would have to make sure I had my VMs split evenly over the NICs.

Now I have many more VMs running with PVE then I could run (efficiently) with vmware-server and have had no network issues whatsoever (other than PVE being somewhat brittle when making changes to the network settings). I guess what I'm looking for now, is a configuration-free way of balancing the loads of the VMs across my various NICs. If I could tie all the nics together on a single bond, and then bridge that bond, this would seem like a solution. I'm not looking for a billion bogobytes of bandwith, I just need each VM not have zero bandwith (when there's bandwith being unused on the next NIC).

Any thoughts on this?

Please can you test that configuration without a bound to see if it is a bondind or routing problem?

Adding 3 devices to a rr bond is not sugessted (performance will not increase much)

- Dietmar

 

[dietmar]

I would start with 2 devices in a bond, and then test with different bonding modes (maybe its an issue with the used switch?), and what network cards do you have?

- Dietmar