Questions about Bridges and Bonds

[DavidSTVZ]

I'm new to VM's, bridges and bonding. The network configuration page suggests some settings but leaves many things unclear to me.

I want to bond two NICs, a 1G and a 10G, in active-backup mode with the 10G as primary. Then I want to use a bridge so that all my VM's use the 10G and failover to the 1G if needed. My organization allows a bridge and can increase MAC counts upon request... but in a bonded situation, I'm not sure if I'm increasing the MAC count for each physical port or just for an IP address.

Here's what the configuration page suggests:

auto lo
iface lo inet loopback

iface eno1 inet manual

iface eno2 inet manual

auto bond0
iface bond0 inet static
      slaves eno1 eno2
      address  192.168.1.2
      netmask  255.255.255.0
      bond_miimon 100
      bond_mode 802.3ad
      bond_xmit_hash_policy layer2+3

auto vmbr0
iface vmbr0 inet static
        address  10.10.10.2
        netmask  255.255.255.0
        gateway  10.10.10.1
        bridge_ports eno1
        bridge_stp off
        bridge_fd 0

In the example, the bond and bridge both use private network addresses, but I'm assuming there needs to be a public address somewhere in this or else the VM's can't access the Internet. Then again, the public addresses used by the VM's themselves may be enough to gain Internet access without exposing the hypervisor to the Internet.

Is it intended for me to use the exact settings above and then somehow use a public IP on the ordinary network interfaces (eno1 and eno2)? Or more likely I need to customize this for my particular network. We have a 10.20.* range for "building devices" that I use for lights out controllers. I can reserve an address (or two) there if needed.

I hope someone can clear this up for me! Thanks.

 

[wolfgang]

Hi,

I want to bond two NICs, a 1G and a 10G, in active-backup mode with the 10G as primary.

In this example you use LACP and not Active-Backup.

You can't use a physical nic twice.

It is possible to use virtual IP's what has the same effect or leave the external addresses away.
The External IP is not needed by the VM's if the VM Gateway is routeble.

But in this case I would use VLan to separate this two networks logical.

 

[DavidSTVZ]

Hmm. Maybe my network operations center can offer some advice.

Or maybe I should do some testing with only the bridge and no bond before getting the bond involved.

 

[DavidSTVZ]

I finally got a working configuration a few days after making this post. I'm not sure if it is completely right because I'm having a network problem on my first test VM. The test VM connects fine, but has intermittent drops in network connectivity.

Other than that, it works great. I can pull the plug on either the 1G or 10G NIC and the other seamlessly keeps the connection going.

I have configured a bonded bridge as follows:

Spoiler: code

auto lo
iface lo inet loopback

auto eno1
allow-hotplug eno1
iface eno1 inet manual
        bond-master bond0

iface eno2 inet manual

iface eno3 inet manual

iface eno4 inet manual

auto enp13s0
allow-hotplug enp13s0
iface enp13s0 inet manual
        bond-master bond0

auto bond0
iface bond0 inet manual
        bond-slaves none
        bond-mode active-backup
        bond-miimon 100
        bond-primary enp13s0

auto vmbr0
iface vmbr0 inet static
        address *.*.*.210                        #note: I have a real address here
        netmask <my.netmask.here>
        gateway <my.gateway.here>
        network <my.network.here>
        broadcast <my.broadcast.here>
        bridge-ports bond0
        bridge-fd 9
        bridge-hello 2
        bridge-maxage 12
        bridge-stp off

My configuration on the guest is using a public IP in the same subnet as my host. Is that a possible source of my intermittent network issue? I stuck with the automatic configuration given by the Debian installer which uses a slightly different format than I used above... but don't see any reason why it shouldn't work.

Spoiler: code

source /etc/network/interfaces.d/*

auto lo
iface lo inet loopback

allow-hotplug ens18
iface ens18 inet static
        address *.*.*.208/21                 #note: I have a real address here as well
        gateway <my.gateway.here>    #        the address is in the same subnet as the host
        dns-nameservers <redacted>
        dns-search <redacted>

 

[DavidSTVZ]

I found an old thread with info that solved my intermittent network issues. I'm getting some duplicate IPv6 address error on the guest OS and disabling duplicate address detection for IPv6 on the guest makes it function seamlessly... though I guess if there really is a duplicate address instead of an error in detection, that could still cause problems:

https://forum.proxmox.com/threads/bond-vmbr-eth0-ipv6-duplicate-address-detected.11291/