network bond not working

Feb 24, 2022
28
0
6
34
Hello,

I am trying to set up a bond for our Proxmox Hypervisor. I am using a brodacom Network card with 2 10gbe sfp+ ports and 2 regular rj45 ports. I would like to have a failover config with the sfp+ connection as primary and the rj45 as secondary. The secondary should only act as a backup in a worst case that the sfp+ would fail or gets unplugged.
Her is my config

Bash:
auto lo
iface lo inet loopback

iface eno3 inet manual

auto eno1
iface eno1 inet manual

auto eno2
iface eno2 inet manual

auto eno4
iface eno4 inet manual

auto bond0
iface bond0 inet manual
        bond-slaves eno1 eno4
        bond-miimon 100
        bond-mode active-backup
        bond-primary eno1

auto vmbr0
iface vmbr0 inet static
        address 10.0.0.10/24
        gateway 10.0.0.1
        bridge-ports eno3
        bridge-stp off
        bridge-fd 0

auto vmbr1
iface vmbr1 inet static
        address 10.0.0.11/24
        bridge-ports bond0
        bridge-stp off
        bridge-fd 0

eno1 is the 10gbit port and eno4 is the 1gbit port. All VMs are connected to the vmbr0 bridge which is currently connected to port eno3 (1gbit) which works fine. As you can see I have a second bridge vmbr1 connected to the bond currently which also works fine (I can access the Proxmox Web panel on 10.0.0.11 but cannot communicate to the internet because you can only have one gateway.). So far so good. However when I try to change the bridge port of the vmbr0 bridge to bond0 instead of eno3 I can't even get to my default gateway. On the Switch side (mikrotik crs354) I have configured the bond of the two ports and setup the active-backup type for bonding.
see config here
1678090051600.png
Here the config which should work but doesn't (only port for vmbr0 bridge is changed from eno3 to bond0). Interestingly not even eno3 via the second vmbr1 bridge works anymore when I try to access it on 10.0.0.11
Bash:
auto lo
iface lo inet loopback

iface eno3 inet manual

auto eno1
iface eno1 inet manual

auto eno2
iface eno2 inet manual

auto eno4
iface eno4 inet manual

auto bond0
iface bond0 inet manual
        bond-slaves eno1 eno4
        bond-miimon 100
        bond-mode active-backup
        bond-primary eno1

auto vmbr0
iface vmbr0 inet static
        address 10.0.0.10/24
        gateway 10.0.0.1
        bridge-ports bond0
        bridge-stp off
        bridge-fd 0

auto vmbr1
iface vmbr1 inet static
        address 10.0.0.11/24
        bridge-ports eno3
        bridge-stp off
        bridge-fd 0
The strange thing is when I try to reproduce this problem on the another server (same model and network card Dell r730) It works directly without issues. Could it be that it is an issue that the vmbr0 bridge already has vms attached to it? because that is the only difference that is between the two servers. what I also noticed on the switch is that the state of the bond never get the flag "slave" only the flag "running" (on the environment where it works, both of these flag exist)

Any input is appreciated.


Thanks
 
Hello, a few thoughts:

I think you should only bond interfaces with the same speed. I dunno if mixing speeds work at all/are reliable.

Also the active-backup modes is one of the modes that assume a 'dumb' switch. So you don't have to configure anything on the switch side. Maybe configuring active-backup on both sides will make both sides assume different active-links and thus make it not connect at all?

Kind regards,
Benedikt
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!