[SOLVED] Linux Bond mixed Speed (10G/1G active-backup) not using 10G

[hungerfisch]

Hi,
I'm having a little trouble configuring my network bonds. I'm using a linux bond bond0. This bond is using eno1 and eno3 for communication.

In my setup eno1 is a 10GBit port connected to a 10GBit Switch. eno3 Is a 1GBit Port connected to a 1GBit Switch. Both Switches are connected using a third switch (thats where the internet comes from, they are to provide redundant access too and make sure the cluster always can talk to itself to keep corosync happy).

Since I want bond0 to use eno1 as standard (I want to use my 10GBIT, we do get 10G internet access) I defined eno1 in bond0 as bond-primary.

The problem though, the bond only offers 1G. I checked and ethtool reports eno1 as 10G, eno3 as 1G and bond0 as 1G. (also tested using iperf and scp, capped at ~112MB/s)
How can I make bond0 utilize my 10G network?

-- cat /etc/network/interfaces --
auto lo
iface lo inet loopback

iface eno1 inet manual
#10G Netzwerk

iface eno2 inet manual
#10G Ceph

iface eno3 inet manual
#1G Netzwerk

iface eno4 inet manual
#1G Ceph

auto bond0
iface bond0 inet manual
bond-slaves eno1 eno3
bond-miimon 100
bond-mode active-backup
bond-primary eno1
#Netzwerk

[...]

Settings for eno1:
Supported ports: [ TP ]
Supported link modes: 100baseT/Half 100baseT/Full
1000baseT/Full
10000baseT/Full
Supported pause frame use: Symmetric Receive-only
Supports auto-negotiation: Yes
Supported FEC modes: Not reported
Advertised link modes: 100baseT/Half 100baseT/Full
1000baseT/Full
10000baseT/Full
Advertised pause frame use: Symmetric Receive-only
Advertised auto-negotiation: Yes
Advertised FEC modes: Not reported
Link partner advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
10000baseT/Full
Link partner advertised pause frame use: No
Link partner advertised auto-negotiation: Yes
Link partner advertised FEC modes: Not reported
Speed: 10000Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 17
Transceiver: internal
Auto-negotiation: on
MDI-X: Unknown
Supports Wake-on: g
Wake-on: d
Current message level: 0x00000000 (0)

Link detected: yes
root@pluto:~#

Settings for bond0:
Supported ports: [ ]
Supported link modes: Not reported
Supported pause frame use: No
Supports auto-negotiation: No
Supported FEC modes: Not reported
Advertised link modes: Not reported
Advertised pause frame use: No
Advertised auto-negotiation: No
Advertised FEC modes: Not reported
Speed: 1000Mb/s
Duplex: Full
Port: Other
PHYAD: 0
Transceiver: internal
Auto-negotiation: off
Link detected: yes
root@pluto:~#

 

[Alwin]

primary

A string (eth0, eth2, etc) specifying which slave is the
primary device. The specified device will always be the
active slave while it is available. Only when the primary is
off-line will alternate devices be used. This is useful when
one slave is preferred over another, e.g., when one slave has
higher throughput than another.

The primary option is only valid for active-backup(1),
balance-tlb (5) and balance-alb (6) mode.

https://www.kernel.org/doc/Documentation/networking/bonding.txt

The option in the debian is then called bond_primary.

 

[hungerfisch]

Thanks, so basically it all comes down to a typo

 

[hungerfisch]

https://www.kernel.org/doc/Documentation/networking/bonding.txt

The option in the debian is then called bond_primary.

Ok, I did "fix" that typo in one of my hosts, but the problem persists and my bond is still reported with 1GBit only, the underlying interfaces is reporting a 10gbit link.

 

[Alwin]

What is cat /proc/net/bonding/bond0 showing?

 

[hungerfisch]

root@eris:~# cat /proc/net/bonding/bond1
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

Bonding Mode: fault-tolerance (active-backup)
Primary Slave: None
Currently Active Slave: eno4
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
Peer Notification Delay (ms): 0

Slave Interface: eno2
MII Status: up
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: b0:83:fe:cb:dc:16
Slave queue ID: 0

Slave Interface: eno4
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: b0:83:fe:cb:dc:1a
Slave queue ID: 0

What is cat /proc/net/bonding/bond0 showing?

Hey,
it's showing the 1Gbit Interface as active interface, also showing Primary Slave: None, this seems to be the issue
(Please notice, bond1 and bond0 run the "same" config, just using different interfaces)



From cat /etc/network/interfaces

auto bond1
iface bond1 inet static
        address  192.168.0.3
        netmask  28
        bond-slaves eno2 eno4
        bond-miimon 100
        bond-mode active-backup
        bond_primary eno2

 

[spirit]

not sure it'll make a difference, but the official debian doc : bond-primary not bond_primary

https://wiki.debian.org/Bonding


(note that if you have installed ifupdown2, bond-primary is not yet supported, support will be available really soon)

 

[hungerfisch]

not sure it'll make a difference, but the official debian doc : bond-primary not bond_primary

https://wiki.debian.org/Bonding


(note that if you have installed ifupdown2, bond-primary is not yet supported, support will be available really soon)

So, my configuration now includes BOTH, but I think the problem here is us using ifupdown2 - which makes me sad now

EDIT:
I've found a workaround I'll try to implement

auto bond0
iface bond0 inet dhcp
        bond-slaves eth0 wlan0
        bond-mode active-backup
        up echo eth0 > /sys/class/net/$IFACE/bonding/primary

 

[hungerfisch]

Is there any information on when to expect ifupdown2 2.0 ?

 

[spirit]

Is there any information on when to expect ifupdown2 2.0 ?

I haved send patched this week, this is already in proxmox git.
(and yes, 2.0 support bond-primary)

So packages should be available really soon

I have already build from proxmox git, if you want to install is manually:

http://odisoweb1.odiso.net/ifupdown2_2.0.1-1+pve1_all.deb

 

[hungerfisch]

I haved send patched this week, this is already in proxmox git.
(and yes, 2.0 support bond-primary)

So packages should be available really soon

I have already build from proxmox git, if you want to install is manually:

http://odisoweb1.odiso.net/ifupdown2_2.0.1-1+pve1_all.deb

Thanks,
just found time to install the update

 

[spirit]

Thanks,
just found time to install the update

maybe my package was buggy,you can try to offical one from pvetest repo :
http://download.proxmox.com/debian/...t/binary-amd64/ifupdown2_2.0.1-1+pve2_all.deb

 

[hungerfisch]

maybe my package was buggy,you can try to offical one from pvetest repo :
http://download.proxmox.com/debian/pve/dists/buster/pvetest/binary-amd64/ifupdown2_2.0.1-1+pve2_all.deb

Actually, it worked, thanks