Bond Problem

[Lip90]

Hello,

I have configured a bond.

auto lo
iface lo inet loopback

auto enp38s0
iface enp38s0 inet manual

auto enp39s0
iface enp39s0 inet manual

iface enp42s0f3u5u3c2 inet manual

auto enp45s0f0
iface enp45s0f0 inet manual

auto enp45s0f1
iface enp45s0f1 inet manual

iface enp45s0f2 inet manual

iface enp45s0f3 inet manual

auto bond0
iface bond0 inet manual
        bond-slaves enp45s0f0 enp45s0f1
        bond-miimon 100
        bond-mode 802.3ad
        bond-xmit-hash-policy layer2+3

auto vmbr0
iface vmbr0 inet static
        address 192.168.1.7/24
        gateway 192.168.1.1
        bridge-ports bond0
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094

I thought with this configuration I would have double the speed. Unfortunately this is not the case. If two clients address my Truenas VM, the data rate is halved.

How do I have to configure the bond so that 2 * 1GbE become 2GbE?

My Switch is a Unifi Switch. The settings are done.

I have already read the forum and the documentation, but I did not find out anything here.

 

[spirit]

Hello,

I have configured a bond.

auto lo
iface lo inet loopback

auto enp38s0
iface enp38s0 inet manual

auto enp39s0
iface enp39s0 inet manual

iface enp42s0f3u5u3c2 inet manual

auto enp45s0f0
iface enp45s0f0 inet manual

auto enp45s0f1
iface enp45s0f1 inet manual

iface enp45s0f2 inet manual

iface enp45s0f3 inet manual

auto bond0
iface bond0 inet manual
        bond-slaves enp45s0f0 enp45s0f1
        bond-miimon 100
        bond-mode 802.3ad
        bond-xmit-hash-policy layer2+3

auto vmbr0
iface vmbr0 inet static
        address 192.168.1.7/24
        gateway 192.168.1.1
        bridge-ports bond0
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094

I thought with this configuration I would have double the speed. Unfortunately this is not the case. If two clients address my Truenas VM, the data rate is halved.

How do I have to configure the bond so that 2 * 1GbE become 2GbE?

My Switch is a Unifi Switch. The settings are done.

I have already read the forum and the documentation, but I did not find out anything here.

you can't use more than 1 link for 1 tcp connecion with lacp bonding.

but mutlple connections can be loadbalanced across multiple links.

bond-xmit-hash-policy layer2+3 -> mean : loadbalance be mac src-ipsrc -mac dst- mac dest.
so with this, only differents vms will be balanced across links. (and depending of their mac address, they could use the same link)

use layer3+4 -> ip src - port src - ip dest - port dest, to loadbalance across multiple links from 1vm.

also, this balancing algorithm is for balancing traffic going out the proxmox server. For incoming traffic, you need to configure this hash algorithm on your physical switch.


without lacp, you could try round-robin bond.

 

[Dunuin]

you can't use more than 1 link for 1 tcp connecion with lacp bonding.

but mutlple connections can be loadbalanced across multiple links.

bond-xmit-hash-policy layer2+3 -> mean : loadbalance be mac src-ipsrc -mac dst- mac dest.
so with this, only differents vms will be balanced across links. (and depending of their mac address, they could use the same link)

use layer3+4 -> ip src - port src - ip dest - port dest, to loadbalance across multiple links from 1vm.

also, this balancing algorithm is for balancing traffic going out the proxmox server. For incoming traffic, you need to configure this hash algorithm on your physical switch.


without lacp, you could try round-robin bond.

Yes, bonding will never increase your speed. It will only increase the bandwidth. Fastest connection you can get is still 1Gbit/s but you can use a 2Gbit/s bandwidth by using two 1Gbit connections in parallel. If your NIC is like a 50 MPH single lane road a bond won't increase the speed limit to 100 MPH. It will just add a second 50 MPH lane. So a single car wouldn't reach the target any faster. Its only useful if you got more cars a single lane could handle to prevent traffic jams.

Using layer3+4 will make it easier to use parallelization but your applications still needs to be able to make use of it by for example splitting datastreams and transfering this data on different ports at the same time.

 

[Lip90]

Thx for your answers.

 

[Lip90]

Unfortunately, I still don't quite understand it. e.g. I have two client PCs, one transmits data to the Truenas, the other transmits data to the Windows VM. As soon as both clients start copying, the speed is halved to 50 Mbit / s.

Is that correct?

I changed the config to layer3+4.

 

[Dunuin]

If you set your LACP to layer3+4, both in PVE and your managed switch, you should get 2x 118 MB/s as long as you are not using the same IPs or ports. You could for example access a SMB and a NFS share at the same time with 118 MB/s each because NFS and SMB are running on different ports. If you want double the speed with just SMB you need to make sure both the client and SMB server use "multi channel support" so connections can be established over multiple ports.