I trying install a cluster with two nodes. Each node with 3 network cards. One card to service client network and 2 others to create a bond to drbd sincronizations and cluster comiunication. Bond work but no performance gain observed. I used iperf to test speed on bond link and value is 1 gb/s like only one card in operation. I tryed all bond protocols including switch assisted. Switch show lacp working as expected.
Bond Performance
- Thread starter vitor costa
- Start date
I recently had this issue.
To see a performance increase, my understanding, is to use 802.3ad, then you MUST set the ports used on your switch to LACP and group or trunk them together.
The other options are for failover options which you should see if you disconnect one port.
For 802.3ad you must set the switch to active-active LACP and trunk the ports together.
What switch is it? I can attest that it is working properly, as I am receiving 1.7mbit throughput from all hosts to NAS.
To see a performance increase, my understanding, is to use 802.3ad, then you MUST set the ports used on your switch to LACP and group or trunk them together.
The other options are for failover options which you should see if you disconnect one port.
For 802.3ad you must set the switch to active-active LACP and trunk the ports together.
What switch is it? I can attest that it is working properly, as I am receiving 1.7mbit throughput from all hosts to NAS.
Hi, with lacp you can't do more than 1gbit/s by tcp connection.
multiple tcp connections will be balanced between 2 links. (but 1tcp connection is never balanced to reach 2gbit)
That can't be true, unless I do not connect to my NAS through TCP. But I can verify 1.8gbit transmission from Proxmox to OmniOS/Napp-IT during a "Move Disk" for a VM.
If I do not connect through TCP, which I don't think it does because my type=NFSv3 and not =NFSv3,tcp; is iPERF a TCP benchmark only?
If I do not connect through TCP, which I don't think it does because my type=NFSv3 and not =NFSv3,tcp; is iPERF a TCP benchmark only?
I am quite sure this is true (for LACP). But maybe you storage layer opens more than one tcp connection.
In this case DRBD with lacp is no way ? In a site a guy sai best use balance-rr protocol w crossover cable (no switch). What u thinj ?
...WAIT...
Set a bond in Napp-IT using eth0 and eth1 going to port 1 and 2 on my DGS-3200.
Set a bond in Proxmox using 802.3ad for eth1 and eth2 going to port 3 and 4 on my DGS-3200.
I set my DGS-3200 to have Active-Active LACP for ports 1-2-3-4
Set DGS-3200 to Port Trunk as follows:
Trunk 1 = Port 1-2
Trunk 2 = Port 3-4
When I move a full VM disk from local storage to NFS I see 1.7-1.8gbit transmission in "top" and in the Napp-IT GUI under the monitor. I do not see this reflected in the Proxmox GUI, but I assume the GUI is not calculating my bond, but my management port which is eth0.
I have not done any additional setting to my OmniOS as in change the type to NFSv3,tcp.
Please explain what is the difference from what the OP is trying to do and/or how I'm not calculating the transmission speed properly.
This is what I did:
Set a bond in Napp-IT using eth0 and eth1 going to port 1 and 2 on my DGS-3200.
Set a bond in Proxmox using 802.3ad for eth1 and eth2 going to port 3 and 4 on my DGS-3200.
I set my DGS-3200 to have Active-Active LACP for ports 1-2-3-4
Set DGS-3200 to Port Trunk as follows:
Trunk 1 = Port 1-2
Trunk 2 = Port 3-4
When I move a full VM disk from local storage to NFS I see 1.7-1.8gbit transmission in "top" and in the Napp-IT GUI under the monitor. I do not see this reflected in the Proxmox GUI, but I assume the GUI is not calculating my bond, but my management port which is eth0.
I have not done any additional setting to my OmniOS as in change the type to NFSv3,tcp.
Please explain what is the difference from what the OP is trying to do and/or how I'm not calculating the transmission speed properly.
AFAIK DRBD use a single tcp connection (at least his results suggests that).
iperf can do multiple tcp connections with -P parameter. I try using it and each connection show half speed, total keep 1 gb/s. So this multiple tcp connections dont work too.
i believe -d do a bidirecional test... Result is 1 gb/s for each side... w -d -P 2 i receive again a SUM with 1 gb/s
lacp is tunnable, depend of network switch model, but loadbalacing can be done by mac address, ip address or port
this is xmit_hash_policy option on linux bonding
http://www.linuxfoundation.org/collaborate/workgroups/networking/bonding
If that can be of any interest, I post my lacp bonding configuration over 4 ethernet cards.
The switch MUST also be configured as LACP on theses ports.
With layer 3/4 xmit hash policy it might interestingly balance multiple tcp connections between the same hosts.
The switch MUST also be configured as LACP on theses ports.
With layer 3/4 xmit hash policy it might interestingly balance multiple tcp connections between the same hosts.
Code:
auto lo
iface lo inet loopback
iface eth0 inet manual
iface eth1 inet manual
iface eth2 inet manual
iface eth3 inet manual
auto bond0
iface bond0 inet manual
slaves eth0 eth1 eth2 eth3
bond-mode 802.3ad
bond-xmit-hash-policy layer3+4
auto vmbr0
iface vmbr0 inet static
address 172.16.20.10
netmask 255.255.0.0
gateway 172.16.50.1
bridge_ports bond0
bridge_stp off
bridge_fd 0
post-up for i in eth0 eth1 eth2 eth3 ; do for o in rx ; do ethtool -K $i $o off; done; doneThat is normal behaviour. LACP is designed to _balance_ bandwidth between the links, but the maximum throughput between two clients is 1Gb/s.
(What is a "client" depend on the method used to compute which link should be used in function of the layer 2/3/4 adresses ;
e.g. with layer 2 you always use the same link between two ethernet cards).
(What is a "client" depend on the method used to compute which link should be used in function of the layer 2/3/4 adresses ;
e.g. with layer 2 you always use the same link between two ethernet cards).