bonding and be2net

[resoli]

Hello,

We are starting migrating our 1.9 cluster to 2.1, but there are problems in setting up bonding on two Emulex NC355i 10Gb ethernet interfaces (be2net driver), onboard on HP Blade BL460c G7.

The two interfaces are connected to two different switches, with a trunk connecting them.

First issue: miimon method does not work, because the driver does not detect link absence, so failover does not work.
Second issue: using arp monitoring works for bonding, but when I put the bonded interfaces into a bridge, both interfaces goes down.

On old 1.9 cluster with pve kernel 2.6.32-6 (version 2.6.32-55+ovzfix-1) we had the problem with miimon, but we were able to workaround with arp monitoring.

We tried to update firmware to latest (4.1.402.20) from hp, without success, and also recompiling driver taking latest code available from HP/Emulex, without any progress.

Somone encountered similar issues, or has any suggestion?

Bye,
rob

 

[dietmar]

Please can you also test with latest available kernel (pve-kernel-2.6.32-14-pve_2.6.32-73)

 

[resoli]

Please can you also test with latest available kernel (pve-kernel-2.6.32-14-pve_2.6.32-73)

Yes, we are at this level, no difference. Anyway, I have tried removing

arp_validate 3

parameter from bonding configuration, and now the bond interface stays up even inside the bridge. Failover works.
if i set that parameter to 1 or 2 the slave interfaces start oscillating between up and down status.

I guess that bridging confuses the validate code inside that version of bonding driver.

rob

 

[resoli]

It seems that bonding with arp monitoring inside a bridge has known issues; see

http://www.linux-kvm.org/page/HOWTO_BONDING

(bottom of the page: Problem with bridge + bonding)

And, moreover:
"[PATCH] bonding: fix arp_validate on bonds inside a bridge"[1]

May be this patch is not yet in pve kernel, or there is a regression in recent kernels?

rob

[1] http://comments.gmane.org/gmane.linux.network/159403

 

[resoli]

Hello,
First issue: miimon method does not work, because the driver does not detect link absence, so failover does not work.

Update: it was a firmware issue: http://h20000.www2.hp.com/bizsuppor...473928&prodTypeId=329290&prodSeriesId=4085948

In short: the interconnection module (passthrough) between the blade host and the switch was not passing the state of the link, making the interface to think it was always up.
The firmware of the Blade enclosure which fixes the problem was updated recently, but, as stated in the HP page, the firmware for the single passththrough modules has to be applied from the enclosure Administration CLI.

rob