be2net driver

  • Thread starter linuxdatacenter
  • Start date
L

linuxdatacenter

Guest
Hi,

Is anyone using emulex oneconnect network adapter successfully with proxmox (the be2net driver in kernel). I am experiencing frequent kernel panics with 2.6.32 and 2.6.35 proxmox kernels.
The hardware is HP Blade bl460c G7
 
Same thing here but only when I bridge a vlan with vmbr0... I'm dying to find a solution to this problem!
 
Hi,

The newest stable kernel from kernel.org (2.6.36.2) solves this problem. I just applied the config from 2.6.35.1-pve and recompiled.

And thanks a lot for the new firmware. I am giving it a shot today. If the machine survives till the end of the week - I will be really happy ;-)

I will keep you informed.
 
Hi,

The new firmware does not solve the problem. I got a kernel panic after a few minutes of uptime.
 
Same thing for me: the firmware does not solve the problem.

Is it safe to the apply linuxdatacenter procedure? (using the newest kernel from kernel.org?)

Or is it possible to have a access to the kernel 2.6.36.2 directly from the proxmox test repo?

Thanks!
 
My experience with 2.6.36.2 so far:

-> multipath works for FC storage (paths are correctly failed over in case of failures)
-> live migration works
-> no hangs
-> no performance degradations

I've been running it for about a 5 days now on 3 servers.
 
Cool! I will try to compile the kernel.

Does OpenVz works with that new kernel?

Thanks!
 
Good news!

I have successfully compiled and installed the kernel 2.6.36.2 which solve the problem with the Emulex. I did like linuxdatacenter and used the config from 2.6.35.1-pve and now my network is stable. I have tried with the kernel 2.6.37 but the Emulex was not detected by it. It is probably due to an error from myself since I'm no guru compiling a kernel... But 2.6.36.2 works great!

Thanks linuxdatacenter for the hint!

Now I have two new problems:
- Routing from kvm to net does not work (http://forum.proxmox.com/threads/4726-My-bridge-does-not-work?p=31203#post31203)
- iscsi timeout when I try to connect a LUN (http://forum.proxmox.com/threads/55...scsi-lun.-timeout-and-DID_TRANSPORT_DISRUPTED)

I hope that I will be able to make it work!
 
Last edited by a moderator:
Using the custom kernel (2.6.36.2) seems to work. I use it for a while and it seems stable...

I'm not very happy about the Emulex card because is it not well supported for Debian... I needed to flash the firmware of the oce10102 because one day, all ports shut and I was not able to reactive them. I try to import Emulex tools built for SLES11 into proxmox without luck. I was force to install windows on my host to flash the firmware (this is very ugly). Now the firmware is up to date but the be2net driver is not.

Using a custom kernel, with an up to date nic firmware which recommends a specific version of the driver (which is not installed) is not really cool. I have 2 cards in my server. If I setup each card without bonding, everything works fine (vlan, routing from KVM to net, iscsi etc...). But when bonding is in the equation, KVM to net routing cease to work and iscsi behaves strangely. And with a custom kernel, I can't get security updates from pve team! :(

So here is my question :) Emulex provides a be2net driver built for the kernel 2.6.32.12. Is it possible to use this driver with the pve-kernel 2.6.32?

http://www.emulex.com/downloads/emulex/oneconnect-ucnas/oce10102-linux-oob-sles-11-spx-core.html

That would be nice because I could get security update from proxmox and use the latest driver from Emulex. Once again, I'm not a kernel guru but I tried to force the driver into the kernel using a virtual pve1.7 installation:

The version of the firmware is 2.103.269.30.

Code:
testpve17:~# modprobe -f --force-vermagic --force-modversion be2net
FATAL: Error inserting be2net (/lib/modules/2.6.32-4-pve/kernel/drivers/net/benet/be2net.ko): Unknown symbol in module, or unknown parameter (see dmesg)
testpve17:~# dmesg
be2net: module_layout: kernel tainted.
Disabling lock debugging due to kernel taint
be2net: Unknown symbol lro_flush_all
be2net: Unknown symbol lro_receive_frags
be2net: Unknown symbol lro_vlan_hwaccel_receive_frags
be2net: disagrees about version of symbol module_layout

The driver from Emulex (2.103.269.28)
Code:
testpve17:~# modinfo be2net.ko #FROM EMULEX, built for SLES11 SP1
filename:       be2net.ko
supported:      external
license:        GPL
author:         ServerEngines Corporation
description:    ServerEngines BladeEngine 10Gbps NIC Driver 2.103.269.28
version:        2.103.269.28
srcversion:     D9941D796750088ACB324E5
alias:          pci:v000019A2d00000710sv*sd*bc*sc*i*
alias:          pci:v000019A2d00000700sv*sd*bc*sc*i*
alias:          pci:v000019A2d00000221sv*sd*bc*sc*i*
alias:          pci:v000019A2d00000211sv*sd*bc*sc*i*
depends:        
supported:      yes
vermagic:       2.6.32.12-0.7-default SMP mod_unload modversions 
parm:           rx_frag_size:Size of receive fragment buffer  - 2048 (default), 4096 or 8192 (uint)
parm:           num_vfs:Number of PCI VFs to initialize (uint)
parm:           msix:Enable and disable the MSIx (By default MSIx is enabled) (uint)

The driver from pve team (2.101.205)
Code:
fl-vm01:~# modinfo /lib/modules/2.6.32-4-pve/kernel/drivers/net/benet/be2net.ko
filename:       /lib/modules/2.6.32-4-pve/kernel/drivers/net/benet/be2net.ko
license:        GPL
author:         ServerEngines Corporation
description:    ServerEngines BladeEngine2 10Gbps NICDriver 2.101.205
version:        2.101.205
srcversion:     199ADD251CB874C3727CC47
alias:          pci:v000019A2d00000710sv*sd*bc*sc*i*
alias:          pci:v000019A2d00000701sv*sd*bc*sc*i*
alias:          pci:v000019A2d00000700sv*sd*bc*sc*i*
alias:          pci:v000019A2d00000221sv*sd*bc*sc*i*
alias:          pci:v000019A2d00000211sv*sd*bc*sc*i*
depends:        
vermagic:       2.6.32-4-pve SMP mod_unload modversions 
parm:           rx_frag_size:Size of a fragment that holds rcvd data. (uint)

The driver from the custom kernel (2.103.175u)
Code:
fl-vm01:~# modinfo /lib/modules/2.6.36.2-customfl/kernel/drivers/net/benet/be2net.ko
filename:       /lib/modules/2.6.36.2-customfl/kernel/drivers/net/benet/be2net.ko
license:        GPL
author:         ServerEngines Corporation
description:    ServerEngines BladeEngine 10Gbps NIC Driver 2.103.175u
version:        2.103.175u
srcversion:     54E65292DC5F63AD3F21648
alias:          pci:v000019A2d00000710sv*sd*bc*sc*i*
alias:          pci:v000019A2d00000700sv*sd*bc*sc*i*
alias:          pci:v000019A2d00000221sv*sd*bc*sc*i*
alias:          pci:v000019A2d00000211sv*sd*bc*sc*i*
depends:        
vermagic:       2.6.36.2-customfl SMP mod_unload modversions 
parm:           rx_frag_size:Size of a fragment that holds rcvd data. (uint)
parm:           num_vfs:Number of PCI VFs to initialize (uint)
 
Last edited by a moderator:
Now I understand that my understanding of the linux kernel is really naive (and the previous post proves it!)
 
I have contacted Emulex in order to get help... Here are their answers:

First mail:
Hi Pierre-Yves,

Thank you for your e-mail regarding the Emulex OneConnect oce10102 CNA.

The drivers available on our web site are for RHEL and SLES linux distributions.

Drivers for Debian other Linux distrubtions are provided in the upstream kernel. You will need to contact Debian and/or Promox for information regarding drivers updates.

Let me know if you need any additonal information.

Best regards,

Applications Engineer, Staff

Second mail:
Hello Pierre-Yves,

Thank you for the additional question.

>On the kernel mainline, the be2net driver version is 2.103.175u. On your site, the version is 2.103.269.28. Is it the same version with different nomenclature?

They are different versions. The drivers on our web site may or may not be the same as drivers provided in the upstream kernel. The latest drivers available for the OS's we support (RHEL, SLES) are availbale on our web site.

>Where can I get the source code of the driver so I can compile a new proxmox kernel myself?

Unfortunately, the be2net source files are not available on our web site. I am not going to be able to help out on this reqquest.

Your support for the be2net driver would be from Debian Linux. You can contact Debian (nott sure how updates are handled), or you can try our Marketing Department at:

Marketing@emulex.com

Best regards,

That's really sad... They expect me to contact the Debian team to do what? The driver already exists in the kernel but is outdated. I'm really happy that Emulex supports linux by publishing their sources in the kernel. A lot of manufacturers don't do that at all. But their source are outdated. Does not sounds good to me.

Emulex also publish up-to-date drivers precompiled for specific linux distribution like RHEL et SLES but I don't want to use those distributions. Is it really hard to publish drivers that are distribution independent? Are the differences between RPM and DEB distro really that big from a drivers perspective?

So now what can I do? I have bought 6 emulex and drivers that are outdated and I can't update them. Emulex won't help me to update the drivers to a more recent version, so I have to dream of the day where a newer version is merge in the kernel and then reconfigure my servers to make the bond works to obtain redundancy.

Sorry if I sound depressed but that's the case :)
 
Sorry if I sound depressed but that's the case :)

Wow, that is really odd:

Unfortunately, the be2net source files are not available on our web site.

So they have a 'good' proprietary driver without source available, and let the community mange the public Linux driver? Or do they provide the source anywhere?
 
As I understand it, they keep the latest source in house (and creates binary rpm only compatible with RHEL&SLES) and merge older version from the same version tree in the kernel mainline. Maybe they do that to keep their latest innovation safe from other manufacturers. I don't know... Their source in the kernel are located here: linux-2.6.37/drivers/net/benet/. Contribution to that code seems to be done from developerxx@emulex.com.

Is there a way I could compile that version (although it is not the latest) with the pve2.6.32 kernel because that version is more stable that the one from 2.6.32? I would like to use openvz container but with my custom 2.6.36 kernel, it is not possible.

Thanks!
 
Last edited by a moderator:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!