[SOLVED] Driver for Intel 553/557-AT (ixgbe) NICs

[spinner]

Hi all.

New to this forum and Proxmox, so please bear with me

I just installed Proxmox VE 5 on a Supermicro X10SDV-6C-TLN4F board.

The 1 gbit NICs where recognized right away but not the 10 gbit ones.

I've been looking a little bit around and from what I've been able to tell, I need to compile my own drivers.

Is this really true? I'd suppose a widely used NIC like that would have the drivers compiled already?

If this holds true, do you know of any guides to follow to do this?

Thanks in advance.

 

[dietmar]

We include the latest ixgbe driver from Intel...

 

[tom]

do you run latest kernel? pls post your:

> pveversion -v

 

[spinner]

Thanks both for your reply.

I've checked with modinfo ixgbe that the driver is actually present:

filename:       /lib/modules/4.10.15-1-pve/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko
version:        5.0.4

do you run latest kernel? pls post your:

> pveversion -v

proxmox-ve: 5.0-15 (running kernel: 4.10.15-1-pve)                                          
pve-manager: 5.0-23 (running version: 5.0-23/af4267bf)                                      
pve-kernel-4.10.15-1-pve: 4.10.15-15                                                        
libpve-http-server-perl: 2.0-5                                                              
lvm2: 2.02.168-pve2                                                                         
corosync: 2.4.2-pve3                                                                        
libqb0: 1.0.1-1                                                                             
pve-cluster: 5.0-10                                                                         
qemu-server: 5.0-12                                                                         
pve-firmware: 2.0-2                                                                         
libpve-common-perl: 5.0-16                                                                  
libpve-guest-common-perl: 2.0-11                                                            
libpve-access-control: 5.0-5                                                                
libpve-storage-perl: 5.0-12                                                                 
pve-libspice-server1: 0.12.8-3                                                              
vncterm: 1.5-2                                                                              
pve-docs: 5.0-6                                                                             
pve-qemu-kvm: 2.9.0-2                                                                       
pve-container: 2.0-14                                                                       
pve-firewall: 3.0-1                                                                         
pve-ha-manager: 2.0-2                                                                       
ksm-control-daemon: 1.2-2                                                                   
glusterfs-client: 3.8.8-1                                                                   
lxc-pve: 2.0.8-3          
lxcfs: 2.0.7-pve2                                                                           
criu: 2.11.1-1~bpo90                                                                        
novnc-pve: 0.6-4                                                                            
smartmontools: 6.5+svn4324-1                                                                
zfsutils-linux: 0.6.5.9-pve16~bpo90

I've tried manually loading the module using modprobe ixgbe as well as include it in /etc/modules and reboot the box, without any luck.

Besides being completely new to Proxmox I also have very limited experience with linux, so I'm probably missing something very basic here.

 

[spinner]

So, looking a bit further into this, I think maybe the device drivers are loaded properly, as >lspci -s 03: -nnvvv actually ends with

Kernel driver in use: ixgbe                                                                                                          
Kernel modules: ixgbe

Also >ip link shows:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000                              
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00                                                                              
2: eno1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000                                    
    link/ether 0c:c4:7a:ca:3b:d2 brd ff:ff:ff:ff:ff:ff                                                                                
3: eno2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr0 state UP mode DEFAULT group default qlen 1000                
    link/ether 0c:c4:7a:ca:3b:d3 brd ff:ff:ff:ff:ff:ff                                                                                
4: rename4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000                                  
    link/ether 0c:c4:7a:ca:58:6a brd ff:ff:ff:ff:ff:ff                                                                                
5: rename5: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000                                  
    link/ether 0c:c4:7a:ca:58:6b brd ff:ff:ff:ff:ff:ff                                                                                
6: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000                      
    link/ether 0c:c4:7a:ca:3b:d3 brd ff:ff:ff:ff:ff:ff

which makes me think that the system has recognized the devices as network devices.

So I guess the question is why they are listed as Unknown in Proxmon and nothing happens when I click Edit in the GUI.

 

[fabian]

please post the output of "journalctl -b -k"

 

[spinner]

Here you go.

 

[spinner]

I think I solved this.

I reinstalled Proxmox to verify that the problem wasn't caused by something gone wrong during the installation. The problem was the same afterwards, but I did notice that I could actually select one of the to 10gbe nics as my management nic during installation. They were still called rename4/rename5 though, which must be due to some glitch in the "new" naming logic used by debian, but it lead me to believe that the nics where actually running as they should, and is was probably a question of getting Proxmox to recognize them in the GUI.

So, following this post https://askubuntu.com/questions/689501/how-to-rename-network-interface-in-15-10 I removed the entries in /etc/network/interfaces and created a file in /etc/udev/rules.d instead, containing the following:

SUBSYSTEM=="net", ACTION=="add", ATTR{address}=="0c:c4:7a:ca:58:6a", NAME="eth0"
SUBSYSTEM=="net", ACTION=="add", ATTR{address}=="0c:c4:7a:ca:58:6b", NAME="eth1"

and rebooted the box.

After the reboot the nics are recognized by Proxmox and I've been able to configure them. Tested with bonding and bridging and everything appears to work fine.

So seems like it's all caused by the naming of the devices during installation.

 

[fabian]

sounds like a bug in the intel drivers (igb and ixgbe) which race against eachother and probably try to rename at the same time using the same naming scheme? I would advise against renaming the devices to eth* (as that is actually the kernels name space) - you could e.g. call them eno10 and eno11 since they are onboard and 10Gbit