SR-IOV VF replace default network interface name?

[raytracy]

I have installed 3 new PVE server with Intel X540-T2 interface and network connection worked well without SR-IOV via physical interface.

I try to enable VF by the following:

Modify /etc/default/grub line as:
--------------------------
GRUB_CMDLINE_LINUX="root=ZFS=rpool/ROOT/pve-1 boot=zfs intel_iommu=on ixgbe.max_vfs=16,16"
--------------------------
Execute update-grub

Modify /etc/network/interface.new as:
----------------------
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
address 10.250.11.102
netmask 255.255.255.0
gateway 10.250.11.254
-----------------------
To remove default vmbr0 and reboot.

After that, I found there is only one server operate normal with all VF follow PF port (vf 0~15 follow eth0, and vf0~15 follow eth1), all other pseudo interface generated by ixgbevf named eth2 to eth33.

The other two servers show very strange situation:

Stuck after grub manager and running for udev initialization about several minutes:

Although there are nic named start from eth0,eth1..., the VF did not follow the real PF port but shown in the middle such as rename13, rename28...not normal ethxx:


the eth0 and eth1 cannot bind to IP address which I specify, and even cannot start up manually:


dmesg show strange 'rename' message which looks like it rename eth0 and eth1 to something else at boot time:

(Note: the 05:00.0 is eth0 PCI address, and 05:00.1 should be eth1 when SR-IOV disabled)

Boot into this situation let me cannot make any network connection.

After that, if I remove the ixgbevf kernel module by rmmod ixgbevf command, all psuedo ethxx will disappear, and leave the strange renamexx and vf (it should be eth0, eth1 and corresponding vf):

The mac address are correct for eth0 and eth1 PF, but the name has changed!!


I cannot understandard what happen on these two servers but there are still one server works fine without above problem. Here is the workable server which vf follow eth0 and eth1 normally, did not renamed and can be connected to network:



All 3 servers are installed with:
pve-manager/4.4-13/7ea56165 (running kernel: 4.4.59-1-pve)
and updated to latest packages.

Any idea or advice?

 

[raytracy]

In addition: Here is result of lspci for ethernet from the two servers:


and my /etc/udev/rules.d looks like this:


I don't know why it did not honor the rules and rename the PF as some thing strange?

 

[wolfgang]

Hi,

AFIK you try to map 16 virtual devices on 16 ports.

GRUB_CMDLINE_LINUX="root=ZFS=rpool/ROOT/pve-1 boot=zfs intel_iommu=on ixgbe.max_vfs=16,16"

ixgbe.max_vfs=16,2

 

[raytracy]

Hi,

AFIK you try to map 16 virtual devices on 16 ports.



ixgbe.max_vfs=16,2

I have try to change the number per your way but got only 16 VF port and 2 VF port:


I have reviewed this doc:
https://github.com/pavel-odintsov/ixgbe-linux-netmap/tree/master/ixgbe-3.23.2.1

It said:

The parameters for the driver are referenced by position. Thus, if you have a
dual port 82599, X540, or X550-based adapter and want N virtual functions per
port, you must specify a number for each port with each parameter separated
by a comma. For example: modprobe ixgbe max_vfs=63,63

So I believe max_vfs=16,16 are corresponding to each PF port.

Is there any thing I have missed or misunderstood?