Cannot use SR-IOV on proxmox

sili · Apr 22, 2016

Hi,
I have upgrade my proxmox
but before upgrade, SR-IOV can't use know

Here is my pveversion

Code:

root@pve:~#  pveversion --verbose
proxmox-ve: 4.1-48 (running kernel: 4.4.6-1-pve)
pve-manager: 4.1-33 (running version: 4.1-33/de386c1a)
pve-kernel-4.4.6-1-pve: 4.4.6-48
pve-kernel-4.2.8-1-pve: 4.2.8-41
pve-kernel-4.2.2-1-pve: 4.2.2-16
lvm2: 2.02.116-pve2
corosync-pve: 2.3.5-2
libqb0: 1.0-1
pve-cluster: 4.0-39
qemu-server: 4.0-71
pve-firmware: 1.1-8
libpve-common-perl: 4.0-59
libpve-access-control: 4.0-16
libpve-storage-perl: 4.0-50
pve-libspice-server1: 0.12.5-2
vncterm: 1.2-1
pve-qemu-kvm: 2.5-13
pve-container: 1.0-61
pve-firewall: 2.0-25
pve-ha-manager: 1.0-28
ksm-control-daemon: 1.2-1
glusterfs-client: 3.5.2-2+deb8u1
lxc-pve: 1.1.5-7
lxcfs: 2.0.0-pve2
cgmanager: 0.39-pve1
criu: 1.6.0-1
zfsutils: 0.6.5-pve9~jessie
openvswitch-switch: 2.3.2-3

and I use i350 to do SR-IOV

Code:

root@pve:~# lspci |grep Eth
03:00.0 Ethernet controller: Intel Corporation Ethernet Connection X552/X557-AT 10GBASE-T
03:00.1 Ethernet controller: Intel Corporation Ethernet Connection X552/X557-AT 10GBASE-T
05:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
05:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
07:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
07:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)

Code:

root@pve:~# lspci -vnnk -s 07:00.1
07:00.1 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] (rev 01)
  Subsystem: Super Micro Computer Inc Device [15d9:1521]
  Flags: bus master, fast devsel, latency 0
  Memory at fba00000 (32-bit, non-prefetchable) [size=128K]
  I/O ports at d000 [size=32]
  Memory at fba40000 (32-bit, non-prefetchable) [size=16K]
  Capabilities: [40] Power Management version 3
  Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
  Capabilities: [70] MSI-X: Enable+ Count=10 Masked-
  Capabilities: [a0] Express Endpoint, MSI 00
  Capabilities: [100] Advanced Error Reporting
  Capabilities: [140] Device Serial Number 0c-c4-7a-ff-ff-c3-9f-06
  Capabilities: [150] Alternative Routing-ID Interpretation (ARI)
  Capabilities: [160] Single Root I/O Virtualization (SR-IOV)
  Capabilities: [1a0] Transaction Processing Hints
  Capabilities: [1d0] Access Control Services
  Kernel driver in use: igb

When I do SR-IOV, I'll get this error

Code:

root@pve:~# echo 2 > /sys/class/net/eth0/device/sriov_numvfs
-bash: echo: write error: Function not implemented

dietmar · Apr 22, 2016

We include the latest driver '5.3.4.4'. Please can you report a bug here:

https://sourceforge.net/p/e1000/bugs/

sili · Apr 22, 2016

hi, I have report bug on sourceforge,
They was replay me this message
"Please contact proxmox as this appears to be a kernel error."

dietmar · Apr 22, 2016

sili said:
hi, I have report bug on sourceforge,

I cannot find your bug report there - please can you send me a link?

sili · Apr 22, 2016

https://sourceforge.net/p/e1000/bugs/522/

dietmar · Apr 22, 2016

That, I will try to clarify the issue there. Please can you test if setting the suggested module flag work?
In /etc/modules, set

igb max_vfs=2,2

dietmar · Apr 22, 2016

I assume the file /sys/class/net/eth0/device/sriov_numvfs does not exist?

sili · Apr 22, 2016

Hi, igb sriov is very Important for me.
If I am try igb after, I'll post reply on this

but I'm try my X552, It's different is X552 use ixgbe driver
I think is maybe the same problem

this is my friend help me to post this
https://forum.proxmox.com/threads/p...541s-soc-x552-x557-sr-iov-issue-failed.26666/

Code:

root@pve:~# pveversion --verbose
proxmox-ve: 4.1-41 (running kernel: 4.2.8-1-pve)
pve-manager: 4.1-22 (running version: 4.1-22/aca130cf)
pve-kernel-4.2.8-1-pve: 4.2.8-41
pve-kernel-4.2.2-1-pve: 4.2.2-16
lvm2: 2.02.116-pve2
corosync-pve: 2.3.5-2
libqb0: 1.0-1
pve-cluster: 4.0-36
qemu-server: 4.0-64
pve-firmware: 1.1-7
libpve-common-perl: 4.0-54
libpve-access-control: 4.0-13
libpve-storage-perl: 4.0-45
pve-libspice-server1: 0.12.5-2
vncterm: 1.2-1
pve-qemu-kvm: 2.5-9
pve-container: 1.0-52
pve-firewall: 2.0-22
pve-ha-manager: 1.0-25
ksm-control-daemon: 1.2-1
glusterfs-client: 3.5.2-2+deb8u1
lxc-pve: 1.1.5-7
lxcfs: 2.0.0-pve2
cgmanager: 0.39-pve1
criu: 1.6.0-1
zfsutils: 0.6.5-pve7~jessie
openvswitch-switch: 2.3.2-2

on proxmox-ve: 4.1-41 (running kernel: 4.2.8-1-pve), I can do SR-IOV in my intel i-350(use igb)

Code:

root@pve:~# modinfo ixgbe |grep vf
parm:           max_vfs:Number of Virtual Functions: 0 = disable (default), 1-63 = enable this many VFs (array of int)
root@pve:~# modinfo igb |grep vf
parm:           max_vfs:Maximum number of virtual functions to allocate per physical function (uint)

I try to compiler new ixgbe driver, but SR-IOV It's not work
after sourceforge reply, I'm try to do ixgbe SR-IOV

command is this, then I can use SR-IOV in X552

Code:

rmmod ixgbe
modprobe ixgbe max_vfs=4,4

dietmar · Apr 22, 2016

It gets complicated and confusing if you start mixing different things in a single thread...

sili · Apr 26, 2016

dietmar said:
I assume the file /sys/class/net/eth0/device/sriov_numvfs does not exist?

Yes, It's not exist.

Code:

root@pve:~# lspci |grep Eth
03:00.0 Ethernet controller: Intel Corporation Ethernet Connection X552/X557-AT 10GBASE-T
03:00.1 Ethernet controller: Intel Corporation Ethernet Connection X552/X557-AT 10GBASE-T
05:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
05:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
07:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
07:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)

Code:

root@pve:/sys/class/net# ls -al
total 0
drwxr-xr-x  2 root root 0 Apr 26 13:46 .
drwxr-xr-x 71 root root 0 Apr 26 13:46 ..
lrwxrwxrwx  1 root root 0 Apr 26 14:01 bond0 -> ../../devices/virtual/net/bond0
lrwxrwxrwx  1 root root 0 Apr 26 13:46 eth0 -> ../../devices/pci0000:00/0000:00:02.2/0000:03:00.1/net/eth0
lrwxrwxrwx  1 root root 0 Apr 26 14:01 eth2 -> ../../devices/pci0000:00/0000:00:1c.0/0000:07:00.0/net/eth2
lrwxrwxrwx  1 root root 0 Apr 26 13:46 eth3 -> ../../devices/pci0000:00/0000:00:1c.0/0000:07:00.1/net/eth3
lrwxrwxrwx  1 root root 0 Apr 26 13:46 eth6 -> ../../devices/pci0000:00/0000:00:03.0/0000:05:00.1/net/eth6
lrwxrwxrwx  1 root root 0 Apr 26 13:46 eth7 -> ../../devices/pci0000:00/0000:00:03.0/0000:05:00.0/net/eth7

Code:

root@pve:/sys/class/net/eth3# ls
addr_assign_type  broadcast  device  dormant  gro_flush_timeout  iflink  name_assign_type  phys_port_id  power  speed  tx_queue_len
address  carrier  dev_id  duplex  ifalias  link_mode  netdev_group  phys_port_name  proto_down  statistics  type
addr_len  carrier_changes  dev_port  flags  ifindex  mtu  operstate  phys_switch_id  queues  subsystem  uevent
root@pve:/sys/class/net/eth2# ls
addr_assign_type  carrier  dev_port  gro_flush_timeout  link_mode  netdev_group  phys_switch_id  speed  type
address  carrier_changes  dormant  ifalias  master  operstate  power  statistics  uevent
addr_len  device  duplex  ifindex  mtu  phys_port_id  proto_down  subsystem  upper_ovs-system
broadcast  dev_id  flags  iflink  name_assign_type  phys_port_name  queues  tx_queue_len

sili · Apr 26, 2016

Hello, I know how to do SR-IOV...
I tried/etc/modules, set igb max_vfs=2,2 but didn't work, but it hints me towards another direction make it worked.

I used following setting and made intel i350 sr-iov enabled.

Code:

echo "options igb max_vfs=2,2" > /etc/modprobe.d/igb.conf
depmod -a
update-initramfs -u

Then Reboot

Code:

root@pve:~# lspci |grep Eth
03:00.0 Ethernet controller: Intel Corporation Ethernet Connection X552/X557-AT 10GBASE-T
03:00.1 Ethernet controller: Intel Corporation Ethernet Connection X552/X557-AT 10GBASE-T
05:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
05:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
07:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
07:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
08:10.0 Ethernet controller: Intel Corporation I350 Ethernet Controller Virtual Function (rev 01)
08:10.1 Ethernet controller: Intel Corporation I350 Ethernet Controller Virtual Function (rev 01)
08:10.4 Ethernet controller: Intel Corporation I350 Ethernet Controller Virtual Function (rev 01)
08:10.5 Ethernet controller: Intel Corporation I350 Ethernet Controller Virtual Function (rev 01)

thanks for all the helps

Search

Search

Cannot use SR-IOV on proxmox

sili

Renowned Member

dietmar

Proxmox Staff Member

sili

Renowned Member

dietmar

Proxmox Staff Member

sili

Renowned Member

dietmar

Proxmox Staff Member

dietmar

Proxmox Staff Member

sili

Renowned Member

dietmar

Proxmox Staff Member

sili

Renowned Member

sili

Renowned Member

We value your privacy