[SOLVED] PVE 4.1 how to passthrough Nic to LXC

[elurex]

Anyone knows how to pass sr-iov VF Nic to LXC in pve 4.1?

The VF nic highlight in *** are the ones that I want to pass to LXC

root@pve-dol1:/sys/class/net/eth6/device#  lspci |grep Eth
03:00.0 Ethernet controller: Intel Corporation Ethernet Connection X552/X557-AT 10GBASE-T
03:00.1 Ethernet controller: Intel Corporation Ethernet Connection X552/X557-AT 10GBASE-T
05:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
05:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
***05:10.1 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
***05:10.3 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
***05:10.5 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
***05:10.7 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
07:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
07:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)

I know first I must set in /etc/pve/lxc/CTID.conf

lxc.cgroup.devices.allow = c XXX:* rwm (where xxx = major *=minor ids)
lxc.mount.entry = /dev/eth0 dev/eth0 none bind,optional,create=file

but I cannot find any VF eth NIC under pve host's /dev/

 

[fireon]

I don't know... why do you would like to do this? This make for me no sense. Make an extra bridge with an specific interface and assign this to your LXC.

 

[elurex]

It's a 10gbe SPF+ which linux bridge or openvswitch will decrease network throughput tremendously

 

[elurex]

AFAIK they should show up on the host in `ip link` / `ifconfig`, so you should be
able to pass the interface via lxc.network.type+lxc.network.link:
eg. if it's enp5s16f1 (which I think would be the default these days for your pci ids):

lxc.network.type: phys
lxc.network.link: enp5s16f1

Note that the order is important, first .type, then .link.

 

[sigxcpu]

http://glennklockwood.blogspot.ro/2013/12/high-performance-virtualization-sr-iov.html

This doesn't say that the standard config will "decrease network throughput tremendously". SR-IOV provides better latency (12% average).

 

[elurex]

that article only applies to amazon env

https://www.openstack.org/summit/va...formance-neutron-using-sr-iov-and-intel-82599

cpu usage... and also network througput...

 

[elurex]

https://software.intel.com/en-us/ar...t-virtualization-sr-iov-with-linux-containers

Intel's How to... and it works.