High PPS network with LXC

[DMD]

I am looking for a way to setup LXC containers that have ability to handle full 10GE worth of small packets. There can be no software bridge in between the container and the network card. What options are available with PVE 4.0?

For example, can I use SR-IOV with LXC under Proxmox? Or PCI passthrough?

 

[manu]

Hi

If you don't want any kind of routing or bridge taking in the Proxmox host, I would recommend PCI Passthrough with a KVM based virtualmachine.
There is a community wiki article about that in https://pve.proxmox.com/wiki/Pci_passthrough

With LXC you can "share" a /dev device between host and guest, but the host has still access to it, so it is not a real passthrough. I know some people have been experimenting with accelereated X serves in containers with passing of /dev devices, I am not sure this would work with networks cards and is all rather experimental.

 

[DMD]

Ok, but then I will run out of PCI devices as I need more VMs/containers than network cards that I have in the server. How is SR-IOV compared to PCI passthrough in terms of performance with lots of small packets under PVE 4.0?

I know VMware has a limit with SR-IOV due to IOTLB limits which they promise to solve with VT-d superpage support. Apprently under VMware SR-IOV can handle less than half as many small packets as native Linux due to this limitation. Is SR-IOV subject to the same limitation under PVE 4.0?

 

[manu]

Hi
Concerning SRV-IO I haven't heard of limitations in Proxmox VE 4.
We don't have yet support for easy configuring of the SRV-IO Virtual Functions in our framework, but since we allow passing extra parameters in both containers and KVMs, there is no reason it should not work.
For KVM you would need to pass the PCI address of the Virtual Function with VFIO, for LXC you would need to create eth devices associated to each Virtual Function and assign the device to containers.