Debian router on proxmox, network performance issues

[Seph]

Hello, i have fresh proxmox install (latest version) on Dell R610 with:
01:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet (rev 20)
01:00.1 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet (rev 20)
02:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet (rev 20)
02:00.1 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet (rev 20)
72GB of ECC Ram
2 x Intel(R) Xeon(R) CPU E5540 @ 2.53GHz (2 Sockets)

and virtual machine running Debian 7 router with IMQ and quagga BGP @3.10.17 kernel

I have tried several diffrent network configurations, including passthrough on nic, virtio bridge with multiqueue or intel E1000.

On iperf tests i am getting great results, but when i connect actual clients to it (that generate around 350mbit traffic) server is able to serve only around 200mbit of traffic with pci passthrough, and around 150mbit with virtio multiqueue=8.
Cpu utilization skyrockets to 80% (8c/16t guest)

I have another exactly same machine running exact copy of debian 7 router natively, getting full 1Gbit speed on wan tests, cpu utilization around 1% with all clients connected.

Is there anything i should know about proxmox or debian config that would let me run this setup?

 

[joshin]

Have you disabled hardware checksum offload on the VM?

 

[Seph]

Haven't tried yet. I will try tomorrow.

 

[mir]

Debian 7

Are you sure it is Debian 7? Debian 7 has been EOL for several years!!

 

[spirit]

>>Cpu utilization skyrockets to 80% (8c/16t guest)

That seem pretty huge for 200mbits. Are you doing a lot of packet per second ?

Is is a custom kernel ? (3.10 on debian ?)

 

[Seph]

Yeah Debian 7 with custom kernel with some custom IMQ, iproute, iptables patches. Doing quite alot of pps as it is wireless network ISP with around 700 clients.

 

[joshin]

Sounds like Vyatta/VyOS. I remember needing to turn off hardware checksums for it to behave.

I ended up being lazy and moved to PFSense.

 

[Seph]

Disabling hardware checksum offload did not help, i will try fresh install, instead of moving with clonezilla image.

 

[joshin]

Disabling hardware checksum offload did not help, i will try fresh install, instead of moving with clonezilla image.

Bummer. What do you have set for the CPU hardware in the VM's settings? Make sure it matches your native system.

 

[Seph]

Tried Host and Nehelem - did not help. Fresh install not good either. Next on my list is replacing network adapter with Intel i350, if that doesn't help i will try to get it running on newer hardware on Dell R620.