Network performance on Host vs guest (using virtio!)

[Dcn]

Hi all,

I'm seeing a huge drop of performance between running Iperf on the host and on a guest, even using virtio:

- Laptop versus PVE Host 942 Mbits/sec
- Laptop versus PVE guest (ubuntu 11.04 with virtio) 370 Mbits/sec

Here's the kvm command visible on "ps aux":

/usr/bin/kvm -monitor unix:/var/run/qemu-server/106.mon,server,nowait -vnc unix:/var/run/qemu-server/106.vnc,password -pidfile /var/run/qemu-server/106.pid -daemonize -
usbdevice tablet -name vm-filesrv -smp sockets=2,cores=2 -nodefaults -boot menu=on -vga cirrus -tdf -k fr -drive file=/var/lib/vz/template/iso/ubuntu-10.04.1-server-
amd64.iso,if=ide,index=2,media=cdrom -drive file=/dev/vg_raid5/vm-106-disk-1,if=virtio,index=0,cache=none,boot=on -m 2048 -netdev
type=tap,id=vlan0d0,ifname=tap106i0d0,script=/var/lib/qemu-server/bridge-vlan,vhost=on -device virtio-net-pci,romfile=,mac=BE:20:B5:BD:AC:6B,netdev=vlan0d0

I've red the performance tuning posts and so far don't understand why my setup has so poor performance while using to good configuration options.
Thanks for your support.

 

[Dcn]

Any idea if it is hardware or software related?

 

[tom]

if you post in detail your setup and your benchmark tool also others can try to do the same and you can compare the values.

so post more details, also the output of 'pveversion -v'

 

[tom]

... (ubuntu 11.04 with virtio)...

ubuntu-10.04.1-server-amd64.iso...

10.04 or 11.04?

 

[tom]

did some tests from my Debian Squeeze Notebook to a pve host and a pve KVM guest Ubuntu 10.04.

root@t400:/home/tom# iperf -c pvehost
------------------------------------------------------------
Client connecting to pvehost, TCP port 5001
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[  3] local 192.168.2.210 port 60525 connected with pvehost port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  1.09 GBytes    933 Mbits/sec

root@t400:/home/tom# iperf -c pveguest
------------------------------------------------------------
Client connecting to pveguest, TCP port 5001
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[  3] local 192.168.2.210 port 42689 connected with pveguest port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  1.03 GBytes    885 Mbits/sec
root@t400:/home/tom#

 

[Dcn]

Sure. I also remade some tests on another iperf client using Ubuntu 11.04.

First a little schema of my setup:

Ubuntu Laptop
|
1m UTP CAT6 cable
|
1GB switch
|
15m UTP CAT6 cable
|
1GB switch
|
1m UTP CAT6 cable
|
PVE with several NIC including one using sky2 driver used for the test
|-> kvm: ubuntu 10.04 LTS using virtio for storage and network NIC


I use the latest stable version of PVE based on 2.6.32 kernel.
I tried using the 2.6.35 kernel without noticing significant performance improvements.

proxmox:~# pveversion -v
pve-manager: 1.9-24 (pve-manager/1.9/6542)
running kernel: 2.6.32-6-pve
proxmox-ve-2.6.35: 1.8-13
pve-kernel-2.6.32-4-pve: 2.6.32-33
pve-kernel-2.6.35-2-pve: 2.6.35-13
pve-kernel-2.6.32-6-pve: 2.6.32-47
qemu-server: 1.1-32
pve-firmware: 1.0-14
libpve-storage-perl: 1.0-19
vncterm: 0.9-2
vzctl: 3.0.29-2pve1
vzdump: 1.2-16
vzprocps: 2.0.11-2
vzquota: 3.0.11-1
pve-qemu-kvm: 0.15.0-1
ksm-control-daemon: 1.0-6


In order to measure the network performance I used Iperf as a client on Ubuntu 11.04 Linux and Iperf as a server on, first, the PVE host, and then on the KVM guest(using 10.04 LTS).
Note the NIC used by the guest is a bridge on Host side, thus stricly the recommanded setup from linux-kvm.org.


First test from Linux laptop to PVE host:

dcn@laptop:~$ iperf -c 192.168.1.150
------------------------------------------------------------
Client connecting to 192.168.1.150, TCP port 5001
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[ 4] local 192.168.1.4 port 49834 connected with 192.168.1.150 port 5001
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-10.0 sec 1.10 GBytes 942 Mbits/sec


Second test from the same laptop but now connecting to the KVM VM running ubuntu 10.04 LTS

dcn@laptop:~$ iperf -c 192.168.1.101
------------------------------------------------------------
Client connecting to 192.168.1.101, TCP port 5001
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[ 5] local 192.168.1.4 port 49833 connected with 192.168.1.101 port 5001
[ ID] Interval Transfer Bandwidth
[ 5] 0.0-10.0 sec 402 MBytes 337 Mbits/sec

Conclusion:

I think the problem comes from software, especially a misconfiguration from my side onthe PVE host - but as described in my first post in this thread I think I've followed the best practices), or a software issue in kvm in itself.

 

[Dcn]

Right it is ubuntu server 10.04 LTS sorry.

$ uname -a
Linux vm-filesrv 2.6.32-24-server #43-Ubuntu SMP Thu Sep 16 16:05:42 UTC 2010 x86_64 GNU/Linux

 

[tom]

...
In order to measure the network performance I used Iperf as a client on the Windows host and Iperf as a server on, first, the PVE host, and then on the KVM guest.
Note the NIC used by the guest is a bridge on Host side, thus stricly the recommanded setup from linux-kvm.org.
..

my conclusion: you try to puzzle me, now you are mentioning a windows host involved?

 

[Dcn]

I'm sorry. My diagram was not displayed as intended and I made some mistakes. Now fixed

 

[udo]

Right it is ubuntu server 10.04 LTS sorry.

$ uname -a
Linux vm-filesrv 2.6.32-24-server #43-Ubuntu SMP Thu Sep 16 16:05:42 UTC 2010 x86_64 GNU/Linux

Hi,
kvm-networkspeed is also related to cpu-power (i don't know how much with virtio). E.G. if your server have more cores with less speed, your client will not reach the same network traffic than the same client on a faster CPU (e.g. less cores with more speed).

Udo

 

[Dcn]

You are right I've notice kind of high cpu usage on both PVE and the VM while doing the iperf tests, although it does not cap.
I have an Intel(R) Core(TM)2 Quad CPU Q9400 @ 2.66GHz CPU on the PVE host and 2 vCores in the VM itself.

Thanks!

 

[maxprox]

Hello,

I also use virtio - just to compare

#########################################################
from one proxmox host to another proxmox host:

root@katapult:~# iperf -c fcprox01
------------------------------------------------------------
Client connecting to fcprox01, TCP port 5001
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[ 3] local 192.168.8.4 port 48574 connected with 192.168.8.6 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 1.10 GBytes 943 Mbits/sec

#########################################################
from a linux VM - Zentyal=>Ubuntu10.4 to a proxmox host

admin@zentyal-vm:~$ iperf -c 192.168.8.4
------------------------------------------------------------
Client connecting to 192.168.8.4, TCP port 5001
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[ 3] local 192.168.8.3 port 57671 connected with 192.168.8.4 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 1.08 GBytes 928 Mbits/sec

#########################################################
from a linux VM - Zentyal=>Ubuntu10.4 to it's own proxmox host

admin@zentyal-vm:~$ iperf -c 192.168.8.6
------------------------------------------------------------
Client connecting to 192.168.8.6, TCP port 5001
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[ 3] local 192.168.8.3 port 53029 connected with 192.168.8.6 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 5.01 GBytes 4.30 Gbits/sec
admin@zentyal-vm:~$

maxprox

 

[tom]

yes, these very good results are the expected one. virtio network with Linux is extremely fast.