good virtio-net performance

U

udo

Distinguished Member
Proxmox Subscriber
Apr 22, 2009
5,981
204
163
Ahrensburg; Germany
Hi,
i make yesterday a test with (for me) suprising result.

On a test-server with the actual pvetest + 2.6.35 kernel i use two kvm linux-guest (devil-linux 1.4RC6) which are connected with an internal bridge:
Code: 
modprobe -o dummy0 dummy
ifconfig dummy0 up
brctl addbr vmbr10
brctl addif vmbr10 dummy0
ifconfig vmbr0 up
First i try the e1000 driver and got following results with iperf (Mbit/s):
Code: 
1 -> 2 | 2 -> 1 # both VMs has 2 CPUs
  814  |  396
  897  |  389
  946  |  387

strange the different - test with only one cpu on both systems
1 -> 2 | 2 -> 1 # both VMs has 1 CPU
  802  |  728
  805  |  758
  814  |  788
       |  811
Now i switched the driver to virtio:
Code: 
1 -> 2 | 2 -> 1 # both VMs has 1 CPU - All Values are [B]GBit/s!![/B]
  4.35 |  4.32
  4.27 |  4.24
  4.23 |  4.33

Nice!! What happens with two cpus?
1 -> 2 | 2 -> 1 # both VMs has 2 CPUs - All Values are [B]GBit/s!![/B]
  3.98 |  4.01
  3.98 |  3.97
  4.06 |  4.20
Of course got the cpu-usage up to 100% (in all cases) but i'm not expect 4Gbit in a kvm-guest.
Not too bad...

Udo
 
Wow, very good.

Could we get somehow this performance beyond one host? I have three hosts with 4 NICs bonded with 802.3ad. But they are in production so I couldn´t test.

thx
macday
 
dietmar said:
Did you use the latest pve-qemu-kvm from pvetest and load the 'vhost_net' module?
Click to expand...
Hi Dietmar,
yes vhost=on are automaticly enabled:
Code: 
# ps aux | grep 141 | grep vhost
root     31232 31.0  0.3 1177820 26600 ?       Sl   15:22   0:03 /usr/bin/kvm -monitor unix:/var/run/qemu-server/141.mon,server,nowait -vnc unix:/var/run/qemu-server/141.vnc,password -pidfile /var/run/qemu-server/141.pid -daemonize -usbdevice tablet -name devil1 -smp sockets=2,cores=1 -nodefaults -boot menu=on,order=d -vga cirrus -tdf -k de -drive file=/var/lib/vz/template/iso/dl1.4rc6.iso,if=ide,index=2,media=cdrom -drive file=/var/lib/vz/images/141/vm-141-disk-1.raw,if=virtio,index=0 -m 1024 -netdev type=tap,id=vlan10d0,ifname=vmtab141i10d0,script=/var/lib/qemu-server/bridge-vlan,[B]vhost=on[/B] -device virtio-net-pci,mac=72:B9:47:D0:FF:73,netdev=vlan10d0

# ps aux | grep 142 | grep vhost
root      4375  5.4  1.6 1193000 128264 ?      Sl   Oct29 148:40 /usr/bin/kvm -monitor unix:/var/run/qemu-server/142.mon,server,nowait -vnc unix:/var/run/qemu-server/142.vnc,password -pidfile /var/run/qemu-server/142.pid -daemonize -usbdevice tablet -name devil2 -smp sockets=2,cores=1 -nodefaults -boot menu=on,order=d -vga cirrus -tdf -k de -drive file=/var/lib/vz/template/iso/dl1.4rc6.iso,if=ide,index=2,media=cdrom -drive file=/var/lib/vz/images/142/vm-142-disk-1.raw,if=virtio,index=0 -m 1024 -netdev type=tap,id=vlan10d0,ifname=vmtab142i10d0,script=/var/lib/qemu-server/bridge-vlan,[B]vhost=on[/B] -device virtio-net-pci,mac=72:B9:47:D1:FA:70,netdev=vlan10d0

Version:
Code: 
# pveversion -v
pve-manager: 1.6-7 (pve-manager/1.6/5277)
running kernel: 2.6.35-1-pve
proxmox-ve-2.6.35: 1.6-7
pve-kernel-2.6.32-3-pve: 2.6.32-14
pve-kernel-2.6.35-1-pve: 2.6.35-7
qemu-server: 1.1-24
pve-firmware: 1.0-9
libpve-storage-perl: 1.0-15
vncterm: 0.9-2
vzctl: 3.0.24-1pve4
vzdump: 1.2-8
vzprocps: 2.0.11-1dso2
vzquota: 3.0.11-1
pve-qemu-kvm: 0.13.0-2
ksm-control-daemon: 1.0-4


Udo
 
Hi,
perhaps that's require a newer virtio-driver inside the guest?
Code: 
modinfo virtio_net
filename:       /lib/modules/2.6.32.23/kernel/drivers/net/virtio_net.ko
license:        GPL
description:    Virtio network driver
alias:          virtio:d00000001v*
depends:        
vermagic:       2.6.32.23 SMP mod_unload modversions 686 
parm:           napi_weight:int
parm:           csum:bool
parm:           gso:bool
But i have make another test and get "only" 1.4GBit/s and only round 30% CPU usage (2 CPUs).
After stop and change to one cpu i got 4.57 GBit/s with up to 148% (in Webfrontend) cpu usage. Normaly 96+104%.


Udo
 
I tried installing the virtio-nic but in my Win2K8SBS-guest as in my Win2K8R2-enterprise-guest there is no new nic recognized. What do I have to do to install the virtio-nic in the guest-systems? In the meantime I'm using the e1000-nic, but this just has a performance of 500 MBit/s with one thread, 1Gbit/s/200MBit/s with option iperf -d.

Thanks,
Yves
 
did you powered off the guest after you added the virtio NIC? a reboot is not enough.
 
Well, at first I just did a reboot, after switching the vm the virtio-adapter was found in the vm with win2k8sbs, but not within win2k8R2 enterprise. I did some benchmarks with iperf (host-system to vm, vm to vm and vice versa) but the perfomance was quite "poor" (around 800/200mbit/s with iperf-option -d, 16kb windowssize linux, 8kb windowssize windows, 400/400mbit/s with windowssize 16kb on both machines)). With the e1000 adapter the performance was even a little bit better (1Gbit/s/200mbit/s with iperf-option -d, 16kb windowssize linux, 8kb windowssize windows, 550/550mbit/s with windowssize 16kb on both machines). CPU-usage was at 30-40%. Using just 1 cpu for each vm didn't make a difference.
 
the jperf download includes the iperf.exe, running well also on 64-bit (cmd).
 
I'm just adding some experience i had hoping it will be usefull for someone.

I have a server running PVE like a charm. I made a Lenny 64bit VM and started trying with virtio disk and network. All went ver well. I had 300Mbps with iperf with a e1000 nic, and with a virtio nic i climbed up to 1gbps!! Same speed as the host! That was very nice.

But i added a second Lenny 64bit VM, and network went definitily down... i lost 85% of pings to any guest!!
After proving many different ideas, i solved it upgrading the host from pve 1.6 to pve 1.7. It seems that the new kernel upgrade made the difference and now i have the two VM running virtio nics very well, with iperf above 900 Mbps

I hope it helps someone.
 
thanks for your posting - virtio on linux is very fast - we just see the same here in our test labs.
 
You must log in or register to reply here.
Top Bottom