Big discovery on virtio performance

[mgiammarco]

Hello,
I have a proxmox cluster of three HP DL380 G9 with 192gb ram each, ssd disks, raid controller with bbu ,10g ethernet and so on.
I am using windows 10 and 2019 VMs on ceph with krbd, writeback, iothread and all options that can improve performance.
So I have installed NVMe disks and built another ceph cluster with only nvme.
No matter what I do i/o performance on windows 10 and 2019 is quite low (tested in severaly ways and with crystal disk mark)
In my office cluster which is with very modest I get much better i/o in my windows vm.
BUT in my windows machine I used FlexVDI virtio drivers (it was a test) which are a 2018 fork of virtio and they are 32bit (not even 64bit drivers)
So I installed a new windows 10 machine with virtiodrivers on cluster above and BUM! performance skyrocketed.
In repeated reads performance is so high that it is pretty clear that it is reading from page cache (krbd). The latest fedora virtio drivers get no improvements by page cache.
So the question is:
- why 0.171 and 0.173 virtio drivers are so slow?
- why 32bit drivers of 1 year ago are so fast?

Can someone test flexvdi drivers and confirm me that I am not dreaming?
Thanks,
Mario

 

[mgiammarco]

I am talking about a 10x improvement in performance... is it strange nobody is interested.

 

[mgiammarco]

How can force windows to install 32bit driver over existing 64 bit?

 

[spirit]

I am talking about a 10x improvement in performance... is it strange nobody is interested.

Maybe this is a regression. you could open a bug directly at
https://github.com/virtio-win/kvm-guest-drivers-windows

are you talking about virtio-blk or virtio-scsi ?

 

[mgiammarco]

virtio-scsi I will now check github

 

[mmenaz]

virtio-scsi I will now check github

Do you think that this is the problem and has been fixed with this commit?
https://github.com/virtio-win/kvm-g...mmit/62e452b94b5277775f7bee7a0ec1f6148f8a1811

 

[spirit]

I don't know, but it's not yet included in latest iso

https://fedorapeople.org/groups/virt/virtio-win/CHANGELOG

But it's not the first time that drivers have regressions.

but it seem to be on big block transfert only.
(do you have crystal mark benchmark to see if it's failing with small block or big block size ?)

 

[xiaolin0199]

From the current use situation, the performance of rbd under windows (virtioscsi) is indeed relatively poor. Whether it is sequential reading and writing of large files or random reading and writing of small files, it can only reach 1/3 to 1/4 of bare rbd. Now, is there still room for driver optimization?

 

[AlexLup]

@mgiammarco - Could you please provide an URL to these drivers?

 

[mac.linux.free]

I think we have to wait for the new compiled binary drivers from Fedora.