VirtIO vs SCSI

tuxis

Active Member
Jan 3, 2014
100
9
38
Ede, NL
www.tuxis.nl
Some VMs get huge disk I/O performance increase, simply by switching the disk bus/device to VirtIO Block.

Results with SCSI:

Code:
[root@acrux testjes]# dd if=/dev/zero of=testfile bs=1M count=1000 oflag=dsync
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB) copied, 93.3164 s, 11.2 MB/s
[root@acrux testjes]# dd if=/dev/zero of=testfile bs=1G count=1 oflag=dsync
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 20.9574 s, 51.2 MB/s


Results with VirtIO block:

Code:
[root@acrux testjes]# dd if=/dev/zero of=testfile bs=1M count=1000 oflag=dsync
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB) copied, 42.9133 s, 24.4 MB/s
[root@acrux testjes]# dd if=/dev/zero of=testfile bs=1G count=1 oflag=dsync
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 5.74577 s, 187 MB/s

Can this huge performance increase be expected by using VirtIO? It seems odd that such a seemingly small change has such a large impact :)

Background information: Ceph cluster with 3 nodes. SCSI controller used is VirtIO SCSI (in both tests).
 

wolfgang

Proxmox Staff Member
Staff member
Oct 1, 2014
6,497
463
103
Hi,

generally, a test with dd is not really meaningful.
Use fio instead.

What I can tell is that the scsi virtio is better maintained and virtio-blk is the older one.
 

tuxis

Active Member
Jan 3, 2014
100
9
38
Ede, NL
www.tuxis.nl
Hi,

generally, a test with dd is not really meaningful.
Use fio instead.

What I can tell is that the scsi virtio is better maintained and virtio-blk is the older one.
I used `fio` afterwards to test the best-performing setups according to `dd`. VirtIO SCSI controller with SCSI bus for disk with SSD emulation and writeback cache is a winner. Actually, stuff like IO thread with VirtIO SCSI single controller makes things perform worse. I was also surprised to see that SSD emulation makes a significant difference..

https://pve.proxmox.com/wiki/Performance_Tweaks states: "Use virtIO for disk and network for best performance."

Do they mean VirtIO controller or VirtIO bus?
 
Last edited:

lilszi

New Member
Jun 25, 2020
1
0
1
32
I used `fio` afterwards to test the best-performing setups according to `dd`. VirtIO SCSI controller with SCSI bus for disk with SSD emulation and writeback cache is a winner. Actually, stuff like IO thread with VirtIO SCSI single controller makes things perform worse. I was also surprised to see that SSD emulation makes a significant difference..

https://pve.proxmox.com/wiki/Performance_Tweaks states: "Use virtIO for disk and network for best performance."

Do they mean VirtIO controller or VirtIO bus?

I just signed up to say thank you for this post, I was having performance issues in the VM's on 6g SSD's pulling only 50MB/s
With your Recommendations, it's now up to 150MB/s
 

gb00s

Member
Aug 4, 2017
19
0
6
41
... stuff like IO thread with VirtIO SCSI single controller makes things perform worse.
On a single bench maybe, but on a real-life system with several i/o-intensive workloads (VM's and databases), why i/o-threading should be a disadvantage if i/o-threading give you several uncorrelated i/o-venues? Without i/o-threading everything stays in line and has to wait until it is to be handled.

I'm curious now ...
 
May 18, 2019
161
7
18
Los Angeles, CA USA
From what I gathered after a lot of research, including some benchmarks

Real physical SATA is the fastest. But this is an emulated environment, so the fastest option is VirtioSCSI.
Within the device, chose writeback cache (the safe option, not the unsafe one)
 
  • Like
Reactions: gb00s and avw

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!