How to use Virtio-blk-data-plane in Proxmox instead Virtio-SCSI?

[Alibek]

Currently with VirtIO-SCSI (VirtIO-SCSI Single with threads) have max IOPS is ~1.8k-2.3k. But Virtio-blk-data-plane may have over 100k IOPS https://www.suse.com/media/white-paper/kvm_virtualized_io_performance.pdf

May I switch to Virtio-blk-data-plane instead VirtIO-SCSI in Proxmox?

 

[wolfgang]

Hi,

virtio-SCSI single is capable to handle over 15k-25K dependence on your CPU clock speed.
I would guess your storage are to slow.

This Virtio-blk-data-plane is based on iSCSI what means you have to create for every vdisk a lun on the host system.
For the normal user, this approach is not needed and will also use on the system more resources in the LIO stack.

But anyway you can do this if you like but you have to set it up yourself.

 

[Alibek]

I would guess your storage are to slow.

You're wrong:


Virtio-blk-data-plane is obsoluted.
In qemu earlier have been flag x-data-plane for virtio-blk instead virtio-blk-data-plane, later this flag and virtio-blk replaced to virtio-scsi (single with threads) and I was test all drivers on pve - maximum result: 2400 iops (inside vm).
Platform:
CPU 2x EPYC 7601
RAM 2TB
Storage1: 4x NVMe SSD Optane (r500k/w500k iops per device)
Storage2: 2x SAS SSD (r80k/w30k iops per device)
Storage3: FC 16Gpbs 18 HDD + 2 SSD Cache + 2 SSD Log, ZFS.

proxmox-ve: 5.3-1 (running kernel: 4.15.18-4-pve)

 

[spirit]

Currently with VirtIO-SCSI (VirtIO-SCSI Single with threads) have max IOPS is ~1.8k-2.3k. But Virtio-blk-data-plane may have over 100k IOPS https://www.suse.com/media/white-paper/kvm_virtualized_io_performance.pdf

May I switch to Virtio-blk-data-plane instead VirtIO-SCSI in Proxmox?

Use virtio disk and enable iothread

 

[wolfgang]

Storage is both HW and software layer.
So the next question is what you use as the software layer at your storage?

I made this test on a Thretripper what nearly the same as you Epyc.
Intel Optane Memory 32GB direct attached on a VM no cache.

fio --filename=/dev/sdb --bs=4K --rw=write --name=test --direct=1 --sync=1 --ioengine=libaio --size=27G --numjobs=1

The result for a virtio-scsi-pci is avg 10412 IOPS
Run status group 0 (all jobs):
WRITE: io=27648MB, aggrb=41648KB/s, minb=41648KB/s, maxb=41648KB/s, mint=679768msec, maxt=679768msec

The result for a virtio-blk bus is avg 10819 IOPS
WRITE: io=27648MB, aggrb=43279KB/s, minb=43279KB/s, maxb=43279KB/s, mint=654159msec, maxt=654159msec

The raw speed on the host is avg 47087 IOPS
WRITE: io=5120.0MB, aggrb=188348KB/s, minb=188348KB/s, maxb=188348KB/s, mint=27836msec, maxt=27836msec