How to make a best use of an SSD drive in VMs?

[shalak]

On my proxmox machine I got two SSD drives natively connected to it. First disk is for proxmox OS, the other I want to use in VMs.

Basically, I want to use half of this second SSD drive in one VM, and other half in another, preferably with auto-adjusting sizes (in case one VM needs more than a half, I cannot predict which will need more at this point).

In proxmox, I've set the second SSD up as an LVM-thin disk.

In VM, I added a new hard disk using SCSI Bus/Device and pointed it to the LVM-Think storege.

Unfortuantely, I'm getting real bad bottlenecks, `sysbench --test=fileio` executed in VM has told me:

Throughput:
    read, MiB/s:                  10.77
    written, MiB/s:               7.18

While on the proxmox host (on the first SSD), I'm seeing:

Throughput:
    read, MiB/s:                  71.33
    written, MiB/s:               47.56

That's pretty much an order of magnitude difference. Toggling the "SSD emulation" doesn't help.

What should I change in my workflow to get better results?

 

[Dunuin]

"virtio SCSI single" was used?
Both SSDs are the same model and size?

 

[shalak]

"virtio SCSI single" was used?

I'm not sure, I use GUI with following settings:

("fast" is the lvm-thin storage)

Both SSDs are the same model and size?

The first one is ADATA SU630, the second one (the one that is bottlenecking) is Seagate IronWolf 125. I would expect it to have better performance than ADATA.

 

[shalak]

I should add that those drives are connected to the server via HPE Smart Array P420i controller, it has this little battery. I wonder how safe would it be to use "writeback" Cache mode

 

[Dunuin]

I'm not sure, I use GUI with following settings:

("fast" is the lvm-thin storage)

You are using an emulated LSI controller. Try "virtio SCSI single" which should be faster.

The first one is ADATA SU630, the second one (the one that is bottlenecking) is Seagate IronWolf 125. I would expect it to have better performance than ADATA.

Then your benchmarks are not comparable. You should benchmark the same SSD model with and without virtualization. Maybe that IronWolf is slow too when directly running that benchmark on the host without virtualizaton.

 

[shalak]

You are using an emulated LSI controller. Try "virtio SCSI single" which should be faster.

I cannot seem to change the SCSI Controller value from GUI and for Bus/Device I can only see:

Then your benchmarks are not comparable. You should benchmark the same SSD model with and without virtualization. Maybe that IronWolf is slow too when directly running that benchmark on the host without virtualizaton.

You're right. I wiped the disk, set up a Directory storage using ext4 and repeated the test:

Throughput:
    read, MiB/s:                  111.64
    written, MiB/s:               74.43

Thus confirming the bottleneck.

 

[Dunuin]

I cannot seem to change the SCSI Controller value from GUI and for Bus/Device I can only see:

 

[shalak]

Thanks, that helped a bit:

With IO Thread disabled:

Throughput:
    read, MiB/s:                  26.73
    written, MiB/s:               17.82

With IO Thread enabled:

Throughput:
    read, MiB/s:                  34.65
    written, MiB/s:               23.10

Is this the best I can aim for? VM overhead leaves us with 1/3rd of the original performance?

 

[Dunuin]

Really depends on the workload. For example doing 1M sequential async writes got way less overhead than 4K random sync writes. Especially with nested filesystems where metadata multiplies. For more individual benchmarks you could use fio.

 

[shalak]

I noticed, that LVM-think works way worse, than Directory storage + qcow2 files for VM disks. What FS for the Directory should I use for such approach? xfs, or ext4? it will only contain the qcow2 files...

 

[Dunuin]

LVM-thin is still very lightweight. Stuff like ZFS will be way worse with easily times the overhead of LVM-Thin. Not sure if ext4 or xfs will be better. Best you benchmark it yourself.