Query about HDD passthru

[NickH]

Following advice a long time ago, I have a couple of VM accessing disks in passthru mode:

qm set 117 -scsi1 /dev/disk/by-id/ata-TOSHIBA_MG09ACA18TE_Y1X0A26JFJDH
qm set 117 -scsi2 /dev/disk/by-id/ata-TOSHIBA_MG09ACA18TE_Z1B0A02KFJDH
qm set 117 -scsi5 /dev/disk/by-id/ata-TOSHIBA_MG09ACA18TE_62X0A100FJDH
qm set 117 -scsi4 /dev/disk/by-id/ata-TOSHIBA_MG09ACA18TE_62X0A0Q9FJDH

qm set 106 -scsi1 /dev/disk/by-id/ata-WDC_WD20EFRX-68EUZN0_WD-WMC4M1334155

The first four are a RAID array and their reshape performance in the VM has been terrible. I am wondering something. These are all SATA disks but I am using the -scsi switch and not a -sata switch. Could this be impacting in performance in the VM?

 

[leesteken]

These are all SATA disks but I am using the -scsi switch and not a -sata switch. Could this be impacting in performance in the VM?

No, but not using the VirtIO SCSI (single) controller will reduce the performance (but it does require drivers for the operating system inside the VM).
VirtIO SCSI Single and enableding IO Thread will reduce CPU overhead and remove the single I/O thread limit.

 

[NickH]

Both guests are ClearOS7 (Centos7 look-alikes) so I assume I have the relevant drivers. How do I use the VirtIO SCSI (single) controller? Is it a question of adding a switch --scsihw virtio-scsi-single? How do I then select the IO Thread option or will that appear in the VM's hard disk settings under Advanced?

 

[leesteken]

Both guests are ClearOS7 (Centos7 look-alikes) so I assume I have the relevant drivers.

GNU/Linux has them usually built-in, yes.

How do I use the VirtIO SCSI (single) controller? Is it a question of adding a switch --scsihw virtio-scsi-single?

VM Settings > Hardware > SCSI Controller (double click or press the Edit button).

How do I then select the IO Thread option or will that appear in the VM's hard disk settings under Advanced?

VM Settings > Hardware > Hard Disk (double click or press the Edit button, for each disk) and enable Advanced.

EDIT: Or use the command line interface for the VM configuration file.

 

[NickH]

Got it. I'll take the machines down after lunch and give it a go. Thanks.

 

[NickH]

Both VM's have changed over now, and I don't have a RAID maintenance operation to do so I can't really test for the moment.

 

[leesteken]

Both VM's have changed over now, and I don't have a RAID maintenance operation to do so I can't really test for the moment.

There is not much more you can do for performance, except maybe experiment with the cache settings, so the results don't really matter.
How much slower was the RAID maintenance compared to running without Proxmox on bare metal?

 

[NickH]

I did't try bare metal, but when I added an HDD to the array to go from a 3 * 18TB disk to 4 * 18TB disk RAID5, estimated reshape times were > 30 days. I did a load of tinkering in the guest o/s and brought it down massively (after 7 days). The maximum rate I was running at (from "cat /proc/mdstat") was about 200,000K/sec:

md127 : active raid5 sdb1[3] sdc1[2] sde1[0] sdd1[1]
      35156183040 blocks super 1.2 level 5, 64k chunk, algorithm 2 [4/4] [UUUU]
      [==============>......]  reshape = 71.2% (12523444480/17578091520) finish=
461.6min speed=182471K/sec

It had been running at c. 40,000K/sec until it hit free space and it had been running at < 10,000K/sec until I tweaked the guest. The RAID was 70% used as I added the disk and was in use as it was reshaping.

Someone I know was setting up a bare metal RAID10 and was hitting > 1,000,000K/sec for an empty array.