Vm running slow with sata drive passthrough

[chattingfun]

Hello,
How to troubleshoot a slow vm?

I configured this vm to have direct access to a seagate sata drive with passthrough.
The drive is setup with a NTFS partition so if I need to take it out and connect it to a windows machine.
It is running current version of ubuntu server. The vm has 1 socket 8 cores cpu, 32gb of memory. The CPU usage is around 5-10% usage but the memory is 90% usage.


Does given a vm direct access to hard drive slow the vm down?

Thank you

 

[Dunuin]

If your disk is slow (for example SMR HDD) the CPU might need to wait for disk IO, your "IO delay" will go up and everything might be become very slow (or even not responding anymore).

 

[chattingfun]

If your disk is slow (for example SMR HDD) the CPU might need to wait for disk IO, your "IO delay" will go up and everything might be become very slow (or even not responding anymore).

The drive is Seagate Exos X20 20TB Enterprise HDD - CMR 3.5 Inch Hyperscale SATA 6Gb/s, 7200 RPM, 512e and 4Kn FastFormat, Low Latency with Enhanced Caching - ST20000NM007D

Is this a slow drive?

 

[Dunuin]

Not for a HDD, but depending on your workload a SSD might be magnitudes (like 100-1000 times) faster. Should be fine for sequential reads/writes. But if your workload needs IOPS performance the HDD might slow everything down.

 

[chattingfun]

Which way is better to configure Proxmox to handle being a NAS?

• Add the secondary drive as a repository where vm hard drive is saved?
• Create a vm running NAS software?
• Create a server vm and configure it for direct access with passthrough?

Which way gives better throughput?

The way I had my Proxmox configure before was to run a TrueNAS VM. Have my Ubuntu server connect to it via a mount.
The TrueNAS ran slow but the Ubuntu server that connected to it ran okay. No other programs was affected by the slowness.

I through If I give direct access to the sata hard drive it would be faster then having more of a overhead with addinal vm in the mix.
So I removed the TrueNAS vm and configure passthrough to the hard drive to the Ubuntu server. Now is runs slow with high memory use when downloading and other programs are slow to respond.

 

[Dunuin]

Which way is better to configure Proxmox to handle being a NAS?

• Add the secondary drive as a repository where vm hard drive is saved?
• Create a vm running NAS software?
• Create a server vm and configure it for direct access with passthrough?

Which way gives better throughput?

Depends...
When using TrueNAS you got ZFS inside the VM, so you probably want as less overhead as possible on the host level.

You are by the way probably not giving the VM direct access to the HDD. That is only possible by passing through the whole disk controller. With single disk passthrough the VM is still working with a virtul disk and got virtualization overhead.

 

[chattingfun]

Depends...
When using TrueNAS you got ZFS inside the VM, so you probably want as less overhead as possible on the host level.

You are by the way probably not giving the VM direct access to the HDD. That is only possible by passing through the whole disk controller. With single disk passthrough the VM is still working with a virtul disk and got virtualization overhead.

The proxmox server has it's own drive. I did configure the second drive as a passthrough to vm 300. Only this vm is using it.

 

[Dunuin]

Yeah, that is still virtualized. Check fdisk -l inside the VM and it will report to be a "QEMU HARDDISK" disk model instead of your Exos X20, as your Ubuntu can only see that 20TB virtual disk. Might be still beneficial, as you don'T need an additional filesystem to store that virtual disk, but you still won't get direct physical access to that HDD.

 

[chattingfun]

It does not list it as QEMU HARDDISK

Disk /dev/vda: 18.19 TiB, 20000588955648 bytes, 39063650304 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 1D650768-4763-435A-93E8-CEC381DF6F8B

Device Start End Sectors Size Type
/dev/vda1 2048 39063648255 39063646208 18.2T Microsoft basic data

 

[chattingfun]

Is there any setting I could change to make it faster like swappiness?

 

[Dunuin]

It does not list it as QEMU HARDDISK

Disk /dev/vda: 18.19 TiB, 20000588955648 bytes, 39063650304 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 1D650768-4763-435A-93E8-CEC381DF6F8B

Device Start End Sectors Size Type
/dev/vda1 2048 39063648255 39063646208 18.2T Microsoft basic data

vda = first paravirtualizated disk driver = virtual virtio SCSI disk

Is there any setting I could change to make it faster like swappiness?

You could try if enabling writeback or writethrough is helping for short bursts of reads/writes. But keep in mind that this will also increase RAM usage as this needs to be cached in RAM too. And especially writeback isn't great for data integrity..especially when not using a UPS/redundant PSU/BBU. As every write cached in RAM will be lost on an power outage.