SAS (SPC-4) disks passthrough to TrueNAS

[franzli]

Hi,

I am having a little problem when passing the physical disk to a VM with TrueNAS. Everything works fine, except for the speed of the disks. When TrueNAS is installed directly on the host, the disks are being recognized as:

da0 at mps0 bus 0 scbus0 target 8 lun 0
da0: <NETAPP X423_HCOBE900A10 NA01> Fixed Direct Access SPC-4 SCSI device
da0: Serial Number KVHNNX1J
da0: 600.000MB/s transfers
da0: Command Queueing enabled
da0: 858483MB (1758174768 512 byte sectors)

While as a VM with disk passthrough I get this:

da0 at vtscsi0 bus 0 scbus0 target 0 lun 2
da0: <QEMU QEMU HARDDISK 2.5+> Fixed Direct Access SPC-3 SCSI device
da0: 300.000MB/s transfers
da0: Command Queueing enabled
da0: 858483MB (1758174768 512 byte sectors)

You can notice the difference.... SPC-4 vs SPC-4. 600.000MB/s vs 300.000MB/s

I am using the Blackmagic Disk speed test and I have a visible difference, exactly half of the speed. 447MB/s physical vs 244MB/s as VM.

This is the config file of the VM:

balloon: 0
boot: order=ide0;ide2;net0
cores: 4
cpu: host
ide0: local-VMdata:vm-137-disk-0,size=8G
ide2: none,media=cdrom
machine: q35
memory: 16384
name: TrueNAS-Test
net0: virtio=8E:9B:D8:15:F5:9B,bridge=vmbr1001,firewall=1
numa: 0
ostype: other
scsi2: /dev/disk/by-id/scsi-35000c5006bd576c7,iothread=1,size=879087384K
scsi3: /dev/disk/by-id/scsi-35000c5006bd5bf8b,iothread=1,size=879087384K
scsi4: /dev/disk/by-id/scsi-35000c5006bc5046f,iothread=1,size=879087384K
scsihw: virtio-scsi-single
smbios1: uuid=70370f13-96c1-4994-8b6b-552b4e9831f7
sockets: 2
tablet: 0
vmgenid: 2e967907-2563-4dcf-8972-ad6cbcdb5005

What am I doing wrong?

The host is a DELL R720 and I can't pass the full controller, because some disks are being used by proxmox.

Thanks in advance!

 

[apoc]

Try enabling IO-threading fot your disks.

Aside that you compare apples with, hm, i would say nuts
Your host has more CPUs, it has less layers to cross (virtualization) etc.. Of course it will be faster.

Virtualization comes at a certain cost. That's just the way it is. You gain: flexibility, isolation and much more.

Aside that your test means nothing. The only thing it shows is throughput. You are testing against a single 10k spindle. The last you should worry about is throughput. Bandwidth is typically enough even with 300mb/s second.

Some questions i want to raise: Is your application actually working like that? What is the block size which is used? The access pattern? Read / write ratio?

Again. The test means nothing...
It is a bit like comparing cars with their HP values. 1000 HP is better? Not if you can't get it on the street.
You currently care about your HP value having some bicycle tires in place

Sorry. An old storage guy is going crazy