Disk Passthrough to VM - 50% Performance Reduction

Apr 17, 2020
33
4
13
40
Testing out disk passthrough to a VM from a SAS HBA and I'm noticing a 50% reduction in speed from writes on the Promox Host vs the VM. This is with the default no cache option as I'm trying to test out direct sync speed.

pve-manager/6.2-4/9824574a (running kernel: 5.4.34-1-pve)

Here is my VM configuration.

Code:
agent: 1,fstrim_cloned_disks=1
boot: c
bootdisk: scsi0
cipassword:
ciuser: cloud-user
cores: 10
ide2: local-zfs-0:vm-3203-cloudinit,media=cdrom
ipconfig0: ip=10.0.3.203/24,gw=10.0.3.1
ipconfig1: ip=10.0.4.203/24
memory: 114688
name: rancher-wkr-1
nameserver: 10.0.0.2
net0: virtio=1E:68:64:EB:9F:DB,bridge=vmbr0,firewall=1,tag=13
net1: virtio=26:B7:27:D5:3E:15,bridge=vmbr1,firewall=1,tag=14
numa: 0
onboot: 1
ostype: l26
scsi0: local-zfs-0:vm-3203-disk-0,size=50G
scsi1: /dev/disk/by-id/scsi-35000c50040b556f3,backup=0,replicate=0,size=1953548336K
scsi2: /dev/disk/by-id/scsi-35000c50034f3106f,backup=0,replicate=0,size=1953548336K
scsi3: /dev/disk/by-id/scsi-35000c50040b56063,backup=0,replicate=0,size=1953548336K
scsi4: /dev/disk/by-id/scsi-35000c50034fe38a3,backup=0,replicate=0,size=1953548336K
scsi5: /dev/disk/by-id/scsi-35000c5004094463f,backup=0,replicate=0,size=1953548336K
scsi6: /dev/disk/by-id/scsi-35000c50033e4f523,backup=0,replicate=0,size=1953514584K
scsihw: virtio-scsi-pci
searchdomain: example.com
serial0: socket
smbios1: uuid=ced463df-4298-432e-8498-56a5e631eb65
sockets: 2
sshkeys:
startup: order=1
vga: serial0
vmgenid: cb291b0f-d9dc-4671-965f-178c808a1cc4

And the test results for /dev/disk/by-id/scsi-35000c50040b556f3
Code:
# This is the Promox Host
root@cloud1:~# dd if=/dev/zero of=/dev/sdd bs=1M count=1024 status=progress oflag=direct
926941184 bytes (927 MB, 884 MiB) copied, 7 s, 132 MB/s
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.9993 s, 134 MB/s

#This is in the VM
cloud-user@rancher-wkr-1:~$ sudo dd if=/dev/zero of=/dev/sdb bs=1M count=1024 status=progress oflag=direct
1061158912 bytes (1.1 GB, 1012 MiB) copied, 16 s, 66.3 MB/s
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 16.206 s, 66.3 MB/s
 
Last edited:
Hi,

dd whit dev/zero is no proper way to benchmark disks.
I would recommend you to use fio instead.
Please report back if you see the same results.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!