KVM kills SSD performance

[tincboy]

I've just installed Proxmox on my new server which has a RAID 10 containing 4 ADATA 256 GB SSD drives,
I've tried to test out write speed on my new server which wrote by speed of 538 MB/s, but running the same test on a KVM VM on top of the same disk, show 1/4 performance
I'm using VIRTIO for disk and I'm in debt if there's any kind of configuration which can fix this issue
Anyone familiar with improving IO performance of VMs by SSD drives?

 

[mir]

What filesystem and mount option to the filesystem?

If filesystem is ext4 use barrier=0 and avoid discard as this will make a sync for every write.
instead add this to /etc/cron.daily

$ cat /etc/cron.daily/fstrim
#!/bin/sh


PATH=/bin:/sbin:/usr/bin:/usr/sbin


ionice -n7 fstrim -v /


ionice -n7 fstrim -v /var/lib/vz

 

[tincboy]

No, I'm using ext3, which one is better for SSD performance?

 

[giner]

More information is needed. What kind of RAID is used? VM OS? VM drive configuration?

 

[tincboy]

I'm using CentOS 6 x64 as guest OS, and the VM has 16GB memory with 16 cores of Xeon CPU,
I guess if I could make the VM to have direct IO access to underlying storage device the issue can be fixed, isn't it?
Is it possible in proxmox to directy connect the device to VM?

 

[spirit]

I'm using CentOS 6 x64 as guest OS, and the VM has 16GB memory with 16 cores of Xeon CPU,
I guess if I could make the VM to have direct IO access to underlying storage device the issue can be fixed, isn't it?
Is it possible in proxmox to directy connect the device to VM?

technicaly, it's possible to use new x-data-plane option in kvm command line

-device virtio-blk-pci,drive=drive-virtio0,id=virtio0,.......,x-data-plane=on .

(it's not supported yet in proxmox gui).

benchmark show around 1millions iops.
but some feature won't work anymore (live backup,move disk, disk io throttling,...)

 

[giner]

I'm using CentOS 6 x64 as guest OS, and the VM has 16GB memory with 16 cores of Xeon CPU,
I guess if I could make the VM to have direct IO access to underlying storage device the issue can be fixed, isn't it?
Is it possible in proxmox to directy connect the device to VM?

What kind of RAID is used? Is this hardware/software RAID?

 

[tincboy]

It's Supermicro RAID 10, I don't guess it's something wrong with RAID controller. because in the host OS, Performance is fine.

 

[giner]

It's Supermicro RAID 10, I don't guess it's something wrong with RAID controller. because in the host OS, Performance is fine.

Is this hardware raid? Model?

 

[giner]

Also you didn't mention guest file system and how did you test performance.

 

[tincboy]

Guest is using ext3 too, and I'm using the same dd command to perform the test
It's hardware raid and it's model is: "Supermicro AOC-USAS2LP-H8iR"

 

[giner]

What cache mode is used for VM drive? Default is cache=none.

Try on both (host and VM):
dd if=/dev/zero of=/some/dir/testfile bs=1M count=1000
dd if=/dev/zero of=/some/dir/testfile bs=1M count=1000 oflag=direct

Also try on VM (need to create a spare virtual drive):
dd if=/dev/zero of=/dev/vdX bs=1M count=1000
dd if=/dev/zero of=/dev/vdX bs=1M count=1000 oflag=direct

 

[tincboy]

technicaly, it's possible to use new x-data-plane option in kvm command line

-device virtio-blk-pci,drive=drive-virtio0,id=virtio0,.......,x-data-plane=on .

(it's not supported yet in proxmox gui).

benchmark show around 1millions iops.
but some feature won't work anymore (live backup,move disk, disk io throttling,...)

I was not able to use this switch, would you please let me know complete sample of it's usage with Proxmox 3.2 ?

 

[tincboy]

-device virtio-blk-pci,drive=drive-virtio0,id=virtio0,.......[/I][/COLOR][COLOR=#545454][I][FONT=arial],[/FONT][/I][/COLOR][COLOR=#545454][I][FONT=arial][B]x[/B][/FONT][/I][/COLOR][COLOR=#545454][I][FONT=arial]-[/FONT][/I][/COLOR][COLOR=#545454][I][FONT=arial][B]data-plane[/B][/FONT][/I][/COLOR][COLOR=#545454][I][FONT=arial]=on[/FONT][/I][/COLOR][COLOR=#545454][I][FONT=arial] .

[/FONT]

I was not able to use this switch, would you please let me know complete sample of it's usage with Proxmox 3.2 ?​

​

 

[linum]

You maybe need to reactivate your LVM volume with:

lvchange /dev/lvm-ssd-01/vm-210-disk-1 -a y

My (working) /etC/pve/qemu-server/210.conf looks like:

args: -drive file=/dev/lvm-ssd-01/vm-210-disk-1,if=none,id=drive-virtio1,aio=native,cache=none -device virtio-blk-pci,drive=drive-virtio1,id=virtio1,bus=pci.0,addr=0xb,scsi=off,config-wce=off,x-data-plane=on

 

[tincboy]

You maybe need to reactivate your LVM volume with:

lvchange /dev/lvm-ssd-01/vm-210-disk-1 -a y

My (working) /etC/pve/qemu-server/210.conf looks like:

args: -drive file=/dev/lvm-ssd-01/vm-210-disk-1,if=none,id=drive-virtio1,aio=native,cache=none -device virtio-blk-pci,drive=drive-virtio1,id=virtio1,bus=pci.0,addr=0xb,scsi=off,config-wce=off,x-data-plane=on

It helped so much,
Would you please let me know what cache method is better for this kind of attach device?