Low results with pveperf (the reason for bad HDD performance?)

[mcflym]

Here are my results of pveperf:

CPU BOGOMIPS: 19152.36
REGEX/SECOND: 962716
HD SIZE: 7.14 GB (/dev/mapper/pve-root)
BUFFERED READS: 15.87 MB/sec
AVERAGE SEEK TIME: 0.57 ms
FSYNCS/SECOND: 95.95
DNS EXT: 113.60 ms
DNS INT: 1.18 ms

I have some performance problems with my harddisks which i passthroughed to my vm via virtio. It is very slow (around 20 or 30 MB/s) and it stucks serveral times... Maybe its caused by the bad performance of the HDD (buffered reads) of my Host?

Its a SSD SATA III on a SATA II Controller.
CPU: XEON 3220 Quad Core
HDD: 5x 2TB SATA II, 1x 4TB SATA II on SATA II and SATA III Controlers
RAM: 4GB

 

[mir]

What are your mount options and file system for the disk?

 

[mcflym]

Here is my fstab of the Host:

/dev/pve/root / ext3 errors=remount-ro 0 1
/dev/pve/data /var/lib/vz ext3 defaults 0 1
UUID=xxxxxxx /boot ext3 defaults 0 1
/dev/pve/swap none swap sw 0 0
proc /proc proc defaults 0 0
/dev/sdb1 /vm_system ext4 defaults 0 0
/dev/sdi1 /vm_backup ext4 defaults 0 0

and this is the conf of the VM:

bootdisk: virtio0
cores: 1
cpu: host
ide2: local:iso/gparted-live-0.16.1-1-i486.iso,media=cdrom,size=133M
keyboard: de
memory: 3072
name: omv
net0: virtio=XX:XX:XX:XX:XX:XX,bridge=vmbr0
onboot: 1
ostype: l26
sockets: 1
startup: order=1,up=25
virtio0: vm_system:100/vm-100-disk-1.raw,format=raw,size=15G
virtio1: /dev/sdc,backup=no
virtio2: /dev/sdd,backup=no
virtio3: /dev/sde,backup=no
virtio4: /dev/sdf,backup=no
virtio5: /dev/sdg,backup=no
virtio6: /dev/sdh,backup=no

and fstab of the VM (its an openmediavault VM):

# / was on /dev/vda1 during installation
UUID=ca0b8e52-0175-47f9-89e3-4edef590ba0a / ext4 errors=remount-ro 0 1
# swap was on /dev/vda5 during installation
UUID=e1823198-986b-43fb-aa14-d0cdc017af89 none swap sw 0 0
/dev/scd0 /media/cdrom0 udf,iso9660 user,noauto 0 0
# >>> [openmediavault]
UUID=198b3802-b6e5-48b5-9916-82580aec2802 /media/198b3802-b6e5-48b5-9916-82580aec2802 ext4 defaults,nofail,acl,user_xattr,noexec,usrjquota=aquota.user,grpjquota=aquota.group,jqf$
UUID=0fd13fc0-f3e2-4b22-a99c-d866b0a07be5 /media/0fd13fc0-f3e2-4b22-a99c-d866b0a07be5 ext4 defaults,nofail,acl,user_xattr,noexec,usrjquota=aquota.user,grpjquota=aquota.group,jqf$
UUID=597c8dd4-c06f-42e0-bb43-705f664622c9 /media/597c8dd4-c06f-42e0-bb43-705f664622c9 ext4 defaults,nofail,acl,user_xattr,noexec,usrjquota=aquota.user,grpjquota=aquota.group,jqf$
UUID=bdf16d7a-084e-4435-9d11-8e381a41bdea /media/bdf16d7a-084e-4435-9d11-8e381a41bdea ext4 defaults,nofail,acl,user_xattr,noexec,usrjquota=aquota.user,grpjquota=aquota.group,jqf$
UUID=3b398216-9709-4cb7-b1bc-d0da59cef4ed /media/3b398216-9709-4cb7-b1bc-d0da59cef4ed ext4 defaults,nofail,acl,user_xattr,noexec,usrjquota=aquota.user,grpjquota=aquota.group,jqf$
UUID=a410c641-a8bc-4939-9d20-9ab72007e9b3 /media/a410c641-a8bc-4939-9d20-9ab72007e9b3 ext4 defaults,nofail,acl,user_xattr,noexec,usrjquota=aquota.user,grpjquota=aquota.group,jqf$
/media/198b3802-b6e5-48b5-9916-82580aec2802// /home/ftp/Media_4 none bind 0 0
/media/198b3802-b6e5-48b5-9916-82580aec2802// /home/ftp/Media_share_5 none bind 0 0
/media/0fd13fc0-f3e2-4b22-a99c-d866b0a07be5// /home/ftp/Media_share_4 none bind 0 0
/media/0fd13fc0-f3e2-4b22-a99c-d866b0a07be5// /home/ftp/Media_3 none bind 0 0
/media/597c8dd4-c06f-42e0-bb43-705f664622c9// /home/ftp/Media_2 none bind 0 0
/media/597c8dd4-c06f-42e0-bb43-705f664622c9// /home/ftp/Media_share_3 none bind 0 0
/media/bdf16d7a-084e-4435-9d11-8e381a41bdea// /home/ftp/Data none bind 0 0
/media/3b398216-9709-4cb7-b1bc-d0da59cef4ed// /home/ftp/Media_share_2 none bind 0 0
/media/3b398216-9709-4cb7-b1bc-d0da59cef4ed// /home/ftp/Media_1 none bind 0 0
# <<< [openmediavault]

 

[mir]

You have two separate problems:
1) Bad SSD performance of your pve root file system
2) Bad performance of your pass-through disks to a VM

1) It is hard to tell why you get those slow numbers.
- What does hdparm -tT /dev/sda tell?
- What does this tell:
- dd if=/dev/zero of=/tmp/test oflag=fdatasync bs=1M count=1024
- dd if=/tmp/test of=/dev/null bs=1M
Try these mount options for /dev/pve/root and /dev/pve/data: relatime,data=ordered,barrier=0,discard or relatime,data=ordered,barrier=0

2) Try to mount your disk pass-throughs as SATA or SCSI and see if this raises the performance. Also try different cache like write-through or writeback

 

[mcflym]

You have two separate problems:
1) Bad SSD performance of your pve root file system
2) Bad performance of your pass-through disks to a VM

1) It is hard to tell why you get those slow numbers.
- What does hdparm -tT /dev/sda tell?
- What does this tell:
- dd if=/dev/zero of=/tmp/test oflag=fdatasync bs=1M count=1024
- dd if=/tmp/test of=/dev/null bs=1M
Try these mount options for /dev/pve/root and /dev/pve/data: relatime,data=ordered,barrier=0,discard or relatime,data=ordered,barrier=0

2) Try to mount your disk pass-throughs as SATA or SCSI and see if this raises the performance. Also try different cache like write-through or writeback

Thanks for your help so far!

here are my results:

1.)
hdparm -tT /dev/sda :
/dev/sda:
Timing cached reads: 7564 MB in 2.00 seconds = 3785.07 MB/sec
Timing buffered disk reads: 180 MB in 3.02 seconds = 59.61 MB/sec

dd doesnt work? "invalid output flag: `fdatasync'"

2.)
I tried nearly all options to mount the devices. There were nearly all the same problems like, stucking and slow speed

BIOS:
SATA AHCI is enabled. Some suggestions to other options there?

I changed my hardware but the strange thing is, that i had the same issues (not so many stucks and not that slow speed, but not really good --> 60-70 MB/s) with the hardware (Celeron Ivy Bridge 2,6 GHZ, 16 MB RAM).



What's the reason for this? Under Windows Server 2012 i had nearly fullspeed with the other hardware, so it only can be some settings in proxmox. Or are there maybe driver issues with SATA controllers?

Thanks in advance for your reply!

Edit:

Made the same pveperf test today, now with that result:

pveperf
CPU BOGOMIPS: 19154.08
REGEX/SECOND: 934697
HD SIZE: 7.14 GB (/dev/mapper/pve-root)
BUFFERED READS: 69.20 MB/sec
AVERAGE SEEK TIME: 0.27 ms
FSYNCS/SECOND: 99.32
DNS EXT: 91.32 ms
DNS INT: 1.13 ms (fritz.box)

Are these values ok, or is it still slow?

the vm-attached HDDs are unchanged (slow)

i changed nothing :?

 

[mcflym]

really crazy things here... now i had 150 MB/s with my VM machine... i now understand nothing :/

 

[mir]

cat /proc/mounts and paste it here.

 

[mcflym]

cat /proc/mounts and paste it here.

Ok...

sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
udev /dev devtmpfs rw,relatime,size=10240k,nr_inodes=502936,mode=755 0 0
devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0
tmpfs /run tmpfs rw,nosuid,noexec,relatime,size=404272k,mode=755 0 0
/dev/mapper/pve-root / ext3 rw,relatime,errors=remount-ro,user_xattr,acl,barrier=0,data=ordered 0 0
tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0
tmpfs /run/shm tmpfs rw,nosuid,nodev,noexec,relatime,size=808540k 0 0
/dev/mapper/pve-data /var/lib/vz ext3 rw,relatime,errors=continue,user_xattr,acl,barrier=0,data=ordered 0 0
/dev/sda1 /boot ext3 rw,relatime,errors=continue,user_xattr,acl,barrier=0,data=ordered 0 0
/dev/sdb1 /vm_system ext4 rw,relatime,barrier=1,data=ordered 0 0
/dev/sdi1 /vm_backup ext4 rw,relatime,barrier=1,data=ordered 0 0
rpc_pipefs /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0
fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0
/dev/fuse /etc/pve fuse rw,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions,allow_other 0 0
beancounter /proc/vz/beancounter cgroup rw,relatime,blkio,name=beancounter 0 0
container /proc/vz/container cgroup rw,relatime,freezer,devices,name=container 0 0
fairsched /proc/vz/fairsched cgroup rw,relatime,cpuacct,cpu,cpuset,name=fairsched 0 0

 

[mir]

comparing to yesterday your mount options is changed:
yesterday: /dev/pve/data ext3 defaults
today: /dev/pve/data ext3 rw,relatime,errors=continue,user_xattr,acl,barrier=0,data=ordered

yesterday: /dev/pve/root ext3 errors=remount-ro
today: /dev/pve/root ext3 rw,relatime,errors=remount-ro,user_xattr,acl,barrier=0,data=ordered

So the mount options did make a difference!

 

[mcflym]

the interesting point is, that i have changed nothing - anyway... everything is fine and i hope it works now (and not only temporary)

THANK YOU anyway!!!