very bad HDD write performance

[ezh]

I've build a test server. AMD Ryzen 2600, 8 Gb RAM, 3 x 2 Tb HDD.
Latest Proxmoz. ZFS filesystem.

All VM vere very-very slow. Tested HDD under Windows VM. Got about 50 kb/s (or smth about) of writing speed.
Tested Ubuntu VM. About the same.

Finally tested Proxmox itself:
root@proxmox-test:~# dd if=/dev/zero of=/tmp/test2.img bs=512 count=1000 oflag=dsync
1000+0 records in
1000+0 records out
512000 bytes (512 kB, 500 KiB) copied, 21.5373 s, 23.8 kB/s

What could be the problem? Should I reinstall Proxmox from scratch?
Should I install natavly some Windows and see the numbers?

Tnx.

 

[micush]

Try EXT4 for the underlying filesystem with RAW VM disk images. Your numbers will improve.

# dd if=/dev/zero of=/mnt/pve/local-datastore/test2.img bs=512 count=1000 oflag=dsync
1000+0 records in
1000+0 records out
512000 bytes (512 kB, 500 KiB) copied, 0.735923 s, 696 kB/s

# dd if=/dev/zero of=/mnt/pve/local-datastore/test2.img bs=1M count=100 oflag=dsync
100+0 records in
100+0 records out
104857600 bytes (105 MB, 100 MiB) copied, 0.967688 s, 108 MB/s

 

[dietmar]

What kind of HD do you use, and what ZFS RAID level?

 

[ezh]

3 x 7200 rpm

 

[micush]

For ZFS, a lot of RAM is required (the more the better as by default it takes half), and for spinning disks it is recommended (basically required) to have an enterprise class SSD to offload ZIL/L2ARC operations to increase performance.

Your performance should improve after that, but it will not most likely be faster than a simpler filesystem setup such as EXT4.

 

[ezh]

I need a RAID 5 setup. PVE does not support mdraid. Should I buy a PCI card then? Smth like https://www.ox.ee/en/product/114245 ?

 

[tom]

If you want good performance, do not use raid5 and no cheap hardware in general.

 

[ezh]

This is my personal test server only. It should perform well, but it will not have any load at all.

 

[frenchfries]

md works fine just cant use it for boot. Maybe go a little ssd for boot and do your raid 5 via software