ZFS freeze VM during high I/O

Telymax

Member
Feb 11, 2019
6
0
21
50
Hi group
This is my configuration:
PVE 5.3
Ryzen 2700
16GB RAM
SSD 250GB (ZFS with default parameters)
VM: Windows 10

When I do a copy inside the VM (for e.g. 5 Gigabyte from a NAS), it becomes almost unusable until the process is finished.

If I use LVM or Ceph on the same hardware, everything works fine.
Does ZFS need any additional settings?

Thanks
Massimo
 
Hi,

If you write default parameters what does this means?
Did you install the ZFS over the installer, on the GUI, self without parameters?
see https://pve.proxmox.com/wiki/ZFS_on_Linux

Anyway, ensure that the zvol has 4k blocksize for Windows VMs, because Windows ignores the default 8k.

Do you have enough memory for ZFS? How much is allocated by running guests?
 
Hi Wolfgang
Yes I installed ZFS over the installer on the GUI without changing anything (I'm a new user and I'm evaluating Proxmox for my virtualization needs).
I already seen the link above, but I don't understand how to set block size to 4k.
I have 16GB of RAM and VM has only 2GB, so I think there is enough RAM for the system.
Could you help me?
Thanks again
 
Last edited:
Searching into the GUI, I found where to set this value (Datacenter > Storage > Add ZFS).
Now, after recreating the VM, Windows 10 seems to be more responsive during heavy load I/O.
I will continue my tests
Thanks for your support
Massimo
 
Last edited:
second round :|

Unfortunately "4k Block Size option" seems to work only with ZFS pools added on other disks beyond the first one (rpool/data).
I tried also to destroy and recreate "/rpool/data" (with 4k option), without success ... the VM located on it, keeps freezing during a heavy I/O.
If I move the VM from /rpool/data to another Zpool (on a second SSD with 4k option), everything works well.
Any other suggestion?

to be continued ...
 
You can set the block size only on the creation of the zvol.
If you what later change this you have to use "move disk" on the hardware tab.

The default created zfspool storage has 8K as blocksize you can change this with

Code:
pvesm set local-zfs  --blocksize 4k
 
ZFS is always slower and it needs twice as many writes compared to other FSs supported by PM, but will always be consistent and will not require journal of fsck and also has a nice caching system: ARC (check out arc_summary).

If you do not need replication or mirroing just use LVM, it will be faster.
If you use ZFS make sure you use a enterprise grade ssd disk. Made all the difference for me.
 
I've already noticed that LVM has good performance compared to other solutions (like VMFS), but I tried to use ZFS exactly for scheduled replication :(
At this point ... I think that I've to migrate my tests to Ceph, discarding ZFS.
(anyway thank you for having shared your experience)

Thanks also to Wolfgang, I tried command line "pvesm set local-zfs --blocksize 4k", but seems that does the same as the GUI
 
Ceph will be more complex (debatable) to maintain and due to increased complexity there is more possibility for errors.
Also it will require a few more nodes.

I would suggest you just use ZFS, with a bit more ram for ARC and with an intel DC ssd. Your life will be easier. :)
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!