ZFS freeze VM during high I/O

Discussion in 'Proxmox VE: Installation and configuration' started by Telymax, Feb 11, 2019.

  1. Telymax

    Telymax New Member

    Joined:
    Feb 11, 2019
    Messages:
    6
    Likes Received:
    0
    Hi group
    This is my configuration:
    PVE 5.3
    Ryzen 2700
    16GB RAM
    SSD 250GB (ZFS with default parameters)
    VM: Windows 10

    When I do a copy inside the VM (for e.g. 5 Gigabyte from a NAS), it becomes almost unusable until the process is finished.

    If I use LVM or Ceph on the same hardware, everything works fine.
    Does ZFS need any additional settings?

    Thanks
    Massimo
     
  2. wolfgang

    wolfgang Proxmox Staff Member
    Staff Member

    Joined:
    Oct 1, 2014
    Messages:
    4,257
    Likes Received:
    269
    Hi,

    If you write default parameters what does this means?
    Did you install the ZFS over the installer, on the GUI, self without parameters?
    see https://pve.proxmox.com/wiki/ZFS_on_Linux

    Anyway, ensure that the zvol has 4k blocksize for Windows VMs, because Windows ignores the default 8k.

    Do you have enough memory for ZFS? How much is allocated by running guests?
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  3. Telymax

    Telymax New Member

    Joined:
    Feb 11, 2019
    Messages:
    6
    Likes Received:
    0
    Hi Wolfgang
    Yes I installed ZFS over the installer on the GUI without changing anything (I'm a new user and I'm evaluating Proxmox for my virtualization needs).
    I already seen the link above, but I don't understand how to set block size to 4k.
    I have 16GB of RAM and VM has only 2GB, so I think there is enough RAM for the system.
    Could you help me?
    Thanks again
     
    #3 Telymax, Feb 12, 2019
    Last edited: Feb 12, 2019
  4. Telymax

    Telymax New Member

    Joined:
    Feb 11, 2019
    Messages:
    6
    Likes Received:
    0
    Searching into the GUI, I found where to set this value (Datacenter > Storage > Add ZFS).
    Now, after recreating the VM, Windows 10 seems to be more responsive during heavy load I/O.
    I will continue my tests
    Thanks for your support
    Massimo
     
    #4 Telymax, Feb 12, 2019
    Last edited: Feb 12, 2019
  5. Telymax

    Telymax New Member

    Joined:
    Feb 11, 2019
    Messages:
    6
    Likes Received:
    0
    second round :|

    Unfortunately "4k Block Size option" seems to work only with ZFS pools added on other disks beyond the first one (rpool/data).
    I tried also to destroy and recreate "/rpool/data" (with 4k option), without success ... the VM located on it, keeps freezing during a heavy I/O.
    If I move the VM from /rpool/data to another Zpool (on a second SSD with 4k option), everything works well.
    Any other suggestion?

    to be continued ...
     
  6. wolfgang

    wolfgang Proxmox Staff Member
    Staff Member

    Joined:
    Oct 1, 2014
    Messages:
    4,257
    Likes Received:
    269
    You can set the block size only on the creation of the zvol.
    If you what later change this you have to use "move disk" on the hardware tab.

    The default created zfspool storage has 8K as blocksize you can change this with

    Code:
    pvesm set local-zfs  --blocksize 4k
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  7. mailinglists

    mailinglists Active Member

    Joined:
    Mar 14, 2012
    Messages:
    315
    Likes Received:
    29
    ZFS is always slower and it needs twice as many writes compared to other FSs supported by PM, but will always be consistent and will not require journal of fsck and also has a nice caching system: ARC (check out arc_summary).

    If you do not need replication or mirroing just use LVM, it will be faster.
    If you use ZFS make sure you use a enterprise grade ssd disk. Made all the difference for me.
     
  8. Telymax

    Telymax New Member

    Joined:
    Feb 11, 2019
    Messages:
    6
    Likes Received:
    0
    I've already noticed that LVM has good performance compared to other solutions (like VMFS), but I tried to use ZFS exactly for scheduled replication :(
    At this point ... I think that I've to migrate my tests to Ceph, discarding ZFS.
    (anyway thank you for having shared your experience)

    Thanks also to Wolfgang, I tried command line "pvesm set local-zfs --blocksize 4k", but seems that does the same as the GUI
     
  9. mailinglists

    mailinglists Active Member

    Joined:
    Mar 14, 2012
    Messages:
    315
    Likes Received:
    29
    Ceph will be more complex (debatable) to maintain and due to increased complexity there is more possibility for errors.
    Also it will require a few more nodes.

    I would suggest you just use ZFS, with a bit more ram for ARC and with an intel DC ssd. Your life will be easier. :)
     
  10. Telymax

    Telymax New Member

    Joined:
    Feb 11, 2019
    Messages:
    6
    Likes Received:
    0
    I know ...I know ...I'll keep in mind your suggestions! :)
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice