unusable command line on zfs backend storage

[samlm0]

Request Increase zfs operations timeout, unusable on high load environment

Unable import disk / resize disk / list volume on zfs base storage using command line, always return timeout

 

[Stoiko Ivanov]

Do you see anything in the logs indicating why the zfs operations take so long?

as explained in:
https://forum.proxmox.com/threads/most-zfs-operations-leads-to-an-timeout.26476/#post-132918
5 seconds should be enough

simply increasing/changing it is not possible, since quite a few services run those commands in the background, and if each zfs invocations needs more than 5 seconds this would lead to problems.

 

[samlm0]

Do you see anything in the logs indicating why the zfs operations take so long?

as explained in:
https://forum.proxmox.com/threads/most-zfs-operations-leads-to-an-timeout.26476/#post-132918
5 seconds should be enough

simply increasing/changing it is not possible, since quite a few services run those commands in the background, and if each zfs invocations needs more than 5 seconds this would lead to problems.

We are developing a VPS control panel based on proxmox, and it has a few issues with the ZFS storage

Some VM running some heavy io task (like use Swap frequently), then the host ZFS command will longer to respond.

If Proxmox reports a timeout error when importing a disk, the ZFS import will not be cancelled at this point. It will continue importing disks

If the zfs pool has too many fragments with heavy io task, the zfs list command will take few minutes to show the result

Summary of the timeout problems we encountered:

1. Create VM via command line with a cloud-init drive report timeout error (if re-run the create command will report cloud-init drive is exists)
2. Import disk via the command line report timeout error, but the disk still importing on the background.
3. Resizing disk via the command line report timeout error, but the disk did not resize

Host setup:

root@node1:~# zfs list | wc -l
322
root@node1:~# zpool list -v
NAME        SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
zfspool    9.05T   529G  8.53T        -         -    28%     5%  2.52x    ONLINE  -
  mirror   9.05T   529G  8.53T        -         -    28%  5.71%      -  ONLINE
    sdb3       -      -      -        -         -      -      -      -  ONLINE
    sdc3       -      -      -        -         -      -      -      -  ONLINE
logs           -      -      -        -         -      -      -      -  -
  nvme1n1   928G  50.8M   928G        -         -     0%  0.00%      -  ONLINE
cache          -      -      -        -         -      -      -      -  -
  nvme0n1   931G   295G   637G        -         -     0%  31.6%      -  ONLINE