ZFS scrub too fast and impacting VM's

[cbr]

Hi all,

We have a storage cluster running with NVME drives using ZFS to have a RaidZ2 pool.

The pool is 84TB and while it is nice to see 84TB scrubbing finished in 39 minutes while reaching speeds up to 2.1GB p/sec, i would like to slow it down a bit or at least doing something to not let VM's crash into a kernel panic due to the fact that virtual disks are not responding.
Not all VM's were impacted, just 2 of the 60 but i suspect that those 2 were doing some R/W at that time and lost the drive.

Is ths possible? Or at least something else except disabling the scrub schedule?

 

[Kingneutron]

I have some very old info from 2017 for this, but you should probably look into the module parm tweaks:

https://openzfs.github.io/openzfs-d...ule Parameters.html#zfs-vdev-scrub-min-active

The tweaks I had were probably for 0.7 or 0.8
zfs_resilver_delay
zfs_top_maxinflight
zfs_resilver_min_time_ms

Since you're running a cluster tho, could you not migrate some of the more critical VMs to another host while the scrub is running, or have separate pools?

 

[cbr]

Well to be honest i already did find some tunables including:

zfs_resilver_delay
zfs_top_maxinflight

These were the most promising but they are removed from the codebase i think, they are not available in /sys/module/zfs/parameters/
Any other suggestions ?

As for the moving of critical vm's, we don't always know what vm's will have IO load since we have customers run on it so that won't be an option for us.

 

[Kingneutron]

> Any other suggestions ?

Search the link I provided for "scrub"

 

[cbr]

Someone from the Proxmox team care to chip in?
This is a thing on all ZFS installs and i was hoping that there was a known official solution...