Does ZFS arc adjust with more VMs?

[Dowd]

I know in the past historically it has used 50% of RAM for ZFS arc and I believe (if I am not mistaken) this amount has been reduced to 10% on new installs (I saw a comment to that affect, but I dont have any new installs to confirm).

Lets say you have 32GB of RAM. 16GB goes to ZFS arc, leaving 16 GB for VMs.

1. What happens if I need more than 16GB? Does it pull from the ZFS arc?
2. Does the system run out of memory if I use more than 16GB?

The other part to this question is that if ZFS arc does in deed change dynamically, e.g., vm uses more than 16GB RAM, is there a way to enforce a minimum amount so that VMs cannot start or use the minimum memory allocated to ZFS arc? I am aware you can edit zfs.conf in modprobe.d with min and max values, but my question is if you have 16GB minimum for min in that value, would that prevent a VM over 16GB from starting because there is no free memory?

 

[sterzy]

this amount has been reduced to 10% on new installs (I saw a comment to that affect, but I dont have any new installs to confirm).

If you set up a new instance with its boot drive on ZFS such a limit will be set, yes.

What happens if I need more than 16GB? Does it pull from the ZFS arc?

It will try to, but ZFS ARC can sometimes be slow with freeing up space in such a scenario. So it will depend on the exact circumstances, but your VM may not start, yes.

Does the system run out of memory if I use more than 16GB?

That should not happen, ARC should free up memory if the system actually runs out. However, it is more likely that your VM won't start in my experience.

but my question is if you have 16GB minimum for min in that value, would that prevent a VM over 16GB from starting because there is no free memory?

Yes, ZFS won't free up more memory in that case. So it won't shrink below these 16 GB [1].

[1]: https://openzfs.github.io/openzfs-docs/Performance and Tuning/Module Parameters.html#zfs-arc-min

 

[Dowd]

If you set up a new instance with its boot drive on ZFS such a limit will be set, yes.


It will try to, but ZFS ARC can sometimes be slow with freeing up space in such a scenario. So it will depend on the exact circumstances, but your VM may not start, yes.


That should not happen, ARC should free up memory if the system actually runs out. However, it is more likely that your VM won't start in my experience.


Yes, ZFS won't free up more memory in that case. So it won't shrink below these 16 GB [1].

[1]: https://openzfs.github.io/openzfs-docs/Performance and Tuning/Module Parameters.html#zfs-arc-min

Interesting, so is there any point to turning min and max for ZFS then since it will pull from the ZFS arc? or should I then really be setting a min that adheres to the 4GB + 1GB/1TB rule? That way no VM will hijack the memory allocated to ZFS arc, thereby reducing performance?