How much RAM for ZFS ARC + VMs on Proxmox?

[speedbird]

Hello,
I'm currently close to building my new Proxmox rig and since ZFS is a first time for me, I was wondering about the need for RAM in my setup.

I will run a 4x480GB SSD setup in RAID10 with ZFS. System memory is 64GB ECC RAM.

Now, I will run 3 VMs for a start, needing 24GB, 8GB and 8GB of RAM respectively.

Proxmox wiki says, it's a good idea to restrict RAM usage of ARC to prevent the host from being exhausted. The page says:

options zfs zfs_arc_max=8589934592

to limit usage to 8GB of System RAM.

Question now is, how much does ZFS actually need to not run into OOM situations, begin swapping or something the like.

I really don't need any unexpected behavior on this machine. How will taking snapshots / backups etc be affected by available ARC memory? Will ZFS purge "old" entries to make room for new ones or will the pool simply fail at some point because there's not enough system memory?

Just to clarify, I do not plan on using Dedup! Only compression is going to be active.

Thanks guys!

 

[guletz]

Hi,

If you reserv 8 Gb Ram for zfs I guess it will be no problem. I do not see any OOM situation because of zfs even with only 1 Gb Ram for zfs. Also you can use arc min like 4 Gb and arc max like 8Gb. And for zvol is better to cache only metadata if you do not want to waste your ram (at zvol level and at the guest VM level).

 

[speedbird]

Hm what do you mean with Zvol caching? Haven't made any kind of setting changes towards that...

However I now tried with 16GB for ARC but when I run Atto Benchmark on a windows 7 vm with 64K size, after the bench is complete, I can see the RAM fill up in no time and the whole system crashes. Only a hard reset can bring it back then. I don't know why that happed. On my test system there's 36GB of RAM, 8GB being fed to windows and 16GB max to ARC. still it crashed the whole system. Strange. Any ideas?

 

[wolfgang]

Hi,

Do you have a separat zfs pool or do you use the rpool?

 

[speedbird]

Do you have a separat zfs pool or do you use the rpool?

Hi Wolfgang,
well, I use the rpool that is created with the installation. Since I can only have 4 drives in the server and want to have RAID10, this seems to be the only option, isn't it?

Thanks for your input.