ZFS ARC Ram question

[nethfel]

Hi all,

I'm considering on a fresh install of 3.4 to install to a RAID 1 array of relatively small disks (depending on what I have on hand, somewhere between 80 and 500 Gig / drive). How much RAM should really be set aside for this for ZFS? I don't plan on hosting any VMs on this particular array, it's just to give some survivability should one of the drives fail, would I really need to set aside 50% of the total RAM for this operation? If so I might just explore setting up mdadm instead, it would just be more straight forward (and potentially easier to maintain) with zfs I think.

 

[Nemesiz]

My home server have 48GB of ram. I set up 10gb for ARC. Works normally for me.
If you dont need file checksum use mdadm. ZFS keeps cache of block and checksums inside ARC. So ZFS speed +- depend on ARC.

p.s. I have 2 pools - 3 disk raid1 and 2 disk mirror. None of them are build for enterprise so ARC boost it.

 

[nethfel]

Ugh, I was afraid someone was going to respond like that. I have to work with what I have available unfortunately (budget is really tight), these boxes only have 16GB of RAM (which for the amount of VMs that will run on them, is fine). I can easily set aside 2GB for ZFS; I might be able to squeeze 4, but anything more than that (and really even @ 4) would make it difficult to impossible to migrate VMs from one of the 3 fresh nodes for maintenance... Guess I'll have to acquire some more RAM, I think I can bump them all up to 20GB without too much added expense - I could probably start now with the 16 I have and set aside 4GB and if I have to do maintenance, shut down non-essential VMs and migrate the essential ones until I can get 20+ into each server..

 

[mir]

IMHO anything less than 8GB dedicated for ZFS will give you a bad experience. ZFS is build for enterprise systems and does not perform well when resources is scarce. In particular scarce RAM can quickly lead to a measurable performance drop.

 

[kobuki]

I use ZFS (ZoL 0.6.3) on some 4 gig nodes on metal and in VM too. They work just fine for light to light-moderate loads. Do not expect high performance, though. ZFS is happily eating any and all RAM it finds.

EDIT: see also http://pve.proxmox.com/wiki/ZFS#Limit_ZFS_memory_usage

 

[nethfel]

Well, that was what I was wondering - how much of a load would there be with it just being the boot drive in a raid 1 of the hypervisor? Granted, I know there will be data coming in/out from the cluster sync and such, but other than that there would be no VMs actually hosted within the ZFS file system.

 

[kobuki]

Oh, sorry, I missed that point. I think with only the OS sitting on a ZFS mirror you should have no problems. But then it's still comparatively very memory-intensive and if you don't need or use ZFS features then I see no point. I'd just stick with MDRAID.