[SOLVED] ZFS on NVME for Containers

[Proxygen]

I have 32GB of RAM and 12 Xeon cores. All the containers I will launch will be the same version of Ubuntu. On top of that, some of the containers will be running identical applications with nearly identical data.

Because of deduplication, I think I could squeeze a lot more data onto the local NVME drive where the containers will reside if I use ZFS.

A) Is this assumption correct?

B) I can't afford to slow down the NVME disk by more than 10% just to get more "space" out of it. Is there a disk IO performance hit when there is plenty of CPU and I can add as much RAM as needed?

C) Is the ZFS cache by default turned on and using the same drive and the ZFS pool, so that I do not need to worry about cache config?

PS: This disk is the only ZFS in the system. The rest is either standard ext4 (root) and LVM (/mnt/backups)

 

[Stoiko Ivanov]

please be careful with enabling deduplication on ZFS - it needs quite a bit of memory (and 32G are not too much to start with for ZFS)
see https://www.solaris-cookbook.eu/lin...ation-useful-data-much-memory-need-zfs-dedup/
and https://www.reddit.com/r/zfs/comments/4ex708/does_it_make_sense_to_use_both_dedup_and/

C) Is the ZFS cache by default turned on and using the same drive and the ZFS pool, so that I do not need to worry about cache config?

ZFS caching system is quite complex:
* it has the in_memory ARC (which by default can use up to 50% of your RAM)
* it can optionally be supplanted by a l2arc (a.k.a. cache) device (usually something much faster than the regular disks) - keep in mind that l2 arc also needs to keep its metadata in memory (ARC)

hope this explains the situation a bit!

 

[guletz]

Hi,

Deduplication is very triky. If your data is mostly unchanged AND the deduplication DB can fit in RAM, then you are ok.


Good Luck!

 

[Proxygen]

I have about 4 instances of 99% identical 25GB and two instances of 99% identical 20 GB. On both the avg file size is 2 to 4MB.

Common wisdom says to expect 1GB per TB of deduplicated disk.

I will bump it up to 96GB once I try to deploy all 6 datasets on ZFS, but I found ZFS using 6 to 8GB with just two 99% identical instances of 20GB...