Tips/Good practice for large RAID/ZFS pools

[halley80]

Hello,
I currently have a Dell R540 blade with 256GB RAM and 6x 7.2TB HDD 7.2K.
(2 x Xeon Bronze 3106@1.7Ghz)

I have created two 7.2TB mirroring pools with a spare.
This in order to host two virtual NAS (1.5TB / 1TB).

This node consumes 160GB of RAM.

The idea is to reduce RAM consumption.
Do you think another ZFS RAID/Pool type could reduce RAM consumption?

Thank you for your answers,
Yours sincerely

 

[UdoB]

The default ram usage for Zfs ARC (Adaptive Replacement Cache = read-cache) is: 50 percent of available memory.

Another pool (you mean vdev!) type does not change this.

You can learn the current configuration with ~# arc_summary. Do this before modifying anything.

I do limit that amount this way: ~# cat /etc/modprobe.d/zfs.conf

#
# 05.2020 zuegeln des ZFS-Standardverhaltens mit 50 % ARC = 32 GiB
#
#  4 GiB = 4294967296
#  8 GiB = 8589934592
# 16 GiB = 17179869184
# 24 BiB = 25769803776
# 32 GiB = 34359738368
#
# Nicht vergessen:
# update-initramfs -u

options zfs zfs_arc_min=8589934592
options zfs zfs_arc_max=17179869184

Best regards

 

[halley80]

Thank you very much for your response.

However, the file /etc/modprobe.d/zfs.conf does not exist on my node.

I have seen the concept of ZFS ARC on https://pve.proxmox.com/wiki/ZFS_on_Linux and my values are (arc_summary) :

zfs_arc_min 0
zfs_arc_max 0

Do you need to create the file and indicate :
options zfs zfs_arc_min=8589934592
options zfs zfs_arc_max=17179869184

Then update-initramfs -u ?

Thanking you for your answers,
with kind regards.

 

[UdoB]

Yes, create the file with your text editor of choice and write those two lines into it. Usually those values are 2^x but thats not a requirement.

Yes, update-initramfs -u ist necessary at least if your root filesystem is also zfs. The initramfs supplies values to the kernel _before_ the "normal" boot phase. It will only get used on reboot, so... that is what you need to do.

Best regards

 

[halley80]

Thank you, now I understand what I wanted to implement for several months.

In my case, a 256GB SSD hard drive can be beneficial for the ZFS-ARC ?
If yes, a "zpool add -f rpool cache /dev/xxx" is sufficient ?

Best regards

 

[UdoB]

Well, the syntax seems right, but "-f" (force) should not be necessary. Unfortunately I can not tell if you will actually gain as much performance win as you might think. As so often: "it depends..."

Sorry, no good answer this time...

 

[halley80]

Perfect. Thank you for your valuable advice.
I will implement all this little by little.

Yours sincerely.
PS : are you interested in a feedback ?

 

[UdoB]

Sure. Add it to this forum entry - any new user in the future may be interested