Does disk storage update need ram update?

[Martini2]

Hello,

i want to add another storage pool to the proxmox server which now has 32GB ram and ~20TB of diskspace using zfs. The ram consumption is ~24GB with max peeks at 29GB.
I will be adding 36TB of diskspace on zfs and assume that the system may become unusable due to lack of ram. Do i need to update the ram as well and if so how much ram do i need?

 

[Dunuin]

ZFS by default will use up to 50% of your hosts RAM. So adding more storage won't increase your RAM usage. But the more storage you got the more RAM you should allow ZFS to use.

Best you look at the output of the arc_summary command. As long as hit rates are high (maybe 90 or 95%) and the cache sizes (metadata and dnode) got enough free space you should be fine. But as always...the more RAM you allow your ARC to use, the faster your pools will be.

 

[Martini2]

this is the shortened output of arc_summary

 ZFS Subsystem Report                            Tue Feb 01 17:16:54 2022
Linux 5.4.162-1-pve                                           2.0.7-pve1
Machine: home (x86_64)                                      2.0.7-pve1

ARC status:                                                      HEALTHY
        Memory throttle count:                                         0

ARC size (current):                                    11.1 %    1.7 GiB
        Target size (adaptive):                        12.8 %    2.0 GiB
        Min size (hard limit):                         6.2 %  1000.3 MiB
        Max size (high water):                           16:1   15.6 GiB
        Most Frequently Used (MFU) cache size:         36.5 %  555.8 MiB
        Most Recently Used (MRU) cache size:           63.5 %  965.5 MiB
        Metadata cache size (hard limit):              75.0 %   11.7 GiB
        Metadata cache size (current):                  3.6 %  437.4 MiB
        Dnode cache size (hard limit):                 10.0 %    1.2 GiB
        Dnode cache size (current):                     2.5 %   29.6 MiB

ARC hash breakdown:
        Elements max:                                             140.2k
        Elements current:                             100.0 %     140.2k
        Collisions:                                                 7.3k
        Chain max:                                                     3
        Chains:                                                     2.3k

ARC misc:
        Deleted:                                                     611
        Mutex misses:                                                  0
        Eviction skips:                                             2.1k

ARC total accesses (hits + misses):                                 2.7M
        Cache hit ratio:                               94.8 %       2.6M
        Cache miss ratio:                               5.2 %     140.8k
        Actual hit ratio (MFU + MRU hits):             90.7 %       2.5M
        Data demand efficiency:                        83.0 %     537.0k
        Data prefetch efficiency:                      25.6 %      19.0k

Cache hits by cache type:
        Most frequently used (MFU):                    73.5 %       1.9M
        Most recently used (MRU):                      22.1 %     570.8k
        Most frequently used (MFU) ghost:               0.0 %          0
        Most recently used (MRU) ghost:                 0.0 %          0
        Anonymously used:                               4.4 %     112.2k

Cache hits by data type:
        Demand data:                                   17.3 %     445.7k
        Demand prefetch data:                           0.2 %       4.9k
        Demand metadata:                               78.4 %       2.0M
        Demand prefetch metadata:                       4.2 %     107.6k

Cache misses by data type:
        Demand data:                                   64.9 %      91.3k
        Demand prefetch data:                          10.0 %      14.1k
        Demand metadata:                               10.5 %      14.8k
        Demand prefetch metadata:                      14.6 %      20.6k

DMU prefetch efficiency:                                           88.2k
        Hit ratio:                                     28.3 %      25.0k
        Miss ratio:                                    71.7 %      63.2k

L2ARC not detected, skipping section
...
VDEV cache disabled, skipping section

ZIL committed transactions:                                       476.9k
        Commit requests:                                            5.6k
        Flushes to stable storage:                                  5.6k
        Transactions to SLOG storage pool:            0 Bytes          0
        Transactions to non-SLOG storage pool:      949.3 MiB      12.7k

does this look OK to add another 36TB as zfs storage? Or will it be better to add the disks as storage with ext4 as filesystem?

 

[Dunuin]

Right now your ARC is using 1.7 GiB and cache utilization and hit rates look fine. So I guess you could try it without upgrading RAM first if you can make sure that ZFS has atleast 6GB RAM available.
If it isn't fast enough you can always upgrade your RAM later.