Weird memory usage issue on Proxmox 9.0

Dec 3, 2024
15
2
3
Before upgrading out main (enterprise-subscription) production cluster, we upgraded our (no-subscription) single-node Proxmox cluster to 9.0.

Now I'm noticing a strange issue with the RAM, where htop claims we're using around 20GB out of 251GB, but the Proxmox GUI and free claim both that the machine is using around 216GB-230GB (it fluctuates).

This is a fairly simple setup using just XFS for most of the storage and ZFS (two HDDs) for *big* storage (secondary mounts on containers/VMs).

What's going on here? How do I fix it? I'd like to solve this issue before we upgrade our big production clusters.
 
Please share
Bash:
top -co%MEM
free -h
arc_summary -s arc
And a picture of htop and node > Summary.
 
Last edited:
```
total used free shared buff/cache available
Mem: 251Gi 205Gi 32Gi 226Mi 17Gi 46Gi
Swap: 0B 0B 0B

------------------------------------------------------------------------
ZFS Subsystem Report Thu Aug 21 10:03:45 2025
Linux 6.14.8-2-pve 2.3.3-pve1
Machine: beherito (x86_64) 2.3.3-pve1

ARC status:
Total memory size: 251.4 GiB
Min target size: 3.1 % 7.9 GiB
Max target size: 99.6 % 250.4 GiB
Target size (adaptive): 75.4 % 188.8 GiB
Current size: 75.4 % 188.7 GiB
Free memory size: 44.8 GiB
Available memory size: 36.4 GiB

ARC structural breakdown (current size): 188.7 GiB
Compressed size: 96.4 % 181.8 GiB
Overhead size: 3.1 % 5.8 GiB
Bonus size: 0.1 % 106.7 MiB
Dnode size: 0.2 % 353.0 MiB
Dbuf size: 0.1 % 146.3 MiB
Header size: 0.2 % 443.3 MiB
L2 header size: 0.0 % 0 Bytes
ABD chunk waste size: < 0.1 % 1.7 MiB

ARC types breakdown (compressed + overhead): 187.7 GiB
Data size: 98.0 % 183.9 GiB
Metadata size: 2.0 % 3.7 GiB

ARC states breakdown (compressed + overhead): 187.7 GiB
Anonymous data size: 0.0 % 0 Bytes
Anonymous metadata size: < 0.1 % 485.5 KiB
MFU data target: 37.4 % 70.2 GiB
MFU data size: 4.6 % 8.7 GiB
MFU evictable data size: 3.5 % 6.6 GiB
MFU ghost data size: 0 Bytes
MFU metadata target: 12.5 % 23.4 GiB
MFU metadata size: 1.7 % 3.2 GiB
MFU evictable metadata size: 1.6 % 3.0 GiB
MFU ghost metadata size: 0 Bytes
MRU data target: 37.6 % 70.6 GiB
MRU data size: 93.4 % 175.3 GiB
MRU evictable data size: 89.1 % 167.1 GiB
MRU ghost data size: 6.2 GiB
MRU metadata target: 12.5 % 23.4 GiB
MRU metadata size: 0.3 % 503.7 MiB
MRU evictable metadata size: < 0.1 % 46.6 MiB
MRU ghost metadata size: 0 Bytes
Uncached data size: 0.0 % 0 Bytes
Uncached metadata size: 0.0 % 0 Bytes

ARC hash breakdown:
Elements: 1.9M
Collisions: 6.5M
Chain max: 4
Chains: 53.6k

ARC misc:
Memory throttles: 0
Memory direct reclaims: 42815
Memory indirect reclaims: 6006185
Deleted: 104.3M
Mutex misses: 8.3k
Eviction skips: 2.8k
Eviction skips due to L2 writes: 0
L2 cached evictions: 0 Bytes
L2 eligible evictions: 12.4 TiB
L2 eligible MFU evictions: < 0.1 % 96.0 KiB
L2 eligible MRU evictions: 100.0 % 12.4 TiB
L2 ineligible evictions: 53.8 MiB
```