High memeory usage ZFS or NFS?

[potetpro]

Testing RAID10 on 4 disks using ZFS, and then sharing this via NFS to the Proxmox nodes.
But suddenly 50/96GB was used, and i can't figure out where it went.
After a reboot it keeps building when i do work on a VM using this network share as disk.

Yes it is used, not just buffer/cache, this is with 0 VM's running.
root@proxmox4:/etc/zfs# free -m
total used free shared buff/cache available
Mem: 96546 50128 46078 72 339 45727
Swap: 8191 0 8191

ps -e -o pid,vsz,comm= | sort -n -k 2

5.03906 MB -bash
5.1875 MB /lib/systemd/systemd-udevd
5.41797 MB /usr/sbin/sshd
6.36328 MB pickup
6.40625 MB sshd:
6.49219 MB qmgr
7.08984 MB /sbin/init
9.625 MB /lib/systemd/systemd-journald
17.1953 MB /sbin/dmeventd
47.8633 MB spiceproxy
50.918 MB spiceproxy
57.8359 MB /usr/bin/pmxcfs
69.8047 MB /usr/sbin/corosync
77.9219 MB pve-firewall
78.0859 MB pvestatd
86.6797 MB pve-ha-lrm
86.8203 MB pve-ha-crm
106.914 MB pvedaemon
108.324 MB pveproxy
116.332 MB pvedaemon
116.594 MB pvedaemon
116.609 MB pvedaemon
118.73 MB pveproxy
118.754 MB pveproxy
118.766 MB pveproxy


top don't show anything else using that much memory

root@proxmox4:/etc/zfs# cat /proc/meminfo
MemTotal: 98863200 kB
MemFree: 47369276 kB
MemAvailable: 47010852 kB
Buffers: 21388 kB
Cached: 226692 kB
SwapCached: 0 kB
Active: 766748 kB
Inactive: 105284 kB
Active(anon): 659112 kB
Inactive(anon): 52680 kB
Active(file): 107636 kB
Inactive(file): 52604 kB
Unevictable: 67372 kB
Mlocked: 67412 kB
SwapTotal: 8388604 kB
SwapFree: 8388604 kB
Dirty: 752 kB
Writeback: 0 kB
AnonPages: 691368 kB
Mapped: 111460 kB
Shmem: 74348 kB
Slab: 590464 kB
SReclaimable: 100744 kB
SUnreclaim: 489720 kB
KernelStack: 9152 kB
PageTables: 13380 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 57820204 kB
Committed_AS: 3182824 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 0 kB
VmallocChunk: 0 kB
HardwareCorrupted: 0 kB
AnonHugePages: 0 kB
ShmemHugePages: 0 kB
ShmemPmdMapped: 0 kB
CmaTotal: 0 kB
CmaFree: 0 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 240124 kB
DirectMap2M: 11161600 kB
DirectMap1G: 91226112 kB
root@proxmox4:/etc/zfs#

root@proxmox4:/etc/zfs# vmstat -s
98863200 K total memory
51266624 K used memory
767084 K active memory
105336 K inactive memory
47247400 K free memory
21580 K buffer memory
327596 K swap cache
8388604 K total swap
0 K used swap
8388604 K free swap
79677 non-nice user cpu ticks
0 nice user cpu ticks
198708 system cpu ticks
5543896 idle cpu ticks
355938 IO-wait cpu ticks
0 IRQ cpu ticks
19650 softirq cpu ticks
0 stolen cpu ticks
53956896 pages paged in
282700713 pages paged out
0 pages swapped in
0 pages swapped out
36549972 interrupts
180846797 CPU context switches
1556280281 boot time
73338 forks

 

[potetpro]

Ok, so according to the wiki page: https://pve.proxmox.com/wiki/ZFS_on_Linux
ZFS uses 50% of the memory available. And this can be changed in the file:
/etc/modprobe.d/zfs.conf

Which does not exist.
But i found a zfs.conf file at:
/lib/modules-load.d/zfs.conf

 

[wolfgang]

Hi

But i found a zfs.conf file at:

If this file does not exists you have to create it.

 

[LnxBil]

Yes it is used, not just buffer/cache, this is with 0 VM's running.

ZFS memory usage (ARC) will be shown as used, not cached. Why is it bad your memory is used? If the memory is needed elsewhere, the ARC is freed.

 

[potetpro]

Anyone?
adding to /lib/modules-load.d/zfs.conf

Hi

If this file does not exists you have to create it.

Yeah, it worked now.

ZFS memory usage (ARC) will be shown as used, not cached. Why is it bad your memory is used? If the memory is needed elsewhere, the ARC is freed.

When it used 50GB, and i tried to start a VM that was assigned 64GB, it would not start.