[SOLVED] Memory usage keeps on growing

[titouan]

Hello,

I have been having some vm crashes for a few weeks now when under high load.
After investigating I realized it was the host who was killing the kvm process because of OOM.

After I found this was the issue I reviewed more closely the memory usage on this host and I can't quite figure out what is going on.

Here is the configuration I am in :
- Host with 32Gb of RAM
- A total of 3 VM with 12Gb of RAM allocated as such : 8Gb, 2Gb, and 2Gb

The free command is reporting a memory usage of ~24Gb :

              total        used        free      shared  buff/cache   available
Mem:           31Gi        24Gi       6,2Gi        51Mi       208Mi       6,0Gi

But when I launch top and sort by memory allocation I can only find the reported memory usage for VM (~10Gb) :

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
 9612 root      20   0 9507500   8,1g   7252 S   6,2  25,7 401:25.11 kvm
16703 root      20   0 2846956 861692   7220 S   0,0   2,6  21:21.81 kvm
 8277 root      20   0 2794220 438624   7212 S   6,2   1,3  20:10.99 kvm
 1772 root      20   0  378016 148752  11036 S   0,0   0,5   0:05.54 pvedaemon worke
11044 www-data  20   0  363252 132288   9520 S   0,0   0,4   0:00.97 pveproxy worker
27045 www-data  20   0  359628 128520   9456 S   0,0   0,4   0:01.42 pveproxy worker
13001 www-data  20   0  359292 128016   9124 S   0,0   0,4   0:00.58 pveproxy worker
 1773 root      20   0  358016 126812   9196 S   0,0   0,4   0:03.21 pvedaemon worke
 1774 root      20   0  357908 126448   8892 S   0,0   0,4   0:01.52 pvedaemon worke
 1869 www-data  20   0  350776 120344   2736 S   0,0   0,4   0:00.23 pveproxy
 1771 root      20   0  349280 118968   2852 S   0,0   0,4   0:00.23 pvedaemon
 1867 root      20   0  330524  95972   7916 S   0,0   0,3   0:01.95 pve-ha-crm
 1877 root      20   0  330180  95300   7580 S   0,0   0,3   0:03.28 pve-ha-lrm
 1748 root      20   0  297148  84316   8792 S   0,0   0,3   0:53.44 pvestatd
 1746 root      20   0  298808  84004   6824 S   0,0   0,3   0:40.19 pve-firewall
........

Can someone help me figure out why does my host uses that much memory that my VM can't run anymore ?

 

[Alwin]

Are you using ZFS? If so, then 50% of the available memory will be used by the ZFS ARC cache.

 

[titouan]

Are you using ZFS? If so, then 50% of the available memory will be used by the ZFS ARC cache.

Yes I am!
I actually do not know the first thing about ZFS, my hosting company configured it for me, is there a way to reduce this memory usage or is it a drawback of ZFS and nothing can be done about it ?

 

[titouan]

Never-mind I found the info I needed on the wiki : https://pve.proxmox.com/wiki/ZFS_on_Linux#_limit_zfs_memory_usage
Thanks @Alwin!