Out of memory

[Luc@RGB]

Hi,

The memory usage is always slowly growing, and if i don't reboot the host, it kill the biggest process, that is a virtual machine, of course.

I don't use ZFS, but anyway, I've changed the zfs_arc_max at 2GB in the /etc/modprobe.d/zfs.conf, but without any change.

the swappiness is also to 0. But i don't think is a swap problem as kvm don't use it. I've checked with:

(echo "COMM PID SWAP"; for file in /proc/*/status ; do awk '/^Pid|VmSwap|Name/{printf $2 " " $3}END{ print ""}' $file; done | grep kB | grep -wv "0 kB" | sort -k 3 -n -r)

pveproxy 1811 117568 kB
pvedaemon 1788 116280 kB
pveproxy worker24789 92484 kB
pveproxy worker30226 91608 kB
pveproxy worker25593 91248 kB
pve-ha-crm 1809 86724 kB
pvedaemon worke13175 83264 kB
pvedaemon worke14949 83000 kB
pvedaemon worke14335 81300 kB
pve-ha-lrm 1819 64252 kB
pve-firewall 1756 54532 kB
spiceproxy 1817 47832 kB
pvestatd 1757 47708 kB
spiceproxy work20060 45408 kB
glances 1356 34256 kB
systemd-journal 832 4760 kB
snmpd 1469 4076 kB
beamium 1357 1248 kB
noderig 1633 1220 kB
master 1729 672 kB
systemd-udevd 856 276 kB

I think the processes that are always growing are the kvms:

top - 13:03:29 up 20 days,  7:38,  1 user,  load average: 2.28, 2.00, 2.09
Tasks: 445 total,   1 running, 442 sleeping,   2 stopped,   0 zombie
%Cpu(s):  4.1 us,  0.4 sy,  0.0 ni, 95.3 id,  0.0 wa,  0.0 hi,  0.2 si,  0.0 st
MiB Mem :  95209.5 total,    620.9 free,  89078.7 used,   5509.8 buff/cache
MiB Swap:   2047.0 total,   1252.7 free,    794.3 used.   5134.6 avail Mem

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
 2198 root      20   0   37.5g  36.4g   7480 S 126.9  39.1  19460:19 kvm
 1852 root      20   0   72.1g  48.6g   5584 S  28.2  52.3  15362:12 kvm

I understand that a kvm use more than the memory assigned to the kvm, but the fact it is always growing, no...
It's my first cluster with Proxmox, the others ones with VMware doesn't have this issue...

HOST:96 GB
KVM100: 44 GB with ballooning (Linux)
KVM101: 36 GB with ballooning (Windows, with drivers)
-> 16 GB free for proxmox without ZFS and without Ceph, and it makes OOM issues...




What can i do?

Thanks

 

[abma]

linux uses all free memory as cache: the easiest would be to slightly reduce the memory for KVM100.

to verify you can manually clear the cache, on the host and/or inside the linux vm:

echo 3 > /proc/sys/vm/drop_caches

 

[Luc@RGB]

Hi,
Just to says it's resolved, i have done 2 changes:
- Active payment susbscription the 18-03
- actualize to PVE 6.3-3 + reboot the 18-03

And now the mem dosen't grow anymore:


I don't know which of this 2 changes resolved the problem...