Why my container is swapping?

[decibel83]

Hi,
I have a container which has very slow performance, and trying to debug this i realised that it's swapping so much even if it has a lot of RAM memory available:


These are the status graphs of the hypervisor:






As you can see the host does not seems to be overloaded.

Another strange thing is that the container is seeing 64 Gb of swap instead of 8:



Swappiness on the host node is 0:

root@node3 ~ # cat /proc/sys/vm/swappiness
0

Could you help me please?
Thanks!

 

[RokaKen]

You may be running into the issue where systemd does not respect the kernel sysctl for swappiness (https://github.com/systemd/systemd/issues/9276) and also discussed on this forum (https://forum.proxmox.com/threads/swappiness-question.42295/)

 

[decibel83]

Thanks!
Will moving to KVM solve this problem?

 

[RokaKen]

Will moving to KVM solve this problem?

No. You'll need to implement the workaround discussed in the links I provided.

 

[decibel83]

No. You'll need to implement the workaround discussed in the links I provided.

Thanks.
Just to be sure: are you referring to the following workaround?

[Service]
ExecStartPost=/bin/find /sys/fs/cgroup -mindepth 2 -maxdepth 2 -name memory.swappiness -exec /bin/sh -c 'sysctl -n vm.swappiness > "{}"' \;
ExecStartPost=/bin/find /sys/fs/cgroup -mindepth 3 -maxdepth 3 -name memory.swappiness -exec /bin/sh -c 'sysctl -n vm.swappiness > "{}"' \;
ExecStartPost=/bin/find /sys/fs/cgroup -mindepth 4 -maxdepth 4 -name memory.swappiness -exec /bin/sh -c 'sysctl -n vm.swappiness > "{}"' \;
ExecStartPost=/bin/find /sys/fs/cgroup -mindepth 5 -maxdepth 5 -name memory.swappiness -exec /bin/sh -c 'sysctl -n vm.swappiness > "{}"' \;
ExecStartPost=/bin/find /sys/fs/cgroup -name memory.swappiness -exec /bin/sh -c 'sysctl -n vm.swappiness > "{}"' \;

Is it correct that I should apply this on the host node for Proxmox itself and containers and on VMs?

Thanks!

 

[RokaKen]

Well, that may work for you. In my case, I implemented the following on all nodes:

cat /etc/sysctl.d/local.conf
vm.overcommit_memory=1
vm.swappiness=0

cat /etc/tmpfiles.d/mem-swap.conf
#Type Path                                              Mode UID  GID  Age Argument
w   /sys/fs/cgroup/memory/system.slice/memory.swappiness -    -    -    -   0

After rebooting each node in sequence, it resolved the problem for me. YMMV.

EDIT: systemd-tmpfiles requires a ".conf" extension or the file is ignored.