[SOLVED] Swappiness question

[gallew]

Update 2: previous statement about updating initramfs is false. System is still swapping.
However, i have located "problem", it seems that proxmox processes, have their own swappiness parameter set (using cgroups), it means, those processes (pvedaemon, pve-ha-cr, pvestatd, pve-firewall, pve-ha-lr, etc) will not honor global swappiness parameter, and this means that those proxmox programs will use swap memory.

root@cd03 ~ # cat /proc/sys/vm/swappiness
0
root@cd03 ~ # cat /sys/fs/cgroup/memory/system.slice/pvedaemon.service/memory.swappiness
60
root@cd03 ~ # smem -s swap -r | head -n 12
  PID User     Command                         Swap      USS      PSS      RSS 
 3381 root     pvedaemo                       91084      204      324     2568 
 3405 root     pve-ha-cr                      71712     8592     9814    15012 
 3385 root     pvedaemon worke                67760    11200    18626    39040 
 3386 root     pvedaemon worke                66992    10452    17841    38084 
 3384 root     pvedaemon worke                63756    12020    19657    40292 
 3234 root     /usr/bin/pmxcfs                50496    17680    24925    36280 
 3357 root     pvestat                        40952    29428    30783    37096 
 3356 root     pve-firewal                    35244    36428    37701    43332 
 3454 root     pve-ha-lr                      23544    56384    57645    63136 
  383 root     /lib/systemd/systemd-journa      324     9856    21684    36348 
29602 root     /usr/bin/kvm -id 120 -chard        0  2098104  2099509  2109448

If globally is set vm.swappiness=0 then other processes (running virtual KVM machines) will not use swap memory.

 

[gallew]

Update 3:
since some proxmox processes (pveproxy for example) gets restarted (in case of ssl cert update for example), it will inherit cgroups swappiness parameter. So in order for proxmox processes not to use swap you'll have to also change cgroup swappiness, otherwise, after process restart, they will swap again.

echo 0 >/sys/fs/cgroup/memory/system.slice/memory.swappiness

Also, i did some testing, seems that system.slice swappiness is honored in Debian kernels (sysctl.conf vm.swappiness = 0 setting), but not in proxmox kernels.
(if set in Debian Stretch default kernel sysctl.conf "vm.swappiness = 0", after reboot /sys/fs/cgroup/memory/system.slice/memory.swappiness is 0, but in proxmox kernel, after reboot, value is 60)
So this is purely proxmox kernel problem.
Please fix!

 

[gallew]

So, until fixed do:

1) create file /etc/rc.local (if it does not exist) with content:

#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.

# fix for proxmox kernel swappiness problem, yes, it must be executed twice
for i in `/usr/bin/find /sys /var/lib 2>/dev/null | /bin/grep slice | /bin/grep swappiness` ; do /bin/echo 0 >"${i}" ; done
for i in `/usr/bin/find /sys /var/lib 2>/dev/null | /bin/grep slice | /bin/grep swappiness` ; do /bin/echo 0 >"${i}" ; done

exit 0

2) make file executable

chmod +x /etc/rc.local

The reason for executing twice is, while first find is running, some processes are still starting, first run sets system.slice memory.swappiness to 0 (since it finds it), but until it does, some process may have been started wit swappiness 60, and they may have been started after find has been run, time window is small, but it happens. (in my case it was postfix). So, since first run sets system.slice memory.swappiness to 0, all new processes will inherit that, and second run takes care of those processes which are left with 60, so after second run all processes have swappiness 0.
Been running for couple of weeks now with this setup, and swap is in all machines 0B

 

[fireon]

@gallew
Very thanks, no i understand why us settings didn't works on new versions. This is strange and really bad and should be fixed shortly.

Please fill out an Bugreport at https://bugzilla.proxmox.com/

 

[bendev]

I had the same issue and the script helped to fix it.
I run version 5.3-11, so this means this is still not solved ?

Thanks!

 

[lexz]

Hello everyone,
I'm very curious this is solved or we still have to use the script?

Hope someone can give us some information.
Thanks.

 

[TechLineX]

Solved??

 

[Whatever]

Not yet for me. On several installation I'm still getting swap usage with enough free memory and vm.swappiness=0

 

[isbear]

I think, this is caused by systemd issue #9276 (google it, cannot post link since I'm noob on this forum).
The second workaround from the discussion (with tmpfiles drop-in) has worked for me.

 

[Flummi-74]

Hello to the forum and sorry for reviving this solved thread.

I was wondering about the effects of the swappiness parameter in my home-cluster nodes too. Through searching I found this interesting and actual articel about the swappiness and the myths about it:

https://www.howtogeek.com/449691/what-is-swapiness-on-linux-and-how-to-change-it/

Hope this helps

Grettings

Flummi

 

[xcdr]

Hi, systemd issue #9276 is labelled as cant-fix.

I searched for recommended default swappiness settings for systemd and I found nothing.

Is there better workaround than rc.local?

 

[fireon]

Disable swap completely and the world is fine

 

[Alwin]

Why swap maybe not so bad after all.
https://chrisdown.name/2018/01/02/in-defence-of-swap.html

 

[proxmoxniggaz]

what is the command to enter to change swappiness in Proxmox NOvnc?