Very high SWAP usage on half of the nodes

[TripKostas]

Hi there.

We are facing an issue lately on 4 out of 7 nodes of our cluster. The swap usage is always on 99% although the RAM is not used up.

More specifically, 400GB ram is installed on one of the hyper visors and the usage is always on 50%, and 8GB Swap always with 99% usage.

On other nodes with virtually the same configuration and specifications the SWAP usage is ~0.5% - 2%.

Any ideas to solve this issue? I am always available to provide any further information needed.

Thank you in advance.

 

[PaulVM]

May be you can get some ideas from: https://forum.proxmox.com/threads/swap-usage.33062/

Regards, P.

 

[TripKostas]

Hi Paul and thank you for the fast response.

Unfortunately the post did not help much. I have no LXC containers, only VMs. On the nodes that have 99% swap, most of the VMs happen to have high swap usage as well.

Trying the swap off and on trick did not help on the VM side and I doubt it will help on the hypervisor as well. You mentioned on the other post that swappiness set to 0 did actually help. Is that truly the case? There are a lot of VMs on the nodes and rebooting nodes is extremely time consuming, so I would like to have more information regarding this behaviour.


All nodes have the following at the moment

Kernel Version Linux 4.4.59-1-pve #1 SMP PVE 4.4.59-87
PVE Manager Version pve-manager/4.4-13/7ea56165

 

[Rhinox]

With 400GB RAM at ~50% usage I wonder why you do not remove swap?

 

[PaulVM]

In my experience, update to the same kernel you are using, swappiness set to 0 AND a reboot solved.
only set the swappiness to 0 having already the last kernel didn't solve (only LXC and mixed LXC/VM configuration).
I temporary solved it with a "swapoff -a && swapon -a", until I could reboot the servers, but I think you already tested it ...
And it isn't a real solution ...
May be the Proxmox staff can give more options ...
Regards, P.

 

[guletz]

Let take an example. I have a Vm with 0.5 Gb Ram and 2 GB swap, on a node with 128 Gb. I run in the guest a hungry ram application, who will use a lot of Ram and it will go to swap. In this case you can solve this with:
- lower the applications who is Ram consuming
- lower the swapping (60 is by default)
- increase the Ram allocated to this Vm

You can use all of them at the same time, or any kind of combination of them. The easiest is the last one(increase the Ram for this Vm if you have any free available). In your case I would try the last one

 

[P...]

Hi There,

My first post , been working with Proxmox from version 2 and also did not see this swap usage eighter up untill Proxmox 5.
The best stable fix a have come up with is downgrade back to 4.4.
The kernel changed in 5 so it must have something to do with that.

Best Regards,

P