[SOLVED] What is the better solution - Swapping on host, or swapping in the VM?

[fireon]

Hello,

i would be interested in how your swap used. In LXC it is clear. If you have Swap on the host and set swap on container then lxc swaps to the host. If no swap is given to the host, the container can't swap.

But what is with KVM? There are three possibilities.

1. Swap on Host and Swap on VM. So when the VM goes out of Memory it's swapping inside the VM, during out of Memory of other VMs and the same time the host is out of memory, swapping continues on the host

2. Swap on Host but not on KVM. If the VM goes out of Memory it will be killed by oom killer. So most more memory on VM is necessary, an overbook of the host could be necessary. So only the host is swapping.

3. The host has now swap, KVM has swap. So the host can't swap. Swapping on out of Memory only in the VM. I have this method on all us ZFS-Machines.

So what of these three steps are used by others? What do you think is the best solution for you, and why?

 

[LnxBil]

One also needs to consider the speed of swap in/out in a clustered environment. If you swap on your host, then you'll most certainly be bound by your local disk bus, which is normally two disks in RAID1 and swapping on your guest is on your storage over some kind of network infrastructure. So you have varying latency and throughput for both kinds of swap.

I implemented a similar, yet different method like your number2:
For most of my VMs I only use zram-based swap and disabled swap-to-disk completely. For my hosts I use zram-based swap as first layer and local disk-based swap as second layer, yet the later is normally never used. I have to say that my VMs normally swap very little because they have plenty of RAM.

 

[Rhinox]

RAM is cheap, so try to avoid swapping, that's the best you can do. But if you have to, then I'd recommend to do swapping in VM only, not on PVE-host. Why?

If VM is swapping, it is aware of it, and before it comes to it, VM-os will try to reduce memory-footprint (depending on its swappiness-setting) and will not try to use swap for disk-cache or i/o-buffers (only real RAM).

If PVE-host is swapping, it has very bad performance impact on all VMs running and PVE-host itself.

But as I said, the best is not to use swap at all. I do not over-commit resources on my PVE-host, so I'm not using swap on host for quite some time, without any problem. And most of my VMs have no swap at all...

 

[fireon]

Thanks for the answers. So i on the side... disable swap completly and buy enough memory. Zram sounds good, i haven'd tested this before.