Sacrifice VM

[StellarStallion]

Hi,

my Proxmox ran out of RAM and killed a VM, which leads to some Questions:

Can I define which VM to kill first?
Would a HA Cluster have moved the VM to another Node?
Can I prevent this behavior, in terms of don't kill VMs, but let the Node die?


Thanks!

 

[LnxBil]

Can I define which VM to kill first?

AFAIK, not via the gui, no. IIRC, you could try to tweak the oom parameter oom_score_adj for the QEMU processes.

Would a HA Cluster have moved the VM to another Node?

No, OOM is a very hard and atomic event. You need to plan ahead.

Can I prevent this behavior, in terms of don't kill VMs, but let the Node die?

If the node dies, all VMs are also killed - or I don't get your point.

 

[UdoB]

my Proxmox ran out of RAM

Yeah..., you should not overcommit RAM. It just does not work well. In all of my (Homelab-) systems RAM is always the most critical resource :-(

If you need to relax that problem a little bit you may look at zram; I prefere this over a static swap-file:

~# apt show zram-tools

Description: utilities for working with zram
 zram is a Linux kernel module that allows you to set up compressed
 filesystems in RAM.
 .
 zram-tools uses this module to set up compressed swap space.
 This is useful on systems with low memory or servers
 running a large amount of services with data that's easily swappable
 but that you may wish to swap back fast without sacrificing disk
 bandwidth.

The recommendation is a clearly prioritized list:

1. max out the technically possible RAM
2. if #1 is already done --> reduce your (RAM-) load to a sane level, e.g. give each VM the amount of RAM it needs to have - not more!
3. only if #1 and #2 is at its limits: think about involving zram

 

[leesteken]

The OOM killer always shoots down the worse offender, which it typically the biggest VM that you care about the most. This is not Proxmox specific and there are Linux guides on the internet on setting OOM-priorities for specific processes, like the kvm process that runs a(nother) VM. This will not really solve your problem if the (big) VM keeps taking memory.
Make sure not to over-commit on memory, remember that Proxmox wants 2GiB (and maybe ZFS also wants memory) and lookup ballooning and KSM settings in the Proxmox manual: https://pve.proxmox.com/pve-docs/pve-admin-guide.html