disable RAM overprovisioning

[IamPepsiGuy]

Is it possible to disable RAM overprovisioning? I'm new to Proxmox. I created and started more VMs that I have physical RAM. I'm familiar with ballooning and the VMs have the ballooning driver installed in the guest, but the startup of the VMs happened too quickly so a VM that was running for a while ended up getting killed by OOM.

I would like to just never overprovision RAM -- but I'd like Proxmox to just fail to start a new VM if there is insufficient RAM to do so (e.g., having Proxmox calculate the max memory of all running VMs plus whatever overhead that Proxmox itself needs, and if there is less remaining then needed by the VM that is requested to start, inform the user that there is insufficient resources to start the VM).

Is this possible? To ensure that OOM never kills a running VM.

 

[datschlatscher]

Proxmox does not include such a feature as of yet.

(e.g., having Proxmox calculate the max memory of all running VMs plus whatever overhead that Proxmox itself needs, and if there is less remaining then needed by the VM that is requested to start, inform the user that there is insufficient resources to start the VM).

This is a little complicated due to features like memory ballooning. When Proxmox starts a VM, it does not have a sure way of telling how much RAM a VM is going to need. It will simply give the VM more memory as requested, as long as it is under the configured limit.

I'm afraid, for now, you will have to simply configure your VMs in such a way that they do not exceed your systems total memory to not invoke the OOM.

 

[IamPepsiGuy]

Just to make sure I'm not missing something, anytime I start a VM on a node (or before I migrate a VM to a node), I need to manually add up all the max memory of all running VMs on target node before I attempt operation?

I was hoping there was a way to turn off RAM overcommitting and ballooning and just allow the system to handling the math to prevent any OOM issues. I don't want my poor math to result in a kill to a critical VM.

 

[datschlatscher]

I was hoping there was a way to turn off RAM overcommitting and ballooning and just allow the system to handling the math to prevent any OOM issues. I don't want my poor math to result in a kill to a critical VM.

Well, if you don't want to rely on your own math skills you can, of course, write a script to help you with that.
Doing this in a universal way to include most use-cases is just not so trivial, though.
From what I can tell, this does not seem like a feature a lot of people request right now, however, if you want, you can open a feature request on our Bugtracker for this, where you and potentially also other users can further outline your use-case: https://bugzilla.proxmox.com/

I need to manually add up all the max memory of all running VMs on target node before I attempt operation?

Nonetheless, if you absolutely want to make sure your VMs do not get killed because of overcommitted RAM, you will have to do something like that.

 

[stibila]

You can configure overcommit handling by Linux kernel. To disable overcommiting memory, create file /etc/sysctl.d/disable_overcommit_mem.conf with content:

vm.overcommit_memory = 2
vm.overcommit_ratio = 100

Then run command sysctl --system

vm.overcommit_ratio is in percents. To leave a bit for OS overhead, you could set vm.overcommit_ratio to lower value, let's say 90.
You can also use vm.overcommit_kbytes instead to specify absolute value.

 

[VictorSTS]

That won't really help because when booting up a VM, QEMU/KVM does not reserve the whole memory configured in the VM.
With vm.overcommit_memory = 2, any VM that malloc()'s when you are over vm.overcommit_ratio will get a no memory error and potentially cause issues with the guest OS, it's applications or QEMU/KVM itself. IMHO this is worse than OOM killing one single VM due to pebkac

 

[stibila]

With vm.overcommit_memory = 2, any memory allocation is allowed only if there is enough actual resources available. In other words, If kernel cannot reserve whole memory that VM allocates, then it won't allow KVM process to start.

I dare you to try it and then report back. You should get:

kvm: cannot set up guest memory 'pc.ram': Cannot allocate memory
TASK ERROR: start failed: QEMU exited with code 1

 

[VictorSTS]

Oh! You right, overcommit applies to virtual memory, not resident memory! OOM killer acts when no more resident memory can be malloc(), so I wrongly assumed vm.overcommit would use resident memory too instead of virtual, even after re-reading kernel doc about it

 

[Ahanae]

I totally get where you're coming from with the OOM issue! While you can't fully disable RAM overprovisioning in Proxmox, you can definitely set limits for your VMs to help manage things better. I've had success by adjusting the memory allocation settings to match my physical RAM more closely. I also recommend keeping an eye on the resource monitor so you can see what’s happening before it gets out of hand. If you configure your cluster properly, it should alert you if there isn’t enough RAM to start a new VM, which can save you from those annoying OOM kills.