Memory ballooning doesn't reduce host RAM usage?

S

sbrocket

New Member
Jun 23, 2025
12
6
3
I'm attempting to understand something unexpected that I'm observing about memory ballooning's behavior and what that implies about its intended usage. I understand that memory ballooning is triggered based on the hypervisor host's RAM usage reaching a set threshold, with the default set to 80%. However, the host's RAM usage which is compared to that threshold includes the maximum RAM allocation of all running VMs...including the max for VMs with ballooning enabled, rather than current amount available to the VM (aka the max minus the current balloon size).

Doesn't that mean that the balloon inflating can never reduce the host's usage below the threshold? And then assuming the host memory pressure exceeded the threshold due to RAM allocated to VMs rather than other processes on the host, doesn't that mean that the VMs with ballooning enabled will always lose available memory until they reach their minimum? And that this ballooned memory will continue to be accounted in host RAM usage, even though it isn't actually available to or being used by anything?

If all that is true, then I don't understand what the intended usefulness of memory ballooning would be. In that scenario it isn't triggered by heavy transient RAM usage of one VM or another, only by the relatively fixed configured maximum for all running VMs. What kinds of scenarios is memory ballooning actually useful for, then? For example, is it only useful to accommodate for host RAM usage exceeding the threshold due to new VMs starting? But if so, I still don't understand why the balloon inflating and reducing available RAM to VMs with ballooning enabled does not then reduce host RAM usage. How else will host usage drop below the threshold...another VM stopping?

To give a contrived but concrete example for discussion purposes, consider a PVE host with 128GiB of RAM and the following VMs configured:
  • VM 100 - running, 64 GiB, ballooning enabled, minimum set to 32 GiB
  • VM 101 - running, 16 GiB, ballooning disabled
  • VM 102 - stopped, 32 GiB, ballooning disabled
Ignore other host RAM usage for this example. Before starting VM 102, VM 100 should have its max 64 GiB available to it. Starting VM 102 will then increase the host's RAM usage from 62.5% (80/128 GiB) to 87.5% (112/128GiB). which will inflate the memory balloon on VM 100. Based on the behavior I've described above and observed in my own usage, VM 100's available memory will then slowly reduce to 32 GiB and host usage will stay at 87.5%. It seems like the 32 GiB "stolen" from 100 by ballooning will go forever unused and unusable to anything else, yet still accounted for in host usage.

It seems like I must have some part of this wrong because that behavior seems pointless, but I can't identify what I have wrong. Could someone please find my error and explain? Thank you.
 
Last edited:
Once the threshold is reached, the host will ask the VMs to return memory to the host. The VMs need to be able to return that memory! That means, a way to communicate with the host via the Ballooning device, and free memory that can be returned.

The return of memory happens in chunks of ~100MiB of memory. Once the threshold is reached, you need to give it some time.

What are the configs of the VMs? Which guest OS? Any PCI passthrough?

Keep in mind, that the Ballooning device itself should be kept enabled, as it is the mechanism by which Proxmox VE queries the memory usage from the guests point of view.
If minimum memory and memory are the same value, no ballooning will happen.
 
  • Like
Reactions: chughes, Ernst T. and Johannes S
aaron said:
Once the threshold is reached, the host will ask the VMs to return memory to the host. The VMs need to be able to return that memory! That means, a way to communicate with the host via the Ballooning device, and free memory that can be returned.
Click to expand...

What I see once the threshold is exceeded is the available memory within my guests with ballooning enabled reducing slowly to their configured minimum, but host usage never reducing by that same amount.

Is that not expected? I’ve reproduced it a couple times now by stopping and starting VMs to go over and then drop below 80% usage. Once over 80% the balloon reduces available memory in the guests where it is enabled, but host usage remains unchanged and over 80% even though plenty of memory was reclaimed to drop it below that. top on the host continues to report the max configured for the guests for the kvm processes.

aaron said:
The return of memory happens in chunks of ~100MiB of memory. Once the threshold is reached, you need to give it some time.
Click to expand...

Yes I’ve observed this, that it takes time to reduce the available memory in the guests. That’s fine and expected.

aaron said:
What are the configs of the VMs? Which guest OS? Any PCI passthrough?

Keep in mind, that the Ballooning device itself should be kept enabled, as it is the mechanism by which Proxmox VE queries the memory usage from the guests point of view.
If minimum memory and memory are the same value, no ballooning will happen.
Click to expand...

The example I gave in the OP is a simplified representation of my actual config. I have 2 VMs with ballooning enabled, each with a set max of 32 GiB and a set min of 16 GiB. They are both modern Linux guests; one is TrueNAS, the other is Linux Mint.

Once I start enough other VMs so that my host usage exceeds 80% (out of 128 GiB), they both slowly reduce to 16 GiB available and stay there, but host usage remains unchanged and above 80%. The 2*16GiB recovered from both of them is 25% of the host max and so should be guaranteed to drop host usage below 80%.

Yes they do each use PCI passthrough; one for a GPU, the other for a HBA. Would that cause the behavior I see where they lose available memory but the host never recovers it?
 
Last edited:
from the docs :
https://pve.proxmox.com/wiki/Dynamic_Memory_Management#Requirements_for_Linux_VM
Note that if you are passing through a physical PCI(e) device or a Virtual Function I/O (VFIO) Mediated device (MDEV) such as a vGPU, then ballooning will not work since these devices are mapped to fixed memory addresses in the host and in the guest. If you do enable ballooning, the KVM process will not release the memory back to the host but the guest may report a reduced memory availability which can be manually recovered using the KVM monitor using the balloon command.
Click to expand...
 
  • Like
Reactions: Johannes S, aaron and sbrocket
aaron said:
... the host will ask the VMs to return memory to the host.
Click to expand...
I think "ask" is putting it too mildly. As I understand it: the memory is simply taken without negotiation or consideration, which can cause OoM-kills inside the VM (if the user sets the minimum memory too low or memory cannot be released quickly enough inside the VM, adding a swapfile inside the VM can help). Or did this behavior change recently?
 
  • Like
Reactions: Johannes S
I took a look at the code again, and AFAIU (which might be wrong), pvestatd initiates the Balloon changes here:
https://git.proxmox.com/?p=pve-mana...fd05ee3c9c5cd1e09d25fe9832eb3364;hb=HEAD#l296

The calculation regarding which VMs are told to give back memory or get memory returned is here: https://git.proxmox.com/?p=pve-mana...103d3ece368ed7edfa4f71e613aff4b334d80;hb=HEAD


Rough summarization: Once the threshold is reached, pvestatd calculates if memory needs to be reclaimed or returned (positive or negative $goal).
For each run (every ~10 seconds), this is done with a $maxchange of 100 MiBs.

Which guests are considered depends on:
  • balloon device enabled
  • lower balloon_min configured
  • enough free space in the VM to reclaim memory
The call issues to the Balloon driver / agent in the VM is this one: https://qemu-project.gitlab.io/qemu/interop/qemu-qmp-ref.html#command-QMP-machine.balloon
The note here is important as it states, that the driver/agent in the VM may not do what we want it to.

Since the pvestatd is running every ~10 seconds, it might not always react quickly enough if the memory pressure in a VM is getting too high, or if the memory pressure on the host is getting too high too quickly.

I personally am not the biggest fan of relying on these very dynamic memory saving features (KSM is another one), as situations can become quite unpredictable. The "easy" solution is unfortunately a very expensive one in 2026 with the ballooned up memory prices thanks to all the planned AI datacenters...
 
Last edited:
  • Like
Reactions: Johannes S, fba and leesteken
I also have found ballooning to be problematic, with many VMs experiencing out of memory situations unexpectedly, even with very generous minimums. A typical VM will consume all available memory for caching, but probably can't release it quickly enough when the balloon driver comes calling.

But still, balloon and KSM are critical features when things are going sideways and you need to squeeze in more VMs.

However, as stated, they are frequently too slow to keep PVE from impacting VMs, so swap is also critical for absorbing spikes in memory usage while waiting for PVE to begin KSM and balloon runs. I personally wouldn't set up a PVE host without a healthy amount of (redundant) swap.
 
  • Like
Reactions: Johannes S
Lowering the threshold, for ballooning and KSM can mitigate issues on the host, as it will start reducing memory more aggressively when the host still has plenty of resources.
 
  • Like
Reactions: Johannes S
You must log in or register to reply here.
Top Bottom
Back