How granular is QEMU Guest Agent's ability to report on RAM usage inside a Guest VM?

Sep 1, 2022
574
217
68
42
The Issue: Based on querying QEMU Guest Agent, PVE 9, shows Total RAM usage of a running VM (as labeled in free -h ) , which is a combination of used and buff/cache.
  • Thus, in guests with workloads that consume all available RAM for buff/cache, there is effectively no visible way to distinguish used RAM that the workload must have versus buffer/cache RAM that the workload will use but can release if the guest needs it for something else.
  • This makes it effectively impossible to determine via experimentation a minimum viable amount of RAM for a guest just by looking at PVE's dashboard for the guest.
  • Monitoring RAM from inside the guest is a must if you want an accurate picture of how RAM is being used.
Code:
Every 2.0s: free -h                                                                                                                         subspace2: Fri Sep 11 12:37:07 2026

               total        used        free      shared  buff/cache   available
Mem:           3.8Gi       1.8Gi       244Mi        61Mi       2.1Gi       2.0Gi
Swap:          1.9Gi       512Ki       1.9Gi

Question: I will freely admit I don't really understand a whole lot about how QEMU Guest Agent works, just what it's meant to do. So, before I file an enhancement request, I wanted to ask if it's even possible for QEMU Guest Agent to expose to PVE buff/cache, available, and, I guess, shared. Being able to toggle those on in the PVE dashboard would make it much easier to see if a guest is actually struggling with the amount of RAM it's been allocated, or just using everything it has for cache.

Context

This was especially confusing to me as a relatively new user when I started up a Debian 13 VM running Unifi OS. I use Debian 13 VMs quite a bit, but I've usually always got quite a bit of free RAM showing at the PVE dashboard level because Debian itself does not use all available RAM as cache (and apparently so far none of the workloads I've deployed there have used it all, either).

But, when I set up Unfi OS (which is a container inside Podman, which I'm also using for the first time) on a Debian 13 guest, no matter how much RAM I gave the VM, PVE's dashboard eventually showed it all being consumed. Debian 13 on its own does not act like that, so I wasted a lot of time looking for memory leaks or misconfiguration or other problems that didn't exist. At one point, I even thought maybe the 8th gen Intel CPU I'm using on the host (my first time using something older than a Ryzen 3700X or 12th gen Intel) was wonky about how it used or reported RAM, and was trying to figure out how to test for a possible bug. I spent a good few hours spinning my wheels before I thought to check free -h and realized Unifi OS, unlike any workload I've run before that doesn't involve ZFS directly, just eats all the RAM you give it.
 
Last edited:
  • Like
Reactions: Gilou
Last edited:
  • Like
Reactions: leesteken
Thanks, @Impact . I accidentally had the ballooning driver off in those VMs. I've fixed that now, and the output is definitely more accurate as to overall used RAM.

I'd also forgotten about that bug report. I'm already subscribed to it. :P It was filed about 5 months ago, but is proposing a fairly significant change to internal data storage and also the GUI itself (and would also have implications for PDM and maybe PBS), so I wouldn't expect a fast turnaround.

Still, I'm looking forward to having more accurate output someday. :)
 
Memory is reported by the ballooning driver/service, not the Guest Agent. Maybe some of this answers your questions. Especially the links.
It should be possible to "accurately" report this in the GUI. Either by using the Guest Agent's exec functionality to check inside the VM or also see this bug report.
CTs have somewhat similar issues, by the way (try dropping cache on the node and see what happens to you CT's memory usage), it's just not as noticeable.

Hallo. On the subject of ballooning (funny name). Is there a way to get it working on free BSD. I got 2 firewalls running with BSD and qemu agent. But the memory reporting aint working?!
 
Still, I'm looking forward to having more accurate output someday. :)

Setup a real monitoring system like zabbix, prometheus, checkmk or icinga in a dedicated vm or lxc and put the agent of your chosen monitoring tool inside your guests. Proxmox dashboards will never have the same features set and reliability.