I came across this thread: https://forum.proxmox.com/threads/memory-usage-not-same-as-in-vm.82640/
which is the exact same issue I have, however the guy didn't end up replying back.
I'm running Ubuntu 20.04.3 with PVE 6.4-4, Ballooning device is enabled in the Memory Settings:
Looks like PVE doesn't detect ram usage in FreeBSD properly, example - VM has 700 MB usage, PVE dashboard shows 1924.
qemu-guest-agent is installed, however, looks like PVE doesn't use it to get the RAM usage:
dommemstat is the command currently implemented in the qemu-guest-agent port...