Memory problem

[gabrielpons]

Hello, first of all, sorry for my English as it is not my native language.

I have a problem with some Proxmox servers and memory RAM.

The sum of the memory of all the virtual machines diverges a lot from the total of the Proxmox server.

All virtual machines have QEMU Guest Agent installed.

Screenshot of the Proxmox server:


Screenshot of the Proxmox server with htop:


Screenshot of virtual machine 401:


Screenshot of virtual machine 402:


Screenshot of virtual machine 406:


Screenshot of virtual machine 416:


The total of all virtual machines is 2916,62MB and the one consumed by the server is 9,85GB, why so much difference?

Any help is greatly appreciated.

 

[Dunuin]

The KVM process won't release memory without ballooning. Lets say your VM is using 1GB of 4GB RAM assigned to that VM. Then the KVM process will use abit more than 1GB. Then your VM is doing something, starts caching and RAM usage goes up to 4GB. The KVM process will now use 4GB. And even if your guest OS frees up RAM and the guests RAM usge goes down to 1GB again, the KVM process will still be at 4GB until your restart that VM. Atleast thats what I am observing here.
I would also like to know why KVM can't reduce its memory footprint when the RAM isn't used any longer by the guest OS.

 

[gabrielpons]

Thank you for your answer, so this is normal?

All virtual machines have balloning enabled:

VM ID 401 -> 1,00GiB/4,00GiB​

VM ID 402 -> 1,00GiB/4,00GiB​

VM ID 406 -> 512,00MiB/1,00GiB​

VM ID 416 -> 512,00MiB/1,00GiB​

 

[leesteken]

It looks about normal. Note that ballooning does not take memory away from VMs until the host memory is at 80%. And it tends to keep it at 80% because it will give memory back to VMs when below 80%. Another thing to keep in mind: ballooning will distribute memory over VMs according to their shares, not how much they want/need.

 

[gabrielpons]

All right, thank you very much

Best regards

 

[Dunuin]

So best to follow the rules:
1.) Dont assign a VM more max RAM than it actually needs so it won't waste too much RAM for caching. Basically all modern OSs will eat every bit of RAM you throw at them. The more RAM you give them, the more they will use. If the OS doesn't need it for services, it will "waste" it for caching.
2.) Plan that a VM process might use more RAM than you assign to it because of virtualization overhead. Virtualizing a 8GB VM might actually use something like 8.5 or 9 GB of RAM. When using a cache mode like writeback that might be even more.
3.) Don't set the min RAM too low or ballooning will force the guest OS to kill processes (OOM killer). Like leesteken already said, ballooning won't care if that VM actually needs that RAM or not. If you set a VM to 2GB min RAM and 8GB max RAM. That VM is using 6GB for system/user services and balooning kicks in when the host exceeds 80% RAM utilization it will slowly remove 6GB of RAM of that VM forcing it to kill enough processes to free up 4 GB of RAM.
4.) Don't overcommit your RAM. If your host got 32GB of RAM you might want to only use 80 to 90% of that so there is always a bit of free RAM as a reserve so spikes won't force PVE to kill VMs or LXCs. That 10-20% also isn't wasted as PVE can still use that RAM for caching to increase disk performance. So you got 26GB of RAM left. Then PVE itself needs 2GB of RAM. So you got a total of 24 GB that you can assign across all VMs/LXCs. Don't assign more than that or you might run sooner or later into OutOfMemory situations where your services or guests get killed.
5.) Keep in mind that you assign GiB and not GB to guests. So 24GB are only 22.3 GiB.
6.) In case you want to switch later to ZFS, ZFS will also need alot of RAM. By default it will use up to 50% of your hosts RAM for caching. How much RAM ZFS needs atleast depends on the amount of Storage you got. A rule of thumb for that could be 2GB + 0.25GB to 1GB per 1 TB of raw storage.

 

[gabrielpons]

Thank you very much for all the advice.