RAM usage well exceeds VM assignment

Sandbo

Active Member
Jul 4, 2019
84
10
28
33
Hi,

Lately I realized that the RAM usage on PVE exceed the amount I allocated for the VMs.
For example, now I am running a single VM and 4 GB RAM has been allocated to it, but the summary of the host says 15GB+ RAM has been used.

I am running PVE 6 with 32 GB RAM and a ZFS storage pool (1.8 TB mirror)

Host summary:
NDIeNWV.png


VM summary:
FWUv6Bm.png


htop somehow shows multiple instance of the same VM and each takes 13.8% of RAM. Is that expected?
13.8% of 32 GB RAM is ~ 4GB.
QgudnbB.png
 
There will be a KVM thread for each core you gave the VM, plus some extra ones to support virtual hardware. All of mine have two additional, same as yours.

IIRC, ZFS uses up to 1/2 of your RAM for ARC cache. See here:
https://pve.proxmox.com/wiki/ZFS_on_Linux#_limit_zfs_memory_usage

I have only limited experience with ZFS and am not currently using it. Hopefully someone more knowledgeable will chime in about ZFS memory usage.
 
There will be a KVM thread for each core you gave the VM, plus some extra ones to support virtual hardware. All of mine have two additional, same as yours.

IIRC, ZFS uses up to 1/2 of your RAM for ARC cache. See here:
https://pve.proxmox.com/wiki/ZFS_on_Linux#_limit_zfs_memory_usage

I have only limited experience with ZFS and am not currently using it. Hopefully someone more knowledgeable will chime in about ZFS memory usage.
Thanks for the ideas, that explained a lot regarding the multiple threads.
I believe the ZFS might be eating up quite a bit of RAM, but then I was expecting to see them in htop which didn't seem to be the case here.
 
I believe the ZFS might be eating up quite a bit of RAM, but then I was expecting to see them in htop which didn't seem to be the case here.

htop basically only shows user land stuff, ZFS and other filesystems are in the kernel space so their memory requirements are not shown here. Also, all space used by filesystem caching (old school block cache and ZFS ARC) is seen as "used" in Linux (and many other OSes) and not as "free" as in Windows. Therefore you have the seen discrepancy of the sum of all VMs memory. Please also refer to https://www.linuxatemyram.com/
 
htop basically only shows user land stuff, ZFS and other filesystems are in the kernel space so their memory requirements are not shown here. Also, all space used by filesystem caching (old school block cache and ZFS ARC) is seen as "used" in Linux (and many other OSes) and not as "free" as in Windows. Therefore you have the seen discrepancy of the sum of all VMs memory. Please also refer to https://www.linuxatemyram.com/
Thanks for the explanation.
It would be great if we can find out the RAM used or even control how much cache is allocated, as with the current behaviour I could run into issue with assigning RAM to the VMs while I thought I had sufficient.

Is there a way to design the size of the cache?
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!