RAM double use

[owear]

Hi there!

I have two VM with windows server 2019 on it with RAM about 100GB for it.
But Proxmox tell me that I have use double RAM resources on it.







root@dell-r640:~# free -m
total used free shared buff/cache available
Mem: 257548 203526 25152 46 28868 52160
Swap: 8191 1375 6816

root@dell-r640:~# vmstat
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
r b swpd free buff cache si so bi bo in cs us sy id wa st
0 0 1408904 25630616 27577268 1983120 0 0 104 43 1 1 1 0 99 0 0



Question: Where is my RAM??))))

 

[_gabriel]

zfs

 

[owear]

zfs

yes, I have two ZFS mirroring for one VM

 

[_gabriel]

This is well known ZFS rule.
By default, ZFS use 50% of the available host RAM as ARC Cache.
Search in the forum and the docs, to limit ARC.

 

[owear]

This is well known ZFS rule.
By default, ZFS use 50% of the available host RAM as ARC Cache.
Search in the forum and the docs, to limit ARC.

If I limiting ARC for 10% of available RAM I will get good performance and stability for ZFS?

 

[heremita]

It's called overhead

Hypervisor Overhead, also known as "virtualization overhead", is the amount of memory that a hypervisor like Proxmox uses for its own operation. This memory is not allocated to any particular VM, but is reserved for hypervisor tasks, such as:
- VM management
- Hypervisor code execution
The memory used by the Hypervisor Overhead does not show up in any VM's RAM usage because it is not allocated to any of them. It is memory reserved for the operation of the hypervisor and is not available for applications or processes running in the VMs.

Overhead increases the server's overall RAM consumption as it adds to the memory allocated to VMs. The impact of Hypervisor Overhead on total RAM consumption depends on several factors:

- Number of VMs
- Resources assigned to VMs
This last point is important, the more memory is allocated to the VMs, the less impact the Hypervisor Overhead will have on the total RAM consumption.

Now if you use zfs this could also be the cause since for each tera disk there are 2 GB of ram that zfs requires, you could control this with KSM to help performance, by default ksm is activated when it is at 20% of free use in RAM memory, this value could be modified so that the ksm is activated much sooner and you can have better performance in the hypervisor in terms of memory use

 

[owear]

This is well known ZFS rule.
By default, ZFS use 50% of the available host RAM as ARC Cache.
Search in the forum and the docs, to limit ARC.

I have already limit ACR for 10GIB, but after reboot node, I have double RAM used for my running VM again((

It works with 1 VM

 

[owear]

Now it is
root@dell-r640:~# arc_summary

------------------------------------------------------------------------
ZFS Subsystem Report Tue Apr 02 20:45:08 2024
Linux 5.15.143-1-pve 2.1.14-pve1
Machine: dell-r640 (x86_64) 2.1.14-pve1

ARC status: HEALTHY
Memory throttle count: 0

ARC size (current): 24.4 % 2.4 GiB
Target size (adaptive): 78.6 % 7.9 GiB
Min size (hard limit): 78.6 % 7.9 GiB
Max size (high water): 1:1 10.0 GiB
Most Frequently Used (MFU) cache size: 9.7 % 233.3 MiB
Most Recently Used (MRU) cache size: 90.3 % 2.1 GiB
Metadata cache size (hard limit): 75.0 % 7.5 GiB
Metadata cache size (current): 5.2 % 402.0 MiB
Dnode cache size (hard limit): 10.0 % 768.0 MiB
Dnode cache size (current): 0.1 % 415.6 KiB

 

[_gabriel]

Depends of usage, but imo 16/32 GB for ARC is ok.
ZFS is faster with datacenter ssd than many ARC Cache.