[SOLVED] Upgrade from 64ram to 128G ram but I am missing 7G

[Venom]

I have something strange, I had this problem before I upgraded as well as I was missing ram then too but I can't seem to figure this out.

# dmesg | grep Memory [    0.323663] Memory: 127494620K/129913696K available (16393K kernel code, 4342K rwdata, 10192K rodata, 2888K init, 4900K bss, 2418816K reserved, 0K cma-reserved) [    0.512565] x86/mm: Memory block size: 2048MB

#  grep MemTotal /proc/meminfo MemTotal:       127674884 kB

# free -h
               total        used        free      shared  buff/cache   available
Mem:           121Gi        46Gi        74Gi        47Mi       902Mi        74Gi
Swap:          8.0Gi          0B       8.0Gi

When I check with "dmidecode --type 17" I also see 4x 32G of my memory modules.

When I convert MemTotal to GB I end up at around 121G, on my desktop I have 32G ram and there I am only missing 1G which I can understand but I can't seem to understand why I am missing 7G here. I came across an old post and there the OP and the person that replied are only missing a few ram. Can someone help me makes sense of why I am missing 7G of my total 128G?

 

[Dunuin]

128 GB = 119.21 GiB = 119.21 Gi
127674884 kB = 118.91 GiB

So actually free -h is reporting that you got too much RAM.

Try free -b to get bytes and then calculate the GB yourself.

 

[Venom]

@Dunuin thanks for your reply!

I think I found it, there was some setting in the bios/uefi where I could disable the onboard graphics chip on the cpu and now I see this.

# free -h
               total        used        free      shared  buff/cache   available
Mem:           125Gi        20Gi       104Gi        47Mi       315Mi       104Gi
Swap:          8.0Gi          0B       8.0Gi

free -b is showing this now.

root@pve:~# free -b
               total        used        free      shared  buff/cache   available
Mem:     134966927360 22096384000 112540565504    49389568   329977856 111805747200
Swap:     8589930496           0  8589930496

When I convert 134966927360(bytes) to gigabytes it equals to 125.7G so that seems equals to the correct ram which I see in "free -h" now. So it seems like it was the graphics card on the cpu where 4G of ram was reserved for. Since I am only missing 3G now which seem closer to my actual total which I would expect.

So I would think this is solved now.

 

[LnxBil]

Just to remark:

There is always less memory available then you actually have due to hardware reservation, potential crashkernel and a lot of internall stuff. The absolute amount increases if your memory increases and you can try to find out more about it, if you read the output of dmesg carefully, there are all memory regions and usage explained in more detail.
You will also waste a lot of memory for 4K-pagetables if you have a lot of RAM. Having more RAM will in general also lower the performance of you system due to a larget memory tree. Having more CPUs with memory will also reduce the memory performance measureable.

This is nothing new and therefore, there exist e.g. hugepages/largepages and also NUMA-pinning to circumvent that. PVE is able to use them, but it is always a trade-off between easy-to-use and performance. As normal pages come in 4K, there is also hardware support for 2M and 1G hugepages. Having 1G hugepages will increase the memory performance by approx. 10% (memory allocation will be blazing fast) and reduce the page table overhead significantly, but it'll restrict you to use just VMs with are multiples of 1G and you need to allocate the hugepages beforehand. IIRC KSM will not work with them either.