bug in showing memory of connected nodes

[zyghom]

I have 3 connected PVE and 4 PBS and all seem ok except I the graph showing total RAM on PBS - see below - doubt I have 2 PiB of RAM ;-)
version:
Datacenter Manager 1.1.7

 

[Bu66as]

Interesting, CPU (1.27 of 16 cores) and storage look plausible, only the memory total is off. Since that widget sums up all 4 PBS, my guess is that one of them reports a bogus total and the others are fine. Can you check each PBS on its own in PDM and see which one shows the weird value? Also, is one of them running in an LXC container or somewhere without a fixed memory limit? Then the total might come from the cgroup and not from real RAM. Just a hunch. Open a report on bugzilla.proxmox.com. Mention which PBS version it is and what free -b shows on that box.

 

[zyghom]

you are right: 2 are in lxc, 2 are in vm
both lxc ones are showing 1 PB of ram

 

[Bu66as]

Thanks for checking. The two with the bogus value are fhuqnap-pbs-qnap1 and pbstn, and both have only a few hundred MiB in use, so the data itself is fine. The total is just wrong. A PVE container always has a memory limit, but inside the container the cgroup is nested and usually reports "unlimited" anyway. My guess is that ends up as the 1 PiB total. What does free -b show inside one of them, and what does cat /sys/fs/cgroup/lxc/<ID>/memory.max show on the PVE host? If both show the configured limit, the real value is available and PBS just reads the wrong source. Either way, put that output in the bug report. It helps more than the screenshot.

 

[zyghom]

free -b it is easy to show:

but with this container being on truenas I have issues to locate "/sys/fs/cgroup/lxc/<ID>/memory.max" as:

9:38:27 root@truenas /home/admin $: ls -la /sys/fs/cgroup/lxc
ls: cannot access '/sys/fs/cgroup/lxc': No such file or directory

 

[Bu66as]

That one helps. 1125899906842624 is exactly 2^50, so the container itself reports 1 PiB in free. PBS isn't calculating anything wrong. It reads what the container tells it, and that's the "no limit set" value. Forget my cgroup path from before, that was for a PVE host and doesn't apply on TrueNAS. My guess is that the container on TrueNAS has no memory limit configured at all. Set a fixed RAM limit on the instance in TrueNAS, restart it and check free -b again. It should show the limit then, and PDM should be fine after the next poll. The other LXC one probably has the same cause. A bug report still makes sense IMO, because PBS/PDM could easily treat 2^50 as "unlimited" instead of adding it to the total.

 

[zyghom]

I have no idea where to set mem limit there

 

[Johannes S]

It seems that TrueNAS doesn't support setting this at the moment If i read their docs correctly: https://www.truenas.com/docs/scale/containers/managingcontainers/

I have no experience with TrueNAS containers lxc feature yet but it looks as if it's still quite limited compared to ProxmoxVE:

• It doesn't seem to be possible to limit memory usage, only CPU
• Nesting support is only possible with priviliged containers which is a big oof security wise

It's a pity, really. Personally I think that NAS systems like TrueNAS, unRAID or OpenMediaVault are for most homeusers the better deal than ProxmoxVE since they their limitations in scope also means that they are somehow easier to manage and still allow the hosting of docker containers (apps in TrueNAS/unRAID lingo) and vms.

Edit: Above link is for the development version. For 25.04 their container backend allows limiting memory: https://www.truenas.com/docs/scale/25.10/scaleuireference/containersscreens/

But they already announced that they will change their backend with the next versions thus I wouldn't feel comfortable that it won't break at some point. ixsystems at their best...

 

[zyghom]

mine is nightly so the newest version on their github or so, let it be, I feel like billioner having PB of RAM (even if it is only on the graph) ;-)