We have a few LXCs (Proxmox 7.2-11) that have memory restrictions set using the normal method available in the Proxmox console. Generally things work great but there is one application we run in the containers that keep consuming memory until the container hits the limit and the OOM killer tries to free some.
We figured out that the application is overestimating the available memory by using the total host memory instead of the container memory. The host has 768GB and the containers have 2GB, so it is a big difference. Inside the containers some utilities (free & /proc/meminfo) show the expected memory:
and some don't (sysinfo) :
The application code is open source so we can see it tries to adjust its memory estimate by looking at the cgroup v2 files inside the container. It finds the user "slice" (I think it is called) and then tries to read from
If I try the same thing on the host I actually find it is set to the expected value:
So, my questions are: Does Proxmox handle limiting memory the same way LXD does? Is it through cgroups v2? What is the best way for a program running in a Proxmox LXC to get the amount of memory available to the container?
Thanks for any insight.
Angus
We figured out that the application is overestimating the available memory by using the total host memory instead of the container memory. The host has 768GB and the containers have 2GB, so it is a big difference. Inside the containers some utilities (free & /proc/meminfo) show the expected memory:
Code:
$ free -h
total used free shared buff/cache available
Mem: 2.0Gi 152Mi 1.1Gi 0.0Ki 755Mi 1.8Gi
Swap: 256Mi 0B 256Mi
$ cat /proc/meminfo | grep MemTotal
MemTotal: 2048000 kBand some don't (sysinfo) :
C:
#include <stdio.h>
#include <sys/sysinfo.h>
/*
get_memory.c
*/
int main(void) {
struct sysinfo si;
sysinfo(&si);
printf("sysinfo() thinks it has %ld bytes of physical memory\n", si.totalram);
return 0;
}
Code:
$ ./get_memory
sysinfo() thinks it has 135083474944 bytes of physical memoryThe application code is open source so we can see it tries to adjust its memory estimate by looking at the cgroup v2 files inside the container. It finds the user "slice" (I think it is called) and then tries to read from
/sys/fs/cgroup/<slice>/memory.max. The problem is this file is always set to max in the container, in fact they are all set to max:
Code:
$ find /sys/fs/cgroup -type f -name memory.max -exec sh -c "cat '{}'" \;
max
max
max
... all max ...
maxIf I try the same thing on the host I actually find it is set to the expected value:
Code:
$ cat /sys/fs/cgroup/lxc/901/memory.max
2097152000So, my questions are: Does Proxmox handle limiting memory the same way LXD does? Is it through cgroups v2? What is the best way for a program running in a Proxmox LXC to get the amount of memory available to the container?
Thanks for any insight.
Angus
