Continuously increasing memory usage until oom-killer kill processes

mathx

Active Member
Jan 15, 2014
155
3
38
Same problem here, all my containers are silently taking more memory in cache until oom-kill kicks in. This happens in all my containers but in pve6. In my case, restarting systemd-journald did not help, however reducing journals log size increased the available memory by the same size I reduced the log. (But the memory keeps decreasing, so it is just a matter of time, I just gained time)
Ive seen other posts around the internet about people hunting for the culprit and finding memory leaks in various apps. If it's not your tmpfs (and there are other tmpfs users other than systemd-journald, note), then it's something else. Keep looking.
 

fabian

Proxmox Staff Member
Staff member
Jan 7, 2016
7,469
1,392
164
This misleading issue is reporting it as buffers/cache instead of in "used". If you google it, there are dozens of posts by others misled by this and trying to flush their caches, which will not help at all. "Used" would be a fairer category to put it in, but Im guessing because of the way kernel structures such things, that may be harder to code. Nonetheless, it's misleading.
lxcfs doesn't do anything here - this is literally how the kernel reports that kind of memory usage (as shared memory!):

https://www.kernel.org/doc/html/latest/filesystems/tmpfs.html

basically the only thing that lxcfs does "special" is to handle total memory and swap according to the version of cgroups employed, the rest comes directly from how the kernel reports memory usage.
Is there a way to restrict tmpfs size in containers?
see above - you have to set this up inside the container (where the mounting actually happens)
 
  • Like
Reactions: mathx

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!