LXC Memory Usage Deb13 PVE9.x

[drevilish]

I'm going to preface this with this worked in Deb12 / PVE8.x.
My application use case ingests a large file, decompiles it, moves on.

I'm using two mountpoints other than the boot disk.
mp0 is a small temp rbd stored on fast disks.
mp1 is a shared mount point on cephfs.

The files being processed are larger than what is available in memory, hence the temp drive.

The memory usage climbs slowly over time to 99% of the limit.
During this time the CPU goes to 99% and the processes/LXC become unresponsive; the only solution is to kill the container or to increase the memory allocation.

By default memory.high = memory.max meaning the LXC isn't told to reduce memory usage within the container before the host refuses to allocate anymore.

If I'm right, LXCs need a better way of handling memory at their OS level for caching and applications.

As a test I've run to see where the memory usage lies.

memory: 4096
lxc.cgroup2.memory.high: 1024

With this set the LXC grabs and hovers around 1.41G usage.

Could we get a UI implementation of memory.high or a global setting for memory.high at 90% of memory.max

 

[drevilish]

I'm getting even more confused now.
Container again bumped into it's top memory limit. I gave it an extra 4G so it would be responsive again. Opened the container, ran free -h

               total        used        free      shared  buff/cache   available
Mem:            12Gi       1.1Gi       1.2Gi        28Ki       2.7Gi        10Gi

Container dashboard reports
Memory usage 67.20% (8.06 GiB of 12.00 GiB)

I bumped the LXC by 4G, but I don't understand why the container thinks it has 10G available while the dashboard thinks it only has 4G.
I'm not sure what if anything, I've done wrong.

CONTAINER $> cat /proc/meminfo
MemTotal:       12582912 kB
MemFree:         1643628 kB
MemAvailable:   12355523 kB
Buffers:               0 kB
Cached:          2292568 kB
SwapCached:            0 kB
Active:          5290688 kB
Inactive:        5602112 kB
Active(anon):     141080 kB
Inactive(anon):    83020 kB
Active(file):    5149608 kB
Inactive(file):  5519092 kB
Unevictable:           0 kB
Mlocked:          175456 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Zswap:                 0 kB
Zswapped:              0 kB
Dirty:                 0 kB
Writeback:             0 kB
AnonPages:        224072 kB
Mapped:                0 kB
Shmem:                28 kB
KReclaimable:     680212 kB
Slab:              43927 kB
SReclaimable:      43195 kB
SUnreclaim:          732 kB
KernelStack:       34176 kB
PageTables:       133956 kB
SecPageTables:     18532 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    32590692 kB
Committed_AS:   67953324 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      738456 kB
VmallocChunk:          0 kB
Percpu:            44576 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
FileHugePages:         0 kB
FilePmdMapped:         0 kB
CmaTotal:              0 kB
CmaFree:               0 kB
Unaccepted:            0 kB
Balloon:               0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:               0 kB
DirectMap4k:      711968 kB
DirectMap2M:    45807616 kB
DirectMap1G:    19922944 kB

HOST $> cat /proc/meminfo
MemTotal:       65181388 kB
MemFree:         7113596 kB
MemAvailable:   21040044 kB
Buffers:         2941460 kB
Cached:          2822304 kB
SwapCached:            0 kB
Active:         25652616 kB
Inactive:       28979520 kB
Active(anon):   20347448 kB
Inactive(anon): 20318208 kB
Active(file):    5305168 kB
Inactive(file):  8661312 kB
Unevictable:      177332 kB
Mlocked:          175456 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Zswap:                 0 kB
Zswapped:              0 kB
Dirty:               112 kB
Writeback:             8 kB
AnonPages:      40670116 kB
Mapped:           306940 kB
Shmem:            159432 kB
KReclaimable:     687852 kB
Slab:            1139136 kB
SReclaimable:     687852 kB
SUnreclaim:       451284 kB
KernelStack:       34160 kB
PageTables:       133692 kB
SecPageTables:     18532 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    32590692 kB
Committed_AS:   67951908 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      716428 kB
VmallocChunk:          0 kB
Percpu:            44576 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
FileHugePages:         0 kB
FilePmdMapped:         0 kB
CmaTotal:              0 kB
CmaFree:               0 kB
Unaccepted:            0 kB
Balloon:               0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:               0 kB
DirectMap4k:      711968 kB
DirectMap2M:    45807616 kB
DirectMap1G:    19922944 kB

Relevant snippet:

HOST $> systemd-cgtop
CGroup                                         Tasks   %CPU   Memory  Input/s Output/s
lxc/131                                           25      -    10.4G        -        -