LXC freezes, ssh 'banner exchange' timeout, high load idle CPU — Debian 13 /tmp tmpfs uses host RAM

[Ivan The Geek]

Posting this because the existing threads on this are all titled by the cause ("tmpfs wrong size"), and nobody hitting it searches for that. They search the symptoms - and those searches currently return threads about slow storage and HA timeouts, which sends you the wrong way for hours. This is what it actually looks like from the outside.

Symptoms

• The container is "up" but unusable. It pings. TCP connections complete.
• ssh fails with Connection timed out during banner exchange.
• RDP / desktop sessions connect but stop repainting.
• On the node: load average huge (21+ on 12 cores) while the CPU is idle.
• It never recovers on its own. Only a container restart clears it.

What is actually happening

Debian 13 moved /tmp to tmpfs. systemd's tmp.mount ships size=50%, and tmpfs resolves that percentage in the kernel from the host's total RAM. A container shares the host kernel, so it inherits the host's idea of 50%. It consults neither lxcfs nor the cgroup memory limit. Measured on one of ours:

host MemTotal (real)                31.3 GiB
CT MemTotal (lxcfs-masked view)      8.0 GiB
CT cgroup memory.max                 8.0 GiB
CT /tmp tmpfs sized itself to       15.6 GiB   <- 50% of the HOST's 31.3

So /tmp can be far larger than the container's entire memory allowance. Ours ranged from 1.9x to 31x the CT's limit - a 512 MiB container had a 15.6 GiB /tmp.

Then anything that fills /tmp - a build, a backup, a large download, a scratch directory - is spending the container's memory, not disk.

Why it freezes instead of just OOM-ing

tmpfs pages (shmem) are only reclaimable to swap. With swap: 0 - the default on many CTs, and unavoidable if the node itself has no swap - they are unevictable. The kernel cannot free them, so it enters permanent direct reclaim: scan, free nothing, retry, forever. That is the frozen state. From the cgroup during ours:

memory.current 7.99 GiB / memory.max 8.00 GiB
shmem 7.52 GiB of that        anon only 0.41 GiB
memory.events: high 6963202  max 11354  oom 64  oom_kill 21

Note that setting swap: N does not help if the node has no swap - container swap is backed by host swap, so memory.swap.max gets set while memory.swap.current stays 0 forever.

The three signals that lie to you

1. df shows free space the whole way down. During the freeze ours reported /tmp as "16G size, 48% used, 8.2G available" - on a container that was already dead. df reports the tmpfs limit, which the CT can never reach.
2. Load looks like CPU. vmstat read us 0, sy 33, id 67 with a run queue of 26 - tasks spinning in kernel reclaim, doing no work. Chasing CPU or storage finds nothing, which is exactly why the symptom searches mislead.
3. Nothing logs it usefully. systemd, dmesg and apt all look ordinary apart from OOM kills of whatever happened to be running.

The one honest signal - check this first

cat /proc/pressure/memory
cat /sys/fs/cgroup/lxc/<ctid>/memory.pressure

If full avg60 is high (ours was 82) then every task is stalled on memory, and you are looking at this problem rather than a storage problem. Then confirm:

findmnt -no SIZE /tmp                       # compare against the CT's memory
grep -E '^(shmem|anon) ' /sys/fs/cgroup/lxc/<ctid>/memory.stat
cat /sys/fs/cgroup/lxc/<ctid>/memory.events  # oom_kill climbing

If shmem is most of memory.current, that is it.

The fix

Cap the tmpfs below the container's memory limit. Inside the CT, create /etc/systemd/system/tmp.mount.d/99-size-cap.conf:

[Mount]
Options=mode=1777,strictatime,nosuid,nodev,size=<25% of CT memory>,nr_inodes=1m

On the filename: systemd only requires *.conf in that directory, and drop-ins are read in lexicographic order. Options= in a mount unit is last-wins rather than append, so the high prefix is deliberate - a cap named 10- would be silently overridden by anything later that also sets Options=, which is the wrong way round for a safety limit.

Two gotchas that will otherwise waste your time:

• systemctl show tmp.mount -p Options reports the live mount, not the unit's configured options. After writing the drop-in and daemon-reload it still prints the old size, so it cannot verify this change. Use systemctl cat, which also lists every drop-in in the order they are applied.
• You cannot remount live from inside an unprivileged CT. mount -o remount re-applies the superblock's existing options including uid=100000 (the idmap base), which is not a valid uid in the container's userns, so it fails with fsconfig() failed: tmpfs: Invalid uid '100000'. Forcing the old mount API with LIBMOUNT_FORCE_MOUNT2=always fails too. Only systemctl restart tmp.mount (which wipes /tmp - check for X11 sockets and live sessions first) or a reboot applies it.

Verify behaviorally rather than by reading the config back: dd past the cap and confirm it stops there. Ours stops at exactly the cap; before the change the same dd wrote everything.

If you would rather /tmp were a normal directory again, Debian's own alternative is systemctl mask tmp.mount and reboot.

Background and tracking

• Bugzilla #6167 - open, UNDECIDED. I have added measurements across ten containers there.
• The /tmp-specific thread: PVE9.x Debian13 LXC /tmp

Worth knowing if you are tempted to wait for a kernel fix: it was proposed in 2017 - Yafang Shao, "mm/shmem: set default tmpfs size according to memcg limit" - and rejected by Shakeel Butt. tmpfs pages are charged to the memcg of the allocator, not the memcg that performed the mount, and mounting is typically done by a controller outside the job's memcg - so sizing from the mounting task's memcg is the wrong hook. Cap your tmpfs rather than waiting.

DISCLAIMER: post created with assistance of Claude.