Proxmox LXC Can't Boot

[Silent10]

Can someone help me with a proxmox LXC please? I am unable to get it to boot, from what I can see everything has free space and i've ran trim and free'd up even more space.

This was previously working for the past year until yesterday (and I don't believe I made any changes)

it wouldn't boot via proxmox gui so I tried to start with debug to log via:
lxc-start -n 108 -F -l DEBUG -o /tmp/lxc-CT108.log

which results in:

cat /tmp/lxc-CT108.log
lxc-start 108 20240924001742.333 INFO     lsm - ../src/lxc/lsm/lsm.c:lsm_init_static:38 - Initialized LSM security driver AppArmor
lxc-start 108 20240924001742.340 INFO     utils - ../src/lxc/utils.c:run_script_argv:587 - Executing script "/usr/share/lxc/hooks/lxc-pve-prestart-hook" for container "108", config section "lxc"
lxc-start 108 20240924001743.755 DEBUG    utils - ../src/lxc/utils.c:run_buffer:560 - Script exec /usr/share/lxc/hooks/lxc-pve-prestart-hook 108 lxc pre-start produced output: closing file '/etc/network/interfaces.tmp.53800' failed - No space left on device

lxc-start 108 20240924001743.758 DEBUG    utils - ../src/lxc/utils.c:run_buffer:560 - Script exec /usr/share/lxc/hooks/lxc-pve-prestart-hook 108 lxc pre-start produced output: error in setup task PVE::LXC::Setup:re_start_hook

lxc-start 108 20240924001743.777 ERROR    utils - ../src/lxc/utils.c:run_buffer:571 - Script exited with status 1
lxc-start 108 20240924001743.777 ERROR    start - ../src/lxc/start.c:lxc_init:845 - Failed to run lxc.hook.pre-start for container "108"
lxc-start 108 20240924001743.777 ERROR    start - ../src/lxc/start.c:__lxc_start:2034 - Failed to initialize container "108"
lxc-start 108 20240924001743.777 INFO     utils - ../src/lxc/utils.c:run_script_argv:587 - Executing script "/usr/share/lxcfs/lxc.reboot.hook" for container "108", config section "lxc"
lxc-start 108 20240924001744.279 INFO     utils - ../src/lxc/utils.c:run_script_argv:587 - Executing script "/usr/share/lxc/hooks/lxc-pve-poststop-hook" for container "108", config section "lxc"
lxc-start 108 20240924001745.101 ERROR    lxc_start - ../src/lxc/tools/lxc_start.c:lxc_start_main:307 - The container failed to start
lxc-start 108 20240924001745.101 ERROR    lxc_start - ../src/lxc/tools/lxc_start.c:lxc_start_main:312 - Additional information can be obtained by setting the --logfile and --logpriority options

I think the relevant part here is likely:
lxc pre-start produced output: closing file '/etc/network/interfaces.tmp.53800' failed - No space left on device

But unless im missing something, i think there is plenty of free space

root@hv:/# df -h
Filesystem            Size  Used Avail Use% Mounted on
udev                  7.7G     0  7.7G   0% /dev
tmpfs                 1.6G  3.2M  1.6G   1% /run
/dev/mapper/pve-root   67G  9.4G   54G  15% /
tmpfs                 7.8G   46M  7.7G   1% /dev/shm
tmpfs                 5.0M     0  5.0M   0% /run/lock
efivarfs              192K   39K  149K  21% /sys/firmware/efi/efivars
/dev/nvme0n1p2        511M  340K  511M   1% /boot/efi
/dev/fuse             128M   24K  128M   1% /etc/pve
tmpfs                 1.6G     0  1.6G   0% /run/user/0
root@hv:/# pvesm status
Name             Type     Status           Total            Used       Available        %
local             dir     active        69706992         9824148        56296184   14.09%
local-lvm     lvmthin     active       144154624        92979732        51174891   64.50%

I also ran:
pct fsck 108

fsck from util-linux 2.38.1
/dev/mapper/pve-vm--108--disk--0: clean, 68954/262144 files, 1044480/1048576 blocks

 

[leesteken]

I think the relevant part here is likely:
lxc pre-start produced output: closing file '/etc/network/interfaces.tmp.53800' failed - No space left on device

That's probably the problem.

I also ran:
pct fsck 108

fsck from util-linux 2.38.1
/dev/mapper/pve-vm--108--disk--0: clean, 68954/262144 files, 1044480/1048576 blocks

Looks like the virtual disk of the container is full (as almost all blocks are used). Make it bigger or mount it on the Proxmox host to delete some files.