[SOLVED] LXC container suddenly not starting

Areckee

New Member
Aug 7, 2023
2
0
1
Hi,
one of my LXC containers (with Docker on Debian) refuses to start since yesterday. I have some unprivileged and some privileged ones and only this one causes problems.
I get these error messages:
Code:
run_buffer: 322 Script exited with status 1
lxc_init: 844 Failed to run lxc.hook.pre-start for container "203"
__lxc_start: 2027 Failed to initialize container "203"
TASK ERROR: startup for container '203' failed

Code:
# lxc-start -n 203 -F -lDEBUG -o lxc-203.log
lxc-start: 203: ../src/lxc/conf.c: run_buffer: 322 Script exited with status 1
lxc-start: 203: ../src/lxc/start.c: lxc_init: 844 Failed to run lxc.hook.pre-start for container "203"
lxc-start: 203: ../src/lxc/start.c: __lxc_start: 2027 Failed to initialize container "203"
lxc-start: 203: ../src/lxc/tools/lxc_start.c: main: 306 The container failed to start
lxc-start: 203: ../src/lxc/tools/lxc_start.c: main: 311 Additional information can be obtained by setting the --logfile and --logpriority options

Code:
# cat lxc-203.log
lxc-start 203 20230807142635.422 INFO     lsm - ../src/lxc/lsm/lsm.c:lsm_init_static:38 - Initialized LSM security driver AppArmor
lxc-start 203 20230807142635.422 INFO     conf - ../src/lxc/conf.c:run_script_argv:338 - Executing script "/usr/share/lxc/hooks/lxc-pve-prestart-hook" for container "203", config section "lxc"
lxc-start 203 20230807142636.103 DEBUG    conf - ../src/lxc/conf.c:run_buffer:311 - Script exec /usr/share/lxc/hooks/lxc-pve-prestart-hook 203 lxc pre-start produced output: closing file '/etc/network/interfaces.tmp.283215' failed - No space left on device

lxc-start 203 20230807142636.105 DEBUG    conf - ../src/lxc/conf.c:run_buffer:311 - Script exec /usr/share/lxc/hooks/lxc-pve-prestart-hook 203 lxc pre-start produced output: error in setup task PVE::LXC::Setup::pre_start_hook

lxc-start 203 20230807142636.116 ERROR    conf - ../src/lxc/conf.c:run_buffer:322 - Script exited with status 1
lxc-start 203 20230807142636.116 ERROR    start - ../src/lxc/start.c:lxc_init:844 - Failed to run lxc.hook.pre-start for container "203"
lxc-start 203 20230807142636.116 ERROR    start - ../src/lxc/start.c:__lxc_start:2027 - Failed to initialize container "203"
lxc-start 203 20230807142636.116 INFO     conf - ../src/lxc/conf.c:run_script_argv:338 - Executing script "/usr/share/lxcfs/lxc.reboot.hook" for container "203", config section "lxc"
lxc-start 203 20230807142636.617 INFO     conf - ../src/lxc/conf.c:run_script_argv:338 - Executing script "/usr/share/lxc/hooks/lxc-pve-poststop-hook" for container "203", config section "lxc"
lxc-start 203 20230807142637.178 DEBUG    conf - ../src/lxc/conf.c:run_buffer:311 - Script exec /usr/share/lxc/hooks/lxc-pve-poststop-hook 203 lxc post-stop produced output: umount: /var/lib/lxc/.pve-staged-mounts/mp0: not mounted.

lxc-start 203 20230807142637.178 DEBUG    conf - ../src/lxc/conf.c:run_buffer:311 - Script exec /usr/share/lxc/hooks/lxc-pve-poststop-hook 203 lxc post-stop produced output: command 'umount -- /var/lib/lxc/.pve-staged-mounts/mp0' failed: exit code 32

lxc-start 203 20230807142637.240 ERROR    lxc_start - ../src/lxc/tools/lxc_start.c:main:306 - The container failed to start
lxc-start 203 20230807142637.240 ERROR    lxc_start - ../src/lxc/tools/lxc_start.c:main:311 - Additional information can be obtained by setting the --logfile and --logpriority options

Code:
# lxc-start --name 203 --foreground
lxc-start: 203: ../src/lxc/conf.c: run_buffer: 322 Script exited with status 1
lxc-start: 203: ../src/lxc/start.c: lxc_init: 844 Failed to run lxc.hook.pre-start for container "203"
lxc-start: 203: ../src/lxc/start.c: __lxc_start: 2027 Failed to initialize container "203"
lxc-start: 203: ../src/lxc/tools/lxc_start.c: main: 306 The container failed to start
lxc-start: 203: ../src/lxc/tools/lxc_start.c: main: 311 Additional information can be obtained by setting the --logfile and --logpriority options

I've read many threads on this forum as well on reddit, applied the solutions (i.e. installed binutils, checked for Debian version in /etc and it's 11.7, # pct fsck 203) but none helped.

Code:
# pveversion -v
proxmox-ve: 7.4-1 (running kernel: 5.15.108-1-pve)
pve-manager: 7.4-16 (running version: 7.4-16/0f39f621)
pve-kernel-5.15: 7.4-4
pve-kernel-5.15.108-1-pve: 5.15.108-2
pve-kernel-5.15.102-1-pve: 5.15.102-1
ceph-fuse: 15.2.17-pve1
corosync: 3.1.7-pve1
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown2: 3.1.0-1+pmx4
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.24-pve2
libproxmox-acme-perl: 1.4.4
libproxmox-backup-qemu0: 1.3.1-1
libproxmox-rs-perl: 0.2.1
libpve-access-control: 7.4.1
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.4-2
libpve-guest-common-perl: 4.2-4
libpve-http-server-perl: 4.2-3
libpve-rs-perl: 0.7.7
libpve-storage-perl: 7.4-3
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 5.0.2-2
lxcfs: 5.0.3-pve1
novnc-pve: 1.4.0-1
proxmox-backup-client: 2.4.3-1
proxmox-backup-file-restore: 2.4.3-1
proxmox-kernel-helper: 7.4-1
proxmox-mail-forward: 0.1.1-1
proxmox-mini-journalreader: 1.3-1
proxmox-widget-toolkit: 3.7.3
pve-cluster: 7.3-3
pve-container: 4.4-6
pve-docs: 7.4-2
pve-edk2-firmware: 3.20230228-4~bpo11+1
pve-firewall: 4.3-5
pve-firmware: 3.6-5
pve-ha-manager: 3.6.1
pve-i18n: 2.12-1
pve-qemu-kvm: 7.2.0-8
pve-xtermjs: 4.16.0-2
qemu-server: 7.4-4
smartmontools: 7.2-pve3
spiceterm: 3.2-2
swtpm: 0.8.0~bpo11+3
vncterm: 1.7-1
zfsutils-linux: 2.1.11-pve1

Code:
# pct config 203
arch: amd64
cores: 2
features: nesting=1
hostname: docker
memory: 2048
net0: name=eth0,bridge=vmbr0,firewall=1,gw=10.10.30.1,hwaddr=3A:1A:56:0B:F2:69,ip=10.10.30.203/24,tag=30,type=veth
onboot: 1
ostype: debian
rootfs: local:203/vm-203-disk-0.raw,size=8G
swap: 512

I'd very much appreciate your help.
 
Last edited:
pre-start produced output: closing file '/etc/network/interfaces.tmp.283215' failed - No space left on device
Hi,
looks like you ran out of storage space. You can mount the containers root filesystem with pct mount 203 and check with df -h && df -hi.
 
  • Like
Reactions: Areckee
Hi,
looks like you ran out of storage space. You can mount the containers root filesystem with pct mount 203 and check with df -h && df -hi.
You are absolutely right. I didn't see it coming :) Thank you for your help. You're doing great job. Keep it that way.
 
Thanks for the discussion here, helped me get on the right track with an Ubuntu 23.04 container `do-release-upgrade -d` bumped to (pre-release of) 23.10 - and it failed to boot.

In my case the issue logged in Proxmox UI was very unhelpful:
Code:
run_buffer: 321 Script exited with status 255
lxc_init: 847 Failed to run lxc.hook.pre-start for container "118"
__lxc_start: 2008 Failed to initialize container "118"
TASK ERROR: startup for container '118' failed

However the `lxc-start -n 118 -F -lDEBUG -o lxc-118.log` suggested above helped with getting a log file which stated clearly:

Code:
lxc-start 118 20231002100729.303 INFO     conf - ../src/lxc/conf.c:run_script_argv:337 - Executing script "/usr/share/lxc/hooks/lxc-pve-prestart-hook" for container "118", config section "lxc"
lxc-start 118 20231002100729.607 DEBUG    conf - ../src/lxc/conf.c:run_buffer:310 - Script exec /usr/share/lxc/hooks/lxc-pve-prestart-hook 118 lxc pre-start produced output: unsupported Ubuntu version '23.10'
lxc-start 118 20231002100729.615 ERROR    conf - ../src/lxc/conf.c:run_buffer:321 - Script exited with status 255

That led me on a grepping spree across host filesystem, with `/usr/share/perl5/PVE/LXC/Setup/Ubuntu.pm` in my case being the culprit - it starts with a list of known release IDs :)

Hope this helps someone else,
Jim Klimov
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!