[SOLVED] LXC container failing to start

C

crider17

New Member
Mar 7, 2021
19
0
1
55
Hi everyone:

I was playing around with adding another drive and now I cannot get my container to spin up.

root@pve:~# lxc-start -n 100 -F -lDEBUG -o lxc-100.log
Code: 
lxc-start: 100: conf.c: run_buffer: 316 Script exited with status 1
lxc-start: 100: start.c: lxc_init: 816 Failed to run lxc.hook.pre-start for container "100"
lxc-start: 100: start.c: __lxc_start: 2007 Failed to initialize container "100"
lxc-start: 100: tools/lxc_start.c: main: 308 The container failed to start
lxc-start: 100: tools/lxc_start.c: main: 313 Additional information can be obtained by setting the --logfile and --logpriority options

root@pve:~# cat lxc-100.log
Code: 
lxc-start 100 20211022224405.397 INFO     lsm - lsm/lsm.c:lsm_init_static:40 - Initialized LSM security driver AppArmor
lxc-start 100 20211022224405.397 INFO     conf - conf.c:run_script_argv:332 - Executing script "/usr/share/lxc/hooks/lxc-pve-prestart-hook" for container "100", config section "lxc"
lxc-start 100 20211022224405.985 DEBUG    conf - conf.c:run_buffer:305 - Script exec /usr/share/lxc/hooks/lxc-pve-prestart-hook 100 lxc pre-start produced output: closing file '/etc/systemd/network/eth0.network.tmp.9856' failed - No space left on device

lxc-start 100 20211022224405.986 DEBUG    conf - conf.c:run_buffer:305 - Script exec /usr/share/lxc/hooks/lxc-pve-prestart-hook 100 lxc pre-start produced output: error in setup task PVE::LXC::Setup::pre_start_hook

lxc-start 100 20211022224405.994 ERROR    conf - conf.c:run_buffer:316 - Script exited with status 1
lxc-start 100 20211022224405.994 ERROR    start - start.c:lxc_init:816 - Failed to run lxc.hook.pre-start for container "100"
lxc-start 100 20211022224405.994 ERROR    start - start.c:__lxc_start:2007 - Failed to initialize container "100"
lxc-start 100 20211022224405.994 INFO     conf - conf.c:run_script_argv:332 - Executing script "/usr/share/lxcfs/lxc.reboot.hook" for container "100", config section "lxc"
lxc-start 100 20211022224406.496 INFO     conf - conf.c:run_script_argv:332 - Executing script "/usr/share/lxc/hooks/lxc-pve-poststop-hook" for container "100", config section "lxc"
lxc-start 100 20211022224407.255 DEBUG    conf - conf.c:run_buffer:305 - Script exec /usr/share/lxc/hooks/lxc-pve-poststop-hook 100 lxc post-stop produced output: umount: /var/lib/lxc/.pve-staged-mounts/mp3: not mounted.

lxc-start 100 20211022224407.256 DEBUG    conf - conf.c:run_buffer:305 - Script exec /usr/share/lxc/hooks/lxc-pve-poststop-hook 100 lxc post-stop produced output: command 'umount -- /var/lib/lxc/.pve-staged-mounts/mp3' failed: exit code 32

lxc-start 100 20211022224407.283 DEBUG    conf - conf.c:run_buffer:305 - Script exec /usr/share/lxc/hooks/lxc-pve-poststop-hook 100 lxc post-stop produced output: umount: /var/lib/lxc/.pve-staged-mounts/mp4: not mounted.

lxc-start 100 20211022224407.284 DEBUG    conf - conf.c:run_buffer:305 - Script exec /usr/share/lxc/hooks/lxc-pve-poststop-hook 100 lxc post-stop produced output: command 'umount -- /var/lib/lxc/.pve-staged-mounts/mp4' failed: exit code 32

lxc-start 100 20211022224407.311 DEBUG    conf - conf.c:run_buffer:305 - Script exec /usr/share/lxc/hooks/lxc-pve-poststop-hook 100 lxc post-stop produced output: umount: /var/lib/lxc/.pve-staged-mounts/mp2: not mounted.

lxc-start 100 20211022224407.312 DEBUG    conf - conf.c:run_buffer:305 - Script exec /usr/share/lxc/hooks/lxc-pve-poststop-hook 100 lxc post-stop produced output: command 'umount -- /var/lib/lxc/.pve-staged-mounts/mp2' failed: exit code 32

lxc-start 100 20211022224407.545 ERROR    lxc_start - tools/lxc_start.c:main:308 - The container failed to start
lxc-start 100 20211022224407.545 ERROR    lxc_start - tools/lxc_start.c:main:313 - Additional information can be obtained by setting the --logfile and --logpriority options
lxc-start 100 20211022224522.404 INFO     lsm - lsm/lsm.c:lsm_init_static:40 - Initialized LSM security driver AppArmor
lxc-start 100 20211022224522.404 INFO     conf - conf.c:run_script_argv:332 - Executing script "/usr/share/lxc/hooks/lxc-pve-prestart-hook" for container "100", config section "lxc"
lxc-start 100 20211022224522.993 DEBUG    conf - conf.c:run_buffer:305 - Script exec /usr/share/lxc/hooks/lxc-pve-prestart-hook 100 lxc pre-start produced output: closing file '/etc/systemd/network/eth0.network.tmp.10112' failed - No space left on device

lxc-start 100 20211022224522.994 DEBUG    conf - conf.c:run_buffer:305 - Script exec /usr/share/lxc/hooks/lxc-pve-prestart-hook 100 lxc pre-start produced output: error in setup task PVE::LXC::Setup::pre_start_hook

lxc-start 100 20211022224523.288 ERROR    conf - conf.c:run_buffer:316 - Script exited with status 1
lxc-start 100 20211022224523.294 ERROR    start - start.c:lxc_init:816 - Failed to run lxc.hook.pre-start for container "100"
lxc-start 100 20211022224523.300 ERROR    start - start.c:__lxc_start:2007 - Failed to initialize container "100"
lxc-start 100 20211022224523.309 INFO     conf - conf.c:run_script_argv:332 - Executing script "/usr/share/lxcfs/lxc.reboot.hook" for container "100", config section "lxc"
lxc-start 100 20211022224523.504 INFO     conf - conf.c:run_script_argv:332 - Executing script "/usr/share/lxc/hooks/lxc-pve-poststop-hook" for container "100", config section "lxc"
lxc-start 100 20211022224524.360 DEBUG    conf - conf.c:run_buffer:305 - Script exec /usr/share/lxc/hooks/lxc-pve-poststop-hook 100 lxc post-stop produced output: umount: /var/lib/lxc/.pve-staged-mounts/mp3: not mounted.

lxc-start 100 20211022224524.361 DEBUG    conf - conf.c:run_buffer:305 - Script exec /usr/share/lxc/hooks/lxc-pve-poststop-hook 100 lxc post-stop produced output: command 'umount -- /var/lib/lxc/.pve-staged-mounts/mp3' failed: exit code 32

lxc-start 100 20211022224524.388 DEBUG    conf - conf.c:run_buffer:305 - Script exec /usr/share/lxc/hooks/lxc-pve-poststop-hook 100 lxc post-stop produced output: umount: /var/lib/lxc/.pve-staged-mounts/mp4: not mounted.

lxc-start 100 20211022224524.390 DEBUG    conf - conf.c:run_buffer:305 - Script exec /usr/share/lxc/hooks/lxc-pve-poststop-hook 100 lxc post-stop produced output: command 'umount -- /var/lib/lxc/.pve-staged-mounts/mp4' failed: exit code 32

lxc-start 100 20211022224524.416 DEBUG    conf - conf.c:run_buffer:305 - Script exec /usr/share/lxc/hooks/lxc-pve-poststop-hook 100 lxc post-stop produced output: umount: /var/lib/lxc/.pve-staged-mounts/mp2: not mounted.

lxc-start 100 20211022224524.418 DEBUG    conf - conf.c:run_buffer:305 - Script exec /usr/share/lxc/hooks/lxc-pve-poststop-hook 100 lxc post-stop produced output: command 'umount -- /var/lib/lxc/.pve-staged-mounts/mp2' failed: exit code 32

lxc-start 100 20211022224524.646 ERROR    lxc_start - tools/lxc_start.c:main:308 - The container failed to start
lxc-start 100 20211022224524.647 ERROR    lxc_start - tools/lxc_start.c:main:313 - Additional information can be obtained by setting the --logfile and --logpriority options

This line stands out as a possible issue but I have no idea what I did to cause it our how to fix it:
xc-start 100 20211022224405.985 DEBUG conf - conf.c:run_buffer:305 - Script exec /usr/share/lxc/hooks/lxc-pve-prestart-hook 100 lxc pre-start produced output: closing file '/etc/systemd/network/eth0.network.tmp.9856' failed - No space left on device

Any help/ideas would be appreciated.
 
Whats your df -h, vgdisplay and zfs list output on your PVE host?

The LXC fails because it can't mount your bind-mounts mp2, mp3 and mp4.
 
Thanks for your reply. My mp2 and mp3 were mapped to two USB drives I think. I included my config file below. I commented out those two mount points and unplugged my USB drives to simplify things. I'm not aware of having an mp4. I don't see it in my config. How do I find it and remove it? The VG "PURPLE" is the new drive I was trying to add.

I appreciate your help. I hope I did not mess this up too bad. It's been pretty solid for a long time.

root@pve:/# df -h
Code: 
Filesystem                  Size  Used Avail Use% Mounted on
udev                        7.8G     0  7.8G   0% /dev
tmpfs                       1.6G  1.1M  1.6G   1% /run
/dev/mapper/pve-root         55G  8.9G   43G  18% /
tmpfs                       7.8G   46M  7.8G   1% /dev/shm
tmpfs                       5.0M     0  5.0M   0% /run/lock
HGST10TB                    1.9T  128K  1.9T   1% /HGST10TB
HGST10TB/subvol-100-disk-0  9.0T  7.2T  1.9T  80% /HGST10TB/subvol-100-disk-0
/dev/fuse                   128M   20K  128M   1% /etc/pve
//192.168.1.17/Plex         5.2T  1.4T  3.8T  28% /mnt/pve/PLEX-ON-WIN7DVR
tmpfs                       1.6G     0  1.6G   0% /run/user/0

root@pve:/# vgdisplay
Code: 
 --- Volume group ---
  VG Name               PURPLE
  System ID             
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  5
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                0
  Open LV               0
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               <5.46 TiB
  PE Size               4.00 MiB
  Total PE              1430511
  Alloc PE / Size       0 / 0   
  Free  PE / Size       1430511 / <5.46 TiB
  VG UUID               JehIHN-4uW7-K10B-GCcO-gFsS-3h67-kwqOWg
   
  --- Volume group ---
  VG Name               pve
  System ID             
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  906
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                7
  Open LV               2
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               <931.01 GiB
  PE Size               4.00 MiB
  Total PE              238338
  Alloc PE / Size       140818 / 550.07 GiB
  Free  PE / Size       97520 / <380.94 GiB
  VG UUID               Q3OipC-dKFT-Vogz-VWXM-UBfW-wFMz-E090aZ

root@pve:/# zfs list
Code: 
NAME                         USED  AVAIL     REFER  MOUNTPOINT
HGST10TB                    7.13T  1.84T       96K  /HGST10TB
HGST10TB/subvol-100-disk-0  7.13T  1.84T     7.13T  /HGST10TB/subvol-100-disk-0

root@pve:/etc/pve/lxc# cat 100.conf
Code: 
#lxc.hook.autodev%3A bash -c 'for char_dev in $(find /sys/dev/char -regextype sed  -regex ".*/1%3A1" -o -regex ".*/4%3A\([3-9]\|[1-5][0-9]\|6[0-3]\)" -o -regex ".*/4%3A\(6[4-9]\|[7-9][0-9]\|1[0-9][0-9]\|2[0-4][0-9]\|25[0-5]\)" -o -regex ".*/10%3A200" -o -regex ".*/116%3A.*" -o -regex ".*/166%3A.*" -o -regex ".*/180%3A\([0-9]\|1[0-5]\)" -o -regex ".*/188%3A.*" -o -regex ".*/189%3A.*" -o -regex ".*/24[0-2]%3A.*"); do  dev="/dev/$(sed -n "/DEVNAME/ s/^.*=\(.*\)$/\1/p" ${char_dev}/uevent)";  mkdir -p $(dirname ${LXC_ROOTFS_MOUNT}${dev});  for link in $(udevadm info --query=property $dev | sed -n "s/DEVLINKS=//p"); do    mkdir -p ${LXC_ROOTFS_MOUNT}$(dirname $link);    cp -dpR $link ${LXC_ROOTFS_MOUNT}${link};  done;  cp -dpR $dev ${LXC_ROOTFS_MOUNT}${dev};done;'
#lxc.cgroup.devices.allow%3A c 189%3A* rwm
#lxc.mount.entry%3A /dev/bus/usb/003 dev/bus/usb/003 none bind,optional,create=file
#mp2: /dev/disk/by-uuid/1620-C3AD,mp=/mnt/usb8tb
#mp3: /dev/disk/by-uuid/b2b3eb19-d563-4645-85a6-ef6cd73303b3,mp=/mnt/backups
arch: amd64
cores: 3
hostname: ubuntu-plex
memory: 12288
mp0: HGST10TB:subvol-100-disk-0,mp=/media,size=10000G
mp1: local-lvm:vm-100-disk-1,mp=/transcode,size=15G
net0: name=eth0,bridge=vmbr0,firewall=1,gw=192.168.1.1,hwaddr=56:61:DC:8E:7D:2F,ip=192.168.1.19/24,type=veth
onboot: 1
ostype: ubuntu
rootfs: local-lvm:vm-100-disk-0,size=60G
startup: order=1
swap: 2048
lxc.autodev: 1
 
So why is it trying to mount mp2 and mp3 when they are commented out in my config? Is there somewhere else I need to remove them? Also, as I said, there should be no MP4.

I plugged back in my USB drives, uncommented the mp2 and mp3 mounts in the config and tried again. Now all it shows is the "mp4: not mounted" error.

So I think I need to find out where it's trying to load that mount point and remove it. Any help would be appreciated. Nothing in the GUI. Nothing in the config.
 
Last edited:
Hm, I don't see any of your storages running out of space explaining the "No space left on device".

Not sure where the "Script exec /usr/share/lxc/hooks/lxc-pve-poststop-hook 100 lxc post-stop produced output: command 'umount -- /var/lib/lxc/.pve-staged-mounts/mp4' failed: exit code 32" is then coming from if you don't got a mp4. But its also only a debug message and not error message so maybe the mountpoints aren't that important.
 
Does anyone know how I can find where my container is trying to load mp4 and remove it? Thanks!
 
I re-added the mp4 in hopes that I would be able to start the container. No good. I'm really grasping at straws here. Can anyone point me in the right direction as to why I am unable to mount mp4?

Code: 
lxc-start 100 20211023200146.396 INFO     lsm - lsm/lsm.c:lsm_init_static:40 - Initialized LSM security driver AppArmor
lxc-start 100 20211023200146.396 INFO     conf - conf.c:run_script_argv:332 - Executing script "/usr/share/lxc/hooks/lxc-pve-prestart-hook" for container "100", config section "lxc"
lxc-start 100 20211023200147.146 DEBUG    conf - conf.c:run_buffer:305 - Script exec /usr/share/lxc/hooks/lxc-pve-prestart-hook 100 lxc pre-start produced output: failed to get device path

lxc-start 100 20211023200147.154 ERROR    conf - conf.c:run_buffer:316 - Script exited with status 2
lxc-start 100 20211023200147.154 ERROR    start - start.c:lxc_init:816 - Failed to run lxc.hook.pre-start for container "100"
lxc-start 100 20211023200147.154 ERROR    start - start.c:__lxc_start:2007 - Failed to initialize container "100"
lxc-start 100 20211023200147.155 INFO     conf - conf.c:run_script_argv:332 - Executing script "/usr/share/lxcfs/lxc.reboot.hook" for container "100", config section "lxc"
lxc-start 100 20211023200147.656 INFO     conf - conf.c:run_script_argv:332 - Executing script "/usr/share/lxc/hooks/lxc-pve-poststop-hook" for container "100", config section "lxc"
lxc-start 100 20211023200148.185 DEBUG    conf - conf.c:run_buffer:305 - Script exec /usr/share/lxc/hooks/lxc-pve-poststop-hook 100 lxc post-stop produced output: umount: /var/lib/lxc/.pve-staged-mounts/mp4: not mounted.

lxc-start 100 20211023200148.186 DEBUG    conf - conf.c:run_buffer:305 - Script exec /usr/share/lxc/hooks/lxc-pve-poststop-hook 100 lxc post-stop produced output: command 'umount -- /var/lib/lxc/.pve-staged-mounts/mp4' failed: exit code 32

lxc-start 100 20211023200148.212 ERROR    lxc_start - tools/lxc_start.c:main:308 - The container failed to start
lxc-start 100 20211023200148.212 ERROR    lxc_start - tools/lxc_start.c:main:313 - Additional information can be obtained by setting the --logfile and --logpriority options
 
Disregard. I just ended up restoring from backup and starting over. For whatever reason I was able to add the new drive without issue this time. Thanks.
 
I had a similar problem and it turned out one of the drives filled its quota (ran out of disk space). I gave it more space and it started right away.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom