All lxc fails to launch after apt update and upgrade.

PmUserZFS

Well-Known Member
Feb 2, 2018
77
2
48
Task viewer: CT 107 - Start

OutputStatus

Stop
Job for pve-container@107.service failed because the control process exited with error code.
See "systemctl status pve-container@107.service" and "journalctl -xe" for details.
TASK ERROR: command 'systemctl start pve-container@107' failed: exit code 1


i have to restart a old backup of the lxcs then I can launch them.



Code:
root@pve:~# systemctl status pve-container@107.service
● pve-container@107.service - PVE LXC Container: 107
   Loaded: loaded (/lib/systemd/system/pve-container@.service; static; vendor preset: enabled)
   Active: failed (Result: exit-code) since Sat 2019-12-07 22:10:04 CET; 3min 29s ago
     Docs: man:lxc-start
           man:lxc
           man:pct
  Process: 20482 ExecStart=/usr/bin/lxc-start -n 107 (code=exited, status=1/FAILURE)

Dec 07 22:10:04 pve systemd[1]: Starting PVE LXC Container: 107...
Dec 07 22:10:04 pve lxc-start[20482]: lxc-start: 107: lxccontainer.c: wait_on_daemonized_start: 865 No such file or directory - Failed to receive the container state
Dec 07 22:10:04 pve lxc-start[20482]: lxc-start: 107: tools/lxc_start.c: main: 329 The container failed to start
Dec 07 22:10:04 pve lxc-start[20482]: lxc-start: 107: tools/lxc_start.c: main: 332 To get more details, run the container in foreground mode
Dec 07 22:10:04 pve lxc-start[20482]: lxc-start: 107: tools/lxc_start.c: main: 335 Additional information can be obtained by setting the --logfile and --logpriority options
Dec 07 22:10:04 pve systemd[1]: pve-container@107.service: Control process exited, code=exited, status=1/FAILURE
Dec 07 22:10:04 pve systemd[1]: pve-container@107.service: Failed with result 'exit-code'.
Dec 07 22:10:04 pve systemd[1]: Failed to start PVE LXC Container: 107.


Code:
-- A start job for unit pve-container@107.service has begun execution.
--
-- The job identifier is 2440.
Dec 07 22:10:04 pve lxc-start[20482]: lxc-start: 107: lxccontainer.c: wait_on_daemonized_start: 865 No such file or directory - Failed to receive the container state
Dec 07 22:10:04 pve lxc-start[20482]: lxc-start: 107: tools/lxc_start.c: main: 329 The container failed to start
Dec 07 22:10:04 pve lxc-start[20482]: lxc-start: 107: tools/lxc_start.c: main: 332 To get more details, run the container in foreground mode
Dec 07 22:10:04 pve lxc-start[20482]: lxc-start: 107: tools/lxc_start.c: main: 335 Additional information can be obtained by setting the --logfile and --logpriority options
Dec 07 22:10:04 pve systemd[1]: pve-container@107.service: Control process exited, code=exited, status=1/FAILURE
-- Subject: Unit process exited
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- An ExecStart= process belonging to unit pve-container@107.service has exited.
--
-- The process' exit code is 'exited' and its exit status is 1.
Dec 07 22:10:04 pve systemd[1]: pve-container@107.service: Failed with result 'exit-code'.
-- Subject: Unit failed
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- The unit pve-container@107.service has entered the 'failed' state with result 'exit-code'.
Dec 07 22:10:04 pve systemd[1]: Failed to start PVE LXC Container: 107.
-- Subject: A start job for unit pve-container@107.service has failed
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- A start job for unit pve-container@107.service has finished with a failure.
--
-- The job identifier is 2440 and the job result is failed.
Dec 07 22:10:04 pve pvedaemon[20480]: command 'systemctl start pve-container@107' failed: exit code 1
Dec 07 22:10:04 pve kernel: lxc-start[20486]: segfault at 50 ip 00007f6295531f8b sp 00007ffd04d71270 error 4 in liblxc.so.1.6.0[7f62954d8000+8a000]
Dec 07 22:10:04 pve kernel: Code: 9b c0 ff ff 4d 85 ff 0f 85 82 02 00 00 66 90 48 8b 73 50 48 8b bb f8 00 00 00 e8 80 78 fa ff 4c 8b 74 24 10 48 89 de 4c 89 f7 <41> ff 56 50 4c 89 f7 48 89 de 41 ff 56 58 48 8b 83 f8 00 00 00 8b
Dec 07 22:10:04 pve pvedaemon[7584]: <root@pam> end task UPID:pve:00005000:00024F75:5DEC152C:vzstart:107:root@pam: command 'systemctl start pve-container@107' failed: exit code 1
 
Code:
root@pve:~# pveversion
pve-manager/6.1-3/37248ce6 (running kernel: 5.3.10-1-pve)
 
seems as if your container is started before the storage is available (seeing the
865 No such file or directory - Failed to receive the container state)

Have you checked if all your storage is avaiable to PVE ?
 
seems as if your container is started before the storage is available (seeing the
865 No such file or directory - Failed to receive the container state)

Have you checked if all your storage is avaiable to PVE ?

Hi, yes its a ZFS fs on a zpool that is availble.

Can one get more info about this output
ec 07 22:10:04 pve lxc-start[20482]: lxc-start: 107: lxccontainer.c: wait_on_daemonized_start: 865 No such file or directory - Failed to receive the container state
 
I had the same issue following an upgrade from 5.4 (i think?) to 6.1 today.

The issue was the dataset for the container was not mounted. When I tried to mount it manually, it turned out the directory wasn't empty (random "dev" device directory). I removed the directories existing in the mount point and remounted. Problem solved.

Hopefully that's your situation as well.
 
1576440533044.png

root@pve:~# lxc-start -n 107 -F -l DEBUG -o /tmp/lxc-107.log
lxc-start: 107: tools/lxc_start.c: main: 289 No container config specified

Code:
root@pve:~# cat /tmp/lxc-107.log
lxc-start 107 20191215200733.668 ERROR    lxc_start - tools/lxc_start.c:main:289 - No container config specified
lxc-start 107 20191215200925.759 ERROR    lxc_start - tools/lxc_start.c:main:289 - No container config specified
lxc-start 107 20191215200931.391 ERROR    lxc_start - tools/lxc_start.c:main:289 - No container config specified
lxc-start 107 20191215200936.995 ERROR    lxc_start - tools/lxc_start.c:main:289 - No container config specified
 
Last edited:
I have tired to restart backups of the 107 before launching it with debug options.

my nextcloud container which I really have to get up and running I havent touched at all.

root@pve:~# lxc-start -n 116 -l DEBUG -o /tmp/lxc-116.log
lxc-start: 116: lxccontainer.c: wait_on_daemonized_start: 865 No such file or directory - Failed to receive the container state
lxc-start: 116: tools/lxc_start.c: main: 329 The container failed to start
lxc-start: 116: tools/lxc_start.c: main: 332 To get more details, run the container in foreground mode
lxc-start: 116: tools/lxc_start.c: main: 335 Additional information can be obtained by setting the --logfile and --logpriority options

Code:
root@pve:~# cat /tmp/lxc-116.log
lxc-start 116 20191215201549.764 INFO     lxccontainer - lxccontainer.c:do_lxcapi_start:994 - Set process title to [lxc monitor] /var/lib/lxc 116
lxc-start 116 20191215201549.764 INFO     lsm - lsm/lsm.c:lsm_init:50 - LSM security driver AppArmor
lxc-start 116 20191215201549.765 INFO     seccomp - seccomp.c:parse_config_v2:789 - Processing "reject_force_umount  # comment this to allow umount -f;  not recomme                          nded"
lxc-start 116 20191215201549.765 INFO     seccomp - seccomp.c:do_resolve_add_rule:535 - Set seccomp rule to reject force umounts
lxc-start 116 20191215201549.765 INFO     seccomp - seccomp.c:parse_config_v2:975 - Added native rule for arch 0 for reject_force_umount action 0(kill)
lxc-start 116 20191215201549.765 INFO     seccomp - seccomp.c:do_resolve_add_rule:535 - Set seccomp rule to reject force umounts
lxc-start 116 20191215201549.765 INFO     seccomp - seccomp.c:parse_config_v2:984 - Added compat rule for arch 1073741827 for reject_force_umount action 0(kill)
lxc-start 116 20191215201549.765 INFO     seccomp - seccomp.c:do_resolve_add_rule:535 - Set seccomp rule to reject force umounts
lxc-start 116 20191215201549.765 INFO     seccomp - seccomp.c:parse_config_v2:994 - Added compat rule for arch 1073741886 for reject_force_umount action 0(kill)
lxc-start 116 20191215201549.765 INFO     seccomp - seccomp.c:do_resolve_add_rule:535 - Set seccomp rule to reject force umounts
lxc-start 116 20191215201549.765 INFO     seccomp - seccomp.c:parse_config_v2:1004 - Added native rule for arch -1073741762 for reject_force_umount action 0(kill)
lxc-start 116 20191215201549.765 INFO     seccomp - seccomp.c:parse_config_v2:789 - Processing "[all]"
lxc-start 116 20191215201549.765 INFO     seccomp - seccomp.c:parse_config_v2:789 - Processing "kexec_load errno 1"
lxc-start 116 20191215201549.765 INFO     seccomp - seccomp.c:parse_config_v2:975 - Added native rule for arch 0 for kexec_load action 327681(errno)
lxc-start 116 20191215201549.765 INFO     seccomp - seccomp.c:parse_config_v2:984 - Added compat rule for arch 1073741827 for kexec_load action 327681(errno)
lxc-start 116 20191215201549.765 INFO     seccomp - seccomp.c:parse_config_v2:994 - Added compat rule for arch 1073741886 for kexec_load action 327681(errno)
lxc-start 116 20191215201549.765 INFO     seccomp - seccomp.c:parse_config_v2:1004 - Added native rule for arch -1073741762 for kexec_load action 327681(errno)
lxc-start 116 20191215201549.765 INFO     seccomp - seccomp.c:parse_config_v2:789 - Processing "open_by_handle_at errno 1"
lxc-start 116 20191215201549.765 INFO     seccomp - seccomp.c:parse_config_v2:975 - Added native rule for arch 0 for open_by_handle_at action 327681(errno)
lxc-start 116 20191215201549.765 INFO     seccomp - seccomp.c:parse_config_v2:984 - Added compat rule for arch 1073741827 for open_by_handle_at action 327681(errno)
lxc-start 116 20191215201549.765 INFO     seccomp - seccomp.c:parse_config_v2:994 - Added compat rule for arch 1073741886 for open_by_handle_at action 327681(errno)
lxc-start 116 20191215201549.765 INFO     seccomp - seccomp.c:parse_config_v2:1004 - Added native rule for arch -1073741762 for open_by_handle_at action 327681(errn                          o)
lxc-start 116 20191215201549.765 INFO     seccomp - seccomp.c:parse_config_v2:789 - Processing "init_module errno 1"
lxc-start 116 20191215201549.765 INFO     seccomp - seccomp.c:parse_config_v2:975 - Added native rule for arch 0 for init_module action 327681(errno)
lxc-start 116 20191215201549.765 INFO     seccomp - seccomp.c:parse_config_v2:984 - Added compat rule for arch 1073741827 for init_module action 327681(errno)
lxc-start 116 20191215201549.765 INFO     seccomp - seccomp.c:parse_config_v2:994 - Added compat rule for arch 1073741886 for init_module action 327681(errno)
lxc-start 116 20191215201549.765 INFO     seccomp - seccomp.c:parse_config_v2:1004 - Added native rule for arch -1073741762 for init_module action 327681(errno)
lxc-start 116 20191215201549.765 INFO     seccomp - seccomp.c:parse_config_v2:789 - Processing "finit_module errno 1"
lxc-start 116 20191215201549.765 INFO     seccomp - seccomp.c:parse_config_v2:975 - Added native rule for arch 0 for finit_module action 327681(errno)
lxc-start 116 20191215201549.765 INFO     seccomp - seccomp.c:parse_config_v2:984 - Added compat rule for arch 1073741827 for finit_module action 327681(errno)
lxc-start 116 20191215201549.765 INFO     seccomp - seccomp.c:parse_config_v2:994 - Added compat rule for arch 1073741886 for finit_module action 327681(errno)
lxc-start 116 20191215201549.765 INFO     seccomp - seccomp.c:parse_config_v2:1004 - Added native rule for arch -1073741762 for finit_module action 327681(errno)
lxc-start 116 20191215201549.765 INFO     seccomp - seccomp.c:parse_config_v2:789 - Processing "delete_module errno 1"
lxc-start 116 20191215201549.765 INFO     seccomp - seccomp.c:parse_config_v2:975 - Added native rule for arch 0 for delete_module action 327681(errno)
lxc-start 116 20191215201549.765 INFO     seccomp - seccomp.c:parse_config_v2:984 - Added compat rule for arch 1073741827 for delete_module action 327681(errno)
lxc-start 116 20191215201549.765 INFO     seccomp - seccomp.c:parse_config_v2:994 - Added compat rule for arch 1073741886 for delete_module action 327681(errno)
lxc-start 116 20191215201549.765 INFO     seccomp - seccomp.c:parse_config_v2:1004 - Added native rule for arch -1073741762 for delete_module action 327681(errno)
lxc-start 116 20191215201549.765 INFO     seccomp - seccomp.c:parse_config_v2:1008 - Merging compat seccomp contexts into main context
lxc-start 116 20191215201549.765 INFO     conf - conf.c:run_script_argv:372 - Executing script "/usr/share/lxc/hooks/lxc-pve-prestart-hook" for container "116", con                          fig section "lxc"
lxc-start 116 20191215201550.194 DEBUG    conf - conf.c:run_buffer:340 - Script exec /usr/share/lxc/hooks/lxc-pve-prestart-hook 116 lxc pre-start produced output: u                          nable to detect OS distribution

lxc-start 116 20191215201550.201 ERROR    conf - conf.c:run_buffer:352 - Script exited with status 2
lxc-start 116 20191215201550.201 ERROR    start - start.c:lxc_init:897 - Failed to run lxc.hook.pre-start for container "116"
lxc-start 116 20191215201550.202 ERROR    start - start.c:__lxc_start:2032 - Failed to initialize container "116"
lxc-start 116 20191215201550.202 DEBUG    lxccontainer - lxccontainer.c:wait_on_daemonized_start:862 - First child 26563 exited
lxc-start 116 20191215201550.202 ERROR    lxccontainer - lxccontainer.c:wait_on_daemonized_start:865 - No such file or directory - Failed to receive the container s                          tate
lxc-start 116 20191215201550.202 ERROR    lxc_start - tools/lxc_start.c:main:329 - The container failed to start
lxc-start 116 20191215201550.202 ERROR    lxc_start - tools/lxc_start.c:main:332 - To get more details, run the container in foreground mode
lxc-start 116 20191215201550.202 ERROR    lxc_start - tools/lxc_start.c:main:335 - Additional information can be obtained by setting the --logfile and --logpriority                           options
 
is your zfs volume mounted?
 
ofc its mounted, I take that for granted..... oh my . it's not ... :/

thnx for all the help :)
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!