Container does not start after a restore

Huub Reemers

Active Member
Mar 10, 2018
10
0
41
71
After a backup in the gui of a CT (100) to a shared nfs drive I restored this backup to another Node pve3.
It seems to be ok but afterwards this new CT (101) would not start.
It gives the following errors.

Job for pve-container@101.service failed because the control process exited with error code.
See "systemctl status pve-container@101.service" and "journalctl -xe" for details.
TASK ERROR: command 'systemctl start pve-container@101' failed: exit code 1

I am new in the Proxmox world, Do you have a solution?

The smiley means ": p" without teh spaces.

systemctl status pve-container@101.service

root@pve3:~# systemctl status pve-container@101.service
pve-container@101.service - PVE LXC Container: 101
Loaded: loaded (/lib/systemd/system/pve-container@.service; static; vendor preset: enabled)
Active: failed (Result: exit-code) since Mon 2018-03-12 13:52:47 CET; 8min ago
Docs: man:lxc-start
man:lxc
man:pct
Process: 23625 ExecStart=/usr/bin/lxc-start -n 101 (code=exited, status=1/FAILURE)

Mar 12 13:52:47 pve3 systemd[1]: Starting PVE LXC Container: 101...
Mar 12 13:52:47 pve3 lxc-start[23625]: lxc-start: 101: lxccontainer.c: wait_on_daemonized_start: 751 No such file or directory - Failed to rec
Mar 12 13:52:47 pve3 lxc-start[23625]: lxc-start: 101: tools/lxc_start.c: main: 371 The container failed to start.
Mar 12 13:52:47 pve3 lxc-start[23625]: lxc-start: 101: tools/lxc_start.c: main: 373 To get more details, run the container in foreground mode.
Mar 12 13:52:47 pve3 lxc-start[23625]: lxc-start: 101: tools/lxc_start.c: main: 375 Additional information can be obtained by setting the --lo
Mar 12 13:52:47 pve3 systemd[1]: pve-container@101.service: Control process exited, code=exited status=1
Mar 12 13:52:47 pve3 systemd[1]: pve-container@101.service: Killing process 23639 (lxc-start) with signal SIGKILL.
Mar 12 13:52:47 pve3 systemd[1]: Failed to start PVE LXC Container: 101.
Mar 12 13:52:47 pve3 systemd[1]: pve-container@101.service: Unit entered failed state.
Mar 12 13:52:47 pve3 systemd[1]: pve-container@101.service: Failed with result 'exit-code'.
------------------------------------------


root@pve3:~# journalctl -xe
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- Unit pvesr.service has begun starting up.
Mar 12 14:15:01 pve3 systemd[1]: Started Proxmox VE replication runner.
-- Subject: Unit pvesr.service has finished start-up
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- Unit pvesr.service has finished starting up.
--
-- The start-up result is done.
Mar 12 14:15:10 pve3 pvedaemon[11553]: starting CT 101: UPID:pve3:00002D21:0000BECD:5AA67D5E:vzstart:101:root@pam:
Mar 12 14:15:10 pve3 pvedaemon[3201]: <root@pam> starting task UPID:pve3:00002D21:0000BECD:5AA67D5E:vzstart:101:root@pam:
Mar 12 14:15:10 pve3 systemd[1]: Starting PVE LXC Container: 101...
-- Subject: Unit pve-container@101.service has begun start-up
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- Unit pve-container@101.service has begun starting up.
Mar 12 14:15:10 pve3 audit[11558]: AVC apparmor="DENIED" operation="mount" info="failed mntpnt match" error=-13 profile="/usr/bin/lxc-start" n
Mar 12 14:15:10 pve3 lxc-start[11556]: lxc-start: 101: lxccontainer.c: wait_on_daemonized_start: 751 No such file or directory - Failed to rec
Mar 12 14:15:10 pve3 lxc-start[11556]: lxc-start: 101: tools/lxc_start.c: main: 371 The container failed to start.
Mar 12 14:15:10 pve3 lxc-start[11556]: lxc-start: 101: tools/lxc_start.c: main: 373 To get more details, run the container in foreground mode.
Mar 12 14:15:10 pve3 lxc-start[11556]: lxc-start: 101: tools/lxc_start.c: main: 375 Additional information can be obtained by setting the --lo
Mar 12 14:15:10 pve3 systemd[1]: pve-container@101.service: Control process exited, code=exited status=1
Mar 12 14:15:10 pve3 systemd[1]: pve-container@101.service: Killing process 11558 (lxc-start) with signal SIGKILL.
Mar 12 14:15:10 pve3 kernel: audit: type=1400 audit(1520860510.743:14): apparmor="DENIED" operation="mount" info="failed mntpnt match" error=-
Mar 12 14:15:10 pve3 systemd[1]: Failed to start PVE LXC Container: 101.
-- Subject: Unit pve-container@101.service has failed
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- Unit pve-container@101.service has failed.
--
-- The result is failed.
Mar 12 14:15:10 pve3 systemd[1]: pve-container@101.service: Unit entered failed state.
Mar 12 14:15:10 pve3 systemd[1]: pve-container@101.service: Failed with result 'exit-code'.
Mar 12 14:15:10 pve3 pvedaemon[11553]: command 'systemctl start pve-container@101' failed: exit code 1
Mar 12 14:15:10 pve3 pvedaemon[3201]: <root@pam> end task UPID:pve3:00002D21:0000BECD:5AA67D5E:vzstart:101:root@pam: command 'systemctl start
 
Last edited:
Mar 12 14:15:10 pve3 audit[11558]: AVC apparmor="DENIED" operation="mount" info="failed mntpnt match" error=-13
...
Mar 12 14:15:10 pve3 lxc-start[11556]: lxc-start: 101: lxccontainer.c: wait_on_daemonized_start: 751 No such file or directory - Failed to rec
These lines makes me think the argument for "rootfs" is incorrect. Edit /etc/pve/lxc/CONTAINERID.conf and confirm that rootfs makes sense in the new environment. Also make sure there is a /var/lib/lxc/CONTAINERID directory.

If that looks ok, use this command to collect more detailed startup information:
Code:
lxc-start -o /tmp/startup.log -l debug -n CONTAINERID

and paste the contents o /tmp/startup.log back here.
 
  • Like
Reactions: Huub Reemers
These lines makes me think the argument for "rootfs" is incorrect. Edit /etc/pve/lxc/CONTAINERID.conf and confirm that rootfs makes sense in the new environment. Also make sure there is a /var/lib/lxc/CONTAINERID directory.

If that looks ok, use this command to collect more detailed startup information:
Code:
lxc-start -o /tmp/startup.log -l debug -n CONTAINERID

and paste the contents o /tmp/startup.log back here.

Thanks voor your help

/etc/pve/lxc/101.conf
------------------------

root@pve3:~# cat /etc/pve/lxc/101.conf

#CentOs7DC1.steendrukmuseum.lan
#192.168.2.11
arch: amd64
cores: 2
hostname: CentOs7DC1
memory: 1024
nameserver: 8.8.8.8 8.8.4.4
net0: name=eth0,bridge=vmbr0,gw=192.168.2.254,hwaddr=7E:C5:14:A4:1B:0F,ip=192.168.2.11/24,type=veth
ostype: centos
rootfs: local-zfs:subvol-101-disk-1,acl=1,size=32G
searchdomain: steendrukmuseum.lan
swap: 1024
root@pve3:~#
----------------------------

Looks ok.

lxc-start -o /tmp/startup.log -l debug -n 101

lxc-start: 101: lxccontainer.c: wait_on_daemonized_start: 751 No such file or directory - Failed to receive the container state
lxc-start: 101: tools/lxc_start.c: main: 371 The container failed to start.
lxc-start: 101: tools/lxc_start.c: main: 373 To get more details, run the container in foreground mode.
lxc-start: 101: tools/lxc_start.c: main: 375 Additional information can be obtained by setting the --logfile and --logpriority options.

cat /tmp/startup.log

lxc-start CONTAINERID 20180313084844.722 INFO lxc_start_ui - tools/lxc_start.c:main:280 - using rcfile /var/lib/lxc/CONTAINERID/config
lxc-start CONTAINERID 20180313084844.722 ERROR lxc_start_ui - tools/lxc_start.c:main:322 - Executing '/sbin/init' with no configuration file may crash the host
lxc-start 101 20180313084859.547 INFO lxc_start_ui - tools/lxc_start.c:main:280 - using rcfile /var/lib/lxc/101/config
lxc-start 101 20180313084859.548 INFO lxc_container - lxccontainer.c:do_lxcapi_start:883 - Attempting to set proc title to [lxc monitor] /var/lib/lxc 101
lxc-start 101 20180313084859.549 ERROR lxc_container - lxccontainer.c:do_lxcapi_start:984 - Permission denied - Failed to make / rslave at startup
lxc-start 101 20180313084859.549 ERROR lxc_container - lxccontainer.c:wait_on_daemonized_start:751 - No such file or directory - Failed to receive the container state
lxc-start 101 20180313084859.549 ERROR lxc_start_ui - tools/lxc_start.c:main:371 - The container failed to start.
lxc-start 101 20180313084859.549 ERROR lxc_start_ui - tools/lxc_start.c:main:373 - To get more details, run the container in foreground mode.
lxc-start 101 20180313084859.549 ERROR lxc_start_ui - tools/lxc_start.c:main:375 - Additional information can be obtained by setting the --logfile and --logpriority options.
lxc-start 101 20180313084919.947 INFO lxc_start_ui - tools/lxc_start.c:main:280 - using rcfile /var/lib/lxc/101/config
lxc-start 101 20180313084919.948 INFO lxc_container - lxccontainer.c:do_lxcapi_start:883 - Attempting to set proc title to [lxc monitor] /var/lib/lxc 101
lxc-start 101 20180313084919.949 ERROR lxc_container - lxccontainer.c:do_lxcapi_start:984 - Permission denied - Failed to make / rslave at startup
lxc-start 101 20180313084919.949 ERROR lxc_container - lxccontainer.c:wait_on_daemonized_start:751 - No such file or directory - Failed to receive the container state
lxc-start 101 20180313084919.949 ERROR lxc_start_ui - tools/lxc_start.c:main:371 - The container failed to start.
lxc-start 101 20180313084919.949 ERROR lxc_start_ui - tools/lxc_start.c:main:373 - To get more details, run the container in foreground mode.
lxc-start 101 20180313084919.949 ERROR lxc_start_ui - tools/lxc_start.c:main:375 - Additional information can be obtained by setting the --logfile and --logpriority options.