Container does not start after a restore

Huub Reemers

Active Member
Mar 10, 2018
10
0
41
70
After a backup in the gui of a CT (100) to a shared nfs drive I restored this backup to another Node pve3.
It seems to be ok but afterwards this new CT (101) would not start.
It gives the following errors.

Job for pve-container@101.service failed because the control process exited with error code.
See "systemctl status pve-container@101.service" and "journalctl -xe" for details.
TASK ERROR: command 'systemctl start pve-container@101' failed: exit code 1

I am new in the Proxmox world, Do you have a solution?

The smiley means ": p" without teh spaces.

systemctl status pve-container@101.service

root@pve3:~# systemctl status pve-container@101.service
pve-container@101.service - PVE LXC Container: 101
Loaded: loaded (/lib/systemd/system/pve-container@.service; static; vendor preset: enabled)
Active: failed (Result: exit-code) since Mon 2018-03-12 13:52:47 CET; 8min ago
Docs: man:lxc-start
man:lxc
man:pct
Process: 23625 ExecStart=/usr/bin/lxc-start -n 101 (code=exited, status=1/FAILURE)

Mar 12 13:52:47 pve3 systemd[1]: Starting PVE LXC Container: 101...
Mar 12 13:52:47 pve3 lxc-start[23625]: lxc-start: 101: lxccontainer.c: wait_on_daemonized_start: 751 No such file or directory - Failed to rec
Mar 12 13:52:47 pve3 lxc-start[23625]: lxc-start: 101: tools/lxc_start.c: main: 371 The container failed to start.
Mar 12 13:52:47 pve3 lxc-start[23625]: lxc-start: 101: tools/lxc_start.c: main: 373 To get more details, run the container in foreground mode.
Mar 12 13:52:47 pve3 lxc-start[23625]: lxc-start: 101: tools/lxc_start.c: main: 375 Additional information can be obtained by setting the --lo
Mar 12 13:52:47 pve3 systemd[1]: pve-container@101.service: Control process exited, code=exited status=1
Mar 12 13:52:47 pve3 systemd[1]: pve-container@101.service: Killing process 23639 (lxc-start) with signal SIGKILL.
Mar 12 13:52:47 pve3 systemd[1]: Failed to start PVE LXC Container: 101.
Mar 12 13:52:47 pve3 systemd[1]: pve-container@101.service: Unit entered failed state.
Mar 12 13:52:47 pve3 systemd[1]: pve-container@101.service: Failed with result 'exit-code'.
------------------------------------------


root@pve3:~# journalctl -xe
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- Unit pvesr.service has begun starting up.
Mar 12 14:15:01 pve3 systemd[1]: Started Proxmox VE replication runner.
-- Subject: Unit pvesr.service has finished start-up
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- Unit pvesr.service has finished starting up.
--
-- The start-up result is done.
Mar 12 14:15:10 pve3 pvedaemon[11553]: starting CT 101: UPID:pve3:00002D21:0000BECD:5AA67D5E:vzstart:101:root@pam:
Mar 12 14:15:10 pve3 pvedaemon[3201]: <root@pam> starting task UPID:pve3:00002D21:0000BECD:5AA67D5E:vzstart:101:root@pam:
Mar 12 14:15:10 pve3 systemd[1]: Starting PVE LXC Container: 101...
-- Subject: Unit pve-container@101.service has begun start-up
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- Unit pve-container@101.service has begun starting up.
Mar 12 14:15:10 pve3 audit[11558]: AVC apparmor="DENIED" operation="mount" info="failed mntpnt match" error=-13 profile="/usr/bin/lxc-start" n
Mar 12 14:15:10 pve3 lxc-start[11556]: lxc-start: 101: lxccontainer.c: wait_on_daemonized_start: 751 No such file or directory - Failed to rec
Mar 12 14:15:10 pve3 lxc-start[11556]: lxc-start: 101: tools/lxc_start.c: main: 371 The container failed to start.
Mar 12 14:15:10 pve3 lxc-start[11556]: lxc-start: 101: tools/lxc_start.c: main: 373 To get more details, run the container in foreground mode.
Mar 12 14:15:10 pve3 lxc-start[11556]: lxc-start: 101: tools/lxc_start.c: main: 375 Additional information can be obtained by setting the --lo
Mar 12 14:15:10 pve3 systemd[1]: pve-container@101.service: Control process exited, code=exited status=1
Mar 12 14:15:10 pve3 systemd[1]: pve-container@101.service: Killing process 11558 (lxc-start) with signal SIGKILL.
Mar 12 14:15:10 pve3 kernel: audit: type=1400 audit(1520860510.743:14): apparmor="DENIED" operation="mount" info="failed mntpnt match" error=-
Mar 12 14:15:10 pve3 systemd[1]: Failed to start PVE LXC Container: 101.
-- Subject: Unit pve-container@101.service has failed
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- Unit pve-container@101.service has failed.
--
-- The result is failed.
Mar 12 14:15:10 pve3 systemd[1]: pve-container@101.service: Unit entered failed state.
Mar 12 14:15:10 pve3 systemd[1]: pve-container@101.service: Failed with result 'exit-code'.
Mar 12 14:15:10 pve3 pvedaemon[11553]: command 'systemctl start pve-container@101' failed: exit code 1
Mar 12 14:15:10 pve3 pvedaemon[3201]: <root@pam> end task UPID:pve3:00002D21:0000BECD:5AA67D5E:vzstart:101:root@pam: command 'systemctl start
 
Last edited:
Strange ... the " : p " without de spaces, appears as a smiley, sorry for this. Can't change it.
 
Mar 12 14:15:10 pve3 audit[11558]: AVC apparmor="DENIED" operation="mount" info="failed mntpnt match" error=-13
...
Mar 12 14:15:10 pve3 lxc-start[11556]: lxc-start: 101: lxccontainer.c: wait_on_daemonized_start: 751 No such file or directory - Failed to rec
These lines makes me think the argument for "rootfs" is incorrect. Edit /etc/pve/lxc/CONTAINERID.conf and confirm that rootfs makes sense in the new environment. Also make sure there is a /var/lib/lxc/CONTAINERID directory.

If that looks ok, use this command to collect more detailed startup information:
Code:
lxc-start -o /tmp/startup.log -l debug -n CONTAINERID

and paste the contents o /tmp/startup.log back here.
 
  • Like
Reactions: Huub Reemers
These lines makes me think the argument for "rootfs" is incorrect. Edit /etc/pve/lxc/CONTAINERID.conf and confirm that rootfs makes sense in the new environment. Also make sure there is a /var/lib/lxc/CONTAINERID directory.

If that looks ok, use this command to collect more detailed startup information:
Code:
lxc-start -o /tmp/startup.log -l debug -n CONTAINERID

and paste the contents o /tmp/startup.log back here.

Thanks voor your help

/etc/pve/lxc/101.conf
------------------------

root@pve3:~# cat /etc/pve/lxc/101.conf

#CentOs7DC1.steendrukmuseum.lan
#192.168.2.11
arch: amd64
cores: 2
hostname: CentOs7DC1
memory: 1024
nameserver: 8.8.8.8 8.8.4.4
net0: name=eth0,bridge=vmbr0,gw=192.168.2.254,hwaddr=7E:C5:14:A4:1B:0F,ip=192.168.2.11/24,type=veth
ostype: centos
rootfs: local-zfs:subvol-101-disk-1,acl=1,size=32G
searchdomain: steendrukmuseum.lan
swap: 1024
root@pve3:~#
----------------------------

Looks ok.

lxc-start -o /tmp/startup.log -l debug -n 101

lxc-start: 101: lxccontainer.c: wait_on_daemonized_start: 751 No such file or directory - Failed to receive the container state
lxc-start: 101: tools/lxc_start.c: main: 371 The container failed to start.
lxc-start: 101: tools/lxc_start.c: main: 373 To get more details, run the container in foreground mode.
lxc-start: 101: tools/lxc_start.c: main: 375 Additional information can be obtained by setting the --logfile and --logpriority options.

cat /tmp/startup.log

lxc-start CONTAINERID 20180313084844.722 INFO lxc_start_ui - tools/lxc_start.c:main:280 - using rcfile /var/lib/lxc/CONTAINERID/config
lxc-start CONTAINERID 20180313084844.722 ERROR lxc_start_ui - tools/lxc_start.c:main:322 - Executing '/sbin/init' with no configuration file may crash the host
lxc-start 101 20180313084859.547 INFO lxc_start_ui - tools/lxc_start.c:main:280 - using rcfile /var/lib/lxc/101/config
lxc-start 101 20180313084859.548 INFO lxc_container - lxccontainer.c:do_lxcapi_start:883 - Attempting to set proc title to [lxc monitor] /var/lib/lxc 101
lxc-start 101 20180313084859.549 ERROR lxc_container - lxccontainer.c:do_lxcapi_start:984 - Permission denied - Failed to make / rslave at startup
lxc-start 101 20180313084859.549 ERROR lxc_container - lxccontainer.c:wait_on_daemonized_start:751 - No such file or directory - Failed to receive the container state
lxc-start 101 20180313084859.549 ERROR lxc_start_ui - tools/lxc_start.c:main:371 - The container failed to start.
lxc-start 101 20180313084859.549 ERROR lxc_start_ui - tools/lxc_start.c:main:373 - To get more details, run the container in foreground mode.
lxc-start 101 20180313084859.549 ERROR lxc_start_ui - tools/lxc_start.c:main:375 - Additional information can be obtained by setting the --logfile and --logpriority options.
lxc-start 101 20180313084919.947 INFO lxc_start_ui - tools/lxc_start.c:main:280 - using rcfile /var/lib/lxc/101/config
lxc-start 101 20180313084919.948 INFO lxc_container - lxccontainer.c:do_lxcapi_start:883 - Attempting to set proc title to [lxc monitor] /var/lib/lxc 101
lxc-start 101 20180313084919.949 ERROR lxc_container - lxccontainer.c:do_lxcapi_start:984 - Permission denied - Failed to make / rslave at startup
lxc-start 101 20180313084919.949 ERROR lxc_container - lxccontainer.c:wait_on_daemonized_start:751 - No such file or directory - Failed to receive the container state
lxc-start 101 20180313084919.949 ERROR lxc_start_ui - tools/lxc_start.c:main:371 - The container failed to start.
lxc-start 101 20180313084919.949 ERROR lxc_start_ui - tools/lxc_start.c:main:373 - To get more details, run the container in foreground mode.
lxc-start 101 20180313084919.949 ERROR lxc_start_ui - tools/lxc_start.c:main:375 - Additional information can be obtained by setting the --logfile and --logpriority options.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!