Containers don't start on bootup

[Dataforce]

I have a strange problem with one of my hosts, after it reboots none of the contains auto-start properly.

The task log shows that proxmox attempts it - There is a "Start all VMs and Containers" task, and all the VMs start fine.

The containers however all fail:

lxc-start: parse.c: lxc_file_for_each_line: 57 Failed to parse config: lxc.include = /usr/share/lxc/config/common.conf.d/

lxc-start: parse.c: lxc_file_for_each_line: 57 Failed to parse config: lxc.include = /usr/share/lxc/config/common.conf

lxc-start: parse.c: lxc_file_for_each_line: 57 Failed to parse config: lxc.include = /usr/share/lxc/config/ubuntu.common.conf

lxc-start: tools/lxc_start.c: main: 284 Failed to create lxc_container
TASK ERROR: command 'lxc-start -n 125250' failed: exit code 1

The containers all start fine if I log into the UI and manually start them.

root@neo-pve:~# pveversion 
pve-manager/4.4-12/e71b7a74 (running kernel: 4.4.35-2-pve)
root@neo-pve:~#

 

[dcsapak]

what is the content of

/usr/share/lxc/config/common.conf
and
/usr/share/lxc/config/ubuntu.common.conf
?

 

[Dataforce]

root@neo-pve:~# cat /usr/share/lxc/config/common.conf
# Default configuration shared by all containers

# Setup the LXC devices in /dev/lxc/
lxc.devttydir = lxc

# Allow for 1024 pseudo terminals
lxc.pts = 1024

# Setup 4 tty devices
lxc.tty = 4

# Drop some harmful capabilities
lxc.cap.drop = mac_admin mac_override sys_time sys_module sys_rawio

# Set the pivot directory
lxc.pivotdir = lxc_putold

# Ensure hostname is changed on clone
lxc.hook.clone = /usr/share/lxc/hooks/clonehostname

# CGroup whitelist
lxc.cgroup.devices.deny = a
## Allow any mknod (but not reading/writing the node)
lxc.cgroup.devices.allow = c *:* m
lxc.cgroup.devices.allow = b *:* m
## Allow specific devices
### /dev/null
lxc.cgroup.devices.allow = c 1:3 rwm
### /dev/zero
lxc.cgroup.devices.allow = c 1:5 rwm
### /dev/full
lxc.cgroup.devices.allow = c 1:7 rwm
### /dev/tty
lxc.cgroup.devices.allow = c 5:0 rwm
### /dev/console
lxc.cgroup.devices.allow = c 5:1 rwm
### /dev/ptmx
lxc.cgroup.devices.allow = c 5:2 rwm
### /dev/random
lxc.cgroup.devices.allow = c 1:8 rwm
### /dev/urandom
lxc.cgroup.devices.allow = c 1:9 rwm
### /dev/pts/*
lxc.cgroup.devices.allow = c 136:* rwm
### fuse
lxc.cgroup.devices.allow = c 10:229 rwm

# Setup the default mounts
lxc.mount.auto = cgroup:mixed proc:mixed sys:mixed
lxc.mount.entry = /sys/fs/fuse/connections sys/fs/fuse/connections none bind,optional 0 0

# Blacklist some syscalls which are not safe in privileged
# containers
lxc.seccomp = /usr/share/lxc/config/common.seccomp

# Lastly, include all the configs from /usr/share/lxc/config/common.conf.d/
lxc.include = /usr/share/lxc/config/common.conf.d/
root@neo-pve:~#

root@neo-pve:~# cat /usr/share/lxc/config/ubuntu.common.conf
# This derives from the global common config
lxc.include = /usr/share/lxc/config/common.conf

# Default mount entries
lxc.mount.entry = /sys/kernel/debug sys/kernel/debug none bind,optional 0 0
lxc.mount.entry = /sys/kernel/security sys/kernel/security none bind,optional 0 0
lxc.mount.entry = /sys/fs/pstore sys/fs/pstore none bind,optional 0 0
lxc.mount.entry = mqueue dev/mqueue mqueue rw,relatime,create=dir,optional 0 0

# When using LXC with apparmor, the container will be confined by default.
# If you wish for it to instead run unconfined, copy the following line
# (uncommented) to the container's configuration file.
#lxc.aa_profile = unconfined

# Uncomment the following line to autodetect squid-deb-proxy configuration on the
# host and forward it to the guest at start time.
#lxc.hook.pre-start = /usr/share/lxc/hooks/squid-deb-proxy-client

# If you wish to allow mounting block filesystems, then use the following
# line instead, and make sure to grant access to the block device and/or loop
# devices below in lxc.cgroup.devices.allow.
#lxc.aa_profile = lxc-container-default-with-mounting

# Extra cgroup device access
## rtc
lxc.cgroup.devices.allow = c 254:0 rm
## tun
lxc.cgroup.devices.allow = c 10:200 rwm
## hpet
lxc.cgroup.devices.allow = c 10:228 rwm
## kvm
lxc.cgroup.devices.allow = c 10:232 rwm
## To use loop devices, copy the following line to the container's
## configuration file (uncommented).
#lxc.cgroup.devices.allow = b 7:* rwm
root@neo-pve:~#

This is the same as other proxmox nodes I have, and hasn't been altered.