suddenly my containers won't start anymore

[amachils]

I have 2 Alpine LXC setup, which worked without fail. After doing a apk update/upgrade, I decided to do a reboot and now both won't start anymore. In dmesg I can see: "audit: type=1400 audit(1469779508.387:3): apparmor="DENIED" operation="change_profile" info="label not found" error=-2 profile="/usr/bin/lxc-start" name="lxc-container-default" pid=6229 comm="lxc
-start"

[edit] I can safely say that the Alpine upgrade had nothing to do with it. There is also a (dormant) Debian container, which hasn't been touched and also won't boot.

[edit2] on the server where LXC doesn't work anymore, this is in the /sys/kernel/security/apparmor/profiles file:
/usr/bin/lxc-start (enforce)
but there should be this:
/usr/bin/lxc-start (enforce)
lxc-container-default-with-nesting (enforce)
lxc-container-default-with-mounting (enforce)
lxc-container-default (enforce)

Where did the rest go?

[edit3] Apparently apparmor didn't start correctly, due to a error in a lxc config file, but on a line that exceeded the number of lines in that file. This config file starts other configs, and somehow system logging logs all this as one..... Isn't systemd logging great?


Regards,

Angelo

 

[wbumiller]

Is your system up to date?

# pveversion -v

And what's the output of:

# apparmor_parser -r -W -T /etc/apparmor.d/lxc-containers

 

[Frank Abbas]

I am having a similar issue. I performed an upgrade now my containers won't start
When I try to start any container I get
lxc-start: lxc_start.c: main: 344 The container failed to start.
lxc-start: lxc_start.c: main: 346 To get more details, run the container in foreground mode.
----------------------------------------------------------------------------------

- I also tried to start the container in foreground Didn't help.
# lxc-start --name 101 --foreground
-------------------------------------------------------------
my pveversion :

pve-kernel-4.2.2-1-pve: 4.2.2-16
lvm2: 2.02.116-pve1
corosync-pve: 2.3.5-1
libqb0: 0.17.2-1
pve-cluster: 4.0-22
qemu-server: 4.0-30
pve-firmware: 1.1-7
libpve-common-perl: 4.0-29
libpve-access-control: 4.0-9
libpve-storage-perl: 4.0-25
pve-libspice-server1: 0.12.5-1
vncterm: 1.2-1
pve-qemu-kvm: 2.4-9
pve-container: 1.0-6
pve-firewall: 2.0-12
pve-ha-manager: 1.0-9https://forum.proxmox.com/threads/suddenly-my-containers-wont-start-anymore.28516/
ksm-control-daemon: 1.2-1
glusterfs-client: 3.5.2-2+deb8u2
lxc-pve: 1.1.3-1
lxcfs: 0.9-pve2
cgmanager: 0.37-pve2
criu: 1.6.0-1
zfsutils: 0.6.5-pve4~jessie
-------------------------------------------

i have used apt-get update, apt-get upgrade, aptitude safe-upgrade. changed source list repositories. NO LUCK PLEASE HELP.

 

[Nodefield]

Also having similar issue:

pveversion:

# pveversion -v
proxmox-ve: 4.2-60 (running kernel: 4.4.15-1-pve)
pve-manager: 4.2-17 (running version: 4.2-17/e1400248)
pve-kernel-4.4.13-1-pve: 4.4.13-56
pve-kernel-4.4.13-2-pve: 4.4.13-58
pve-kernel-4.4.15-1-pve: 4.4.15-60
pve-kernel-4.4.10-1-pve: 4.4.10-54
lvm2: 2.02.116-pve2
corosync-pve: 2.4.0-1
libqb0: 1.0-1
pve-cluster: 4.0-43
qemu-server: 4.0-85
pve-firmware: 1.1-8
libpve-common-perl: 4.0-71
libpve-access-control: 4.0-19
libpve-storage-perl: 4.0-56
pve-libspice-server1: 0.12.8-1
vncterm: 1.2-1
pve-qemu-kvm: 2.6-1
pve-container: 1.0-72
pve-firewall: 2.0-29
pve-ha-manager: 1.0-33
ksm-control-daemon: 1.2-1
glusterfs-client: 3.5.2-2+deb8u2
lxc-pve: 2.0.3-4
lxcfs: 2.0.2-pve1
cgmanager: 0.39-pve1
criu: 1.6.0-1

Starting container in foreground:

# lxc-start --name 100 --foreground
readline() on closed filehandle $fd at /usr/share/lxc/hooks/lxc-pve-autodev-hook line 32.
Failed to mount cgroup at /sys/fs/cgroup/systemd: Permission denied
[!!!!!!] Failed to mount API filesystems, freezing.
Freezing execution.

Empty output from apparmor_parser:

# apparmor_parser -r -W -T /etc/apparmor.d/lxc-containers

 

[Nodefield]

Seems to work with:

proxmox-ve: 4.2-58 (running kernel: 4.4.13-2-pve)
pve-manager: 4.2-17 (running version: 4.2-17/e1400248)

 

[Nodefield]

I had custom setting for this container that caused the issue:

lxc.aa_profile: lxc-container-default-with-nfs

This was set for the container to get NFS mounting support. Obviously this stopped working with the recent PVE upgrades.

After removing that custom AppArmor profile, everything works.

Still have to figure out how to get NFS mounting working again for this container, though.