[SOLVED] Apparmor preventing LXCs starting after update

bradgy

Active Member
Mar 10, 2018
3
0
41
41
I ran an apt update && apt dist-upgrade on my home server after a few weeks of uptime as part of its routine maintenance but its LXCs are failing to start after the reboot. All the VMs are still working.

journalctl -xe output:

Code:
-- Unit pve-container@200.service has begun starting up.
Mar 10 13:57:03 bjserver audit[1951]: AVC apparmor="DENIED" operation="mount" info="failed mntpnt match" error=-13 profile="/usr/bin/lxc-start" name="/" pid=1951 comm="lxc-start" flags="rw, rslave"
Mar 10 13:57:03 bjserver kernel: audit: type=1400 audit(1520708223.230:19): apparmor="DENIED" operation="mount" info="failed mntpnt match" error=-13 profile="/usr/bin/lxc-start" name="/" pid=1951 comm="lxc-start
Mar 10 13:57:03 bjserver lxc-start[1949]: lxc-start: 200: lxccontainer.c: wait_on_daemonized_start: 751 No such file or directory - Failed to receive the container state
Mar 10 13:57:03 bjserver lxc-start[1949]: lxc-start: 200: tools/lxc_start.c: main: 368 The container failed to start.
Mar 10 13:57:03 bjserver lxc-start[1949]: lxc-start: 200: tools/lxc_start.c: main: 370 To get more details, run the container in foreground mode.
Mar 10 13:57:03 bjserver lxc-start[1949]: lxc-start: 200: tools/lxc_start.c: main: 372 Additional information can be obtained by setting the --logfile and --logpriority options.
Mar 10 13:57:03 bjserver systemd[1]: pve-container@200.service: Control process exited, code=exited status=1
Mar 10 13:57:03 bjserver systemd[1]: pve-container@200.service: Killing process 1951 (lxc-start) with signal SIGKILL.
Mar 10 13:57:03 bjserver systemd[1]: Failed to start PVE LXC Container: 200.
-- Subject: Unit pve-container@200.service has failed
-- Defined-By: systemd
-- Support: (url)
--
-- Unit pve-container@200.service has failed.
--
-- The result is failed.
Mar 10 13:57:03 bjserver systemd[1]: pve-container@200.service: Unit entered failed state.
Mar 10 13:57:03 bjserver systemd[1]: pve-container@200.service: Failed with result 'exit-code'.
Mar 10 13:57:03 bjserver pvedaemon[1946]: command 'systemctl start pve-container@200' failed: exit code 1
Mar 10 13:57:03 bjserver pvedaemon[1273]: <root@pam> end task UPID:bjserver:0000079A:0000409C:5AA42A7F:vzstart:200:root@pam: command 'systemctl start pve-container@200' failed: exit code 1
Mar 10 13:57:03 bjserver pve-guests[1293]: <root@pam> end task UPID:bjserver:00000510:00000441:5AA429E4:startall::root@pam: OK

I'm not super knowledgeable about containers, just enough to get myself into trouble, so any ideas on fixes are appreciated.
 
+1, I have encountered the same issue, as well as a friend.

As a temporary fix, you can downgrade apparmor to 2.10.95-4 from backports:
https: //packages.debian.org/jessie-backports/apparmor

Download the appropriate package for your system

Install it using:
Code:
dpkg -i filename.deb

And hold it, to make sure it does not get upgraded with an apt upgrade:
Code:
apt-mark hold apparmor

If you want to upgrade it, unhold it:
Code:
apt-mark unhold apparmor

Now, start apparmor
Code:
systemctl start apparmor

Hope this will work for you as well!
 
  • Like
Reactions: Mateus Marques
Hot dang.
This worked!
I'd seen other posts talking about downgrading the apparmor package but they were from late last year so I naively thought that they wouldn't apply any more.
Thanks for making a post r33int! You are a lifesaver.
 
Hot dang.
This worked!
I'd seen other posts talking about downgrading the apparmor package but they were from late last year so I naively thought that they wouldn't apply any more.
Thanks for making a post r33int! You are a lifesaver.
Glad I helped you! :D
 
please don't install packages from jessie-backports on PVE 5 systems!

the AppArmor issue has been solved in an lxc-pve update last week, so unless you have incorrect PVE repositories configured it should not be possible to hit it. PVE 4 is not affected at all, since the relevant AppArmor change was only for Debian Stretch.
 
I guess neither of these two have either
a) a proper subscription or
b) like me, use the pve-no-subscription repo.

If you only upgraded using the default apt entries (And you did not have a valid sub), it would be possible to get into this situation I guess?
 
I guess neither of these two have either
a) a proper subscription or
b) like me, use the pve-no-subscription repo.

If you only upgraded using the default apt entries (And you did not have a valid sub), it would be possible to get into this situation I guess?

yes, if you don't have PVE repositories configured which you can access, you won't get PVE updates.
 
yes, if you don't have PVE repositories configured which you can access, you won't get PVE updates.

I just installed a fresh 5.1-3 version of Proxmox and at first it worked. Then I updated with a simple apt-get update and apt-get full-upgrade and I got this error and could not start any LXC containers. (Never tried KVM) I needed to manually add
deb http://download.proxmox.com/debian/pve stretch pve-no-subscription then run apt-get full-upgrade (and update ofcourse) to fix it. I am writing this as a heads up to all other people who has the same problem and as sort of bug report.
 
Thanks for the info Fabian and others. I've learnt something valuable here. I thought I took things pretty slowly setting everything up but there's always more documentation to read and understand; in this case, setting up repositories correctly. I'll look into a subscription as well in order to support the work the Proxmox team does.

edit: for anyone reading this later... setting up the pve-no-subscription repo as described here solved my issue with apparmor.
 
But, it's safe add pve-no-subscription in production servers ? Can i patch this bug without add this repo?
 
But, it's safe add pve-no-subscription in production servers ? Can i patch this bug without add this repo?
Proxmox do not recommend using the pve-no-subscription repository for production servers, as packages are not always heavily tested/validated.

The subscription repository is recommended for commercial/production servers as the packages provided there are tested much more.

Realistically, most of the time you're probably going to be okay using the pve-no-subscription repository; however if you're using the server for commercial purposes, I'd recommend paying for the subscription repo personally.
 
Proxmox do not recommend using the pve-no-subscription repository for production servers, as packages are not always heavily tested/validated.

The subscription repository is recommended for commercial/production servers as the packages provided there are tested much more.

Realistically, most of the time you're probably going to be okay using the pve-no-subscription repository; however if you're using the server for commercial purposes, I'd recommend paying for the subscription repo personally.

Thank you!
 
What's the path forward for non-subscribers? I hit this bug last night, thankfully stumbled upon this post for a quick fix. My cluster's not used for anything commercial / "production" per se, just a homelab, so paying the subscription fee for my three processors isn't ideal.

I haven't added or removed any repos, just using the stock ones that come with 5.1-35.
 
What's the path forward for non-subscribers? I hit this bug last night, thankfully stumbled upon this post for a quick fix. My cluster's not used for anything commercial / "production" per se, just a homelab, so paying the subscription fee for my three processors isn't ideal.

I haven't added or removed any repos, just using the stock ones that come with 5.1-35.

if you don't have a subscription, you need to enable the no-subscription repository like described in the docs. otherwise, you don't get updates, and thus also no bug and security fixes.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!