Priviledge Container: disabling apparmor does not work

[elendiir]

Hi guys,

I am quite new to Proxmox. I do have a privileged container (need to mount NFS within a docker container). Everything was running fine for 2 months. Yesterday I did update the LXC and now the paperless container running inside the LXC wont start anymore.

This is the error message when running docker-compose up -d:

Starting paperless_broker_1 ... error
ERROR: for paperless_broker_1 Cannot start service broker: AppArmor enabled on system but the docker-default profile could not be loaded: running `/usr/sbin/apparmor_parser apparmor_parser -Kr /var/lib/docker/tmp/docker-default2635317771` failed with output: apparmor_parser: Unable to replace "docker-default". Permission denied; attempted to load a profile while confined?
error: exit status 243

I tried to disable apparmor for this container by adding the following line at the end of the file /etc/pve/lxc/${container_id}.conf as stated in the documentation:
lxc.apparmor.profile = unconfined
and than I rebooted the LXC

I also ensured that apparmor is not installed on the LXC (Ubuntu 22.04) using this command
sudo apt purge apparmor apparmor-utils auditd

But still apparmor is enabled: docker info gives me:
Security Options:
apparmor
seccomp

When I try to install the apparmor packages I end up with the following errors:
sudo apt install apparmor apparmor-utils auditd

Created symlink /etc/systemd/system/multi-user.target.wants/auditd.service -> /lib/systemd/system/auditd.service.
Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142.
Setting up apparmor (3.0.4-2ubuntu2.1) ...
apparmor_parser: Unable to replace "lsb_release". Permission denied; attempted to load a profile while confined?
apparmor_parser: Unable to replace "kmod". Permission denied; attempted to load a profile while confined?
apparmor_parser: Unable to replace "nvidia_modprobe". Permission denied; attempted to load a profile while confined?
Created symlink /etc/systemd/system/sysinit.target.wants/apparmor.service -> /lib/systemd/system/apparmor.service.
Reloading AppArmor profiles
/sbin/apparmor_parser: Unable to replace "kmod". Permission denied; attempted to load a profile while confined?
/sbin/apparmor_parser: Unable to replace "nvidia_modprobe". Permission denied; attempted to load a profile while confined?
/sbin/apparmor_parser: Unable to replace "lsb_release". Permission denied; attempted to load a profile while confined?
/sbin/apparmor_parser: Unable to replace "/usr/bin/man". Permission denied; attempted to load a profile while confined?
Skipping profile in /etc/apparmor.d/disable: usr.sbin.rsyslogd
/sbin/apparmor_parser: Unable to replace "tcpdump". Permission denied; attempted to load a profile while confined?
/sbin/apparmor_parser: Unable to replace "/usr/lib/NetworkManager/nm-dhcp-client.action". Permission denied; attempted to load a profile while confined?
/sbin/apparmor_parser: Unable to replace "lsb_release". Permission denied; attempted to load a profile while confined?
/sbin/apparmor_parser: Unable to replace "kmod". Permission denied; attempted to load a profile while confined?
/sbin/apparmor_parser: Unable to replace "nvidia_modprobe". Permission denied; attempted to load a profile while confined?
/sbin/apparmor_parser: Unable to replace "/usr/bin/man". Permission denied; attempted to load a profile while confined?
Skipping profile in /etc/apparmor.d/disable: usr.sbin.rsyslogd
/sbin/apparmor_parser: Unable to replace "tcpdump". Permission denied; attempted to load a profile while confined?
/sbin/apparmor_parser: Unable to replace "/usr/lib/NetworkManager/nm-dhcp-client.action". Permission denied; attempted to load a profile while confined?
Error: At least one profile failed to load
Setting up apparmor-utils (3.0.4-2ubuntu2.1) ...
Processing triggers for man-db (2.10.2-1) ...
apt install apparmor apparmor-utils auditd

Can anybody please help me either to disable apparmor or to get it install correctly.

Thanks a lot
Fritz

 

[elendiir]

Good morning!

I spent several hours without being able to get the system running again. So I reverted back to the last backup.
Installing the system updates bumps Docker within the LXC from 20.10.23 to 23.0.0. With 20.10.23 the container are starting. With 23.0.0 the container wont start because of the apparmor error.

I still have no idea how to sort things out correctly or how to disable apparmor on the LCX. Especially since all apparmor packages have been prurged from the LCX.

Any help is welcome.
Fritz

 

[maeries]

Same problem here though with a bit different behaviour. With apparmor installed I get:

$ docker-compose up -d
Building with native build. Learn about native build in Compose here: https://docs.docker.com/go/compose-native-build/
Starting pihole ... error

ERROR: for pihole  Cannot start service pihole: AppArmor enabled on system but the docker-default profile could not be loaded: running `/sbin/apparmor_parser apparmor_parser -Kr /var/lib/docker/tmp/docker-default926933200` failed with output: apparmor_parser: Unable to replace "docker-default".  Permission denied; attempted to load a profile while confined?

error: exit status 243

ERROR: for pihole  Cannot start service pihole: AppArmor enabled on system but the docker-default profile could not be loaded: running `/sbin/apparmor_parser apparmor_parser -Kr /var/lib/docker/tmp/docker-default926933200` failed with output: apparmor_parser: Unable to replace "docker-default".  Permission denied; attempted to load a profile while confined?

error: exit status 243
ERROR: Encountered errors while bringing up the project.

With apparmor removed:

$ docker-compose up -d
Building with native build. Learn about native build in Compose here: https://docs.docker.com/go/compose-native-build/
Starting pihole ... error

ERROR: for pihole  Cannot start service pihole: AppArmor enabled on system but the docker-default profile could not be loaded: running `apparmor_parser apparmor_parser --version` failed with output:
error: exec: "apparmor_parser": executable file not found in $PATH

ERROR: for pihole  Cannot start service pihole: AppArmor enabled on system but the docker-default profile could not be loaded: running `apparmor_parser apparmor_parser --version` failed with output:
error: exec: "apparmor_parser": executable file not found in $PATH
ERROR: Encountered errors while bringing up the project.

Note that it says AppArmor enabled on system, while it's not installed. I also ran systemctl disable apparmor.service beforehand

Putting lxc.apparmor.profile = unconfined into the config doesn't do anything.

I'm also on a privileged LXC

 

[maeries]

Found a solution. Run docker with --security-opt apparmor:unconfined respectively put

security_opt:
      - apparmor:unconfined

as in

services:
  pihole:
    container_name: pihole
    image: pihole/pihole:latest
    security_opt:
      - apparmor:unconfined

in your docker-compose.yml

 

[maeries]

Also what I found and what might lead to a better solution is that

$ sudo aa-status
apparmor module is loaded.
17 profiles are loaded.
17 profiles are in enforce mode.
   /usr/bin/lxc-start
   /usr/bin/man
   /usr/lib/cups/backend/cups-pdf
   /usr/sbin/cups-browsed
   /usr/sbin/cupsd
   /usr/sbin/cupsd//third_party
   lsb_release
   lxc-container-default
   lxc-container-default-cgns
   lxc-container-default-with-mounting
   lxc-container-default-with-nesting
   man_filter
   man_groff
   nvidia_modprobe
   nvidia_modprobe//kmod
   swtpm
   tcpdump
0 profiles are in complain mode.
3 processes have profiles defined.
3 processes are in enforce mode.
   /usr/sbin/cups-browsed (153)
   /usr/sbin/cupsd (89)
   /usr/lib/cups/notifier/dbus (2059) /usr/sbin/cupsd
0 processes are in complain mode.
0 processes are unconfined but have a profile defined.

shows no profile for docker. Could it be that that's what's missing?

 

[elendiir]

Thanks a lot Maeries,

as far as I do remember disabling apparmor in my docker compose configuration was part of one of my many tries. I will check this when I am back home from my business trip.

 

[elendiir]

Maeries,

I just added the apparmor:unconfined into the yml-file. And now it works. I think I did something wrong the last time.
Thanks for your support! This removes a lot of headaches.

Cheers
Fritz

 

[nhasian]

I'd like to know what changed. all my docker containers stopped running until I added --security-opt apparmor:unconfined to each of them.

 

[elendiir]

@nhasian
My privileged LCX haven been running fine over half a year. The containers in the LCX stopped working after upgrading Docker from version 20.10.23 to 23.0.0. After deploying this version I needed to add apparmor:unconfined in order to be able to start the containers again.

 

[Giovanni]

anyone find a solution other than editing every single container and restarting it with that option? It feels like there must be something on the OS wide level that could be changed to avoid this hassle.