Privileged Containers and AppArmor: Any advice appreciated!

[R2_D2]

Hey there,

I work at a small startup where we're currently trying to evaluate whether Proxmox might be able to allow us to run an LXC for each of our data scientists to use and RDP into. Unfortunately, during our testing we're running into some issues that are making this quite difficult. This is due to some admittedly difficult requirements on our end, namely we need:

• A single NVIDIA GPU to be shared between containers.
• To mount NFS volumes so our data scientists can access large datasets.
• A bind mount so that data scientists can access fast scratch space.
• Each container to work with LDAP via SSSD.
• Docker to be able to be run inside each container.

As much as we want to use unprivileged containers, it is our understanding that these pretty much necessitate privilege. We've evaluated this trade-off quite a bit and are doing what we can to mitigate the security issues (e.g. this machine will be on its own VLAN and will only be accessed by trusted users), but wanted to confirm that satisfying these requirements isn't possible using unprivileged containers? Or if it is possible, any guidance on how to go about this would be greatly appreciated, especially with the user mapping that might be required (we're a bit worried this would end up being a maintenance nightmare).

Additionally, we've been having some issues with AppArmor in the Ubuntu 20.04 template in privileged containers. I've added the output of `systemctl status apparmor` below. As you can see, it fails to load in privileged containers (yes, nesting is enabled). This is preventing Docker from running as it tries to load its own profile, and snap is also broken as it too relies on AppArmor. Given that it's not running anyways, we could just uninstall AppArmor entirely to solve the Docker issue, but this leaves the snap issue unresolved (and breaks things like chromium-browser, which requires snap even if installed using apt). Is it even possible to use AppArmor within privileged containers? Or is there anything we can do to get it working? Any help here would be appreciated.

Many thanks for your time and effort helping us with this!

* apparmor.service - Load AppArmor profiles
     Loaded: loaded (/lib/systemd/system/apparmor.service; disabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Wed 2022-08-03 15:51:58 MDT; 53min ago
       Docs: man:apparmor(7)
             https://gitlab.com/apparmor/apparmor/wikis/home/
    Process: 40677 ExecStart=/lib/apparmor/apparmor.systemd reload (code=exited, status=1/FAILURE)
   Main PID: 40677 (code=exited, status=1/FAILURE)
        CPU: 22.433s

Aug 03 15:51:53 [hostname] apparmor.systemd[40715]: /sbin/apparmor_parser: Unable to replace "chromium".  Permission denied; attempted to load a profile while confined?
Aug 03 15:51:53 [hostname] apparmor.systemd[40715]: /sbin/apparmor_parser: Unable to replace "/usr/lib/lightdm/lightdm-guest-session".  Permission denied; attempted to load a profile while confined?
Aug 03 15:51:53 [hostname] apparmor.systemd[40770]: /sbin/apparmor_parser: Unable to replace "/usr/lib/cups/backend/cups-pdf".  Permission denied; attempted to load a profile while confined?
Aug 03 15:51:58 [hostname] apparmor.systemd[40754]: /sbin/apparmor_parser: Unable to replace "gpg".  Permission denied; attempted to load a profile while confined?
Aug 03 15:51:58 [hostname] apparmor.systemd[40754]: /sbin/apparmor_parser: Unable to replace "libreoffice-soffice".  Permission denied; attempted to load a profile while confined?
Aug 03 15:51:58 [hostname] apparmor.systemd[40677]: Error: At least one profile failed to load
Aug 03 15:51:58 [hostname] systemd[1]: apparmor.service: Main process exited, code=exited, status=1/FAILURE
Aug 03 15:51:58 [hostname] systemd[1]: apparmor.service: Failed with result 'exit-code'.
Aug 03 15:51:58 [hostname] systemd[1]: Failed to start Load AppArmor profiles.
Aug 03 15:51:58 [hostname] systemd[1]: apparmor.service: Consumed 22.433s CPU time.

 

[eckoflyte]

I am having the exact same issue.

 

[rholighaus]

Having the same issue too.
Any updates?