5.3 and lxc nesting

L

limone

Well-Known Member
Aug 1, 2017
89
9
48
30
Hi,

I'm very happy too see, that you can now run docker inside a lxc container.
So I tried that with my mailcow installation, which I ran before using
Code: 
lxc.apparmor.profile: unconfined
lxc.cgroup.devices.allow: a
lxc.cap.drop:
in the containers config.
By enabling nesting I also removed that extra config, as it shouldn't be needed anymore.

So I started the container, but there seems to be a problem with apparmor :(
Code: 
docker-compose up -d
WARNING: The WATCHDOG_NOTIFY_EMAIL variable is not set. Defaulting to a blank string.
Starting mailcowdockerized_memcached-mailcow_1 ...
Starting mailcowdockerized_unbound-mailcow_1   ...
Starting mailcowdockerized_watchdog-mailcow_1  ...
Starting mailcowdockerized_memcached-mailcow_1 ... error
Starting mailcowdockerized_dovecot-mailcow_1   ...
Starting mailcowdockerized_dockerapi-mailcow_1 ...
Starting mailcowdockerized_clamd-mailcow_1     ...
Starting mailcowdockerized_sogo-mailcow_1      ...
Starting mailcowdockerized_postfix-mailcow_1   ...
Starting mailcowdockerized_ipv6nat_1           ... error
Starting mailcowdockerized_watchdog-mailcow_1  ... error

ERROR: for mailcowdockerized_ipv6nat_1  Cannot start service ipv6nat: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:402: container init caused \"process_linux.go:367: setting cgroup config for procHooks process caused \\\"failed to write a *:* rwm to devices.allow: write /sys/fs/cgroup/devices/docker/dcacee67f17cc3d92d6fb1742b1786f7758ff65cff88cf7ac26c1f875af243a0/devices.allow: operation not permitted\\\"\"": unknown

ERROR: for mailcowdockerized_memcached-mailcow_1  Cannot start service memcached-mailcow: AppArmor enabled on system but the docker-default profile could not be loaded: running `/sbin/apparmor_parser apparmor_parser -Kr /var/lib/docker/tmp/docker-default562853185` failed with output: apparmor_parser: Unable to rStarting mailcowdockerized_mysql-mailcow_1     ... error

error: exit status 243
Starting mailcowdockerized_unbound-mailcow_1   ... error
ERROR: for mailcowdockerized_watchdog-mailcow_1  Cannot start service watchdog-mailcow: AppArmor enabled on system but the docker-default profile could not be loaded: running `/sbin/apparmor_parser apparmor_parser -Kr /var/lib/docker/tmp/docker-default036280447` failed with output: apparmor_parser: Unable to replace "docker-default".  Permission denied; attempted to load a profile while confined?

error: exit status 243

ERROR: for mailcowdockerized_mysql-mailcow_1  Cannot start service mysql-mailcow: AppArmor enabled on system but the docker-default profile could not be loaded: running `/sbin/apparmor_parser apparmor_parser -Kr /var/lib/docker/tmp/docker-default135030688` failed with output: apparmor_parser: Unable to replace "docker-default".  Permission denied; attempted to load a profile while confined?

Starting mailcowdockerized_dockerapi-mailcow_1 ... error

ERROR: for mailcowdockerized_unbound-mailcow_1  Cannot start service unbound-mailcow: AppArmor enabled on system but the docker-default profile could not be loaded: running `/sbin/apparmor_parser apparmor_parser -Kr /var/lib/docker/tmp/docker-default099745233` failed with output: apparmor_parser: Unable to replace "docker-default".  Permission denied; attempted to load a profile while confined?

error: exit status 243

Starting mailcowdockerized_sogo-mailcow_1      ... error
eplace "docker-default".  Permission denied; attempted to load a profile while confined?

error: exit status 243
Starting mailcowdockerized_clamd-mailcow_1     ... error
ERROR: for mailcowdockerized_sogo-mailcow_1  Cannot start service sogo-mailcow: AppArmor enabled on system but the docker-default profile could not be loaded: running `/sbin/apparmor_parser apparmor_parser -Kr /var/lib/docker/tmp/docker-default265822617` failed with output: apparmor_parser: Unable to replace "docker-default".  Permission denied; attempted to load a profile while confined?
Starting mailcowdockerized_dovecot-mailcow_1   ... error
error: exit status 243

ERROR: for mailcowdockerized_clamd-mailcow_1  Cannot start service clamd-mailcow: AppArmor enabled on system but the docker-default profile could not be loaded: running `/sbin/apparmor_parser apparmor_parser -Kr /var/lib/docker/tmp/docker-default207691873` failed with output: apparmor_parser: Unable to replace "docker-default".  Permission denied; attempted to load a profile while confined?

error: exit status 243

ERROR: for mailcowdockerized_dovecot-mailcow_1  Cannot start service dovecot-mailcow: AppArmor enabled on system but the docker-default profile could not be loaded: running `/sbin/apparmor_parser apparmor_parser -Kr /var/lib/docker/tmp/docker-default976620509` failed with output: apparmor_parser: Unable to replace "docker-default".  Permission denied; attempted to load a profile while confined?

Starting mailcowdockerized_redis-mailcow_1     ... error

ERROR: for mailcowdockerized_redis-mailcow_1  Cannot start service redis-mailcow: AppArmor enabled on system but the docker-default profile could not be loaded: running `/sbin/apparmor_parser apparmor_parser -Kr /var/lib/docker/tmp/docker-default040636575` failed with output: apparmor_parser: Unable to replace "Starting mailcowdockerized_postfix-mailcow_1   ... error

error: exit status 243

ERROR: for mailcowdockerized_postfix-mailcow_1  Cannot start service postfix-mailcow: AppArmor enabled on system but the docker-default profile could not be loaded: running `/sbin/apparmor_parser apparmor_parser -Kr /var/lib/docker/tmp/docker-default673294117` failed with output: apparmor_parser: Unable to replace "docker-default".  Permission denied; attempted to load a profile while confined?

error: exit status 243

ERROR: for ipv6nat  Cannot start service ipv6nat: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:402: container init caused \"process_linux.go:367: setting cgroup config for procHooks process caused \\\"failed to write a *:* rwm to devices.allow: write /sys/fs/cgroup/devices/docker/dcacee67f17cc3d92d6fb1742b1786f7758ff65cff88cf7ac26c1f875af243a0/devices.allow: operation not permitted\\\"\"": unknown

ERROR: for memcached-mailcow  Cannot start service memcached-mailcow: AppArmor enabled on system but the docker-default profile could not be loaded: running `/sbin/apparmor_parser apparmor_parser -Kr /var/lib/docker/tmp/docker-default562853185` failed with output: apparmor_parser: Unable to replace "docker-default".  Permission denied; attempted to load a profile while confined?

error: exit status 243

ERROR: for watchdog-mailcow  Cannot start service watchdog-mailcow: AppArmor enabled on system but the docker-default profile could not be loaded: running `/sbin/apparmor_parser apparmor_parser -Kr /var/lib/docker/tmp/docker-default036280447` failed with output: apparmor_parser: Unable to replace "docker-default".  Permission denied; attempted to load a profile while confined?

error: exit status 243

ERROR: for mysql-mailcow  Cannot start service mysql-mailcow: AppArmor enabled on system but the docker-default profile could not be loaded: running `/sbin/apparmor_parser apparmor_parser -Kr /var/lib/docker/tmp/docker-default135030688` failed with output: apparmor_parser: Unable to replace "docker-default".  Permission denied; attempted to load a profile while confined?

error: exit status 243

ERROR: for unbound-mailcow  Cannot start service unbound-mailcow: AppArmor enabled on system but the docker-default profile could not be loaded: running `/sbin/apparmor_parser apparmor_parser -Kr /var/lib/docker/tmp/docker-default099745233` failed with output: apparmor_parser: Unable to replace "docker-default".  Permission denied; attempted to load a profile while confined?

error: exit status 243

ERROR: for dockerapi-mailcow  Cannot start service dockerapi-mailcow: AppArmor enabled on system but the docker-default profile could not be loaded: running `/sbin/apparmor_parser apparmor_parser -Kr /var/lib/docker/tmp/docker-default664241045` failed with output: apparmor_parser: Unable to replace "docker-default".  Permission denied; attempted to load a profile while confined?

error: exit status 243

ERROR: for sogo-mailcow  Cannot start service sogo-mailcow: AppArmor enabled on system but the docker-default profile could not be loaded: running `/sbin/apparmor_parser apparmor_parser -Kr /var/lib/docker/tmp/docker-default265822617` failed with output: apparmor_parser: Unable to replace "docker-default".  Permission denied; attempted to load a profile while confined?

error: exit status 243

ERROR: for clamd-mailcow  Cannot start service clamd-mailcow: AppArmor enabled on system but the docker-default profile could not be loaded: running `/sbin/apparmor_parser apparmor_parser -Kr /var/lib/docker/tmp/docker-default207691873` failed with output: apparmor_parser: Unable to replace "docker-default".  Permission denied; attempted to load a profile while confined?

error: exit status 243

ERROR: for dovecot-mailcow  Cannot start service dovecot-mailcow: AppArmor enabled on system but the docker-default profile could not be loaded: running `/sbin/apparmor_parser apparmor_parser -Kr /var/lib/docker/tmp/docker-default976620509` failed with output: apparmor_parser: Unable to replace "docker-default".  Permission denied; attempted to load a profile while confined?

error: exit status 243

ERROR: for redis-mailcow  Cannot start service redis-mailcow: AppArmor enabled on system but the docker-default profile could not be loaded: running `/sbin/apparmor_parser apparmor_parser -Kr /var/lib/docker/tmp/docker-default040636575` failed with output: apparmor_parser: Unable to replace "docker-default".  Permission denied; attempted to load a profile while confined?

error: exit status 243

ERROR: for postfix-mailcow  Cannot start service postfix-mailcow: AppArmor enabled on system but the docker-default profile could not be loaded: running `/sbin/apparmor_parser apparmor_parser -Kr /var/lib/docker/tmp/docker-default673294117` failed with output: apparmor_parser: Unable to replace "docker-default".  Permission denied; attempted to load a profile while confined?

error: exit status 243
ERROR: Encountered errors while bringing up the project.

So, is it not as easy as I thought?
 
Code: 
-- Subject: Unit containerd.service has begun start-up
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- Unit containerd.service has begun starting up.
Dec 21 13:29:54 dockertest systemd[1]: docker.service: Failed to reset devices.list: Operation not permitted
Dec 21 13:29:54 dockertest systemd[1]: Starting Docker Application Container Engine...
-- Subject: Unit docker.service has begun start-up
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- Unit docker.service has begun starting up.
Dec 21 13:29:54 dockertest modprobe[2718]: modprobe: ERROR: ../libkmod/libkmod.c:586 kmod_search_moddep() could not open moddep file '/lib/modules/4.15.18-9-pve/modules.dep.bin'
Dec 21 13:29:54 dockertest modprobe[2718]: modprobe: FATAL: Module overlay not found in directory /lib/modules/4.15.18-9-pve
Dec 21 13:29:54 dockertest systemd[1]: containerd.service: Control process exited, code=exited status=1
Dec 21 13:29:54 dockertest systemd[1]: containerd.service: Failed with result 'exit-code'.
Dec 21 13:29:54 dockertest systemd[1]: Failed to start containerd container runtime.
-- Subject: Unit containerd.service has failed
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- Unit containerd.service has failed.
--
-- The result is RESULT.
Dec 21 13:29:54 dockertest systemd[1]: Dependency failed for Docker Application Container Engine.
-- Subject: Unit docker.service has failed
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- Unit docker.service has failed.
--
-- The result is RESULT.
Dec 21 13:29:54 dockertest systemd[1]: docker.service: Job docker.service/start failed with result 'dependency'.
Dec 21 13:29:54 dockertest systemd[1]: Stopped Docker Application Container Engine.
-- Subject: Unit docker.service has finished shutting down
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- Unit docker.service has finished shutting down.

But when I run lsmod on proxmox itself it shows overlay
Code: 
root@proxmox:~# lsmod | grep overlay
overlay                77824  16

privileged/unprivileged container doesn't change anything..
 
Hi.

The problem is the new containerd service, which tries to load the overlay module inside of LXC. This doesn't work and isn't needed.

You have to edit the containerd service as follows:
  1. run command systemctl edit containerd.service and enter following content:
    Code: 
    [Service]
ExecStartPre=
  2. save and run systemctl daemon-reload
  3. That's it. Now docker should start.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom