LXC nested doesn't work in Proxmx 4 beta 1 (problem with Cgroups)

[Krzysztof Majk]

Hello,
I'm testing new Promox 4 beta 1 and I have problems with running nested LXC and Docker in Proxmox LXC Ubuntu 14.04/Debian 8 container.

File /usr/share/lxc/config/ubuntu.common.conf uncommented apparmor profile for nesting:

lxc.aa_profile = lxc-container-default-with-nesting (tested with "unconfined" too - the same)

When I try to run nested (new ubuntu LXC container in a running LXC container on Proxmox) it throws error :

lxc-start --name u1 --logfile /tmp/test --logpriority debug
lxc-start: cgmanager.c: cgm_create: 631 cgroup error?  100 cgroups with this name already running
lxc-start: start.c: lxc_spawn: 861 failed creating cgroups
lxc-start: start.c: __lxc_start: 1080 failed to spawn 'u1'
lxc-start: lxc_start.c: main: 342 The container failed to start.
lxc-start: lxc_start.c: main: 346 Additional information can be obtained by setting the --logfile and --logpriority options

/tmp/test log file:

 lxc-start 1437899440.222 INFO     lxc_start_ui - lxc_start.c:main:265 - using rcfile /var/lib/lxc/u1/config      lxc-start 1437899440.222 WARN     lxc_log - log.c:lxc_log_init:316 - lxc_log_init called with log already initialized
      lxc-start 1437899440.222 WARN     lxc_cgmanager - cgmanager.c:cgm_get:954 - do_cgm_get exited with error
      lxc-start 1437899440.222 INFO     lxc_lsm - lsm/lsm.c:lsm_init:48 - LSM security driver AppArmor
      lxc-start 1437899440.223 INFO     lxc_seccomp - seccomp.c:parse_config_v2:285 - processing: .[all].
      lxc-start 1437899440.223 INFO     lxc_seccomp - seccomp.c:parse_config_v2:285 - processing: .kexec_load errno 1.
      lxc-start 1437899440.223 INFO     lxc_seccomp - seccomp.c:parse_config_v2:358 - Adding non-compat rule for kexec_load action 327681
      lxc-start 1437899440.223 INFO     lxc_seccomp - seccomp.c:parse_config_v2:285 - processing: .open_by_handle_at errno 1.
      lxc-start 1437899440.223 INFO     lxc_seccomp - seccomp.c:parse_config_v2:358 - Adding non-compat rule for open_by_handle_at action 327681
      lxc-start 1437899440.223 INFO     lxc_seccomp - seccomp.c:parse_config_v2:285 - processing: .init_module errno 1.
      lxc-start 1437899440.223 INFO     lxc_seccomp - seccomp.c:parse_config_v2:358 - Adding non-compat rule for init_module action 327681
      lxc-start 1437899440.223 INFO     lxc_seccomp - seccomp.c:parse_config_v2:285 - processing: .finit_module errno 1.
      lxc-start 1437899440.223 INFO     lxc_seccomp - seccomp.c:parse_config_v2:358 - Adding non-compat rule for finit_module action 327681
      lxc-start 1437899440.223 INFO     lxc_seccomp - seccomp.c:parse_config_v2:285 - processing: .delete_module errno 1.
      lxc-start 1437899440.223 INFO     lxc_seccomp - seccomp.c:parse_config_v2:358 - Adding non-compat rule for delete_module action 327681
      lxc-start 1437899440.223 DEBUG    lxc_conf - conf.c:lxc_create_tty:3665 - allocated pty '/dev/pts/5' (5/6)
      lxc-start 1437899440.223 DEBUG    lxc_conf - conf.c:lxc_create_tty:3665 - allocated pty '/dev/pts/6' (7/8)
      lxc-start 1437899440.223 DEBUG    lxc_conf - conf.c:lxc_create_tty:3665 - allocated pty '/dev/pts/8' (9/10)
      lxc-start 1437899440.223 DEBUG    lxc_conf - conf.c:lxc_create_tty:3665 - allocated pty '/dev/pts/9' (11/12)
      lxc-start 1437899440.223 INFO     lxc_conf - conf.c:lxc_create_tty:3676 - tty's configured
      lxc-start 1437899440.223 DEBUG    lxc_start - start.c:setup_signal_fd:247 - sigchild handler set
      lxc-start 1437899440.223 DEBUG    lxc_console - console.c:lxc_console_peer_default:500 - opening /dev/tty for console peer
      lxc-start 1437899440.223 DEBUG    lxc_console - console.c:lxc_console_peer_default:506 - using '/dev/tty' as console
      lxc-start 1437899440.223 DEBUG    lxc_console - console.c:lxc_console_sigwinch_init:179 - 7574 got SIGWINCH fd 17
      lxc-start 1437899440.223 DEBUG    lxc_console - console.c:lxc_console_winsz:88 - set winsz dstfd:14 cols:181 rows:48
      lxc-start 1437899440.223 INFO     lxc_start - start.c:lxc_init:443 - 'u1' is initialized
      lxc-start 1437899440.223 DEBUG    lxc_start - start.c:__lxc_start:1058 - Not dropping cap_sys_boot or watching utmp
      lxc-start 1437899440.224 DEBUG    lxc_conf - conf.c:instantiate_veth:3003 - instantiated veth 'vethRXB0YE/vethN7GBTR', index is '24'
      lxc-start 1437899440.224 INFO     lxc_cgroup - cgroup.c:cgroup_init:62 - cgroup driver cgmanager initing for u1
      lxc-start 1437899440.259 ERROR    lxc_cgmanager - cgmanager.c:cgm_create:631 - cgroup error?  100 cgroups with this name already running
      lxc-start 1437899440.259 ERROR    lxc_start - start.c:lxc_spawn:861 - failed creating cgroups
      lxc-start 1437899440.279 ERROR    lxc_start - start.c:__lxc_start:1080 - failed to spawn 'u1'
      lxc-start 1437899440.279 ERROR    lxc_start_ui - lxc_start.c:main:342 - The container failed to start.
      lxc-start 1437899440.279 ERROR    lxc_start_ui - lxc_start.c:main:346 - Additional information can be obtained by setting the --logfile and --logpriority options.

Error when trying to run Docker daemon:

docker -e lxc -d
INFO[0000] Listening for HTTP on unix (/var/run/docker.sock)
ERRO[0000] 'overlay' not found as a supported filesystem on this host. Please ensure kernel is new enough and has overlay support loaded.
WARN[0000] Running modprobe bridge nf_nat failed with message: , error: exit status 1
WARN[0000] Your kernel does not support cgroup memory limit: mountpoint for memory not found
WARN[0000] mountpoint for cpu not found
FATA[0000] Error mounting devices cgroup: mountpoint for devices not found

Steps to reproduce:
Install proxmox 4 beta 1
Make a new LXC container with Ubuntu 14.04/Debian 8 template
Install lxc in a container
Try to run a new nested lxc container

Thanks for any help.

 

[psc]

I'm trying this in beta 2 now:

lxc.mount.auto = cgroup
lxc.aa_profile = unconfined

restarted lxc on the host:

I get a slightly different error:

# docker -d -e lxc
Warning: '-d' is deprecated, it will be removed soon. See usage.
WARN[0000] please use 'docker daemon' instead.
INFO[0000] Listening for HTTP on unix (/var/run/docker.sock)
ERRO[0000] 'overlay' not found as a supported filesystem on this host. Please ensure kernel is new enough and has overlay support loaded.
INFO[0000] Option DefaultDriver: bridge
INFO[0000] Option DefaultNetwork: bridge
WARN[0000] Running modprobe bridge nf_nat br_netfilter failed with message: modprobe: ERROR: ../libkmod/libkmod.c:556 kmod_search_moddep() could not open moddep file '/lib/modules/4.2.0-1-pve/modules.dep.bin'
modprobe: ERROR: ../libkmod/libkmod.c:556 kmod_search_moddep() could not open moddep file '/lib/modules/4.2.0-1-pve/modules.dep.bin'
modprobe: ERROR: ../libkmod/libkmod.c:556 kmod_search_moddep() could not open moddep file '/lib/modules/4.2.0-1-pve/modules.dep.bin'
, error: exit status 1
INFO[0000] Firewalld running: false
WARN[0000] Your kernel does not support cgroup memory limit: mountpoint for memory not found
WARN[0000] mountpoint for cpu not found
FATA[0000] Error starting daemon: Devices cgroup isn't mounted

 

[psc]

I can start an older version of docker
# apt-get install docker.io

# docker -d -e lxc
2015/09/22 07:57:01 docker daemon: 1.0.1 990021a; execdriver: lxc; graphdriver:
[ab338efc] +job serveapi(unix: ///var/run/docker.sock)
[ab338efc] +job initserver()
[ab338efc.initserver()] Creating server
2015/09/22 07:57:01 Listening for HTTP on unix (/var/run/docker.sock)
[ab338efc] +job init_networkdriver()
[ab338efc] -job init_networkdriver() = OK (0)
2015/09/22 07:57:01 WARNING: mountpoint not found
Loading containers: : done.
[ab338efc.initserver()] Creating pidfile
[ab338efc.initserver()] Setting up signal traps
[ab338efc] -job initserver() = OK (0)
[ab338efc] +job acceptconnections()
[ab338efc] -job acceptconnections() = OK (0)

But then still not start a container

$ sudo docker run -it --rm busybox /bin/bash
lxc-start: cgmanager.c: cgm_create: 631 cgroup error? 100 cgroups with this name already running
lxc-start: start.c: lxc_spawn: 861 failed creating cgroups
lxc-start: start.c: __lxc_start: 1080 failed to spawn '64d4a0944f48afe57be407fd28a80006eb828bb6fe3b8634bc995c10e33ee3c3'
lxc-start: lxc_start.c: main: 342 The container failed to start.
lxc-start: lxc_start.c: main: 346 Additional information can be obtained by setting the --logfile and --logpriority options.