CGroups (or equivalent) inside a LXC container

[jinjer]

I have a question regarding the new lxc containers in 4.0.

I would like to be able to run separate processes inside a LXC container, in their own cgroups, so that a single process cannot take down a container (or the server for that matter).

On bare metal this is done using cgroups. On proxmox, cgroups are already used for containers. It it possible to extend the cgroups paradigm inside the container ?

 

[dietmar]

It it possible to extend the cgroups paradigm inside the container ?

AFAIK this is already done (we use cgmanager)

 

[jinjer]

I'm not a big expert in cgroups. For some reason the nested config is not working inside the lxc container.

I tried this reference with an ubuntu 14.04 guest

A lxc-start does this:

lxc-start -n 6000lxc-start: lxc_start.c: main: 344 The container failed to start.
lxc-start: lxc_start.c: main: 346 To get more details, run the container in foreground mode.
lxc-start: lxc_start.c: main: 348 Additional information can be obtained by setting the --logfile and --logpriority options.

My config looks like this:

lxc.include = /usr/share/lxc/config/ubuntu.common.conf
lxc.aa_profile = lxc-default-with-nesting
lxc.arch = amd64
lxc.cgroup.cpu.cfs_period_us = 100000
lxc.cgroup.cpu.cfs_quota_us = 100000
lxc.cgroup.cpu.shares = 1024
lxc.cgroup.memory.limit_in_bytes = 536870912
lxc.cgroup.memory.memsw.limit_in_bytes = 1073741824

I tried other profiles from apparmor (/etc/apparmor.d/lxc/), however none of them seems to work.

On the other hand, if I start the LXC container without specifying the lxc.aa_profile parameter, I see that the cgroup fs is mounted inside the container, but cgm tests (package cgm-utils) fails with the following logs:

root@lxc1:/usr/share/cgmanager/tests# ./runtests.sh
Note: real uid is 1000 gid is 1000 user is
mount: block device cgroup is write-protected, mounting read-only
mount: cannot mount block device cgroup read-only
rmdir: failed to remove '/sys/fs/cgroup/b': No such file or directory
rmdir: failed to remove '/sys/fs/cgroup/xxx/b': No such file or directory
rmdir: failed to remove '/sys/fs/cgroup/zzz/b': No such file or directory
rmdir: failed to remove '/sys/fs/cgroup/zzz': No such file or directory
mkdir: cannot create directory '/sys/fs/cgroup/xxx': File exists
umount: /sys/fs/cgroup: device is busy.
        (In some cases useful info about processes that use
         the device is found by lsof(8) or fuser(1))
Test 1: getValue
Test 2 (getMyCgroup)
Test 3 (Create)
Test 5 (../ getValue)
Test 6 (movepid)
1
method return sender=(null sender) -> dest=(null destination) reply_serial=1
2
4
method return sender=(null sender) -> dest=(null destination) reply_serial=1
5
Test 7 (non-root movepid)
Error org.freedesktop.DBus.Error.InvalidArgs: invalid request
Test 8 (movepid self)
method return sender=(null sender) -> dest=(null destination) reply_serial=1
Test 9 (Chown)
mount: block device cgroup is write-protected, mounting read-only
mount: cannot mount block device cgroup read-only
Chowned zzz, but cannot verify the result
umount: /tmp/tmp.ePNDljkI6J: not mounted
test 10: unpriv movepid to chowned directory
method return sender=(null sender) -> dest=(null destination) reply_serial=1
test 11: unpriv setvalue to first cgroup
test 12: valid unpriv cgroup creation
test 13: valid unprivileged setvalue
   int32 1
   int32 -1
Test 14 (Remove)
Test 14 (nrtasks)
./test15.sh: line 22:  2032 Killed                  sleep 200
test 16: INvalid unprivileged setvalue
   int32 1
   int32 -1
test 17: chown
mount: block device cgroup is write-protected, mounting read-only
mount: cannot mount block device cgroup read-only
   int32 1
   int32 -1
Cannot verify results
   int32 1
   int32 -1
Error org.freedesktop.DBus.Error.InvalidArgs: invalid request
test 17 (chmod) passed
umount: /tmp/tmp.LCpkNbHNxk: not mounted
test 18: api_version
method return sender=(null sender) -> dest=(null destination) reply_serial=1
   variant       int32 4
test 19: escape
method return sender=(null sender) -> dest=(null destination) reply_serial=1
   int32 1
method return sender=(null sender) -> dest=(null destination) reply_serial=1
method return sender=(null sender) -> dest=(null destination) reply_serial=1
Error org.freedesktop.DBus.Error.InvalidArgs: invalid request
Test 20: ListChildren
method return sender=(null sender) -> dest=(null destination) reply_serial=1
   array [
      string "xxx"
      string "zzz"
      string "ooga"
      string "escapetest"
      string "test21_cg"
      string "testchown"
   ]
method return sender=(null sender) -> dest=(null destination) reply_serial=1
   int32 -1
Error org.freedesktop.DBus.Error.InvalidArgs: invalid request
method return sender=(null sender) -> dest=(null destination) reply_serial=1
   int32 1
method return sender=(null sender) -> dest=(null destination) reply_serial=1
   array [
   ]
Test 21: remove_on_empty
Error org.freedesktop.DBus.Error.InvalidArgs: Invalid arguments to Remove method
Error org.freedesktop.DBus.Error.InvalidArgs: Invalid arguments to Remove method
method return sender=(null sender) -> dest=(null destination) reply_serial=1
   int32 1
method return sender=(null sender) -> dest=(null destination) reply_serial=1
   int32 1
method return sender=(null sender) -> dest=(null destination) reply_serial=1
method return sender=(null sender) -> dest=(null destination) reply_serial=1
Error org.freedesktop.DBus.Error.InvalidArgs: invalid request
./test21.sh: line 21:  2099 Terminated              sleep 200
Failed to remove-on-empty
Test 21 failed.

cgproxy seems to be working on the container, but cgm ping fails:

root@lxc1:~# ps auxw|grep cgproot      8233  0.0  0.0  15232  1136 ?        Ss   14:27   0:00 /sbin/cgproxy --sigstop
root      8244  0.0  0.0   8868   808 pts/4    S+   14:27   0:00 grep --color=auto cgp
root@lxc1:~# cgm ping
method return sender=(null sender) -> dest=(null destination) reply_serial=1

Any ideas?

 

[jinjer]

bump