Debian buster LXC container

aalmenar · Jan 8, 2019

Hi everyone,

Just dropping a question i cant find around the forum.

I managed to create an image using dab but since some days ago, im unable to install/start any service on the container. Im constantly getting apparmor errors like this one when installing mariadb:

[510907.402767] audit: type=1400 audit(1546941509.083:189): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-9002_</var/lib/lxc>" name="/" pid=30859 comm="(install)" flags="rw, rslave"

Also with a clean container, trying to install apache2 also becomes practically impossible:

[511748.194258] audit: type=1400 audit(1546942349.893:190): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-9002_</var/lib/lxc>" name="/" pid=1698 comm="(pachectl)" flags="rw, rslave"

They were working nicely until 3 days ago.

Another service is dovecot with the same error:

[519311.829786] audit: type=1400 audit(1546949913.651:193): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-9002_</var/lib/lxc>" name="/" pid=24770 comm="(dovecot)" flags="rw, rslave"

The reason its to test software with the next debian version running on containers.

Any help will be appreciated.

Regards.

oguz · Jan 8, 2019

Can you post your container config? /etc/pve/lxc/CTID.conf

aalmenar · Jan 8, 2019

Hi oguz,

Yes:

Code:

arch: amd64
cores: 2
hostname: host10.mydomain.com
memory: 2048
net0: name=eth0,bridge=vmbr0,gw=192.0.2.1,gw6=2001:db8::,hwaddr=0A:C6:94:AB:B3:7A,ip=192.0.2.10/24,ip6=2001:db8:ffff::3/56,tag=10,type=veth
ostype: debian
rootfs: local-lvm:vm-9002-disk-0,quota=1,size=30G
swap: 0

oguz said:
Can you post your container config? /etc/pve/lxc/CTID.conf

oguz · Jan 8, 2019

I think we will need more information about your template to have a better idea of the problem.

Maybe it would be helpful to see the template's dab config file.

aalmenar said:
They where working nicely until 3 days ago.

Have you updated anything before this started happening? If yes, which packages?

aalmenar · Jan 8, 2019

The Template is like this:

Makefile:

Code:

BASEDIR:=$(shell dab basedir)

.DEFAULT_GOAL := all

.PHONY: all
all: info/init_ok global finalize

finalize:
    echo "10.0 " > ${BASEDIR}/etc/debian_version
    dab finalize

info/init_ok: dab.conf
    dab init
    touch $@
    dab bootstrap --minimal

.PHONY: clean
clean:
    dab clean
    rm -f *~

.PHONY: dist-clean
dist-clean:
    dab dist-clean
    rm -f *~

dab.conf:

Code:

Suite: buster
CacheDir: ../cache
Source: [URL]http://http.debian.net/debian[/URL] SUITE main contrib
Source: [URL]http://http.debian.net/debian[/URL] SUITE-updates main contrib
Source: [URL]http://http.debian.net/debian-security[/URL] SUITE/updates main contrib
Architecture: amd64
Name: debian-10.0-minimal
Version: 20190108-1
Section: system
Maintainer: Someone <someone@mydomain.com>
Infopage: [URL]http://mydomain.com[/URL]
Description: Debian 10 (minimal)
 A small Debian Buster system.

oguz said:
I think we will need more information about your template to have a better idea of the problem.

Maybe it would be helpful to see the template's dab config file.

Have you updated anything before this started happening? If yes, which packages?

Since it's kind of moving target, lots of packages, but installing a template made today and trying to exec anything of those packages give the same error. (I believe it's due to systemd update) since apache2 can start if on systemd unit i change PrivateTmp=true to false, but in mariadb this value is set to false and doesn't start.

fabian · Jan 9, 2019

might be related to the systemd v240 migration to testing on januaray 1st?

t.lamprecht · Jan 9, 2019

fabian said:
might be related to the systemd v240 migration to testing on januaray 1st?

Yes, systemd 240 enforces now a few things which were earlier not, the PrivateTmp option from the apache2 unit file failed also in stretch (would need a "lxc.apparmor.raw: mount options=(rw,make-rslave) -> **," config option, which is potentialy dangerous as it allows a lot more), but while in stretch systemd just continued, in buster it just fails. Also some other default things like rbind mounting root to "/run/systemd/unit-root/" fails and is not easily possible to allow through apparmor in a sane unproblematic way...

aalmenar · Jan 9, 2019

Yes. Earlier I though of systemd update as the main issue on this, but maybe there is a workaround, or just it is not possible?

My apparmor knowledge is scarce to say the least.

t.lamprecht said:
Yes, systemd 240 enforces now a few things which were earlier not, the PrivateTmp option from the apache2 unit file failed also in stretch (would need a "lxc.apparmor.raw: mount options=(rw,make-rslave) -> **," config option, which is potentialy dangerous as it allows a lot more), but while in stretch systemd just continued, in buster it just fails. Also some other default things like rbind mounting root to "/run/systemd/unit-root/" fails and is not easily possible to allow through apparmor in a sane unproblematic way...

t.lamprecht · Jan 9, 2019

aalmenar said:
Yes. Earlier I though of systemd update as the main issue on this, but maybe there is a workaround, or just it is not possible?

For now I'd suggest to downgrade systemd to 239 and pin that one. A colleague is already onto checking a bug in the apparmor parser, which may then help at least for the rslave mount issue.
For the rest I cannot tell anything for sure, besides that until buster gets stable this should get fixed (knocks wood).

aalmenar · Jan 9, 2019

I'd try but that package (systemd_239) is no longer available in testing.... will have to wait for now.

t.lamprecht said:
For now I'd suggest to downgrade systemd to 239 and pin that one. A colleague is already onto checking a bug in the apparmor parser, which may then help at least for the rslave mount issue.
For the rest I cannot tell anything for sure, besides that until buster gets stable this should get fixed (knocks wood).

sigo · Jan 9, 2019

You can enable feature "Nesting" (on tab Options) for LXC container. In my case (I'm also using sid) the same problem was solved.

aalmenar · Jan 9, 2019

Yes, it's working, at least for now i can continue testing. Thank you.

sigo said:
You can enable feature "Nesting" (on tab Options) for LXC container. In my case (I'm also using sid) the same problem was solved.

aalmenar · Jan 9, 2019

@t.lamprecht should i file a bug on debian on systemd package ?

t.lamprecht said:
For now I'd suggest to downgrade systemd to 239 and pin that one. A colleague is already onto checking a bug in the apparmor parser, which may then help at least for the rslave mount issue.
For the rest I cannot tell anything for sure, besides that until buster gets stable this should get fixed (knocks wood).

aalmenar · Jan 9, 2019

After filling a bug with debian systemd maintainers the response was this:

Can you disable AppArmor confinement and try again?
(with lxc 3.x that means setting lxc.apparmor.profile = unconfined)

Or use the config options from
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=916644

Please also tell us the lxc and apparmor versions you are using

Looping in the AA mantainer.

fabian · Jan 11, 2019

aalmenar said:
After filling a bug with debian systemd maintainers the response was this:

Can you disable AppArmor confinement and try again?
(with lxc 3.x that means setting lxc.apparmor.profile = unconfined)

Or use the config options from
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=916644

Please also tell us the lxc and apparmor versions you are using

Looping in the AA mantainer.

I guess you are referring to https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=918839 ? In general it is a good idea to mention that your are not running Debian in a Debian bug report to avoid wasting Debian maintainer time

PVE uses a custom LXC build and a different apparmor kernel version and feature set than Debian. In any case, I put some more info into the bug report

davidg1982 · Jun 27, 2019

sigo said:
You can enable feature "Nesting" (on tab Options) for LXC container. In my case (I'm also using sid) the same problem was solved.

How do I do this, specifically?

oguz · Jun 27, 2019

hi,

davidg1982 said:
How do I do this, specifically?

go to gui, you'll find it under Container -> Options -> Features -> Nesting

davidg1982 · Jun 27, 2019

oguz said:
hi,

go to gui, you'll find it under Container -> Options -> Features -> Nesting

Mine seems to not be editable. As in, no features are present. It's also an unprivileged container. Does that make a difference?

Edit:
I need privileged permissions. My bad.

davidg1982 · Jun 27, 2019

sigo said:
You can enable feature "Nesting" (on tab Options) for LXC container. In my case (I'm also using sid) the same problem was solved.

I want to understand the security implications of this. Why would I want to run nesting or not?

oguz · Jun 28, 2019

davidg1982 said:
I want to understand the security implications of this. Why would I want to run nesting or not?

it lets you mount /proc and /sys and some other stuff in rw mode, which could be dangerous. you're also allowed a greater range of syscalls than regular containers.

Debian buster LXC container

Member

Proxmox Retired Staff

Member

Proxmox Retired Staff

Member

Proxmox Staff Member

Proxmox Staff Member

Member

Proxmox Staff Member

Member

Active Member

Member

Member

Member

Proxmox Staff Member

Active Member

Proxmox Retired Staff

Active Member

Active Member

Proxmox Retired Staff