LXC postqueue BUG: warning: close: Permission denied

[yena]

Hello,
i think LCX DEBIAN 8 container have a Bug:

# mailq
# postqueue: warning: close: Permission denied

Proxmox Kernel:
Linux prx15 4.2.2-1-pve #1 SMP Mon Oct 5 18:23:31 CEST 2015 x86_64 GNU/Linux

Some Solutions ?

Thansk

 

[MasterTH]

hi,

facing the same issue. all proxmox software is up to date.
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1446906 - seems to be the an really old issue.

kind regards

 

[MasterTH]

also found this:
http://www.claudiokuenzler.com/blog/558/ubuntu-trusty-mailq-container-permission-denied-kernel-issue

may this helps an dev to fix the issue

 

[MasterTH]

found another bug report and the same dmesg output inside the lxc container:
[126657.474390] audit: type=1400 audit(1446622766.591:3392930): apparmor="DENIED" operation="file_perm" profile="lxc-container-default" name="public/showq" pid=13755 comm="postqueue" requested_mask="r" denied_mask="r" fsuid=2001 ouid=0
[126657.474399] audit: type=1400 audit(1446622766.591:3392931): apparmor="DENIED" operation="file_perm" profile="lxc-container-default" name="public/showq" pid=13755 comm="postqueue" requested_mask="r" denied_mask="r" fsuid=2001 ouid=0

Here's the link:
https://bugs.launchpad.net/ubuntu/utopic/+source/linux/+bug/1390223

can someone of the dev have a look into please?

and another one:
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1446906

 

[dietmar]

can someone of the dev have a look into please?

As you see, Wolfang is already working on that.

 

[wbumiller]

It's a general apparmor3 issue. Install an ubuntu wily, `apt-get install postfix`, then try to run `aa-exec -p $someprofile mailq` and you'll get a permission denied error. Running postfix with strace causes it to sometimes work. It might be a race condition somewhere in the kernel side apparmor code... :-\

 

[MasterTH]

hmm.. ok - is there a solution for this?
disabling apparmor??

 

[wbumiller]

That's a possible workaround, not a solution ;-)
We're looking into it but for now it looks like it's up to the apparmor devs to fix this.

 

[MasterTH]

sometimes a workaround is also a solution

 

[lhorace]

While this may not be a show-stopper IE. postfix still runs fine (albiet permission denied errors), it would be great to get a workaround to this issue (beside's disabling apparmor all together)?

 

[MasterTH]

disabling apparmor doesn't allow the lxc container to start anymore:

pct start 101
lxc-start: lxc_start.c: main: 344 The container failed to start.
lxc-start: lxc_start.c: main: 346 To get more details, run the container in foreground mode.
lxc-start: lxc_start.c: main: 348 Additional information can be obtained by setting the --logfile and --logpriority options.

this is what is in /var/log/lxc/101.log

      lxc-start 1447142931.734 ERROR    lxc_conf - conf.c:instantiate_veth:2730 - failed to create veth pair (veth101i1 and vethQ7UN1F): File exists
      lxc-start 1447142931.773 ERROR    lxc_conf - conf.c:lxc_create_network:3047 - failed to create netdev
      lxc-start 1447142931.773 ERROR    lxc_start - start.c:lxc_spawn:954 - failed to create the network
      lxc-start 1447142931.773 ERROR    lxc_start - start.c:__lxc_start:1211 - failed to spawn '101'
      lxc-start 1447142937.117 ERROR    lxc_start_ui - lxc_start.c:main:344 - The container failed to start.
      lxc-start 1447142937.117 ERROR    lxc_start_ui - lxc_start.c:main:346 - To get more details, run the container in foreground mode.
      lxc-start 1447142937.117 ERROR    lxc_start_ui - lxc_start.c:main:348 - Additional information can be obtained by setting the --logfile and --logpriority options.
      lxc-start 1447142957.589 ERROR    lxc_conf - conf.c:instantiate_veth:2730 - failed to create veth pair (veth101i0 and veth56ATUV): File exists
      lxc-start 1447142957.609 ERROR    lxc_conf - conf.c:lxc_create_network:3047 - failed to create netdev
      lxc-start 1447142957.609 ERROR    lxc_start - start.c:lxc_spawn:954 - failed to create the network
      lxc-start 1447142957.609 ERROR    lxc_start - start.c:__lxc_start:1211 - failed to spawn '101'
      lxc-start 1447142962.949 ERROR    lxc_start_ui - lxc_start.c:main:344 - The container failed to start.
      lxc-start 1447142962.949 ERROR    lxc_start_ui - lxc_start.c:main:346 - To get more details, run the container in foreground mode.
      lxc-start 1447142962.949 ERROR    lxc_start_ui - lxc_start.c:main:348 - Additional information can be obtained by setting the --logfile and --logpriority options.

 

[wbumiller]

That looks like an unrelated bug. Trying to start it twice should work. (That one however we can work around ... though I'd much rather catch it. (seems to be somewhere in lxc's shutdown, failing to remove old interfaces))

 

[MasterTH]

i tried it twice
i'll give it a second try.

 

[MasterTH]

the container is restarting after a few minutes of waiting. but the permission denied error message is still present.

the problem is, that mails that are not could be delivered (mailadress not available or sth else) are sent back to sender every hour and are not been deleted from the queue.


EDIT:
This is still, even after disabling apparmor (had not rebootet the whole hardware-server (should i've done this))

 

[Ovidiu]

quick question:

I am seign the same, I had a problem with a new mailserver so the queue got full, not that I fixed the problem, I am getting all those emails and it looks to me like they get sent to me but then postfix cannot delete them so I seem to be getting the same emails over and over again as I am unable to flush the queue.

Or am I misreading this bug?

 

[ipp-2nd]

the bug is still existing. i think the apparmor devs should have a look into it, but it will take some time.

we were facing the same issue and i helped us with disabling the lxc in apparmor.

simple ln -s both lxc* files in /etc/apparmor to the disabled folder and after that restart the daemon or reboot the whole server. This will fix the permission problems in lxc.
but be aware - this could be a security issue if you are not sure what your customers are doing inside the containers. If you would like to be safe use kvm instead (for us this wasn't possible)


kind regards
IP-Projects GmbH - Second Level Support

 

[Ovidiu]

THX! I'll certainly give this idea a try, the containers are for myself, not for clients, I just need separation between different projects of mine.
One container contains ISPCFG3 where I host web sites and email but there is no root access except chrooted access so there isn't much that can go wrong there if I disable apparmor for this container either.

 

[wbumiller]

So here's a quick update: I've talked to the AppArmor devs and it's now confirmed. They initially didn't think much it since the bug was reported for LXC and looked exactly like a different issue caused by missing flags in the profile. This, however, is not the case and the problem seems to be a bit more tricky to solve, but some ideas are already floating around. Unfortunately I cannot say much about the time needed to deal with this. It seems to be in the kernel so the fix will require a reboot of your nodes.
Meanwhile we're still experimenting with user namespaces for unprivileged containers. With these disabling AppArmor wouldn't be that much of a problem as there the root user of a container maps to an unprivileged normal user on the host.

 

[Ovidiu]

@ipp-2nd

OK, got home and wanted to give it a try but unsure what exactly you meant.

i.e. in another post I saw how to fix apparmor interfering with named:

ln -s /etc/apparmor.d/usr.sbin.named /etc/apparmor.d/disable/
Reload named profile in apparmor
apparmor_parser -R /etc/apparmor.d/usr.sbin.named

Restart bind
service bind9

Can you give me more details? Are you talking about the /etc/apparmor.d/ or the /etc/apparmor/ folder? And which lxc* files exactly are you talking?

 

[ipp-2nd]

exactly i meant the apparmor.d folder