cpanel dovecot resource issue with apparmor

mathx

Renowned Member
Jan 15, 2014
177
3
83
Had a situation where constraints from apparmor were causing problems with cpanel's dovecot. The container is NOT unpriviledged and not protected.

The cpanel support guy said I need

Code:
lxc.aa_profile = unconfined

But from what I understand in PVE, we want to pick a profile in /etc/apparmor.d/lxc, so I have

lxc.apparmor.profile: lxc-default-with-mounting (as cpanel uses simfs)

but this causes the container to not start:

Code:
Sep 27 00:58:34 theserver systemd[1]: Starting PVE LXC Container: 741...
Sep 27 00:58:34 theserver systemd-udevd[31096]: Could not generate persistent MAC address for veth9WU17P: No such file or directory
Sep 27 00:58:35 theserver lxc-start[31058]: lxc-start: 741: lxccontainer.c: wait_on_daemonized_start: 865 Received container state "ABORTING" instead of "RUNNING"
Sep 27 00:58:35 theserver lxc-start[31058]: lxc-start: 741: tools/lxc_start.c: main: 330 The container failed to start
Sep 27 00:58:35 theserver lxc-start[31058]: lxc-start: 741: tools/lxc_start.c: main: 333 To get more details, run the container in foreground mode
Sep 27 00:58:35 theserver lxc-start[31058]: lxc-start: 741: tools/lxc_start.c: main: 336 Additional information can be obtained by setting the --logfile and --logpriority options
Sep 27 00:58:35 theserver systemd[1]: pve-container@741.service: Control process exited, code=exited status=1
Sep 27 00:58:35 theserver systemd[1]: pve-container@741.service: Killing process 31064 (3) with signal SIGKILL.
Sep 27 00:58:35 theserver systemd[1]: Failed to start PVE LXC Container: 741.
Sep 27 00:58:35 theserver systemd[1]: pve-container@741.service: Unit entered failed state.
Sep 27 00:58:35 theserver systemd[1]: pve-container@741.service: Failed with result 'exit-code'.
Sep 27 00:58:35 theserver pvestatd[5970]: unable to get PID for CT 741 (not running?)
Sep 27 00:58:35 theserver pct[31056]: command 'systemctl start pve-container@741' failed: exit code 1
 
Forgot to include pve version:


pve-manager/5.4-13/aee6f0ec (running kernel: 4.15.18-20-pve)
 
of course now it all suddenly works if i have no apparmor profile defined with pct start. (no dovecot alerts)


if i start it with the manual lxc-start per the url above, it starts with the apparmor profile (despite some warnings?) :( will advise if this repeats. I have another container with the same problem but imap is not in use but the server is sensitive, can reboot it this weekend and see output.

#pveversion -v
proxmox-ve: 5.4-2 (running kernel: 4.15.18-20-pve)
pve-manager: 5.4-13 (running version: 5.4-13/aee6f0ec)
pve-kernel-4.15: 5.4-8
pve-kernel-4.15.18-20-pve: 4.15.18-46
pve-kernel-4.15.18-12-pve: 4.15.18-36
pve-kernel-4.15.18-10-pve: 4.15.18-32
corosync: 2.4.4-pve1
criu: 2.11.1-1~bpo90
glusterfs-client: 3.8.8-1
ksm-control-daemon: 1.2-2
libjs-extjs: 6.0.1-2
libpve-access-control: 5.1-12
libpve-apiclient-perl: 2.0-5
libpve-common-perl: 5.0-55
libpve-guest-common-perl: 2.0-20
libpve-http-server-perl: 2.0-14
libpve-storage-perl: 5.0-44
libqb0: 1.0.3-1~bpo9
lvm2: 2.02.168-pve6
lxc-pve: 3.1.0-6
lxcfs: 3.0.3-pve1
novnc-pve: 1.0.0-3
proxmox-widget-toolkit: 1.0-28
pve-cluster: 5.0-38
pve-container: 2.0-40
pve-docs: 5.4-2
pve-edk2-firmware: 1.20190312-1
pve-firewall: 3.0-22
pve-firmware: 2.0-7
pve-ha-manager: 2.0-9
pve-i18n: 1.1-4
pve-libspice-server1: 0.14.1-2
pve-qemu-kvm: 3.0.1-4
pve-xtermjs: 3.12.0-1
qemu-server: 5.0-54
smartmontools: 6.5+svn4324-1
spiceterm: 3.0-5
vncterm: 1.5-3
zfsutils-linux: 0.7.13-pve1~bpo2

console:
Code:
lxc-start: 741: cgroups/cgfsng.c: mkdir_eexist_on_last: 1301 File exists - Failed to create directory "/sys/fs/cgroup/cpuset//lxc/741"
                                                                                                                                      lxc-start: 741: cgroups/cgfsng.c: container_create_path_for_hierarchy: 1353 Failed to create cgroup "/sys/fs/cgroup/cpuset//lxc/741"
                            lxc-start: 741: cgroups/cgfsng.c: cgfsng_payload_create: 1526 Failed to create cgroup "/sys/fs/cgroup/cpuset//lxc/741"
                                                                                                                                                  systemd 219 running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 -SECCOMP +BLKID +ELFUTILS +KMOD +IDN)
Detected virtualization lxc.
Detected architecture x86-64.

Welcome to CentOS Linux 7 (Core)!

output to logfile attached.
 

Attachments

  • lxc.log
    18.5 KB · Views: 3
Last edited:
aha!! it does not work with the apparmor profile and pct start however!

Code:
#pct start 741
Job for pve-container@741.service failed because the control process exited with error code.
See "systemctl status pve-container@741.service" and "journalctl -xe" for details.
command 'systemctl start pve-container@741' failed: exit code 1

Code:
* pve-container@741.service - PVE LXC Container: 741
   Loaded: loaded (/lib/systemd/system/pve-container@.service; static; vendor preset: enabled)
   Active: failed (Result: exit-code) since Fri 2019-09-27 10:40:57 EDT; 2min 34s ago
     Docs: man:lxc-start
           man:lxc
           man:pct
  Process: 17275 ExecStart=/usr/bin/lxc-start -n 741 (code=exited, status=1/FAILURE)

Sep 27 10:40:56 deimos systemd[1]: Starting PVE LXC Container: 741...
Sep 27 10:40:57 deimos lxc-start[17275]: lxc-start: 741: lxccontainer.c: wait_on_daemonized_start: 865 Received container state "ABORTING" instead of "RUNNING"
Sep 27 10:40:57 deimos lxc-start[17275]: lxc-start: 741: tools/lxc_start.c: main: 330 The container failed to start
Sep 27 10:40:57 deimos lxc-start[17275]: lxc-start: 741: tools/lxc_start.c: main: 333 To get more details, run the container in foreground mode
Sep 27 10:40:57 deimos lxc-start[17275]: lxc-start: 741: tools/lxc_start.c: main: 336 Additional information can be obtained by setting the --logfile and --logpriority options
Sep 27 10:40:57 deimos systemd[1]: pve-container@741.service: Control process exited, code=exited status=1
Sep 27 10:40:57 deimos systemd[1]: pve-container@741.service: Killing process 17281 (3) with signal SIGKILL.
Sep 27 10:40:57 deimos systemd[1]: Failed to start PVE LXC Container: 741.
Sep 27 10:40:57 deimos systemd[1]: pve-container@741.service: Unit entered failed state.

Code:
Sep 27 10:40:56 deimos systemd[1]: Starting PVE LXC Container: 741...
Sep 27 10:40:56 deimos systemd-udevd[17460]: Could not generate persistent MAC address for veth0FINEH: No such file or directory
Sep 27 10:40:57 deimos lxc-start[17275]: lxc-start: 741: lxccontainer.c: wait_on_daemonized_start: 865 Received container state "ABORTING" instead of "RUNNING"
Sep 27 10:40:57 deimos lxc-start[17275]: lxc-start: 741: tools/lxc_start.c: main: 330 The container failed to start
Sep 27 10:40:57 deimos lxc-start[17275]: lxc-start: 741: tools/lxc_start.c: main: 333 To get more details, run the container in foreground mode
Sep 27 10:40:57 deimos lxc-start[17275]: lxc-start: 741: tools/lxc_start.c: main: 336 Additional information can be obtained by setting the --logfile and --logpriority options
Sep 27 10:40:57 deimos systemd[1]: pve-container@741.service: Control process exited, code=exited status=1
Sep 27 10:40:57 deimos systemd[1]: pve-container@741.service: Killing process 17281 (3) with signal SIGKILL.
Sep 27 10:40:57 deimos systemd[1]: Failed to start PVE LXC Container: 741.
 
This seems to have caused or is related to another issue I posted about previously, now some elements of /proc and /sys are unavailable:


# pct list
can't open '/sys/fs/cgroup/cpuacct/lxc/741/ns/cpuacct.stat' - No such file or directory

and when I enter into an existing container i get

printer:~# service mysql start
cat: /proc/cmdline: No such file or directory

had to ssh in. Something very strange is going on on this server.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!