cpanel dovecot resource issue with apparmor

[mathx]

Had a situation where constraints from apparmor were causing problems with cpanel's dovecot. The container is NOT unpriviledged and not protected.

The cpanel support guy said I need

lxc.aa_profile = unconfined

But from what I understand in PVE, we want to pick a profile in /etc/apparmor.d/lxc, so I have

lxc.apparmor.profile: lxc-default-with-mounting (as cpanel uses simfs)

but this causes the container to not start:

Sep 27 00:58:34 theserver systemd[1]: Starting PVE LXC Container: 741...
Sep 27 00:58:34 theserver systemd-udevd[31096]: Could not generate persistent MAC address for veth9WU17P: No such file or directory
Sep 27 00:58:35 theserver lxc-start[31058]: lxc-start: 741: lxccontainer.c: wait_on_daemonized_start: 865 Received container state "ABORTING" instead of "RUNNING"
Sep 27 00:58:35 theserver lxc-start[31058]: lxc-start: 741: tools/lxc_start.c: main: 330 The container failed to start
Sep 27 00:58:35 theserver lxc-start[31058]: lxc-start: 741: tools/lxc_start.c: main: 333 To get more details, run the container in foreground mode
Sep 27 00:58:35 theserver lxc-start[31058]: lxc-start: 741: tools/lxc_start.c: main: 336 Additional information can be obtained by setting the --logfile and --logpriority options
Sep 27 00:58:35 theserver systemd[1]: pve-container@741.service: Control process exited, code=exited status=1
Sep 27 00:58:35 theserver systemd[1]: pve-container@741.service: Killing process 31064 (3) with signal SIGKILL.
Sep 27 00:58:35 theserver systemd[1]: Failed to start PVE LXC Container: 741.
Sep 27 00:58:35 theserver systemd[1]: pve-container@741.service: Unit entered failed state.
Sep 27 00:58:35 theserver systemd[1]: pve-container@741.service: Failed with result 'exit-code'.
Sep 27 00:58:35 theserver pvestatd[5970]: unable to get PID for CT 741 (not running?)
Sep 27 00:58:35 theserver pct[31056]: command 'systemctl start pve-container@741' failed: exit code 1

 

[mathx]

Forgot to include pve version:


pve-manager/5.4-13/aee6f0ec (running kernel: 4.15.18-20-pve)

 

[oguz]

hi,

please post the output of pveversion -v, and provide a debug log[0] of the container start.

[0]: https://pve.proxmox.com/pve-docs/chapter-pct.html#_obtaining_debugging_logs

 

[mathx]

of course now it all suddenly works if i have no apparmor profile defined with pct start. (no dovecot alerts)


if i start it with the manual lxc-start per the url above, it starts with the apparmor profile (despite some warnings?) will advise if this repeats. I have another container with the same problem but imap is not in use but the server is sensitive, can reboot it this weekend and see output.

#pveversion -v
proxmox-ve: 5.4-2 (running kernel: 4.15.18-20-pve)
pve-manager: 5.4-13 (running version: 5.4-13/aee6f0ec)
pve-kernel-4.15: 5.4-8
pve-kernel-4.15.18-20-pve: 4.15.18-46
pve-kernel-4.15.18-12-pve: 4.15.18-36
pve-kernel-4.15.18-10-pve: 4.15.18-32
corosync: 2.4.4-pve1
criu: 2.11.1-1~bpo90
glusterfs-client: 3.8.8-1
ksm-control-daemon: 1.2-2
libjs-extjs: 6.0.1-2
libpve-access-control: 5.1-12
libpve-apiclient-perl: 2.0-5
libpve-common-perl: 5.0-55
libpve-guest-common-perl: 2.0-20
libpve-http-server-perl: 2.0-14
libpve-storage-perl: 5.0-44
libqb0: 1.0.3-1~bpo9
lvm2: 2.02.168-pve6
lxc-pve: 3.1.0-6
lxcfs: 3.0.3-pve1
novnc-pve: 1.0.0-3
proxmox-widget-toolkit: 1.0-28
pve-cluster: 5.0-38
pve-container: 2.0-40
pve-docs: 5.4-2
pve-edk2-firmware: 1.20190312-1
pve-firewall: 3.0-22
pve-firmware: 2.0-7
pve-ha-manager: 2.0-9
pve-i18n: 1.1-4
pve-libspice-server1: 0.14.1-2
pve-qemu-kvm: 3.0.1-4
pve-xtermjs: 3.12.0-1
qemu-server: 5.0-54
smartmontools: 6.5+svn4324-1
spiceterm: 3.0-5
vncterm: 1.5-3
zfsutils-linux: 0.7.13-pve1~bpo2

console:

lxc-start: 741: cgroups/cgfsng.c: mkdir_eexist_on_last: 1301 File exists - Failed to create directory "/sys/fs/cgroup/cpuset//lxc/741"
                                                                                                                                      lxc-start: 741: cgroups/cgfsng.c: container_create_path_for_hierarchy: 1353 Failed to create cgroup "/sys/fs/cgroup/cpuset//lxc/741"
                            lxc-start: 741: cgroups/cgfsng.c: cgfsng_payload_create: 1526 Failed to create cgroup "/sys/fs/cgroup/cpuset//lxc/741"
                                                                                                                                                  systemd 219 running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 -SECCOMP +BLKID +ELFUTILS +KMOD +IDN)
Detected virtualization lxc.
Detected architecture x86-64.

Welcome to CentOS Linux 7 (Core)!

output to logfile attached.

 

[mathx]

aha!! it does not work with the apparmor profile and pct start however!

#pct start 741
Job for pve-container@741.service failed because the control process exited with error code.
See "systemctl status pve-container@741.service" and "journalctl -xe" for details.
command 'systemctl start pve-container@741' failed: exit code 1

* pve-container@741.service - PVE LXC Container: 741
   Loaded: loaded (/lib/systemd/system/pve-container@.service; static; vendor preset: enabled)
   Active: failed (Result: exit-code) since Fri 2019-09-27 10:40:57 EDT; 2min 34s ago
     Docs: man:lxc-start
           man:lxc
           man:pct
  Process: 17275 ExecStart=/usr/bin/lxc-start -n 741 (code=exited, status=1/FAILURE)

Sep 27 10:40:56 deimos systemd[1]: Starting PVE LXC Container: 741...
Sep 27 10:40:57 deimos lxc-start[17275]: lxc-start: 741: lxccontainer.c: wait_on_daemonized_start: 865 Received container state "ABORTING" instead of "RUNNING"
Sep 27 10:40:57 deimos lxc-start[17275]: lxc-start: 741: tools/lxc_start.c: main: 330 The container failed to start
Sep 27 10:40:57 deimos lxc-start[17275]: lxc-start: 741: tools/lxc_start.c: main: 333 To get more details, run the container in foreground mode
Sep 27 10:40:57 deimos lxc-start[17275]: lxc-start: 741: tools/lxc_start.c: main: 336 Additional information can be obtained by setting the --logfile and --logpriority options
Sep 27 10:40:57 deimos systemd[1]: pve-container@741.service: Control process exited, code=exited status=1
Sep 27 10:40:57 deimos systemd[1]: pve-container@741.service: Killing process 17281 (3) with signal SIGKILL.
Sep 27 10:40:57 deimos systemd[1]: Failed to start PVE LXC Container: 741.
Sep 27 10:40:57 deimos systemd[1]: pve-container@741.service: Unit entered failed state.

Sep 27 10:40:56 deimos systemd[1]: Starting PVE LXC Container: 741...
Sep 27 10:40:56 deimos systemd-udevd[17460]: Could not generate persistent MAC address for veth0FINEH: No such file or directory
Sep 27 10:40:57 deimos lxc-start[17275]: lxc-start: 741: lxccontainer.c: wait_on_daemonized_start: 865 Received container state "ABORTING" instead of "RUNNING"
Sep 27 10:40:57 deimos lxc-start[17275]: lxc-start: 741: tools/lxc_start.c: main: 330 The container failed to start
Sep 27 10:40:57 deimos lxc-start[17275]: lxc-start: 741: tools/lxc_start.c: main: 333 To get more details, run the container in foreground mode
Sep 27 10:40:57 deimos lxc-start[17275]: lxc-start: 741: tools/lxc_start.c: main: 336 Additional information can be obtained by setting the --logfile and --logpriority options
Sep 27 10:40:57 deimos systemd[1]: pve-container@741.service: Control process exited, code=exited status=1
Sep 27 10:40:57 deimos systemd[1]: pve-container@741.service: Killing process 17281 (3) with signal SIGKILL.
Sep 27 10:40:57 deimos systemd[1]: Failed to start PVE LXC Container: 741.

 

[mathx]

This seems to have caused or is related to another issue I posted about previously, now some elements of /proc and /sys are unavailable:


# pct list
can't open '/sys/fs/cgroup/cpuacct/lxc/741/ns/cpuacct.stat' - No such file or directory

and when I enter into an existing container i get

printer:~# service mysql start
cat: /proc/cmdline: No such file or directory

had to ssh in. Something very strange is going on on this server.

 

[mathx]

This seems related to an issue Im having again now:

https://forum.proxmox.com/threads/p...cgroup-cpuacct-lxc-673-ns-cpuacct-stat.68430/