[SOLVED] Failed to start CT - ( Failed to start Load AppArmor profiles )

Kudaki

New Member
Oct 13, 2021
6
0
1
47
Hello !! Thank you very much for the forum. I have installed a server with proxmox and this week when I update it to version 7, I find that the ct does not start. It shows me an apparmor error, I have searched a lot on google and this same forum, I have tried to execute the various solutions of the posts that I have read, and unfortunately none have given me the solution. Because they still can't start the ct. I have created a new test ct with debian and it does not work for me either. if anyone could give me a hand i would be very grateful. I leave here the different outputs that the commands in proxmox give me. Thank you.

root@pve:/# pveversion -v
proxmox-ve: 7.0-2 (running kernel: 5.11.22-5-pve)
pve-manager: 7.0-13 (running version: 7.0-13/7aa7e488)
pve-kernel-helper: 7.1-2
pve-kernel-5.11: 7.0-8
pve-kernel-5.4: 6.4-6
pve-kernel-5.11.22-5-pve: 5.11.22-10
pve-kernel-5.11.22-4-pve: 5.11.22-9
pve-kernel-5.4.140-1-pve: 5.4.140-1
pve-kernel-5.4.34-1-pve: 5.4.34-2
ceph-fuse: 14.2.21-1
corosync: 3.1.5-pve1
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown: 0.8.36+pve1
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.22-pve1
libproxmox-acme-perl: 1.3.0
libproxmox-backup-qemu0: 1.2.0-1
libpve-access-control: 7.0-5
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.0-10
libpve-guest-common-perl: 4.0-2
libpve-http-server-perl: 4.0-3
libpve-storage-perl: 7.0-12
libqb0: 1.0.5-1
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 4.0.9-4
lxcfs: 4.0.8-pve2
novnc-pve: 1.2.0-3
proxmox-backup-client: 2.0.11-1
proxmox-backup-file-restore: 2.0.11-1
proxmox-mini-journalreader: 1.2-1
proxmox-widget-toolkit: 3.3-6
pve-cluster: 7.0-3
pve-container: 4.0-10
pve-docs: 7.0-5
pve-edk2-firmware: 3.20210831-1
pve-firewall: 4.2-4
pve-firmware: 3.3-2
pve-ha-manager: 3.3-1
pve-i18n: 2.5-1
pve-qemu-kvm: 6.0.0-4
pve-xtermjs: 4.12.0-1
qemu-server: 7.0-16
smartmontools: 7.2-pve2
spiceterm: 3.2-2
vncterm: 1.7-1
zfsutils-linux: 2.0.5-pve1





root@pve:/# pct start 105
apparmor_prepare: 1083 Cannot use generated profile: apparmor_parser not available
lxc_init: 850 Failed to initialize LSM
__lxc_start: 2007 Failed to initialize container "105"
startup for container '105' failed
 

oguz

Proxmox Staff Member
Staff member
Nov 19, 2018
4,895
617
118
hi,

can you please also post outputs from the following:
* apparmor_parser --version
* pct config 105
* cat /var/lib/lxc/105/config
 

Kudaki

New Member
Oct 13, 2021
6
0
1
47
Thank you very much for answering me.
I don't speak much English, I am using google translator.
These are the outputs that the commands you have requested give me.

root@pve:~# apparmor_parser --version
AppArmor parser version 2.13.6
Copyright (C) 1999-2008 Novell Inc.
Copyright 2009-2018 Canonical Ltd.

root@pve:~# pct config 105
arch: amd64
cores: 1
hostname: Issabel.Persigranca.Backup
memory: 3072
net0: name=eth0,bridge=vmbr0,firewall=1,gw=192.168.2.1,hwaddr=66:F5:3D:07:4D:9,ip=192.168.2.129/24,type=veth
onboot: 0
ostype: centos
rootfs: local-lvm:vm-105-disk-0,size=20G
swap: 0
unprivileged: 1


root@pve:~# cat /var/lib/lxc/105/config
lxc.cgroup.relative = 0
lxc.cgroup.dir.monitor = lxc.monitor/105
lxc.cgroup.dir.container = lxc/105
lxc.cgroup.dir.container.inner = ns
lxc.arch = amd64
lxc.include = /usr/share/lxc/config/centos.common.conf
lxc.include = /usr/share/lxc/config/centos.userns.conf
lxc.seccomp.profile = /var/lib/lxc/105/rules.seccomp
lxc.apparmor.profile = generated
lxc.apparmor.raw = deny mount -> /proc/,
lxc.apparmor.raw = deny mount -> /sys/,
lxc.mount.auto = sys:mixed
lxc.monitor.unshare = 1
lxc.idmap = u 0 100000 65536
lxc.idmap = g 0 100000 65536
lxc.tty.max = 2
lxc.environment = TERM=linux
lxc.uts.name = Issabel.Persigranca.Backup
lxc.cgroup2.memory.max = 3221225472
lxc.cgroup2.memory.swap.max = 0
lxc.rootfs.path = /var/lib/lxc/105/rootfs
lxc.net.0.type = veth
lxc.net.0.veth.pair = veth105i0
lxc.net.0.hwaddr = 66:F5:3D:07:4D:9
lxc.net.0.name = eth0
lxc.net.0.script.up = /usr/share/lxc/lxcnetaddbr
lxc.cgroup2.cpuset.cpus = 0
root@pve:~#

The error keeps showing up on proxmox version 7.0-13. Currently I get to have the container running on another server with proxmox 6.2-4. The error message is as follows.

run_apparmor_parser: 919 Failed to run apparmor_parser on "/var/lib/lxc/105/apparmor/lxc-105_<-var-lib-lxc>": AppArmor parser error for /var/lib/lxc/105/apparmor/lxc-105_<-var-lib-lxc> in /var/lib/lxc/105/apparmor/lxc-105_<-var-lib-lxc> at line 1: Could not open 'tunables/global'
apparmor_prepare: 1089 Failed to load generated AppArmor profile
lxc_init: 850 Failed to initialize LSM
__lxc_start: 2007 Failed to initialize container "105"
TASK ERROR: startup for container '105' failed
 
Last edited:

oguz

Proxmox Staff Member
Staff member
Nov 19, 2018
4,895
617
118
thank you for the outputs.

could you also run pct start 105 --debug and post the log here?
 

Kudaki

New Member
Oct 13, 2021
6
0
1
47
Here it is !!! Thank you very much for the help!!!

root@pve:~# pct start 105 --debug
run_apparmor_parser: 919 Failed to run apparmor_parser on "/var/lib/lxc/105/appa rmor/lxc-105_<-var-lib-lxc>": AppArmor parser error for /var/lib/lxc/105/apparmo r/lxc-105_<-var-lib-lxc> in /var/lib/lxc/105/apparmor/lxc-105_<-var-lib-lxc> at line 1: Could not open 'tunables/global'
apparmor_prepare: 1089 Failed to load generated AppArmor profile
lxc_init: 850 Failed to initialize LSM
__lxc_start: 2007 Failed to initialize container "105"
r "105", config section "lxc"
DEBUG terminal - terminal.c:lxc_terminal_peer_default:665 - No such device - The process does not have a controlling terminal
DEBUG seccomp - seccomp.c:parse_config_v2:656 - Host native arch is [32212255 34]
INFO seccomp - seccomp.c:parse_config_v2:807 - Processing "reject_force_umou nt # comment this to allow umount -f; not recommended"
INFO seccomp - seccomp.c:do_resolve_add_rule:524 - Set seccomp rule to rejec t force umounts
INFO seccomp - seccomp.c:do_resolve_add_rule:524 - Set seccomp rule to rejec t force umounts
INFO seccomp - seccomp.c:do_resolve_add_rule:524 - Set seccomp rule to rejec t force umounts
INFO seccomp - seccomp.c:parse_config_v2:807 - Processing "[all]"
INFO seccomp - seccomp.c:parse_config_v2:807 - Processing "kexec_load errno 1"
INFO seccomp - seccomp.c:do_resolve_add_rule:564 - Adding native rule for sy scall[246:kexec_load] action[327681:errno] arch[0]
INFO seccomp - seccomp.c:do_resolve_add_rule:564 - Adding compat rule for sy scall[246:kexec_load] action[327681:errno] arch[1073741827]
INFO seccomp - seccomp.c:do_resolve_add_rule:564 - Adding compat rule for sy scall[246:kexec_load] action[327681:errno] arch[1073741886]
INFO seccomp - seccomp.c:parse_config_v2:807 - Processing "open_by_handle_at errno 1"
INFO seccomp - seccomp.c:do_resolve_add_rule:564 - Adding native rule for sy scall[304:eek:pen_by_handle_at] action[327681:errno] arch[0]
INFO seccomp - seccomp.c:do_resolve_add_rule:564 - Adding compat rule for sy scall[304:eek:pen_by_handle_at] action[327681:errno] arch[1073741827]
INFO seccomp - seccomp.c:do_resolve_add_rule:564 - Adding compat rule for sy scall[304:eek:pen_by_handle_at] action[327681:errno] arch[1073741886]
INFO seccomp - seccomp.c:parse_config_v2:807 - Processing "init_module errno 1"
INFO seccomp - seccomp.c:do_resolve_add_rule:564 - Adding native rule for sy scall[175:init_module] action[327681:errno] arch[0]
INFO seccomp - seccomp.c:do_resolve_add_rule:564 - Adding compat rule for sy scall[175:init_module] action[327681:errno] arch[1073741827]
INFO seccomp - seccomp.c:do_resolve_add_rule:564 - Adding compat rule for sy scall[175:init_module] action[327681:errno] arch[1073741886]
INFO seccomp - seccomp.c:parse_config_v2:807 - Processing "finit_module errn o 1"
INFO seccomp - seccomp.c:do_resolve_add_rule:564 - Adding native rule for sy scall[313:finit_module] action[327681:errno] arch[0]
INFO seccomp - seccomp.c:do_resolve_add_rule:564 - Adding compat rule for sy scall[313:finit_module] action[327681:errno] arch[1073741827]
INFO seccomp - seccomp.c:do_resolve_add_rule:564 - Adding compat rule for sy scall[313:finit_module] action[327681:errno] arch[1073741886]
INFO seccomp - seccomp.c:parse_config_v2:807 - Processing "delete_module err no 1"
INFO seccomp - seccomp.c:do_resolve_add_rule:564 - Adding native rule for sy scall[176:delete_module] action[327681:errno] arch[0]
INFO seccomp - seccomp.c:do_resolve_add_rule:564 - Adding compat rule for sy scall[176:delete_module] action[327681:errno] arch[1073741827]
INFO seccomp - seccomp.c:do_resolve_add_rule:564 - Adding compat rule for sy scall[176:delete_module] action[327681:errno] arch[1073741886]
INFO seccomp - seccomp.c:parse_config_v2:807 - Processing "ioctl errno 1 [1, 0x9400,SCMP_CMP_MASKED_EQ,0xff00]"
INFO seccomp - seccomp.c:do_resolve_add_rule:547 - arg_cmp[0]: SCMP_CMP(1, 7 , 65280, 37888)
INFO seccomp - seccomp.c:do_resolve_add_rule:564 - Adding native rule for sy scall[16:ioctl] action[327681:errno] arch[0]
INFO seccomp - seccomp.c:do_resolve_add_rule:547 - arg_cmp[0]: SCMP_CMP(1, 7 , 65280, 37888)
INFO seccomp - seccomp.c:do_resolve_add_rule:564 - Adding compat rule for sy scall[16:ioctl] action[327681:errno] arch[1073741827]
INFO seccomp - seccomp.c:do_resolve_add_rule:547 - arg_cmp[0]: SCMP_CMP(1, 7 , 65280, 37888)
INFO seccomp - seccomp.c:do_resolve_add_rule:564 - Adding compat rule for sy scall[16:ioctl] action[327681:errno] arch[1073741886]
INFO seccomp - seccomp.c:parse_config_v2:807 - Processing "keyctl errno 38"
INFO seccomp - seccomp.c:do_resolve_add_rule:564 - Adding native rule for sy scall[250:keyctl] action[327718:errno] arch[0]
INFO seccomp - seccomp.c:do_resolve_add_rule:564 - Adding compat rule for sy scall[250:keyctl] action[327718:errno] arch[1073741827]
INFO seccomp - seccomp.c:do_resolve_add_rule:564 - Adding compat rule for sy scall[250:keyctl] action[327718:errno] arch[1073741886]
INFO seccomp - seccomp.c:parse_config_v2:1017 - Merging compat seccomp conte xts into main context
ERROR apparmor - lsm/apparmor.c:run_apparmor_parser:919 - Failed to run appar mor_parser on "/var/lib/lxc/105/apparmor/lxc-105_<-var-lib-lxc>": AppArmor parse r error for /var/lib/lxc/105/apparmor/lxc-105_<-var-lib-lxc> in /var/lib/lxc/105 /apparmor/lxc-105_<-var-lib-lxc> at line 1: Could not open 'tunables/global'
ERROR apparmor - lsm/apparmor.c:apparmor_prepare:1089 - Failed to load genera ted AppArmor profile
ERROR start - start.c:lxc_init:850 - Failed to initialize LSM
ERROR start - start.c:__lxc_start:2007 - Failed to initialize container "105"
WARN cgfsng - cgroups/cgfsng.c:cgfsng_payload_destroy:538 - Uninitialized li mit cgroup
WARN cgfsng - cgroups/cgfsng.c:cgfsng_monitor_destroy:904 - Uninitialized mo nitor cgroup
INFO conf - conf.c:run_script_argv:332 - Executing script "/usr/share/lxcfs/ lxc.reboot.hook" for container "105", config section "lxc"
startup for container '105' failed
 

oguz

Proxmox Staff Member
Staff member
Nov 19, 2018
4,895
617
118
the error seems to be:
Code:
run_apparmor_parser: 919 Failed to run apparmor_parser on "/var/lib/lxc/105/apparmor/lxc-105_<-var-lib-lxc>": AppArmor parser error for /var/lib/lxc/105/apparmor/lxc-105_<-var-lib-lxc> in /var/lib/lxc/105/apparmor/lxc-105_<-var-lib-lxc> at line 1: Could not open 'tunables/global'

do you have this file on your system: /etc/apparmor.d/tunables/global

you could check the output of the following command: ls -al /etc/apparmor.d/tunables/

should look like the following:
Code:
root@pve:~# ls -al /etc/apparmor.d/tunables/
total 52
drwxr-xr-x 1 root root  242 Aug  4 11:32 .
drwxr-xr-x 1 root root  296 Aug  4 11:38 ..
-rw-r--r-- 1 root root  624 Apr  3  2021 alias
-rw-r--r-- 1 root root  376 Apr  3  2021 apparmorfs
-rw-r--r-- 1 root root  804 Apr  3  2021 dovecot
-rw-r--r-- 1 root root  744 Apr  3  2021 global
-rw-r--r-- 1 root root  983 Apr  3  2021 home
drwxr-xr-x 1 root root   32 Aug  4 11:32 home.d
-rw-r--r-- 1 root root 1391 Apr  3  2021 kernelvars
-rw-r--r-- 1 root root  631 Apr  3  2021 multiarch
drwxr-xr-x 1 root root   20 Aug  4 11:32 multiarch.d
-rw-r--r-- 1 root root  440 Apr  3  2021 proc
-rw-r--r-- 1 root root   23 Apr  3  2021 run
-rw-r--r-- 1 root root  405 Apr  3  2021 securityfs
-rw-r--r-- 1 root root  819 Apr  3  2021 share
-rw-r--r-- 1 root root  378 Apr  3  2021 sys
-rw-r--r-- 1 root root  868 Apr  3  2021 xdg-user-dirs
drwxr-xr-x 1 root root   20 Aug  4 11:32 xdg-user-dirs.d

also for the next time you're posting outputs please use [code][/code] tags :)
 

Kudaki

New Member
Oct 13, 2021
6
0
1
47
It seems that the error is there, indeed the directory does not look like that.
Let's see if I don't screw up when I post the result. :)

Code:
root@pve:/# ls -al /etc/apparmor.d/tunables/
total 24
drwxrwxrwx 5 root root 4096 Oct 14 12:00 .
drwxrwxrwx 7 root root 4096 Oct 14 12:00 ..
drwxr-xr-x 2 root root 4096 Oct 14 12:00 home.d
drwxr-xr-x 2 root root 4096 Oct 14 12:00 multiarch.d
-rw-r--r-- 1 root root  428 Oct 14 11:07 proc
drwxr-xr-x 2 root root 4096 Oct 14 11:49 xdg-user-dirs.d
 

oguz

Proxmox Staff Member
Staff member
Nov 19, 2018
4,895
617
118
It seems that the error is there, indeed the directory does not look like that.
Let's see if I don't screw up when I post the result. :)

Code:
root@pve:/# ls -al /etc/apparmor.d/tunables/
total 24
drwxrwxrwx 5 root root 4096 Oct 14 12:00 .
drwxrwxrwx 7 root root 4096 Oct 14 12:00 ..
drwxr-xr-x 2 root root 4096 Oct 14 12:00 home.d
drwxr-xr-x 2 root root 4096 Oct 14 12:00 multiarch.d
-rw-r--r-- 1 root root  428 Oct 14 11:07 proc
drwxr-xr-x 2 root root 4096 Oct 14 11:49 xdg-user-dirs.d
okay, please try the following:

Code:
$ mkdir foo
$ cd foo
$ wget https://debian-repo.stanford.edu/debian/pool/main/a/apparmor/apparmor_2.13.6.orig.tar.gz
$ tar xzvf *.tar.gz
$ cp -r apparmor*/profiles/apparmor.d/tunables/ /etc/apparmor.d

and then try starting the container. if it still doesn't work you can try with the --debug option again (maybe other files are missing as well...)
 

Kudaki

New Member
Oct 13, 2021
6
0
1
47
Many thanks !!!! at the end the container has started. I get a message of warnings -1, but it has managed to start. I guess now, to remove that warnings message, Do I have to do ... "pct enter 105" from the console and update the container with "yum update"?

Code:
root@pve:/foo# pct start 105
WARN: old systemd (< v232) detected, container won't run in a pure cgroupv2 environment! Please see documentation -> container -> cgroup version.
Task finished with 1 warning(s)!
root@pve:/foo#
 

oguz

Proxmox Staff Member
Staff member
Nov 19, 2018
4,895
617
118
Many thanks !!!! at the end the container has started.
great! you're welcome :)

I get a message of warnings -1, but it has managed to start. I guess now, to remove that warnings message, Do I have to do ... "pct enter 105" from the console and update the container with "yum update"?
yes, i guess you were using centos 7 so that's why you'd get that warning. see our documentation section [0] for our recommendations (one of them is upgrading to centos 8)

[0]: https://pve.proxmox.com/pve-docs/chapter-pct.html#pct_cgroup_compat
 

Kudaki

New Member
Oct 13, 2021
6
0
1
47
Thank you !!!
The container does not take me from the internet, so I am mounting everything again in Centos v8, and so it will surely go!
You can give the issue as solved, thank you very much for all the help.
A huge greeting from the Canary Islands !!!
 

oguz

Proxmox Staff Member
Staff member
Nov 19, 2018
4,895
617
118
The container does not take me from the internet, so I am mounting everything again in Centos v8, and so it will surely go!
you should be able to run ifup eth0 inside the container and then it can work

You can give the issue as solved, thank you very much for all the help.
great you're welcome :)
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!