Unable to start any LXCs since DropBox was installed.

Bchewy

New Member
Jan 30, 2022
7
0
1
23
Hello all!

I run a simple proxmox server on the cloud, with some containers to make things more fun, and just experimenting different cloud technologies. However, ever since I've installed dropbox (I installed it on my root server - i know, bad choice!!) - It automatically took up majority of the space on my root drive (19gb), and now my containers don't ever start.

The context for dropbox is: I used it (and rsync) to keep my dropbox and folders in sync. Basically wanted to back up some underlying files, like the lxc config files - which i think may have been what has botched the containers, although the odd thing is: they're still there and i don' think that would be the issue?

Inital rsync command used to sync dropbox and the /var/lib/vz;
Code:
rsync -cogrtuv --delete --chown=root:root /var/lib/vz /root/Dropbox/ProxmoxBackups/devmount

I checked if apparmor was running on my root server, and this seems to be an error i get too :/
systemctl status apparmor
Code:
   Loaded: loaded (/lib/systemd/system/apparmor.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Wed 2022-08-17 04:27:57 UTC; 4s ago
     Docs: man:apparmor(7)
           https://gitlab.com/apparmor/apparmor/wikis/home/
  Process: 8285 ExecStart=/lib/apparmor/apparmor.systemd reload (code=exited, status=1/FAILURE)
 Main PID: 8285 (code=exited, status=1/FAILURE)

Aug 17 04:27:57 ns5010417 apparmor.systemd[8285]: /sbin/apparmor_parser: Unable to replace "smbldap-useradd".  Profile doesn't conform to protocol
Aug 17 04:27:57 ns5010417 apparmor.systemd[8285]: /sbin/apparmor_parser: Unable to replace "sanitized_helper".  Profile doesn't conform to protocol
Aug 17 04:27:57 ns5010417 apparmor.systemd[8285]: /sbin/apparmor_parser: Unable to replace "/usr/bin/pidgin".  Profile doesn't conform to protocol
Aug 17 04:27:57 ns5010417 apparmor.systemd[8285]: /sbin/apparmor_parser: Unable to replace "sanitized_helper".  Profile doesn't conform to protocol
Aug 17 04:27:57 ns5010417 apparmor.systemd[8285]: /sbin/apparmor_parser: Unable to replace "/usr/bin/totem".  Profile doesn't conform to protocol
Aug 17 04:27:57 ns5010417 apparmor.systemd[8285]: /sbin/apparmor_parser: Unable to replace "/usr/bin/totem-audio-preview".  Profile doesn't conform to protocol
Aug 17 04:27:57 ns5010417 apparmor.systemd[8285]: Error: At least one profile failed to load
Aug 17 04:27:57 ns5010417 systemd[1]: apparmor.service: Main process exited, code=exited, status=1/FAILURE
Aug 17 04:27:57 ns5010417 systemd[1]: apparmor.service: Failed with result 'exit-code'.
Aug 17 04:27:57 ns5010417 systemd[1]: Failed to start Load AppArmor profiles.


The error i get:
pct start 102 --debug
Code:
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
    LANGUAGE = (unset),
    LC_ALL = (unset),
    LC_TERMINAL = "iTerm2",
    LC_CTYPE = "UTF-8",
    LANG = "en_US.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to a fallback locale ("en_US.UTF-8").
run_apparmor_parser: 882 Failed to run apparmor_parser on "/var/lib/lxc/102/apparmor/lxc-102_<-var-lib-lxc>": apparmor_parser: Unable to replace "lxc-102_</var/lib/lxc>".  Profile doesn't conform to protocol
apparmor_prepare: 1052 Failed to load generated AppArmor profile
lxc_init: 832 Failed to initialize LSM
__lxc_start: 1945 Failed to initialize container "102"
g script "/usr/share/lxc/hooks/lxc-pve-prestart-hook" for container "102", config section "lxc"
DEBUG    terminal - terminal.c:lxc_terminal_peer_default:665 - No such device - The process does not have a controlling terminal
DEBUG    seccomp - seccomp.c:parse_config_v2:656 - Host native arch is [3221225534]
INFO     seccomp - seccomp.c:parse_config_v2:807 - Processing "reject_force_umount  # comment this to allow umount -f;  not recommended"
INFO     seccomp - seccomp.c:do_resolve_add_rule:524 - Set seccomp rule to reject force umounts
INFO     seccomp - seccomp.c:do_resolve_add_rule:524 - Set seccomp rule to reject force umounts
INFO     seccomp - seccomp.c:do_resolve_add_rule:524 - Set seccomp rule to reject force umounts
INFO     seccomp - seccomp.c:parse_config_v2:807 - Processing "[all]"
INFO     seccomp - seccomp.c:parse_config_v2:807 - Processing "kexec_load errno 1"
INFO     seccomp - seccomp.c:do_resolve_add_rule:566 - Adding native rule for syscall[246:kexec_load] action[327681:errno] arch[0]
INFO     seccomp - seccomp.c:do_resolve_add_rule:566 - Adding compat rule for syscall[246:kexec_load] action[327681:errno] arch[1073741827]
INFO     seccomp - seccomp.c:do_resolve_add_rule:566 - Adding compat rule for syscall[246:kexec_load] action[327681:errno] arch[1073741886]
INFO     seccomp - seccomp.c:parse_config_v2:807 - Processing "open_by_handle_at errno 1"
INFO     seccomp - seccomp.c:do_resolve_add_rule:566 - Adding native rule for syscall[304:open_by_handle_at] action[327681:errno] arch[0]
INFO     seccomp - seccomp.c:do_resolve_add_rule:566 - Adding compat rule for syscall[304:open_by_handle_at] action[327681:errno] arch[1073741827]
INFO     seccomp - seccomp.c:do_resolve_add_rule:566 - Adding compat rule for syscall[304:open_by_handle_at] action[327681:errno] arch[1073741886]
INFO     seccomp - seccomp.c:parse_config_v2:807 - Processing "init_module errno 1"
INFO     seccomp - seccomp.c:do_resolve_add_rule:566 - Adding native rule for syscall[175:init_module] action[327681:errno] arch[0]
INFO     seccomp - seccomp.c:do_resolve_add_rule:566 - Adding compat rule for syscall[175:init_module] action[327681:errno] arch[1073741827]
INFO     seccomp - seccomp.c:do_resolve_add_rule:566 - Adding compat rule for syscall[175:init_module] action[327681:errno] arch[1073741886]
INFO     seccomp - seccomp.c:parse_config_v2:807 - Processing "finit_module errno 1"
INFO     seccomp - seccomp.c:do_resolve_add_rule:566 - Adding native rule for syscall[313:finit_module] action[327681:errno] arch[0]
INFO     seccomp - seccomp.c:do_resolve_add_rule:566 - Adding compat rule for syscall[313:finit_module] action[327681:errno] arch[1073741827]
INFO     seccomp - seccomp.c:do_resolve_add_rule:566 - Adding compat rule for syscall[313:finit_module] action[327681:errno] arch[1073741886]
INFO     seccomp - seccomp.c:parse_config_v2:807 - Processing "delete_module errno 1"
INFO     seccomp - seccomp.c:do_resolve_add_rule:566 - Adding native rule for syscall[176:delete_module] action[327681:errno] arch[0]
INFO     seccomp - seccomp.c:do_resolve_add_rule:566 - Adding compat rule for syscall[176:delete_module] action[327681:errno] arch[1073741827]
INFO     seccomp - seccomp.c:do_resolve_add_rule:566 - Adding compat rule for syscall[176:delete_module] action[327681:errno] arch[1073741886]
INFO     seccomp - seccomp.c:parse_config_v2:807 - Processing "keyctl errno 38"
INFO     seccomp - seccomp.c:do_resolve_add_rule:566 - Adding native rule for syscall[250:keyctl] action[327718:errno] arch[0]
INFO     seccomp - seccomp.c:do_resolve_add_rule:566 - Adding compat rule for syscall[250:keyctl] action[327718:errno] arch[1073741827]
INFO     seccomp - seccomp.c:do_resolve_add_rule:566 - Adding compat rule for syscall[250:keyctl] action[327718:errno] arch[1073741886]
INFO     seccomp - seccomp.c:parse_config_v2:1017 - Merging compat seccomp contexts into main context
ERROR    apparmor - lsm/apparmor.c:run_apparmor_parser:882 - Failed to run apparmor_parser on "/var/lib/lxc/102/apparmor/lxc-102_<-var-lib-lxc>": apparmor_parser: Unable to replace "lxc-102_</var/lib/lxc>".  Profile doesn't conform to protocol
ERROR    apparmor - lsm/apparmor.c:apparmor_prepare:1052 - Failed to load generated AppArmor profile
ERROR    start - start.c:lxc_init:832 - Failed to initialize LSM
ERROR    start - start.c:__lxc_start:1945 - Failed to initialize container "102"
DEBUG    conf - conf.c:idmaptool_on_path_and_privileged:2741 - The binary "/usr/bin/newuidmap" does have the setuid bit set
DEBUG    conf - conf.c:idmaptool_on_path_and_privileged:2741 - The binary "/usr/bin/newgidmap" does have the setuid bit set
DEBUG    conf - conf.c:lxc_map_ids:2809 - Functional newuidmap and newgidmap binary found
startup for container '102' failed

I'm more or less aware that the apparmor config/profile was unable to load, and hence the container wasn't able to start. I've also debugged a little but I can't seem to find out what's the problem.

apparmor_parser --version
Code:
AppArmor parser version 2.13.2
Copyright (C) 1999-2008 Novell Inc.
Copyright 2009-2018 Canonical Ltd.

pct config 102
Code:
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
    LANGUAGE = (unset),
    LC_ALL = (unset),
    LC_TERMINAL = "iTerm2",
    LC_CTYPE = "UTF-8",
    LANG = "en_US.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to a fallback locale ("en_US.UTF-8").
arch: amd64
cores: 4
description: ID 100 template is the first template for ubuntu 18.04 configured for OVH Failover IPs%0A
hostname: bchewyme
memory: 16000
net0: name=eth0,bridge=vmbr0,firewall=1,hwaddr=02:00:00:c2:ca:5e,ip=dhcp,ip6=dhcp,type=veth
onboot: 1
ostype: ubuntu
rootfs: local:102/vm-102-disk-0.raw,size=32G
swap: 1000
unprivileged: 1
lxc.cgroup.devices.allow: c 10:200 rwm
lxc.mount.entry: /dev/net/tun dev/net/tun none bind,create=file

I would greatly appreciate any help, still a novice and learning as I go, thank you!
 
Last edited:
please include
- pveversion -v
- df -h
- systemctl list-units --failed
 
  • Like
Reactions: Bchewy
I installed it on my root server - i know, bad choice!!
Yes, uninstall and your LX(C) containers will hopefully work again. Never install stuff on your PVE host that will potentially break something, especially if it's not included in vanilla Debian.

I'd recommend installing DropBox inside of a container and bind-mount your container backup directly directly there so that you don't need the rsyncing and most importantly, you won't f**k up your PVE host.
 
  • Like
Reactions: Bchewy
please include
- pveversion -v
- df -h
- systemctl list-units --failed
Hello, Thanks so much for responding, I've listed the commands you mentioned below!

pveversion -v
Code:
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
    LANGUAGE = (unset),
    LC_ALL = (unset),
    LC_TERMINAL = "iTerm2",
    LC_CTYPE = "UTF-8",
    LANG = "en_US.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to a fallback locale ("en_US.UTF-8").
proxmox-ve: 6.4-1 (running kernel: 5.10.0-0.bpo.9-rt-amd64)
pve-manager: 6.4-13 (running version: 6.4-13/9f411e79)
pve-kernel-5.4: 6.4-12
pve-kernel-helper: 6.4-12
pve-kernel-5.4.162-1-pve: 5.4.162-2
ceph-fuse: 12.2.11+dfsg1-2.1+b1
corosync: 3.1.5-pve2~bpo10+1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: 0.8.35+pve1
libjs-extjs: 6.0.1-10
libknet1: 1.22-pve2~bpo10+1
libproxmox-acme-perl: 1.1.0
libproxmox-backup-qemu0: 1.1.0-1
libpve-access-control: 6.4-3
libpve-apiclient-perl: 3.1-3
libpve-common-perl: 6.4-4
libpve-guest-common-perl: 3.1-5
libpve-http-server-perl: 3.2-3
libpve-storage-perl: 6.4-1
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.6-2
lxcfs: 4.0.6-pve1
novnc-pve: 1.1.0-1
proxmox-backup-client: 1.1.13-2
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.6-1
pve-cluster: 6.4-1
pve-container: 3.3-6
pve-docs: 6.4-2
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-4
pve-firmware: 3.3-2
pve-ha-manager: 3.1-1
pve-i18n: 2.3-1
pve-qemu-kvm: 5.2.0-6
pve-xtermjs: 4.7.0-3
pve-zsync: 2.2
qemu-server: 6.4-2
smartmontools: 7.2-pve2
spiceterm: 3.1-1
vncterm: 1.6-2
zfsutils-linux: 2.0.7-pve1

df -H
XML:
Filesystem           Size  Used Avail Use% Mounted on
udev                  68G     0   68G   0% /dev
tmpfs                 14G   26M   14G   1% /run
/dev/md2              21G  4.2G   16G  21% /
tmpfs                 68G   48M   68G   1% /dev/shm
tmpfs                5.3M     0  5.3M   0% /run/lock
tmpfs                 68G     0   68G   0% /sys/fs/cgroup
/dev/mapper/vg-data  482G  125G  332G  28% /var/lib/vz
/dev/nvme1n1p1       536M  164k  536M   1% /boot/efi
/dev/fuse             32M   37k   32M   1% /etc/pve
tmpfs                 14G     0   14G   0% /run/user/0

systemctl list-units --failed
Code:
  UNIT                         LOAD   ACTIVE SUB    DESCRIPTION
● apparmor.service             loaded failed failed Load AppArmor profiles
● pve-container@101.service    loaded failed failed PVE LXC Container: 101
● pve-container@102.service    loaded failed failed PVE LXC Container: 102
● pve-container@108.service    loaded failed failed PVE LXC Container: 108
● systemd-modules-load.service loaded failed failed Load Kernel Modules

LOAD   = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB    = The low-level unit activation state, values depend on unit type.

5 loaded units listed. Pass --all to see loaded but inactive units, too.
To show all installed unit files use 'systemctl list-unit-files'.

Yes, uninstall and your LX(C) containers will hopefully work again. Never install stuff on your PVE host that will potentially break something, especially if it's not included in vanilla Debian.

I'd recommend installing DropBox inside of a container and bind-mount your container backup directly directly there so that you don't need the rsyncing and most importantly, you won't f**k up your PVE host.
I've just uninstalled it and deleted all the files, to no avail, is there a possible way I can rebuild the apparmor profile files? It seems the main issue here is the apparmor profiles
 
Last edited:
you can check with debsums whether any files got corrupted (it can only check files shipped by packages though).
 
you can check with debsums whether any files got corrupted (it can only check files shipped by packages though).
I tried this with debsums

My proxmox master is:
lsb_release -a
Code:
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 10 (buster)
Release:        10
Codename:       buster

debsums -s
Code:
debsums: changed file /lib/systemd/system/networking.service (from ifupdown package)

That was the only thing that returned but i don't suppose it has anything to do with the the apparmor profile issue?
Please advise, and thank you greatly for all the help!
 
totally missed the running kernel, and yeah, that will definitely cause problems!
 
I'll give it a try, and update, will also need to learn a little more about the underlying kernel and how it affect Proxmox. Thanks both!
 
IT FIXED IT!!

Thanks so much for the help you both,


I assume: having proxmox running on a seperate kernel would break a lot of things, and my guess is that when installing dropbox headless somewhere in there it tried to reboot into the other kernel and hence breaking tons of stuff?

My root server also ran out of space, and that could possibly be why?
 
for modern kernels it mostly works okay (except for ZFS being built-in in the PVE case and not in the mainline kernel case), but for old kernel versions like that our Ubuntu-based kernel had a lot of essential patches for things like namespaces and apparmor that made running containers pretty much impossible on stock kernels.
 
  • Like
Reactions: Bchewy
for modern kernels it mostly works okay (except for ZFS being built-in in the PVE case and not in the mainline kernel case), but for old kernel versions like that our Ubuntu-based kernel had a lot of essential patches for things like namespaces and apparmor that made running containers pretty much impossible on stock kernels.
Thanks for sharing fabian!
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!