[SOLVED] LXC ID in broken state

liquidox

Renowned Member
Sep 21, 2016
37
4
73
80
I seem to have gotten a specific LXC ID in a broken state.

There have been 2-3 containers on this ID, at some point removed, then a new one cloned, which then has taken this free ID.

This has led to the following state, /var/log/syslog:
Jun 18 11:22:28 zoar pvestatd[31428]: lxc status update error: can't open '/sys/fs/cgroup/blkio/lxc/107/ns/blkio.throttle.io_service_bytes' - No such file or directory
Jun 18 11:22:28 zoar pvestatd[31428]: lxc console cleanup error: can't open '/sys/fs/cgroup/blkio/lxc/107/ns/blkio.throttle.io_service_bytes' - No such file or directory

There's several dashed entries for LXC ID 107 in the cgroup folders, this appears to cause the issue.

find /sys/fs/cgroup/ -name "107-*"
/sys/fs/cgroup/memory/lxc/107-3
/sys/fs/cgroup/memory/lxc/107-1
/sys/fs/cgroup/memory/lxc/107-2
/sys/fs/cgroup/perf_event/lxc/107-3
/sys/fs/cgroup/perf_event/lxc/107-1
/sys/fs/cgroup/perf_event/lxc/107-2
/sys/fs/cgroup/rdma/lxc/107-3
/sys/fs/cgroup/rdma/lxc/107-1
/sys/fs/cgroup/rdma/lxc/107-2
/sys/fs/cgroup/cpuset/lxc/107-3
/sys/fs/cgroup/cpuset/lxc/107-1
/sys/fs/cgroup/cpuset/lxc/107-2
/sys/fs/cgroup/cpu,cpuacct/lxc/107-3
/sys/fs/cgroup/cpu,cpuacct/lxc/107-1
/sys/fs/cgroup/cpu,cpuacct/lxc/107-2
/sys/fs/cgroup/blkio/lxc/107-3
/sys/fs/cgroup/blkio/lxc/107-1
/sys/fs/cgroup/blkio/lxc/107-2
/sys/fs/cgroup/pids/lxc/107-3
/sys/fs/cgroup/pids/lxc/107-1
/sys/fs/cgroup/pids/lxc/107-2
/sys/fs/cgroup/hugetlb/lxc/107-3
/sys/fs/cgroup/hugetlb/lxc/107-2

pveversion:
proxmox-ve: 5.4-1 (running kernel: 4.15.18-15-pve)
pve-manager: 5.4-6 (running version: 5.4-6/aa7856c5)
pve-kernel-4.15: 5.4-3
pve-kernel-4.15.18-15-pve: 4.15.18-40
pve-kernel-4.15.18-14-pve: 4.15.18-39
pve-kernel-4.15.18-12-pve: 4.15.18-36
corosync: 2.4.4-pve1
criu: 2.11.1-1~bpo90
glusterfs-client: 3.8.8-1
ksm-control-daemon: 1.2-2
libjs-extjs: 6.0.1-2
libpve-access-control: 5.1-10
libpve-apiclient-perl: 2.0-5
libpve-common-perl: 5.0-52
libpve-guest-common-perl: 2.0-20
libpve-http-server-perl: 2.0-13
libpve-storage-perl: 5.0-43
libqb0: 1.0.3-1~bpo9
lvm2: 2.02.168-pve6
lxc-pve: 3.1.0-3
lxcfs: 3.0.3-pve1
novnc-pve: 1.0.0-3
proxmox-widget-toolkit: 1.0-28
pve-cluster: 5.0-37
pve-container: 2.0-39
pve-docs: 5.4-2
pve-edk2-firmware: 1.20190312-1
pve-firewall: 3.0-22
pve-firmware: 2.0-6
pve-ha-manager: 2.0-9
pve-i18n: 1.1-4
pve-libspice-server1: 0.14.1-2
pve-qemu-kvm: 3.0.1-2
pve-xtermjs: 3.12.0-1
qemu-server: 5.0-52
smartmontools: 6.5+svn4324-1
spiceterm: 3.0-5
vncterm: 1.5-3
zfsutils-linux: 0.7.13-pve1~bpo2

At some point in time, the 107 container has had this in it's config, possibly related?
lxc.apparmor.profile: unconfined
lxc.cgroup.devices.allow: a
lxc.mount.auto: proc:rw

Any ideas how to fix this, can I (and should I) safely remove the "107-x" folders?

Note that this breaks more than just LXC 107. When 107 is turned on in this state, the overview of containers within Proxmox UI turns grey, all of them!
 
Last edited:
hi,

i guess you did an upgrade recently. you will need to reboot your pve node(s).
 
you're welcome. you can mark this thread [SOLVED] by editing your first post
 
I had the same problem, the solution was to delete the following paths:

find /sys/fs/cgroup/systemd/ID* -type d | tac | xargs rmdir
find /sys/fs/cgroup/systemd/lxc/ID* -type d | tac | xargs rmdir
find /sys/fs/cgroup/perf_event//lxc/ID* -type d | tac | xargs rmdir
find /var/lib/lxcfs/cgroup/blkio/lxc/ID* -type d | tac | xargs rmdir
find /var/lib/lxcfs/cgroup/cpuset/lxc/ID* -type d | tac | xargs rmdir
find /var/lib/lxcfs/cgroup/devices/lxc/ID* -type d | tac | xargs rmdir
find /var/lib/lxcfs/cgroup/freezer/lxc/ID* -type d | tac | xargs rmdir
find /var/lib/lxcfs/cgroup/hugetlb/lxc/ID* -type d | tac | xargs rmdir
find /var/lib/lxcfs/cgroup/memory/lxc/ID* -type d | tac | xargs rmdir
find /var/lib/lxcfs/cgroup/net_cls,net_prio/lxc/ID* -type d | tac | xargs rmdir
find /var/lib/lxcfs/cgroup/pids/lxc/ID* -type d | tac | xargs rmdir
find /sys/fs/cgroup/rdma//lxc/ID* -type d | tac | xargs rmdir
find /var/lib/lxcfs/cgroup/blkio/lxc/ID* -type d | tac | xargs rmdir

Best Regards