LCX not starting after upgrading PVe from 9.1.x to 9.2.2 - Mount fails

[elendiir]

Hi,
I just have upgraded our PVE environment from 9.1.x (not sure which version it has been) to 9.2.2. using the productive repro. PVE-Container version is 6.1.10
Now I am not able to start any LXCs anymore.

Starting the LCX in debug shows me the following errors:

lxc-start 501 20260612093837.936 DEBUG    utils - ../src/lxc/utils.c:run_buffer:558 - Script exec /usr/share/lxcfs/lxc.mount.hook '501' 'lxc' 'mount'
produced output: mount: /usr/lib/x86_64-linux-gnu/lxc/rootfs/proc/cpuinfo: bind /var/lib/lxcfs/proc/cpuinfo failed.
       dmesg(1) may have more information after failed mount system call.
lxc-start 501 20260612093837.936 ERROR    utils - ../src/lxc/utils.c:run_buffer:569 - Script exited with status 32
lxc-start 501 20260612093837.936 ERROR    conf - ../src/lxc/conf.c:lxc_setup:3845 - Failed to run mount hooks
lxc-start 501 20260612093837.936 ERROR    start - ../src/lxc/start.c:do_start:1466 - Failed to setup container "501"
lxc-start 501 20260612093837.936 ERROR    sync - ../src/lxc/sync.c:sync_wait:34 - An error occurred in another process (expected sequence number 4)
lxc-start 501 20260612093837.936 DEBUG    network - ../src/lxc/network.c:lxc_delete_network:4493 - Deleted network devices

This is one example LCX.conf:

arch: amd64
cores: 2
features: keyctl=1,nesting=1
hostname: pihole
memory: 2048
net0: name=eth0,bridge=vmbr0,gw=10.0.0.254,hwaddr=BC:24:11:39:41:85,ip=10.0.0.8/24,ip6=auto,type=veth
onboot: 1
ostype: debian
rootfs: local-lvm:vm-501-disk-0,size=10G
startup: order=1,up=10,down=10
swap: 512
tags: 
unprivileged: 1

The LCXs have been working fine before the update. It happens on all our 3 servers.

Any idea what to do? I have found older threads having a similar issue. But all of those seems to be fixed with 9.1.x already.

Thanks
Fritz

 

[Impact]

Have you checked dmesg as it says?

 

[elendiir]

Sorry I forgot to post this part. I am not sure what part is of interest. I think this line might show the error?

[Jun12 13:24] audit: type=1400 audit(1781263485.458:15832): apparmor="DENIED" operation="getattr" class="posix_mqueue" profile="/usr/bin/lxc-st>
[  +0.051371] audit: type=1400 audit(1781263485.510:15833): apparmor="DENIED" operation="getattr" class="posix_mqueue" profile="/usr/bin/lxc-st>
[  +0.064925] EXT4-fs (dm-6): mounted filesystem 2fe804d6-8a10-4df5-b260-b496f8a968ed r/w with ordered data mode. Quota mode: none.
[  +0.233953] audit: type=1400 audit(1781263485.808:15834): apparmor="STATUS" operation="profile_load" profile="/usr/bin/lxc-start" name="lxc-5>
[  +0.494820] vmbr0: port 2(veth501i0) entered blocking state
[  +0.000006] vmbr0: port 2(veth501i0) entered disabled state
[  +0.000027] veth501i0: entered allmulticast mode
[  +0.000052] veth501i0: entered promiscuous mode
[  +0.036034] eth0: renamed from vethojCVBk
[  +0.125635] audit: type=1400 audit(1781263486.465:15835): apparmor="STATUS" operation="profile_remove" profile="/usr/bin/lxc-start" name="lxc>
[  +0.021103] vmbr0: port 2(veth501i0) entered disabled state
[  +0.000210] veth501i0 (unregistering): left allmulticast mode
[  +0.000004] veth501i0 (unregistering): left promiscuous mode
[  +0.000004] vmbr0: port 2(veth501i0) entered disabled state
[  +0.421669] EXT4-fs (dm-6): unmounting filesystem 2fe804d6-8a10-4df5-b260-b496f8a968ed.
[Jun12 13:26] audit: type=1400 audit(1781263597.426:15836): apparmor="DENIED" operation="getattr" class="posix_mqueue" profile="/usr/bin/lxc-st>
[  +0.055083] audit: type=1400 audit(1781263597.481:15837): apparmor="DENIED" operation="getattr" class="posix_mqueue" profile="/usr/bin/lxc-st>
[  +0.075805] EXT4-fs (dm-6): mounted filesystem 2fe804d6-8a10-4df5-b260-b496f8a968ed r/w with ordered data mode. Quota mode: none.
[  +0.218860] audit: type=1400 audit(1781263597.776:15838): apparmor="STATUS" operation="profile_load" profile="/usr/bin/lxc-start" name="lxc-5>
[  +0.496341] vmbr0: port 2(veth501i0) entered blocking state
[  +0.000007] vmbr0: port 2(veth501i0) entered disabled state
[  +0.000028] veth501i0: entered allmulticast mode
[  +0.000057] veth501i0: entered promiscuous mode
[  +0.040510] eth0: renamed from veth8dD3mA
[  +0.123692] audit: type=1400 audit(1781263598.437:15839): apparmor="STATUS" operation="profile_remove" profile="/usr/bin/lxc-start" name="lxc>
[  +0.021143] vmbr0: port 2(veth501i0) entered disabled state
[  +0.000212] veth501i0 (unregistering): left allmulticast mode
[  +0.000003] veth501i0 (unregistering): left promiscuous mode
[  +0.000003] vmbr0: port 2(veth501i0) entered disabled state
[  +0.435085] EXT4-fs (dm-6): unmounting filesystem 2fe804d6-8a10-4df5-b260-b496f8a968ed.

 

[Impact]

Does apt -U dist-upgrade go through without warnings/errors? What's systemctl --failed say? Does it behave/look different after a reboot of the node?