After updating yesterday I woke up to half of my containers refusing to start after they were backed up overnight.
logfile shows this on all of them:
Storage for the containers is an NFS share from TrueNas on the same machine. Mapallusers/group is set to owner (root). Temporarily setting permissions to 777 on the truenas dataset ACL doesn't change anything. The storage is visible in PVE and the half of the LXCs that boot are able to access it just fine, so I don't think it's an issue on the truenas side.
From what I've seen so far being unprivileged vs privileged isn't a factor since there are running containers that are unprivileged. Restoring from backup also fails to boot.
Not sure where to go from here
Code:
run_buffer: 571 Script exited with status 1
lxc_init: 845 Failed to run lxc.hook.pre-start for container "122"
__lxc_start: 2046 Failed to initialize container "122"
TASK ERROR: startup for container '122' failed
logfile shows this on all of them:
Code:
xc-start 122 20260202221806.881 INFO confile - ../src/lxc/confile.c:set_config_idmaps:2295 - Read uid map: type u nsid 0 hostid 100000 range 65536
lxc-start 122 20260202221806.881 INFO confile - ../src/lxc/confile.c:set_config_idmaps:2295 - Read uid map: type g nsid 0 hostid 100000 range 65536
lxc-start 122 20260202221806.881 INFO lsm - ../src/lxc/lsm/lsm.c:lsm_init_static:38 - Initialized LSM security driver AppArmor
lxc-start 122 20260202221806.881 INFO utils - ../src/lxc/utils.c:run_script_argv:587 - Executing script "/usr/share/lxc/hooks/lxc-pve-prestart-hook" for container "122", config section "lxc"
lxc-start 122 20260202221807.280 DEBUG utils - ../src/lxc/utils.c:run_buffer:560 - Script exec /usr/share/lxc/hooks/lxc-pve-prestart-hook 122 lxc pre-start produced output: failed to propagate access mode to mountpoint: Operation not permitted
lxc-start 122 20260202221807.289 ERROR utils - ../src/lxc/utils.c:run_buffer:571 - Script exited with status 1
lxc-start 122 20260202221807.289 ERROR start - ../src/lxc/start.c:lxc_init:845 - Failed to run lxc.hook.pre-start for container "122"
lxc-start 122 20260202221807.289 ERROR start - ../src/lxc/start.c:__lxc_start:2046 - Failed to initialize container "122"
lxc-start 122 20260202221807.289 INFO utils - ../src/lxc/utils.c:run_script_argv:587 - Executing script "/usr/share/lxc/hooks/lxc-pve-poststop-hook" for container "122", config section "lxc"
lxc-start 122 20260202221807.696 INFO utils - ../src/lxc/utils.c:run_script_argv:587 - Executing script "/usr/share/lxcfs/lxc.reboot.hook" for container "122", config section "lxc"
lxc-start 122 20260202221808.198 ERROR lxc_start - ../src/lxc/tools/lxc_start.c:lxc_start_main:307 - The container failed to start
lxc-start 122 20260202221808.198 ERROR lxc_start - ../src/lxc/tools/lxc_start.c:lxc_start_main:312 - Additional information can be obtained by setting the --logfile and --logpriority options
lxc-start 122 20260202223331.459 INFO confile - ../src/lxc/confile.c:set_config_idmaps:2295 - Read uid map: type u nsid 0 hostid 100000 range 65536
lxc-start 122 20260202223331.459 INFO confile - ../src/lxc/confile.c:set_config_idmaps:2295 - Read uid map: type g nsid 0 hostid 100000 range 65536
lxc-start 122 20260202223331.459 INFO lsm - ../src/lxc/lsm/lsm.c:lsm_init_static:38 - Initialized LSM security driver AppArmor
lxc-start 122 20260202223331.459 INFO utils - ../src/lxc/utils.c:run_script_argv:587 - Executing script "/usr/share/lxc/hooks/lxc-pve-prestart-hook" for container "122", config section "lxc"
lxc-start 122 20260202223332.795 DEBUG utils - ../src/lxc/utils.c:run_buffer:560 - Script exec /usr/share/lxc/hooks/lxc-pve-prestart-hook 122 lxc pre-start produced output: failed to propagate access mode to mountpoint: Operation not permitted
lxc-start 122 20260202223332.167 ERROR utils - ../src/lxc/utils.c:run_buffer:571 - Script exited with status 1
lxc-start 122 20260202223332.168 ERROR start - ../src/lxc/start.c:lxc_init:845 - Failed to run lxc.hook.pre-start for container "122"
lxc-start 122 20260202223332.168 ERROR start - ../src/lxc/start.c:__lxc_start:2046 - Failed to initialize container "122"
lxc-start 122 20260202223332.168 INFO utils - ../src/lxc/utils.c:run_script_argv:587 - Executing script "/usr/share/lxc/hooks/lxc-pve-poststop-hook" for container "122", config section "lxc"
lxc-start 122 20260202223332.431 INFO utils - ../src/lxc/utils.c:run_script_argv:587 - Executing script "/usr/share/lxcfs/lxc.reboot.hook" for container "122", config section "lxc"
lxc-start 122 20260202223332.933 ERROR lxc_start - ../src/lxc/tools/lxc_start.c:lxc_start_main:307 - The container failed to start
lxc-start 122 20260202223332.933 ERROR lxc_start - ../src/lxc/tools/lxc_start.c:lxc_start_main:312 - Additional information can be obtained by setting the --logfile and --logpriority options
Storage for the containers is an NFS share from TrueNas on the same machine. Mapallusers/group is set to owner (root). Temporarily setting permissions to 777 on the truenas dataset ACL doesn't change anything. The storage is visible in PVE and the half of the LXCs that boot are able to access it just fine, so I don't think it's an issue on the truenas side.
From what I've seen so far being unprivileged vs privileged isn't a factor since there are running containers that are unprivileged. Restoring from backup also fails to boot.
Not sure where to go from here


