[SOLVED] PVE8 with LXC upgraded to Debian 13 won't start, restore on PVE9 fails

M

Mr.Doubt

Member
Mar 6, 2022
85
32
23
Hello,

I know, PVE8 is old. Upgrade to 9 is planed.
Today I've upgraded LXC from Debian 13 to Debian 13.1. The containers won't start after Upgrade.

Code: 
pveversion -v
proxmox-ve: 8.4.0 (running kernel: 6.8.12-11-pve)
pve-manager: 8.4.12 (running version: 8.4.12/c2ea8261d32a5020)
proxmox-kernel-helper: 8.1.4
proxmox-kernel-6.8.12-14-pve-signed: 6.8.12-14
proxmox-kernel-6.8: 6.8.12-14
proxmox-kernel-6.8.12-13-pve-signed: 6.8.12-13
proxmox-kernel-6.8.12-11-pve-signed: 6.8.12-11
proxmox-kernel-6.5.13-6-pve-signed: 6.5.13-6
proxmox-kernel-6.5: 6.5.13-6
pve-kernel-5.3: 6.1-6
pve-kernel-5.4.162-1-pve: 5.4.162-2
pve-kernel-5.3.18-3-pve: 5.3.18-3
pve-kernel-5.3.10-1-pve: 5.3.10-1
amd64-microcode: 3.20250311.1~deb12u1
ceph-fuse: 16.2.15+ds-0+deb12u1
corosync: 3.1.9-pve1
criu: 3.17.1-2+deb12u2
glusterfs-client: 10.3-5
ifupdown: residual config
ifupdown2: 3.2.0-1+pmx11
ksm-control-daemon: 1.5-1
libjs-extjs: 7.0.0-5
libknet1: 1.30-pve2
libproxmox-acme-perl: 1.6.0
libproxmox-backup-qemu0: 1.5.2
libproxmox-rs-perl: 0.3.5
libpve-access-control: 8.2.2
libpve-apiclient-perl: 3.3.2
libpve-cluster-api-perl: 8.1.2
libpve-cluster-perl: 8.1.2
libpve-common-perl: 8.3.4
libpve-guest-common-perl: 5.2.2
libpve-http-server-perl: 5.2.2
libpve-network-perl: 0.11.2
libpve-rs-perl: 0.9.4
libpve-storage-perl: 8.3.7
libqb0: 1.0.5-1
libspice-server1: 0.15.1-1
lvm2: 2.03.16-2
lxc-pve: 6.0.0-1
lxcfs: 6.0.0-pve2
novnc-pve: 1.6.0-2
proxmox-backup-client: 3.4.6-1
proxmox-backup-file-restore: 3.4.6-1
proxmox-backup-restore-image: 0.7.0
proxmox-firewall: 0.7.1
proxmox-kernel-helper: 8.1.4
proxmox-mail-forward: 0.3.3
proxmox-mini-journalreader: 1.5
proxmox-offline-mirror-helper: 0.6.7
proxmox-widget-toolkit: 4.3.13
pve-cluster: 8.1.2
pve-container: 5.3.0
pve-docs: 8.4.1
pve-edk2-firmware: 4.2025.02-4~bpo12+1
pve-esxi-import-tools: 0.7.4
pve-firewall: 5.1.2
pve-firmware: 3.16-3
pve-ha-manager: 4.0.7
pve-i18n: 3.4.5
pve-qemu-kvm: 9.2.0-7
pve-xtermjs: 5.5.0-2
qemu-server: 8.4.1
smartmontools: 7.3-pve1
spiceterm: 3.3.0
swtpm: 0.8.0+pve1
vncterm: 1.8.0
zfsutils-linux: 2.2.8-pve1
Code: 
 pct config 114
arch: amd64
cores: 2
features: nesting=1
hostname: testhost
memory: 512
net0: name=eth0,bridge=vmbr0,gw=192.168.0.1,hwaddr=4A:6A:14:xx:xx:xx,ip=192.168.96.123/24,type=veth
ostype: debian
rootfs: local-lvm:vm-114-disk-0,size=4G
searchdomain: 
swap: 512
unprivileged: 1
Code: 
lxc-start -n 114 -F -l DEBUG -o /tmp/lxc-114.log
lxc-start: 114: ../src/lxc/utils.c: run_buffer: 571 Script exited with status 25
lxc-start: 114: ../src/lxc/start.c: lxc_init: 845 Failed to run lxc.hook.pre-start for container "114"
lxc-start: 114: ../src/lxc/start.c: __lxc_start: 2034 Failed to initialize container "114"
lxc-start: 114: ../src/lxc/tools/lxc_start.c: lxc_start_main: 307 The container failed to start
lxc-start: 114: ../src/lxc/tools/lxc_start.c: lxc_start_main: 312 Additional information can be obtained by setting the --logfile and --logpriority options
Code: 
cat /tmp/lxc-114.log
lxc-start 114 20250906172907.468 INFO     confile - ../src/lxc/confile.c:set_config_idmaps:2273 - Read uid map: type u nsid 0 hostid 100000 range 65536
lxc-start 114 20250906172907.468 INFO     confile - ../src/lxc/confile.c:set_config_idmaps:2273 - Read uid map: type g nsid 0 hostid 100000 range 65536
lxc-start 114 20250906172907.469 INFO     lsm - ../src/lxc/lsm/lsm.c:lsm_init_static:38 - Initialized LSM security driver AppArmor
lxc-start 114 20250906172907.469 INFO     utils - ../src/lxc/utils.c:run_script_argv:587 - Executing script "/usr/share/lxc/hooks/lxc-pve-prestart-hook" for container "114", config section "lxc"
lxc-start 114 20250906172908.513 DEBUG    utils - ../src/lxc/utils.c:run_buffer:560 - Script exec /usr/share/lxc/hooks/lxc-pve-prestart-hook 114 lxc pre-start produced output: unsupported debian version '13.1'

lxc-start 114 20250906172908.534 ERROR    utils - ../src/lxc/utils.c:run_buffer:571 - Script exited with status 25
lxc-start 114 20250906172908.534 ERROR    start - ../src/lxc/start.c:lxc_init:845 - Failed to run lxc.hook.pre-start for container "114"
lxc-start 114 20250906172908.534 ERROR    start - ../src/lxc/start.c:__lxc_start:2034 - Failed to initialize container "114"
lxc-start 114 20250906172908.534 INFO     utils - ../src/lxc/utils.c:run_script_argv:587 - Executing script "/usr/share/lxcfs/lxc.reboot.hook" for container "114", config section "lxc"
lxc-start 114 20250906172909.378 INFO     utils - ../src/lxc/utils.c:run_script_argv:587 - Executing script "/usr/share/lxc/hooks/lxc-pve-poststop-hook" for container "114", config section "lxc"
lxc-start 114 20250906172909.570 DEBUG    utils - ../src/lxc/utils.c:run_buffer:560 - Script exec /usr/share/lxc/hooks/lxc-pve-poststop-hook 114 lxc post-stop produced output: umount: /var/lib/lxc/.pve-staged-mounts/mp2: not mounted.

lxc-start 114 20250906172909.570 DEBUG    utils - ../src/lxc/utils.c:run_buffer:560 - Script exec /usr/share/lxc/hooks/lxc-pve-poststop-hook 114 lxc post-stop produced output: command 'umount -- /var/lib/lxc/.pve-staged-mounts/mp2' failed: exit code 32

lxc-start 114 20250906172909.577 DEBUG    utils - ../src/lxc/utils.c:run_buffer:560 - Script exec /usr/share/lxc/hooks/lxc-pve-poststop-hook 114 lxc post-stop produced output: umount: /var/lib/lxc/.pve-staged-mounts/mp0: not mounted.

lxc-start 114 20250906172909.577 DEBUG    utils - ../src/lxc/utils.c:run_buffer:560 - Script exec /usr/share/lxc/hooks/lxc-pve-poststop-hook 114 lxc post-stop produced output: command 'umount -- /var/lib/lxc/.pve-staged-mounts/mp0' failed: exit code 32

lxc-start 114 20250906172909.609 DEBUG    utils - ../src/lxc/utils.c:run_buffer:560 - Script exec /usr/share/lxc/hooks/lxc-pve-poststop-hook 114 lxc post-stop produced output: umount: /var/lib/lxc/.pve-staged-mounts/mp1: not mounted.

lxc-start 114 20250906172909.609 DEBUG    utils - ../src/lxc/utils.c:run_buffer:560 - Script exec /usr/share/lxc/hooks/lxc-pve-poststop-hook 114 lxc post-stop produced output: command 'umount -- /var/lib/lxc/.pve-staged-mounts/mp1' failed: exit code 32

lxc-start 114 20250906172909.657 ERROR    lxc_start - ../src/lxc/tools/lxc_start.c:lxc_start_main:307 - The container failed to start
lxc-start 114 20250906172909.657 ERROR    lxc_start - ../src/lxc/tools/lxc_start.c:lxc_start_main:312 - Additional information can be obtained by setting the --logfile and --logpriority options

Trying to restore the vzdump on PVE9 it fails too
Code: 
pveversion -v
proxmox-ve: 9.0.0 (running kernel: 6.14.8-2-pve)
pve-manager: 9.0.6 (running version: 9.0.6/49c767b70aeb6648)
proxmox-kernel-helper: 9.0.4
pve-kernel-5.15: 7.4-4
proxmox-kernel-6.14.11-1-pve-signed: 6.14.11-1
proxmox-kernel-6.14: 6.14.11-1
proxmox-kernel-6.14.8-2-pve-signed: 6.14.8-2
proxmox-kernel-6.8.12-13-pve-signed: 6.8.12-13
proxmox-kernel-6.8: 6.8.12-13
pve-kernel-5.15.108-1-pve: 5.15.108-1
pve-kernel-5.15.30-2-pve: 5.15.30-3
ceph-fuse: 19.2.3-pve1
corosync: 3.1.9-pve2
criu: 4.1.1-1
frr-pythontools: 10.3.1-1+pve4
ifupdown2: 3.3.0-1+pmx10
intel-microcode: 3.20250512.1
ksm-control-daemon: 1.5-1
libjs-extjs: 7.0.0-5
libproxmox-acme-perl: 1.7.0
libproxmox-backup-qemu0: 2.0.1
libproxmox-rs-perl: 0.4.1
libpve-access-control: 9.0.3
libpve-apiclient-perl: 3.4.0
libpve-cluster-api-perl: 9.0.6
libpve-cluster-perl: 9.0.6
libpve-common-perl: 9.0.9
libpve-guest-common-perl: 6.0.2
libpve-http-server-perl: 6.0.4
libpve-network-perl: 1.1.6
libpve-rs-perl: 0.10.10
libpve-storage-perl: 9.0.13
libspice-server1: 0.15.2-1+b1
lvm2: 2.03.31-2+pmx1
lxc-pve: 6.0.4-2
lxcfs: 6.0.4-pve1
novnc-pve: 1.6.0-3
proxmox-backup-client: 4.0.14-1
proxmox-backup-file-restore: 4.0.14-1
proxmox-backup-restore-image: 1.0.0
proxmox-firewall: 1.1.2
proxmox-kernel-helper: 9.0.4
proxmox-mail-forward: 1.0.2
proxmox-mini-journalreader: 1.6
proxmox-offline-mirror-helper: 0.7.1
proxmox-widget-toolkit: 5.0.5
pve-cluster: 9.0.6
pve-container: 6.0.9
pve-docs: 9.0.8
pve-edk2-firmware: 4.2025.02-4
pve-esxi-import-tools: 1.0.1
pve-firewall: 6.0.3
pve-firmware: 3.16-4
pve-ha-manager: 5.0.4
pve-i18n: 3.5.2
pve-qemu-kvm: 10.0.2-4
pve-xtermjs: 5.5.0-2
qemu-server: 9.0.19
smartmontools: 7.4-pve1
spiceterm: 3.4.0
swtpm: 0.8.0+pve2
vncterm: 1.9.0
zfsutils-linux: 2.3.4-pve1
From the task Window
Code: 
recovering backed-up configuration from 'pve-backup:backup/vzdump-lxc-114-2025_09_06-19_24_55.tar.zst'
  Logical volume "vm-100-disk-0" created.
  Logical volume pve/vm-100-disk-0 changed.
Creating filesystem with 1048576 4k blocks and 262144 inodes
Filesystem UUID: 4931a919-a557-494c-ab5a-cff9168b8b24
Superblock backups stored on blocks: 
    32768, 98304, 163840, 229376, 294912, 819200, 884736
restoring 'pve4-special:backup/vzdump-lxc-114-2025_09_06-19_24_55.tar.zst' now..
extracting archive '/mnt/pve/pve-backup/dump/vzdump-lxc-114-2025_09_06-19_24_55.tar.zst'
Total bytes read: 2129541120 (2.0GiB, 228MiB/s)
merging backed-up and given configuration..
  Logical volume "vm-100-disk-0" successfully removed.
TASK ERROR: unable to restore CT 100 - unsupported debian version '13.1'

If you need more debug information, tell me how.
 
Done further investigation

Restore a backup from the container, as it was Debian 13 works on the PVE9 from above.
Doing an upgrade to 13.1 on the PVE9 host and the container dosn't start anymore

Code: 
lxc-start -n 100 -F -l DEBUG -o /tmp/lxc-100.log
lxc-start: 100: ../src/lxc/utils.c: run_buffer: 571 Script exited with status 25
lxc-start: 100: ../src/lxc/start.c: lxc_init: 845 Failed to run lxc.hook.pre-start for container "100"
lxc-start: 100: ../src/lxc/start.c: __lxc_start: 2034 Failed to initialize container "100"
lxc-start: 100: ../src/lxc/tools/lxc_start.c: lxc_start_main: 307 The container failed to start
lxc-start: 100: ../src/lxc/tools/lxc_start.c: lxc_start_main: 312 Additional information can be obtained by setting the --logfile and --logpriority options

Code: 
cat /tmp/lxc-100.log
lxc-start 100 20250906175134.104 INFO     confile - ../src/lxc/confile.c:set_config_idmaps:2273 - Read uid map: type u nsid 0 hostid 100000 range 65536
lxc-start 100 20250906175134.104 INFO     confile - ../src/lxc/confile.c:set_config_idmaps:2273 - Read uid map: type g nsid 0 hostid 100000 range 65536
lxc-start 100 20250906175134.104 INFO     lsm - ../src/lxc/lsm/lsm.c:lsm_init_static:38 - Initialized LSM security driver AppArmor
lxc-start 100 20250906175134.104 INFO     utils - ../src/lxc/utils.c:run_script_argv:587 - Executing script "/usr/share/lxc/hooks/lxc-pve-prestart-hook" for container "100", config section "lxc"
lxc-start 100 20250906175134.902 DEBUG    utils - ../src/lxc/utils.c:run_buffer:560 - Script exec /usr/share/lxc/hooks/lxc-pve-prestart-hook 100 lxc pre-start produced output: unsupported debian version '13.1'

lxc-start 100 20250906175134.921 ERROR    utils - ../src/lxc/utils.c:run_buffer:571 - Script exited with status 25
lxc-start 100 20250906175134.922 ERROR    start - ../src/lxc/start.c:lxc_init:845 - Failed to run lxc.hook.pre-start for container "100"
lxc-start 100 20250906175134.922 ERROR    start - ../src/lxc/start.c:__lxc_start:2034 - Failed to initialize container "100"
lxc-start 100 20250906175134.922 INFO     utils - ../src/lxc/utils.c:run_script_argv:587 - Executing script "/usr/share/lxc/hooks/lxc-pve-poststop-hook" for container "100", config section "lxc"
lxc-start 100 20250906175135.561 DEBUG    utils - ../src/lxc/utils.c:run_buffer:560 - Script exec /usr/share/lxc/hooks/lxc-pve-poststop-hook 100 lxc post-stop produced output: umount: /var/lib/lxc/.pve-staged-mounts/mp0: not mounted.

lxc-start 100 20250906175135.561 DEBUG    utils - ../src/lxc/utils.c:run_buffer:560 - Script exec /usr/share/lxc/hooks/lxc-pve-poststop-hook 100 lxc post-stop produced output: command 'umount -- /var/lib/lxc/.pve-staged-mounts/mp0' failed: exit code 32

lxc-start 100 20250906175135.566 DEBUG    utils - ../src/lxc/utils.c:run_buffer:560 - Script exec /usr/share/lxc/hooks/lxc-pve-poststop-hook 100 lxc post-stop produced output: umount: /var/lib/lxc/.pve-staged-mounts/mp2: not mounted.

lxc-start 100 20250906175135.566 DEBUG    utils - ../src/lxc/utils.c:run_buffer:560 - Script exec /usr/share/lxc/hooks/lxc-pve-poststop-hook 100 lxc post-stop produced output: command 'umount -- /var/lib/lxc/.pve-staged-mounts/mp2' failed: exit code 32

lxc-start 100 20250906175135.571 DEBUG    utils - ../src/lxc/utils.c:run_buffer:560 - Script exec /usr/share/lxc/hooks/lxc-pve-poststop-hook 100 lxc post-stop produced output: umount: /var/lib/lxc/.pve-staged-mounts/mp1: not mounted.

lxc-start 100 20250906175135.571 DEBUG    utils - ../src/lxc/utils.c:run_buffer:560 - Script exec /usr/share/lxc/hooks/lxc-pve-poststop-hook 100 lxc post-stop produced output: command 'umount -- /var/lib/lxc/.pve-staged-mounts/mp1' failed: exit code 32

lxc-start 100 20250906175135.610 INFO     utils - ../src/lxc/utils.c:run_script_argv:587 - Executing script "/usr/share/lxcfs/lxc.reboot.hook" for container "100", config section "lxc"
lxc-start 100 20250906175136.113 ERROR    lxc_start - ../src/lxc/tools/lxc_start.c:lxc_start_main:307 - The container failed to start
lxc-start 100 20250906175136.113 ERROR    lxc_start - ../src/lxc/tools/lxc_start.c:lxc_start_main:312 - Additional information can be obtained by setting the --logfile and --logpriority options
 
Did you see this issue - you have the same error messsage I had (unsupported debian version '13.1') - editing /usr/share/perl5/PVE/LXC/Setup/Debian.pm to change the check to <=13.1 fixed it for me and allowed my updated 13.1 LXCs to start.

Edit: also this one - looks like lots of people having this problem today with the 13.1 release
 
Last edited:
Solved
Edit line 39 in /usr/share/perl5/PVE/LXC/Setup/Debian.pm
Code: 
    die "unsupported debian version '$version'\n" if !($version >= 4 && $version <= 14);
Restart works.
Restore works after systemctl restart pvedaemon.service
 
  • Like
Reactions: HaveBlue
Well, not to burst anyone's buble but it’s a stop-gap and will be overwritten by the next pve-container update. I’d only do it if you need a container right now and can’t wait for the packaged fix.
 
CanadianDjiinn said:
Well, not to burst anyone's buble but it’s a stop-gap and will be overwritten by the next pve-container update. I’d only do it if you need a container right now and can’t wait for the packaged fix.
Click to expand...
It was already fixed two weeks ago in pve-container update 6.0.10 (and 5.3.1 for PVE 8) - looks like the latest code only throws a warning, and then starts the container anyway even if Debian version is higher than max tested version (currently 14), instead of dying as was the case previously.
 
Last edited:
  • Like
Reactions: CanadianDjiinn
ycct said:
It was already fixed two weeks ago in pve-container update 6.0.10 (and 5.3.1 for PVE 8) - looks like the latest code only throws a warning, and then starts the container anyway even if Debian version is higher than max tested version (currently 14), instead of dying as was the case previously.
Click to expand...
Thank you, I guess I am behind on this.
 
You must log in or register to reply here.
Top Bottom