Restore CT Failing; "error reading pxar archive: unexpected offset"

J

jasonsansone

Well-Known Member
May 17, 2021
174
51
48
Oklahoma City, OK
www.sansonehowell.com
recovering backed-up configuration from 'PBS:backup/ct/100/2025-05-25T00:14:55Z'
restoring 'PBS:backup/ct/100/2025-05-25T00:14:55Z' now..
Error: error extracting archive - encountered unexpected error during extraction: error reading pxar archive: unexpected offset 121669021136, smaller than already consumed payload 121669419904
TASK ERROR: unable to restore CT 109 - command '/usr/bin/proxmox-backup-client restore '--crypt-mode=none' ct/100/2025-05-25T00:14:55Z root.pxar /var/lib/lxc/109/rootfs --allow-existing-dirs --repository root@pam@pbs.redacted.com:Backups' failed: exit code 255

I need to restore a backup, but I seem to encounter this error no matter what I try. I have attempt to restore manually from the CLI, with a larger fsroot and with ignore errors flag. Neither changed the outcome. I have searched for this particular error, and can't find any reference in the forums. Ideas?
 
Hi,
please share the output of pveversion --verbose. Also, did you already try to run a snapshot verification on the PBS for the snapshot in question. Does a restore of the snapshot before this one work? Also, please share the backup task log for the backup run which created this backup.
 
Seven daily backups exist for the container, covering May 18 through May 24. All backups were verified at the time of creation and have been manually re-verified. Backing storage is ZFS, which has been scrubbed. Restoration of any backup would be satisfactory, but the results for each has been the same. The container hosts, among other things, a sqlite database which for unknown reasons began to grow exponentially out of control until the container ran out of available space and locked up. The May 19 backup is the seemingly appropriate size, but subsequent backups are substantially larger than the size of the lxc container. The db itself is likely corrupted, but backups of the db exists within the container backup which can be easily restored if the entire container can be restored. I am sure that some corruption exists as to individual files, but ignoring those checks seems to have no effect. Unfortunately, backup task logs were pruned.

proxmox-ve: 8.4.0 (running kernel: 6.14.0-2-pve)
pve-manager: 8.4.1 (running version: 8.4.1/2a5fa54a8503f96d)
proxmox-kernel-helper: 8.1.1
proxmox-kernel-6.14.5-1-bpo12-pve-signed: 6.14.5-1~bpo12+1
proxmox-kernel-6.14: 6.14.5-1~bpo12+1
proxmox-kernel-6.14.0-2-pve-signed: 6.14.0-2
proxmox-kernel-6.11.11-2-pve-signed: 6.11.11-2
proxmox-kernel-6.11: 6.11.11-2
proxmox-kernel-6.8.12-11-pve-signed: 6.8.12-11
proxmox-kernel-6.8: 6.8.12-11
ceph-fuse: 17.2.8-pve2
corosync: 3.1.9-pve1
criu: 3.17.1-2+deb12u1
glusterfs-client: 10.3-5
ifupdown: residual config
ifupdown2: 3.2.0-1+pmx11
intel-microcode: 3.20250512.1~deb12u1
ksm-control-daemon: 1.5-1
libjs-extjs: 7.0.0-5
libknet1: 1.30-pve2
libproxmox-acme-perl: 1.6.0
libproxmox-backup-qemu0: 1.5.1
libproxmox-rs-perl: 0.3.5
libpve-access-control: 8.2.2
libpve-apiclient-perl: 3.3.2
libpve-cluster-api-perl: 8.1.0
libpve-cluster-perl: 8.1.0
libpve-common-perl: 8.3.1
libpve-guest-common-perl: 5.2.2
libpve-http-server-perl: 5.2.2
libpve-network-perl: 0.11.2
libpve-rs-perl: 0.9.4
libpve-storage-perl: 8.3.6
libqb0: 1.0.5-1
libspice-server1: 0.15.1-1
lvm2: 2.03.16-2
lxc-pve: 6.0.0-1
lxcfs: 6.0.0-pve2
novnc-pve: 1.6.0-2
proxmox-backup-client: 3.4.1-1
proxmox-backup-file-restore: 3.4.1-1
proxmox-firewall: 0.7.1
proxmox-kernel-helper: 8.1.1
proxmox-mail-forward: 0.3.2
proxmox-mini-journalreader: 1.4.0
proxmox-offline-mirror-helper: 0.6.7
proxmox-widget-toolkit: 4.3.11
pve-cluster: 8.1.0
pve-container: 5.2.6
pve-docs: 8.4.0
pve-edk2-firmware: 4.2025.02-3
pve-esxi-import-tools: 0.7.4
pve-firewall: 5.1.1
pve-firmware: 3.15-4
pve-ha-manager: 4.0.7
pve-i18n: 3.4.4
pve-qemu-kvm: 9.2.0-5
pve-xtermjs: 5.5.0-2
qemu-server: 8.3.12
smartmontools: 7.3-pve1
spiceterm: 3.3.0
swtpm: 0.8.0+pve1
vncterm: 1.8.0
zfsutils-linux: 2.2.7-pve2

proxmox-backup unknown running kernel: 6.14.0-2-pve
proxmox-backup-server 3.4.1-1 running version: 3.4.1
ifupdown2 3.2.0-1+pmx11
libjs-extjs 7.0.0-5
proxmox-backup-docs 3.4.1-1
proxmox-backup-client unknown
proxmox-mail-forward 0.3.2
proxmox-mini-journalreader 1.4.0
proxmox-offline-mirror-helper 0.6.7
proxmox-widget-toolkit 4.3.11
pve-xtermjs 5.5.0-2
smartmontools 7.3-pve1
zfsutils-linux 2.2.7-pve2

2025-05-26T16:03:56-05:00: verify Backups:ct/109/2025-05-18T08:01:56Z
2025-05-26T16:03:56-05:00: check pct.conf.blob
2025-05-26T16:03:56-05:00: check root.ppxar.didx
2025-05-26T16:18:05-05:00: verified 117951.98/153437.28 MiB in 849.10 seconds, speed 138.91/180.71 MiB/s (0 errors)
2025-05-26T16:18:05-05:00: check root.mpxar.didx
2025-05-26T16:18:05-05:00: verified 13.46/67.58 MiB in 0.20 seconds, speed 66.56/334.29 MiB/s (0 errors)
2025-05-26T16:18:05-05:00: TASK OK
 
Are you able to download the full container as archive via the File restore dialog? To do so, go to the PBS storage as listed in the PVE WebUI, select the Backup pane, select the snapshot and click the File restore button. There, select the top level root.mpxar.didx archive and download the contents as compressed tar archive.

If that works you can either recreate an similar container from scratch and copy over the file contents via e.g. rsync or try to inject the container config file into the tar archive and restore from that.

Edit: another option might be to mount the pxar archive on the PVE host via proxmox-backup-client mount ... and sync the contents from there via rsync to a new container with similar configuration. The container filesystem can be mounted on the host as well via pct mount <VMID>, so rsync can run locally via rsync -av snapshot-mountpoint cointainer-mountpoint
 
Last edited:
Could you maybe also provide us the raw mpxar and ppxar file which you can download in the PBS WebUI via some secure channel (e.g. via a link to a nextcloud share as private message). This might help us debug and understand what is wrong here.
 
The mpxar would be a good starting point for further investigation. Depending on what the actual issue is, it might not be enough however.

Chris said:
Are you able to download the full container as archive via the File restore dialog? To do so, go to the PBS storage as listed in the PVE WebUI, select the Backup pane, select the snapshot and click the File restore button. There, select the top level root.mpxar.didx archive and download the contents as compressed tar archive.

If that works you can either recreate an similar container from scratch and copy over the file contents via e.g. rsync or try to inject the container config file into the tar archive and restore from that.

Edit: another option might be to mount the pxar archive on the PVE host via proxmox-backup-client mount ... and sync the contents from there via rsync to a new container with similar configuration. The container filesystem can be mounted on the host as well via pct mount <VMID>, so rsync can run locally via rsync -av snapshot-mountpoint cointainer-mountpoint
Click to expand...
Did you try the suggested steps? Were you able to extract/restore the required contents by this?
 
You must log in or register to reply here.
Top Bottom