Single-file restore and LVM

Same here, but not only with LVM, all backend storage are concerned.

Code:
proxmox-file-restore failed: Error: starting VM timed out (500)

Code:
()
proxmox-ve: 7.0-2 (running kernel: 5.11.22-1-pve)
pve-manager: 7.0-10 (running version: 7.0-10/d2f465d3)
pve-kernel-5.11: 7.0-5
pve-kernel-helper: 7.0-5
pve-kernel-5.4: 6.4-4
pve-kernel-5.11.22-2-pve: 5.11.22-4
pve-kernel-5.11.22-1-pve: 5.11.22-2
pve-kernel-5.4.124-1-pve: 5.4.124-1
pve-kernel-5.4.34-1-pve: 5.4.34-2
ceph: 16.2.5-pve1
ceph-fuse: 16.2.5-pve1
corosync: 3.1.2-pve2
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown: not correctly installed
ifupdown2: 3.1.0-1+pmx2
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.21-pve1
libproxmox-acme-perl: 1.2.0
libproxmox-backup-qemu0: 1.2.0-1
libpve-access-control: 7.0-4
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.0-5
libpve-guest-common-perl: 4.0-2
libpve-http-server-perl: 4.0-2
libpve-storage-perl: 7.0-9
libqb0: 1.0.5-1
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 4.0.9-4
lxcfs: 4.0.8-pve2
novnc-pve: 1.2.0-3
proxmox-backup-client: 2.0.5-2
proxmox-backup-file-restore: 2.0.5-2
proxmox-mini-journalreader: 1.2-1
proxmox-widget-toolkit: 3.3-5
pve-cluster: 7.0-3
pve-container: 4.0-8
pve-docs: 7.0-5
pve-edk2-firmware: 3.20200531-1
pve-firewall: 4.2-2
pve-firmware: 3.2-4
pve-ha-manager: 3.3-1
pve-i18n: 2.4-1
pve-qemu-kvm: 6.0.0-2
pve-xtermjs: 4.12.0-1
qemu-server: 7.0-10
smartmontools: 7.2-pve2
spiceterm: 3.2-2
vncterm: 1.7-1
zfsutils-linux: 2.0.5-pve1
Yup ... on all nodes. So file restore is not currently working on actual version proxmox-backup-file-restore: 2.0.5-2.

Maybe problem in this two commits ?
- https://git.proxmox.com/?p=proxmox-backup.git;a=commit;h=d9822cd3cb30494b2fdabd7a3670e940ad8199ae
- https://git.proxmox.com/?p=proxmox-backup.git;a=commit;h=66501529a2efbd5a426eba56ff8604db157b7814

Any Dev :) ?
 
FYI: The minimal environment running in the file-restore VM missed the backup system user, which only got a hard requirement there very recently through a not too obvious side effect.
Fixed in proxmox-backup-file-restore version 2.0.7-1 available on pvetest repository at time of writing, as the update is rather small and contained we may move it to no-subscription soon, it only broke there.
 
Last edited:
  • Like
Reactions: xPakrikx and ahoi_
Hello, I get the timeout message and I have installed proxmox-backup-file-restore version 2.0.7-1.

After clicking OK in the dialog the list of files was loaded correctly.

Here is my pveversion output
Code:
proxmox-ve: 7.0-2 (running kernel: 5.11.22-3-pve)
pve-manager: 7.0-10 (running version: 7.0-10/d2f465d3)
pve-kernel-5.11: 7.0-6
pve-kernel-helper: 7.0-6
pve-kernel-5.4: 6.4-5
pve-kernel-5.11.22-3-pve: 5.11.22-6
pve-kernel-5.4.128-1-pve: 5.4.128-1
pve-kernel-5.4.106-1-pve: 5.4.106-1
pve-kernel-4.15: 5.4-19
pve-kernel-4.15.18-30-pve: 4.15.18-58
pve-kernel-4.15.17-1-pve: 4.15.17-9
ceph-fuse: 14.2.21-1
corosync: 3.1.2-pve2
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown: residual config
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.21-pve1
libproxmox-acme-perl: 1.2.0
libproxmox-backup-qemu0: 1.2.0-1
libpve-access-control: 7.0-4
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.0-5
libpve-guest-common-perl: 4.0-2
libpve-http-server-perl: 4.0-2
libpve-storage-perl: 7.0-9
libqb0: 1.0.5-1
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 4.0.9-4
lxcfs: 4.0.8-pve2
novnc-pve: 1.2.0-3
proxmox-backup-client: 2.0.7-1
proxmox-backup-file-restore: 2.0.7-1
proxmox-mini-journalreader: 1.2-1
proxmox-widget-toolkit: 3.3-6
pve-cluster: 7.0-3
pve-container: 4.0-8
pve-docs: 7.0-5
pve-edk2-firmware: 3.20200531-1
pve-firewall: 4.2-2
pve-firmware: 3.2-4
pve-ha-manager: 3.3-1
pve-i18n: 2.4-1
pve-qemu-kvm: 6.0.0-2
pve-xtermjs: 4.12.0-1
qemu-server: 7.0-11
smartmontools: 7.2-pve2
spiceterm: 3.2-2
vncterm: 1.7-1
zfsutils-linux: 2.0.5-pve1
 
Last edited:
Still get the same problem:

Centos8 VM backup with XFS LV:


proxmox-file-restore failed: Error: mounting 'drive-ide0.img.fidx/part/["2"]' failed: all mounts failed or no supported file system (500)
 
Still get the same problem:

Centos8 VM backup with XFS LV:


proxmox-file-restore failed: Error: mounting 'drive-ide0.img.fidx/part/["2"]' failed: all mounts failed or no supported file system (500)
Which version are you on? (pveversion -v) Are you sure the filesystem on the backup is valid and readable? The error itself only says that mounting failed. You can also check the log of the last file-restore at /var/log/proxmox-backup/file-restore/qemu.log.
 
root@pvelan1:~# pveversion -v
proxmox-ve: 6.4-1 (running kernel: 5.4.128-1-pve)
pve-manager: 6.4-13 (running version: 6.4-13/9f411e79)
pve-kernel-5.4: 6.4-5
pve-kernel-helper: 6.4-5
pve-kernel-5.4.128-1-pve: 5.4.128-2
pve-kernel-5.4.124-1-pve: 5.4.124-2
pve-kernel-5.4.106-1-pve: 5.4.106-1
ceph-fuse: 12.2.11+dfsg1-2.1+b1
corosync: 3.1.2-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: residual config
ifupdown2: 3.0.0-1+pve4~bpo10
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.20-pve1
libproxmox-acme-perl: 1.1.0
libproxmox-backup-qemu0: 1.1.0-1
libpve-access-control: 6.4-3
libpve-apiclient-perl: 3.1-3
libpve-common-perl: 6.4-3
libpve-guest-common-perl: 3.1-5
libpve-http-server-perl: 3.2-3
libpve-storage-perl: 6.4-1
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.6-2
lxcfs: 4.0.6-pve1
novnc-pve: 1.1.0-1
proxmox-backup-client: 1.1.13-2
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.6-1
pve-cluster: 6.4-1
pve-container: 3.3-6
pve-docs: 6.4-2
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-4
pve-firmware: 3.2-4
pve-ha-manager: 3.1-1
pve-i18n: 2.3-1
pve-qemu-kvm: 5.2.0-6
pve-xtermjs: 4.7.0-3
qemu-server: 6.4-2
smartmontools: 7.2-pve2
spiceterm: 3.1-1
vncterm: 1.6-2
zfsutils-linux: 2.0.5-pve1~bpo10+1


This is what i found in the log:

==> qemu.log <==
EXT4-fs (vda2): VFS: Can't find ext4 filesystem
EXT4-fs (vda2): VFS: Can't find ext4 filesystem
EXT2-fs (vda2): error: can't find an ext2 filesystem on dev vda2.
FAT-fs (vda2): bogus number of reserved sectors
FAT-fs (vda2): bogus number of reserved sectors
VFS: could not find a valid V7 on vda2.
ntfs: (device vda2): read_ntfs_boot_sector(): Primary boot sector is invalid.
ntfs: (device vda2): read_ntfs_boot_sector(): Mount option errors=recover not used. Aborting without trying to recover.
ntfs: (device vda2): ntfs_fill_super(): Not an NTFS volume.
ufs: ufs_fill_super(): bad magic number
befs: (vda2): invalid magic header
F2FS-fs (vda2): Can't find valid F2FS filesystem in 1th superblock
F2FS-fs (vda2): Can't find valid F2FS filesystem in 2th superblock
[2021-09-13T14:45:04.072Z ERROR proxmox_backup::server::rest] GET /api2/json/list?path=ZHJpdmUtaWRlMC5pbWcuZmlkeC9wYXJ0LzI%3D: 400 Bad Request: [client 0.0.0.0:807] mounting 'drive-ide0.img.fidx/part/["2"]' failed: all mounts failed or no supported file system
 
After upgrading PVE from 6.4.13 to PVE 7.0-11 the "file restore" feauture is not working at all.
Using PVE 6.4.13 the "file restore" features was working apart restoring from a file system with LVM (but probably this is "normal" after reading the thread !)

This is the error that appear on "File restore":
Code:
Proxmox-file-restore failed: Error: Starting VM failed. See output above for more information. (500)

This is the version of PVE that I'm using
Code:
# pveversion -v
proxmox-ve: 7.0-2 (running kernel: 5.11.22-4-pve)
pve-manager: 7.0-11 (running version: 7.0-11/63d82f4e)
pve-kernel-5.11: 7.0-7
pve-kernel-helper: 7.0-7
pve-kernel-5.4: 6.4-6
pve-kernel-5.11.22-4-pve: 5.11.22-8
pve-kernel-5.4.140-1-pve: 5.4.140-1
pve-kernel-5.4.128-1-pve: 5.4.128-2
pve-kernel-4.15: 5.4-19
pve-kernel-4.15.18-30-pve: 4.15.18-58
pve-kernel-4.15.18-9-pve: 4.15.18-30
ceph: 16.2.5-pve1
ceph-fuse: 16.2.5-pve1
corosync: 3.1.5-pve1
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown: residual config
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.22-pve1
libproxmox-acme-perl: 1.3.0
libproxmox-backup-qemu0: 1.2.0-1
libpve-access-control: 7.0-4
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.0-6
libpve-guest-common-perl: 4.0-2
libpve-http-server-perl: 4.0-2
libpve-storage-perl: 7.0-11
libqb0: 1.0.5-1
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 4.0.9-4
lxcfs: 4.0.8-pve2
novnc-pve: 1.2.0-3
proxmox-backup-client: 2.0.9-2
proxmox-backup-file-restore: 2.0.9-2
proxmox-mini-journalreader: 1.2-1
proxmox-widget-toolkit: 3.3-6
pve-cluster: 7.0-3
pve-container: 4.0-9
pve-docs: 7.0-5
pve-edk2-firmware: 3.20200531-1
pve-firewall: 4.2-3
pve-firmware: 3.3-1
pve-ha-manager: 3.3-1
pve-i18n: 2.5-1
pve-qemu-kvm: 6.0.0-4
pve-xtermjs: 4.12.0-1
qemu-server: 7.0-13
smartmontools: 7.2-pve2
spiceterm: 3.2-2
vncterm: 1.7-1
zfsutils-linux: 2.0.5-pve1

I'm testing "File Restore" from two different storages that are placed on a PBS version 2.0-4 and from a PBS version 1.1-13 but the issue is equal.

My log file is "empty"
Code:
# cat /var/log/proxmox-backup/file-restore/qemu.log
[2021-09-20T15:50:02+02:00] PBS file restore VM log

Any suggestion ?
 
Could you try to run the file-restore command manually on the CLI of your PVE node?
Code:
proxmox-file-restore list vm/100/2021-09-20T16:20:30Z /drive-scsi0.img.fidx/ --repository root@pam@127.0.0.1:datastore
(substitute your own IP/datastore/VM combo of course)
 
Could you try to run the file-restore command manually on the CLI of your PVE node?
Code:
proxmox-file-restore list vm/100/2021-09-20T16:20:30Z /drive-scsi0.img.fidx/ --repository root@pam@127.0.0.1:datastore
(substitute your own IP/datastore/VM combo of course)
I tried this:
Code:
export PBS_PASSWORD="MyPassword"
export PBS_REPOSITORY="backup@pbs@MyIpAddress:storage1"

proxmox-file-restore list vm/105/2021-09-20T13:28:53Z /drive-virtio0.img.fidx/

the command returns this
Code:
certificate validation failed - Certificate fingerprint was not confirmed.
qemu-system-x86_64: -drive file=pbs:repository=backup@pbs@10.255.253.200:8007:storage1,,snapshot=vm/105/2021-09-20T13:28:53Z,,archive=drive-virtio0.img.fidx,read-only=on,if=none,id=drive0: PBS connect failed: command error: error trying to connect: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:../ssl/statem/statem_clnt.c:1913:
Error: Starting VM failed. See output above for more information.
 
Last edited:
I tried this:
Code:
export PBS_PASSWORD="MyPassword"
export PBS_REPOSITORY="backup@pbs@MyIpAddress:storage1"

proxmox-file-restore list vm/105/2021-09-20T13:28:53Z /drive-virtio0.img.fidx/

the command returns this
Code:
certificate validation failed - Certificate fingerprint was not confirmed.
qemu-system-x86_64: -drive file=pbs:repository=backup@pbs@10.255.253.200:8007:storage1,,snapshot=vm/105/2021-09-20T13:28:53Z,,archive=drive-virtio0.img.fidx,read-only=on,if=none,id=drive0: PBS connect failed: command error: error trying to connect: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:../ssl/statem/statem_clnt.c:1913:
Error: Starting VM failed. See output above for more information.
You need to export PBS_FINGERPRINT=00:00:00... as well, with the fingerprint shown on the front page of your PBS install, or use a certificate that is in the system trust store.
 
You need to export PBS_FINGERPRINT=00:00:00... as well, with the fingerprint shown on the front page of your PBS install, or use a certificate that is in the system trust store.
Thank's Stefan for the suggestion, now executing proxmox-file-restore exit this message:
Code:
CID '10' in use by other VM, attempting next one
CID '11' in use by other VM, attempting next one
CID '12' in use by other VM, attempting next one
Error: starting VM timed out

I think the problem depends on the PVE 7.0.11 that I'm using for the test are virtualized (nested virtualization) on another PVE 6.x
In the next day I'll try on a phisical PVE 7.x.

updated:
I confirm that the problem exists only in my test machine with nested virtualization !!
With a "phisical" PVE 7.x it's all ok.
Thank's to all :)
 
Last edited:
Code:
proxmox-file-restore list vm/100/2021-09-20T16:20:30Z /drive-scsi0.img.fidx/ --repository root@pam@127.0.0.1:datastore
(substitute your own IP/datastore/VM combo of course)
Hi Stefan,

Does the file-restore CLI work with containers? If so, I can't figure out what should follow on this line.
Code:
proxmox-file-restore list ct/150/2021-09-21T12:54:01Z

I have these working variables in .bashrc:
Code:
export PBS_REPOSITORY=localhost:backup1
export PBS_PASSWORD=3E.............
export PBS_FINGERPRINT=9b:94:d8:8c:5e:bc:.........
--
pbs user: root@pam
 
Last edited:
Does the file-restore CLI work with containers? If so, I can't figure out what should follow on this line.
It does, you need to append the folder you want to list. For starters, just try root /. If that doesn't help, what error message are you getting?
 
  • Like
Reactions: moxfan
It does, you need to append the folder you want to list. For starters, just try root /. If that doesn't help, what error message are you getting?

Ah, yes. Excellent, it works!

Are there plans to support wildcards in file selections? That would be so good.

Code:
root@srv:~# proxmox-file-restore extract ct/150/2021-09-21T12:54:01Z /root.pxar.didx/*.orig /root/origs/ --verbose
Error: error opening '[47, 114, 111, 111, 116, 47, 42, 46, 111, 114, 105, 103]'
 
Would make sense as a feature request for VM restore, though at least you can restore entire folders.

Which is proving very useful as it's restoring the source folder directly to the target path and the restore action is taking much less time to finish.

With either proxmox-file-restore or the interactive shell, is there a workaround so that the files in the target path can be overwritten with those in the archive or removed if they don't exist in the archive ?

Code:
pxar:/ > list-selected
/root/chattrtest
/root/chattrtest/immut2.txt
pxar:/ > restore-selected /usr/temp/pxartest
Error: failed to create file "immut2.txt": EEXIST: File exists
 
Last edited:
With either proxmox-file-restore or the interactive shell, is there a workaround so that the files in the target path can be overwritten with those in the archive or removed if they don't exist in the archive ?
Not at the moment, but you could work around by extracting to a seperate folder and using rsync (with --delete) to merge them.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!