File restore (NTFS) - all mount failed or no supported file system

[fabian]

please try the same but with the kernel driver ("mount -t ntfs3 ...")!

 

[dtruffo]

Test done

mount -t ntfs3 /dev/loop0p2 /XX -o ro

mount: /XX: mount(2) system call failed: No such file or directory.
dmesg(1) may have more information after failed mount system call.

dmesg says...
[65802.072898] loop0: detected capacity change from 0 to 17825792000
[65802.127805] loop0: p1 p2
[65831.280929] ntfs3: Max link count 4000
[65831.280935] ntfs3: Enabled Linux POSIX ACLs support
[65831.280936] ntfs3: Read-only LZX/Xpress compression included


Could it be that volume is too big ? (~9TB)


Journalctl -xe output

░░ The job identifier is 3076.
Jul 10 12:03:48 pve3 sshd[1106081]: pam_env(sshd:session): deprecated reading of user environment enabled
Jul 10 12:04:28 pve3 kernel: loop0: detected capacity change from 0 to 17825792000
Jul 10 12:04:28 pve3 kernel: loop0: p1 p2
Jul 10 12:04:57 pve3 kernel: ntfs3: Max link count 4000
Jul 10 12:04:57 pve3 kernel: ntfs3: Enabled Linux POSIX ACLs support
Jul 10 12:04:57 pve3 kernel: ntfs3: Read-only LZX/Xpress compression included
Jul 10 12:08:26 pve3 systemd[1]: run-pbs\x2dloopdev-root\x5cx40pam\x5cx4010.10.60.10\x5cx3a8007\x5cx3aPROXMOX\x5cx2dBACKUP\x5cx3avm\x2d126\x2d2024\x5cx2d07\x5cx2d09T22\x5cx3a14\x5cx3a26Z\x2ddrive\x5cx2dscsi0.img.mount: Deactivated successfully.
░░ Subject: Unit succeeded
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░
░░ The unit run-pbs\x2dloopdev-root\x5cx40pam\x5cx4010.10.60.10\x5cx3a8007\x5cx3aPROXMOX\x5cx2dBACKUP\x5cx3avm\x2d126\x2d2024\x5cx2d07\x5cx2d09T22\x5cx3a14\x5cx3a26Z\x2ddrive\x5cx2dscsi0.img.mount has successfully entered the 'dead' state.
lines 3143-3221/3221 (END)

 

[fabian]

possibly, or it has some option set that the kernel driver doesn't understand/handle correctly..

 

[dtruffo]

Interesting.

So, since Kernel is the one installed by Proxmox installation CD and SUBSCRIPTION updates how can I solve ?

Yesterday solution was proxmox-backup-client version (not available on SUBSCRIPTION repository), today kernel (proxmox delivered).

Hummmm not sure Proxmox knows what to do.

 

[fabian]

this is simply a limitation with the way the file restore works - it can only restore what the linux kernel inside the restore VM understands.. anything else, you need to manually handle (via map and either passthrough into a VM or manual restore on the host).

 

[mishki]

you need to manually handle (via map and either passthrough into a VM or manual restore on the host).

how to do it correctly? how to passthrough to VM?

 

[fabian]

map the backup, e.g.:

$ proxmox-backup-client map vm/800/2023-01-16T13:09:55Z drive-scsi0.img --repository test@pbs@localhost:tank
Password for "test@pbs": 
Image 'test@pbs@localhost:8007:tank:vm/800/2023-01-16T13:09:55Z/drive-scsi0.img' mapped on /dev/loop0

configure VM to use the mapped backup as disk:

qm set 800 -scsi1 /dev/loop0,snapshot=1
update VM 800: -scsi1 /dev/loop0,snapshot=1

you need to replace
- the VMID "800"
- the loop device "loop0" (taken from the output of the `map` command)
- the disk slot "scsi1" (use a currently not used one!)

when you are done, remove the disk again, then unmap the loop device using "proxmox-backup-client unmap, e.g.:

$ qm set 800 -delete scsi1
update VM 800: -delete scsi1
$ proxmox-backup-client unmap loop0

 

[t.oster]

Hi,
we have the same problem. The backups of a windows server 2022 are not file-restorable suddenly. Older backups work but since a specific date, we get the error in the UI: mounting 'drive-virtio0.img.fidx/part/2' failed: all mounts failed or no supported file system (400)
The file /var/log/proxmox-backup/file-restore/qemu.log shows:

[2024-10-01T06:02:54.884Z WARN  proxmox_restore_daemon::proxmox_restore_daemon::disk] mount error on '/dev/vda2' (ntfs3) - ENOENT: No such file or directory

Dmesg shows nothing.
However manually mounting with -t ntfs works:

proxmox-backup-client map vm/108/2024-09-30T18:00:05Z drive-virtio0.img --repository backupuser@pbs@bielefeld:dwd
Image 'backupuser@pbs@bielefeld:8007:dwd:vm/108/2024-09-30T18:00:05Z/drive-virtio0.img' mapped on /dev/loop0
mount -t ntfs /dev/loop0p2 /tmp/test
mount: /tmp/test: WARNING: source write-protected, mounted read-only.

mount entry shows

/dev/loop0p2 on /tmp/test type ntfs (ro,relatime,uid=0,gid=0,fmask=0177,dmask=077,nls=utf8,errors=continue,mft_zone_multiplier=1)

Anything I can do to help you reproduce the problem? Or debug further what the difference between the manual mounting and the steps proxmox does are? The strange thing is that it works on older backups of the very same machine.
BTW checkdisk says everything is OK.

 

[t.oster]

I just noticed:
When I sucessfully open another backup, the log shows:
[2024-10-01T06:12:04.300Z INFO proxmox_restore_daemon::proxmox_restore_daemon::disk] mounting '/dev/vda2' succeeded, fstype: 'ntfs3'
so I guess proxmox is now using ntfs3 instead of ntfs?
If I manually try this
mount -t ntfs3 /dev/loop0p2 /tmp/test
I get
mount: /tmp/test: mount(2) system call failed: No such file or directory.
dmesg(1) may have more information after failed mount system call.

 

[fabian]

could you post the full log file and the VM config from the backup?

 

[t.oster]

Here is the VM config and the logs when trying to open the file-restore.

 

[t.oster]

and the client.log, if that was your question

 

[fabian]

is there anything in dmesg after you try mounting on the host with ntfs3? which kernel version are you running there?

 

[t.oster]

Kernel 6.8.12-2-pve
Dmesg does nothing when mounting (even with -o foce). but here is an strace if that helps:

 

[t.oster]

And an strace with mount -t ntfs (not ntfs3)

 

[fabian]

I wonder if the FS in question is somehow unclean and the ENOENT is just a badly bubbled up error..

 

[t.oster]

but wouldn't mount -t ntfs then throw an error? it mounts OK with mount -t ntfs. I have to check if I can shutdown the machine and create a backup with a cleanly shut down file-system, but since it is a production machine, I have to check.

 

[fabian]

not necessarily - `ntfs` and `ntfs3` are totally different drivers kernel side..

 

[tyx]

I'm having the same issue as t.oster (down to the old backups working fine), with the only difference being that I cannot mount with either `ntfs` or `ntfs3`, just with `ntfs-3g` .
The chkdsk was also fine last I checked, but I'll check again just to be sure… what more could be done to help find the source of the issue?

 

[fabian]

nothing much other than improving the Linux NTFS drivers (and maybe providing minimal reproducing disk images to help with that)..