[TUTORIAL] Work Around for Deduped NTFS File Restores

P

PwrBank

Active Member
Nov 12, 2024
237
120
43
As some of you have probably ran into, if you try to run a file restore on a NTFS partition that has NTFS dedupe on, you get this error:

  1. The NTFS volume won't mount with the kernel ntfs3 driver ("Failed to initialize $Secure (-2)").
  2. Files deduplicated by Windows Server (reparse tag 0x80000013) mount as broken symlinks ("unsupported reparse tag 0x80000013") and can't be read from a mounted backup at all.

Both have workable solutions. This tutorial covers manually mapping and mounting a snapshot, and building the ntfs-3g dedup plugin that reconstructs deduplicated files read-only, no full VM restore required.

Tested on: Proxmox VE 9 (Debian 13), ntfs-3g 2022.10.3, proxmox-backup-client 4.2.1, Windows Server volume using dedup format 0x0103.

Part 1: Manually mount a snapshot​


Step 1: map the drive image​


Code: 
export PBS_REPOSITORY='user@pam!token@server:8007:datastore'
export PBS_PASSWORD='<api token secret>'

# Self-signed cert? Note: PBS client >= 4.x expects the fingerprint in
# COLON-separated lowercase hex; bare hex is rejected.
export PBS_FINGERPRINT='aa:bb:cc:dd:ee:...'

proxmox-backup-client map vm/100/2026-01-01T00:00:00Z drive-scsi2.img \
    --repository "$PBS_REPOSITORY" -ns <namespace>

This creates /dev/loop0 with one partition per drive (e.g. /dev/loop0p1). Check with lsblk /dev/loop0.

Step 2: mount with ntfs-3g, not the kernel driver​


The kernel ntfs3 driver fails on many real-world volumes:

Code: 
# mount /dev/loop0p1 /tmp/restore
mount: /tmp/restore: fsconfig() failed: No such file or directory.
# dmesg:
ntfs3(loop0p1): Failed to initialize $Secure (-2).

An explicit
Code: 
mount -t ntfs
doesn't help either; it routes back to ntfs3 (visible in dmesg). Use the FUSE driver instead:

Code: 
apt install ntfs-3g
mkdir -p /tmp/restore
mount -t ntfs-3g -o ro /dev/loop0p1 /tmp/restore

Step 3: copy what you need, then clean up​


Code: 
cp "/tmp/restore/path/to/file" /tmp/
umount /tmp/restore
proxmox-backup-client unmap /dev/loop0

Part 2: Reading Windows Server deduplicated files​


Windows Server dedup replaces file content with a reparse point (tag 0x80000013); the real data lives as chunks under System Volume Information\Dedup on the same volume. No Linux driver understands this, so deduplicated files mount as broken symlinks and are normally only recoverable via a full VM restore.

The fix: the ntfs-3g dedup plugin, originally written by Jean-Pierre André (long-time ntfs-3g maintainer), kept alive in the smx-smx fork, with an open PR adding dedup format 0x0103 (Windows Server 2019+). It hooks ntfs-3g's read path and reconstructs files from the dedup chunk store on the fly. It works on any block device ntfs-3g can mount, including a PBS-mapped loop device.

Build (on the PVE host)​


Code: 
apt install -y ntfs-3g ntfs-3g-dev build-essential autoconf automake libtool pkg-config libfuse3-dev

# Hosts with fuse3 only: provide the legacy pkg-config name so ./configure passes
ln -sf /usr/lib/x86_64-linux-gnu/pkgconfig/fuse3.pc /usr/lib/x86_64-linux-gnu/pkgconfig/fuse.pc

cd /root
git clone https://github.com/smx-smx/ntfs-3g-dedup-plugin.git
cd ntfs-3g-dedup-plugin
# Add dedup format 0x0103 support (Windows Server 2019+)
git fetch origin pull/1/head:dedup103 && git checkout dedup103

# The fork's Makefile also builds an experimental .NET variant. The C plugin is
# self-contained, so drop it:
sed -i 's/ -DUSE_DOTNET//' Makefile.am
sed -i '/src\/dotnet.c/d' Makefile.am
perl -0pi -e 's/src\/dedup\.c \\\n/src\/dedup.c\n/' Makefile.am

# fuse3 >= 3.17 dropped support for FUSE API versions < 30:
sed -i 's/#define FUSE_USE_VERSION 26/#define FUSE_USE_VERSION 30/' src/dedup.c

autoreconf -fi && ./configure && make

Install​


Code: 
# Where does ntfs-3g look for plugins?
strings $(which ntfs-3g) | grep ntfs-plugin
# e.g. /usr/lib/x86_64-linux-gnu/ntfs-3g/ntfs-plugin-%08lx.so

mkdir -p /usr/lib/x86_64-linux-gnu/ntfs-3g
cp .libs/ntfs-plugin-80000013.so /usr/lib/x86_64-linux-gnu/ntfs-3g/

Use​


Map and mount as in Part 1, then:

Code: 
ls -l "/tmp/restore/path/to/dir/" | grep <fragment>
# before: lrwxrwxrwx ... -> unsupported reparse tag 0x80000013
# after:  -r-xr-xr-x 2 root root 848346 Jul  6 15:56 <file>

cp "/tmp/restore/path/to/dir/"*<fragment>* /tmp/restore.xlsx
file /tmp/restore.xlsx      # Microsoft Excel 2007+ (or Zip archive)
unzip -t /tmp/restore.xlsx  # No errors detected in compressed data

The plugin loads lazily on first access to a deduplicated file; no ntfs-3g restart needed. If it can't handle a given volume, ntfs-3g logs an error and the file stays a broken symlink, so check dmesg/ntfs-3g output.

Verification​


Real-world result (the reason I wrote this up), an 848 KB .xlsx that had been deduplicated on a Windows Server, restored from a PBS snapshot entirely through the plugin. Previously I was needing to restore the entire VM (which is multiple TBs) and get the file off. Which as you can imagine is quite annoying for a less than 1MB file...

Notes / limitations​


  • Read-only reconstruction; the plugin never writes to the volume.
  • Format support: 0x0100-0x0102 (Server 2012-2016) in the released code; the PR branch adds 0x0103 (Server 2019+).
  • The plugin .so links the host's libntfs-3g; only copy it between hosts with the same libntfs-3g soname (check with ldd).
  • Always mount backups read-only, and test on a copy first.

Credits​




Is it possible to get this built into the GUI File Restore? At the moment it's pretty much useless on deduped NTFS partitions.
 
Moayad said:
For your information, this matches the known [0] `ntfs3` file-restore limitation, and an `ntfs-3g` fallback patch [1] series has been proposed but is not released yet.
Click to expand...

In this case, ntfs-3g didn't work without the patch specific PR for deduping. I haven't tried it on servers older than version 2019 of Windows Server, but maybe the ntfs-3g works on them.
 
You must log in or register to reply here.
Top Bottom
Back