Hello
We have PBS 4.1.0 server used for backing up a number of PVE VMs and file-level sets via backup agents. It is used successfully for about a month. Total size of PBS ZFS datastore is 30TB,78% of free space.
Yesterday i try to set up backup script at our mail storage server containing a huge set (~32M, 3.7TB) of files in wide directory tree. The mail server is physical (not VM) under CentOS 9 and have proxmox-backup-client 4.0.15 installed. I make LVM snapshots to have consistent backup, mount the snapshots and run proxmox-backup-client. All backups but one completed successfully. Backup process for one of big "archive" mail partition works for 15h and have failed at ~90%. No any system errors logged on the mail and/or backup servers.
Backup script fragment:
Output of script (fragment):
The failed backup directory looks normal in the native LV filesystem. Snapshot LVM was removed but no any FS (xfs) errors logged so it seems FS state is not the cause. Yes, the directory is big enough containing 265123 files. Can it be a problem for PBS archive format or for file change detection algorithm?
Or any ideas more?
We have PBS 4.1.0 server used for backing up a number of PVE VMs and file-level sets via backup agents. It is used successfully for about a month. Total size of PBS ZFS datastore is 30TB,78% of free space.
Yesterday i try to set up backup script at our mail storage server containing a huge set (~32M, 3.7TB) of files in wide directory tree. The mail server is physical (not VM) under CentOS 9 and have proxmox-backup-client 4.0.15 installed. I make LVM snapshots to have consistent backup, mount the snapshots and run proxmox-backup-client. All backups but one completed successfully. Backup process for one of big "archive" mail partition works for 15h and have failed at ~90%. No any system errors logged on the mail and/or backup servers.
Backup script fragment:
Bash:
mount -o remount,ro "/var/imap_arch"
let errors+=$?
/sbin/lvcreate -ay -L 100G -p r -n "$CYR_ARCH_SNAP_LV" -s "$CYR_ARCH_VG/$CYR_ARCH_LV"
local result=$?
mount -o remount,rw "/var/imap_arch"
let errors+=$?
if [ $result -ne 0 ]; then
let errors+=$result
else
mount -o ro,norecovery,nouuid "/dev/$CYR_ARCH_VG/$CYR_ARCH_SNAP_LV" "$CYR_ARCH_SNAP_MNT"
let errors+=$?
fi
...
if [ $errors -eq 0 ]; then
export PBS_PASSWORD_FILE
export PBS_FINGERPRINT
/usr/bin/proxmox-backup-client backup \
"cyrus-imap-db.pxar:$CYR_DB_SNAP_MNT" \
"cyrus-imap-main.pxar:$CYR_MAIN_SNAP_MNT" \
"cyrus-imap-arch.pxar:$CYR_ARCH_SNAP_MNT" \
--repository "$PBS_TOKEN@backup.solvo.ru:DATA" -ns MAIL --backup-id "`hostname -s`-cyrus" --backup-type host --change-detection-mode=metadata
let errors+=$?Output of script (fragment):
Code:
Logical volume "imapmain.snap" created.
Logical volume "imaparch.snap" created.
Logical volume "imaplib.snap" created.
Starting backup: [MAIL]:host/rhino-cyrus/2026-02-12T21:46:40Z
Client name: rhino
Starting backup protocol: Fri Feb 13 00:46:40 2026
No previous manifest available.
...
Upload directory '/mnt/imaparch.snap' to 'backup-agent@pbs!cyrus-imap@backup.solvo.ru:8007:DATA' as cyrus-imap-arch.mpxar.didx
processed 4.208 GiB in 1m, uploaded 4.169 GiB
...
processed 3.081 TiB in 14h 47m 1s, uploaded 2.754 TiB
processed 3.082 TiB in 14h 48m 1s, uploaded 2.754 TiB
unclosed encoder dropped
closed encoder dropped with state
unfinished encoder state dropped
cyrus-imap-arch.ppxar: had to backup 2.754 TiB of 3.082 TiB (compressed 1.403 TiB) in 53310.74 s (average 54.177 MiB/s)
cyrus-imap-arch.ppxar: backup was done incrementally, reused 335.65 GiB (10.6%)
Error: upload failed: error at "imap/V/user/nladokhin/Support/Bugs"The failed backup directory looks normal in the native LV filesystem. Snapshot LVM was removed but no any FS (xfs) errors logged so it seems FS state is not the cause. Yes, the directory is big enough containing 265123 files. Can it be a problem for PBS archive format or for file change detection algorithm?
Or any ideas more?
and ask the user to clean it. But it not the biggest one...