proxmox-backup-client catalog shell + restore-selected is extremely slow for tiny file restores, while web UI download is fast

[blueowl]

I’m seeing extremely poor performance when restoring files from a pxar archive with proxmox-backup-client catalog shell and restore-selected.

Environment

• Client package: proxmox-backup-client 4.1.5-1
• Installed from: http://download.proxmox.com/debian/pbs-client trixie/main
• Backup server: proxmox-backup-server 4.1.6-1 running version: 4.1.6

Problem
I entered the catalog shell and selected a path inside the pxar archive:

proxmox-backup-client catalog shell host/pbs-host/2026-04-14T21:56:47Z data1.pxar --ns data
select /pwrp/hllprd/src/db
restore-selected /tmp/testrestore/test10

Initially, running this as a normal user appeared to hang indefinitely. After leaving it overnight, it eventually failed with:
Error: failed to set ownershipCaused by: Operation not permitted (os error 1)


When I ran the catalog shell with "sudo -E" prefixed, it did complete and preserved ownership correctly:
-rw-rw-r-- 1 user pwrp 278K Oct 15 2025 testrestore/pwrp/hllprd/src/db/cvolhllprd.wb_load

The restored file above is only about 278 KB, but the restore took about 90 minutes!
The same directory, restored/downloaded from the PBS web UI, completed in about 30 seconds.

So there seems to be a very large performance gap between the web UI and the proxmox-backup-client tool.

Debugging
With PBS_LOG=debug, the client produced a huge number of repeating HTTP/2 log lines like:
Connection: send frame=Headers { stream_id: StreamId(3935), flags: (0x5: END_HEADERS | END_STREAM) } peer=ClientConnection: received frame=Headers { stream_id: StreamId(3935), flags: (0x4: END_HEADERS) } peer=ClientConnection: received frame=Data { stream_id: StreamId(3935), flags: (0x1: END_STREAM) } peer=Client
This continued with thousands of stream IDs, couldn't pull anything useful from it.

While the restore was running, iostat -x 1 on the PBS server showed only a short burst of activity and then mostly idle disks/CPU. The server did not appear to be under meaningful sustained load.

iperf3 from client to PBS:
[ 5] 0.00-10.00 sec 869 MBytes 729 Mbits/sec 1001 retr sender
[ 5] 0.00-10.00 sec 868 MBytes 728 Mbits/sec receiver

Client benchmark
proxmox-backup-client benchmark from the client gave:
Uploaded 110 chunks in 5 seconds.
Time per request: 52411 microseconds.
TLS speed: 80.03 MB/s
SHA256 speed: 4452.42 MB/s
Compression speed: 639.65 MB/s
Decompress speed: 913.52 MB/s
AES256/GCM speed: 8130.86 MB/s
Verify speed: 792.19 MB/s

Setup
Client:

• 1 Gbit NIC
• MTU 1500
• Ping to PBS: avg 0.248 ms, 0% loss
• Intel Ultra 5 225T + integrated graphics

PBS server:

• bond0 at 20 Gbit
• member links at 10 Gbit each
• MTU 1500
• Intel Xeon Silver 4214 CPU @ 2.20GHz

So, what's up with this? How can the proxmox-backup-client be so unfathomably slow?

 

[Chris]

Hi,
please share the reader task log for this restore from the PBS host. Also, can you try to mount the archive instead, and restore the file from the mounted archive? How is the performance in that case?

Edit: Maybe we must rethink the choice of extractor/decoder for the catalog shell, or find a better entry point(s) for the sequential decoder based on selections.

 

[blueowl]

Hi!
The reader task log is "too large for the server to process" even when split 4 ways with "split tasklog --number=4". If it's of high importance i uploaded it to limewire: https://limewire.com/d/mSfSI#wyOJLL0CkH

Using "sudo -E proxmox-backup-client mount host/backup data1.pxar /mnt/pbs" and then copying the file over as root works like any other network share, but the entire browsing of the mounted directory has to be done as root or with sudo commands, losing the simplicity of the catalog shell. The catalog shell is what fits my usecase best, but i could redirect users to the web ui for downloads until it's fixed.

 

[Chris]

Please open an issue at https://bugzilla.proxmox.com/, so this is not lost in the forum and you get status notifications on updates.