Benchmarking PBS with repository flag

L

luckman212

Well-Known Member
Jun 22, 2017
30
2
48
Can anyone help me figure out how to benchmark with the --repository flag? No matter what I try, it doesn't work...

https://pbs.proxmox.com/docs/backup-client.html#benchmarking

Code: 
# whoami
root
# proxmox-backup-client benchmark --repository 192.168.20.55:syno-pbs
Password for "root@pam": *******************
Error: Permission denied (os error 13)

I have also tried the following for repo URLs
Code: 
proxmox-backup-client benchmark --repository 'backup@pbs!token01'
proxmox-backup-client benchmark --repository 'root@pam!token02@localhost:8007:syno-pbs'
proxmox-backup-client benchmark --repository 'root@pam@localhost:8007:syno-pbs'
proxmox-backup-client benchmark --repository 192.168.20.55:syno-pbs
 
luckman212 said:
Error: Permission denied (os error 13)
Click to expand...
Hi,
from this error I would rather guess that your pbs datastore is not set up correctly, the Proxmox Backup Server instance unable to write contents to it. Given the datastore name I'm guessing the datastore is located on a network share? Check permissions and ownership.
 
Thanks, yes it is on a NFS share, I think it set up correctly, and working fine... backups have been executing for >2 months...

1743430576593.png
 
Thank you for trying to help. No it is not in an LXC - I'm running it from a root shell on my PBS directly, which is a full Debian bookworm VM installed from the official PBS ISO, running on Synology VMM.
 
What do you get when enabling debug output via PBS_LOG=debug proxmox-backup-client benchmark --repository <repo>?
 
This:
Code: 
Linux pbs 6.8.12-9-pve #1 SMP PREEMPT_DYNAMIC PMX 6.8.12-9 (2025-03-16T19:18Z) x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Mon Mar 31 11:06:48 2025 from 192.168.20.40
root@pbs:~# PBS_LOG=debug proxmox-backup-client benchmark --repository syno-pbs
Password for "root@pam": *******************
Connecting to backup server
resolving host="localhost"
connecting to 127.0.0.1:8007
connected to 127.0.0.1:8007
flushed 212 bytes
parsed 3 headers
incoming body is content-length (280 bytes)
incoming body completed
pooling idle connection for ("https", localhost:8007)
reuse idle connection for ("https", localhost:8007)
flushed 513 bytes
parsed 3 headers
incoming body is content-length (31 bytes)
incoming body completed
pooling idle connection for ("https", localhost:8007)
Error: Permission denied (os error 13)
 
Okay looks like the client acts fine up to that error. Can you also check the systemd journal on the PBS server for errors around the time you are executing the command? Anything of interest in there?
 
This is what I see...

Code: 
root@pbs:~# journalctl -f
Mar 31 11:45:51 pbs systemd[1210]: Listening on gpg-agent-extra.socket - GnuPG cryptographic agent and passphrase cache (restricted).
Mar 31 11:45:51 pbs systemd[1210]: Listening on gpg-agent-ssh.socket - GnuPG cryptographic agent (ssh-agent emulation).
Mar 31 11:45:51 pbs systemd[1210]: Listening on gpg-agent.socket - GnuPG cryptographic agent and passphrase cache.
Mar 31 11:45:51 pbs systemd[1210]: Reached target sockets.target - Sockets.
Mar 31 11:45:51 pbs systemd[1210]: Reached target basic.target - Basic System.
Mar 31 11:45:51 pbs systemd[1210]: Reached target default.target - Main User Target.
Mar 31 11:45:51 pbs systemd[1210]: Startup finished in 177ms.
Mar 31 11:45:51 pbs systemd[1]: Started user@0.service - User Manager for UID 0.
Mar 31 11:45:51 pbs systemd[1]: Started session-6.scope - Session 6 of User root.
Mar 31 11:45:51 pbs sshd[1207]: pam_env(sshd:session): deprecated reading of user environment enabled

#this is right after I run `PBS_LOG=debug proxmox-backup-client benchmark --repository syno-pbs` in the other shell
Mar 31 11:47:22 pbs proxmox-backup-proxy[615]: GET //api2/json/backup?backup-id=benchmark&backup-time=1743436026&backup-type=host&benchmark=true&debug=false&store=syno-pbs: 400 Bad Request: [client [::ffff:127.0.0.1]:45118] Permission denied (os error 13)
 
So it seems that the error occurs while upgrading to the http/2 connection during initiation of the backup protocol. This step also creates the backup group and snapshot directory under <datastore-base>/host/benchmark. I suspect your Proxmox Backup Server instance has issues writing to that folder or the subfolder. Please check the contents via the output of ls -lahR <datastore-base>/host/benchmark, given your datastores base path. If that folder does not exist, check the output of ls -lahR <datastore-base>/host/
 
My datastore is mounted via NFS, I don't have a /host dir in there at all (?)

Code: 
root@pbs:/# find / -type d -name host
/usr/lib/modules/6.8.12-8-pve/kernel/drivers/ufs/host
/usr/lib/modules/6.8.12-8-pve/kernel/drivers/memstick/host
/usr/lib/modules/6.8.12-8-pve/kernel/drivers/mmc/host
/usr/lib/modules/6.8.12-8-pve/kernel/drivers/usb/host
/usr/lib/modules/6.8.12-8-pve/kernel/drivers/nvme/host
/usr/lib/modules/6.8.12-8-pve/kernel/drivers/bus/mhi/host
/usr/lib/modules/6.8.12-9-pve/kernel/drivers/ufs/host
/usr/lib/modules/6.8.12-9-pve/kernel/drivers/memstick/host
/usr/lib/modules/6.8.12-9-pve/kernel/drivers/mmc/host
/usr/lib/modules/6.8.12-9-pve/kernel/drivers/usb/host
/usr/lib/modules/6.8.12-9-pve/kernel/drivers/nvme/host
/usr/lib/modules/6.8.12-9-pve/kernel/drivers/bus/mhi/host
/usr/lib/modules/6.8.12-4-pve/kernel/drivers/ufs/host
/usr/lib/modules/6.8.12-4-pve/kernel/drivers/memstick/host
/usr/lib/modules/6.8.12-4-pve/kernel/drivers/mmc/host
/usr/lib/modules/6.8.12-4-pve/kernel/drivers/usb/host
/usr/lib/modules/6.8.12-4-pve/kernel/drivers/nvme/host
/usr/lib/modules/6.8.12-4-pve/kernel/drivers/bus/mhi/host
/var/lib/proxmox-backup/rrdb/host

root@pbs:/# find /mnt/syno-pbs -type d -name host
(no output)
 
What's the output of stat /mnt/syno-pbs?
 
Here it is
Code: 
root@pbs:~# stat /mnt/syno-pbs
  File: /mnt/syno-pbs
  Size: 64            Blocks: 0          IO Block: 4096   directory
Device: 0,43    Inode: 256         Links: 1
Access: (0555/dr-xr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2025-04-01 00:02:40.550134005 -0400
Modify: 2025-03-02 03:08:03.465257977 -0500
Change: 2025-03-02 03:08:03.465257977 -0500
 Birth: -
 
luckman212 said:
(0555/dr-xr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
Click to expand...
So it is due to incorrect file permissions after all. Above should be (0755/drwxr-xr-x) Uid: ( 34/ backup) Gid: ( 34/ backup) as otherwise the PBS instance cannot create/modify datastore contents at the datastore root level.
 
Ah ha! Thanks for spotting that. You are right of course.

This fixed it:

Code: 
root@nas:~# chmod 0755 /volume1/pbs
root@nas:~# chown 34:34 /volume1/pbs

root@pbs:~# proxmox-backup-client benchmark --repository syno-pbs
Password for "root@pam": *******************
Uploaded 372 chunks in 5 seconds.
Time per request: 13518 microseconds.
TLS speed: 310.26 MB/s
SHA256 speed: 282.56 MB/s
Compression speed: 220.45 MB/s
Decompress speed: 278.34 MB/s
AES256/GCM speed: 2079.49 MB/s
Verify speed: 137.72 MB/s
┌───────────────────────────────────┬────────────────────┐
│ Name                              │ Value              │
╞═══════════════════════════════════╪════════════════════╡
│ TLS (maximal backup upload speed) │ 310.26 MB/s (25%)  │
├───────────────────────────────────┼────────────────────┤
│ SHA256 checksum computation speed │ 282.56 MB/s (14%)  │
├───────────────────────────────────┼────────────────────┤
│ ZStd level 1 compression speed    │ 220.45 MB/s (29%)  │
├───────────────────────────────────┼────────────────────┤
│ ZStd level 1 decompression speed  │ 278.34 MB/s (23%)  │
├───────────────────────────────────┼────────────────────┤
│ Chunk verification speed          │ 137.72 MB/s (18%)  │
├───────────────────────────────────┼────────────────────┤
│ AES256 GCM encryption speed       │ 2079.49 MB/s (57%) │
└───────────────────────────────────┴────────────────────┘

Thanks so much for the assist!
 
You must log in or register to reply here.
Top Bottom