Very slow performance while iterating backups

[SRU]

Hi,
we have some proxmox backup servers version 2.2.x with mixed discs:

• system discs with SSD/NVME
• data discs with sata 16tb drives.

if i try to restore a bigger vm, i always get timeouts from the backup server while searching for the latest backups.
Every first try always ends in a timeout no matter wether using the GUI or the CLI.

pvesm list pmb-04 -vmid 20110
proxmox-backup-client failed: Error: http request timed out

The second try works out fine returning a valid result.

root@p21:~/bin # time pvesm list pmb-04 -vmid 20110
real 1m43.270s
user 0m0.709s
sys 0m0.115s

Verify jobs last many hours up to 1.5 days.

I would like to move the content from datastore/vm to the systemdiscs as these have no load.
The volume is not so big and i think it could increase the performance for restore and verify jobs.

Maybe the usage of a database with some indices for the index informations can optimze this search and increase the performance ?

This backup server has:

• AMD Ryzen 7 3700X 8-Core Processor
• 128gb ram
• 2x 1tb nvme
• 10x16tb sata discs
• backup volume used: 20tb

i know the best approach would have been using NVMEs or at least SSDs for performance reasons, but my provider has no server with that huge storage as SSD / NVME.

• Any hints on how to reorganize the volumes and disks to reach performance while iterating backups ?
• Any hints on how to speed up verify jobs ?

Thanks in advance,
SRU

 

[ph0x]

Read about the zfs special devices, those help a lot with this use case.

 

[hancok62]

you're using nvme disk's as cache devices or for the system only?

something useful for that:
https://pbs.proxmox.com/docs/system-requirements.html

If HDDs are used: Using a metadata cache is highly recommended, for example, add a ZFS special device mirror.

 

[SRU]

Thanks for the answers to both of you, we will do research on that!