Improve loading time

[vosatka.milan]

I have one PVE node with ~80VM and ~10CT. I do daily snapshot backups on PBE (4 x Intel(R) Core(TM) i5-6600 CPU @ 3.30GHz 32GB RAM, SATA SSD for system, 2x 8TB HDD in ZFS mirror for datastore. Its 34 days running smoothly and Im having about 2200 snapshots in total now.

The issue is the list of backups is loading very long time. Like if I open the VM in PVE and click on Backup, it takes about 30 second for the process bar (process circle) to finish. Then the screen is empty for another 10 seconds, so it looks I have no backups at all and then finally the list of backups appears. After this "initial" load the list is loaded normally again and it takes like 2 or 3 seconds for all other attempts and different machines.

Do you have any recommendation how to avoid this extreme delay for first loading of backups list, when users are panicking there are no backups?

 

[Matthias.]

That's most likely because one of your storages (NFS probably) is slow to respond or unreachable

 

[vosatka.milan]

All my other storages (none of them NFS) are reachable with no delay in loading. It definitely looks the slow storage here is the proxmox backup server.
Today I got "communication failure" while waiting to load the backups. After refresh its loading correctly with no delay again. I think it used to be ok, when the number of backups was like 100. But now with 2300 snapshots the first load is freezing.

 

[Einar Stenberg]

Snapshots and backups are two very diffrent things..
Assuming you are indeed talking about backups as you mentioned PBS in second post;
I have seen similar behaviour when the PBS server have to little resources, like the cpu being pinned at 100% by some task, or if the storage you are using has very high latency. this was for my old setup, in my new one I use a ssd cache in front of my spinning rust, and also a more powerfull cpu and have not had the issue since.

While I was having the issue though the temporary fix I used was to quickly click back and worth between "summary" and "backups" multiple times until it loaded... hehe, though idk if that still works in the newer versions.

 

[vosatka.milan]

Well for backups I use "Snapshot" Mode so I think PVE makes snapshot of VM, creates diff and send it with VM config file to PBS, so not so much different, but yes we are talking about backups. *PBS itself is labeling particular backups as "Snapshots" - as on the screenshot =>

The CPU and RAM stay very low. Yeah the storage is probably the bottle neck - IOPS for datastorage is getting to ~500 during the loading action, which may be the limit for 2x HDD in ZFS mirror. But for practically cold backups I see no point using SSD as primary storage. Small SSD as cache could help, but still I have feeling it would mean resolving a software issue with buying new HW.

The fact this laggy load happens only for first time (probably the first one after backup jobs run) and all other subsequent queries even from different sessions and different users are just OK, leads me to assumption there is something like "backups list" processing / caching, which could be done automatically after backup jobs are done. (in other words kind of simulating PVE asking for backups list after backups jobs are done, so all following queries would be loaded normally).

* the correct number of backups on storage is 5000+

 

[Harliff]

Let me upvote this issue.
The backup list (on a PVE VM) wont loaded at all in 70% times.
My PBS server built using HDDs only.

 

[Harliff]

Workaround:

# crontab -l | grep -v '^#'
@hourly /usr/sbin/pvesm list my-pbs-server > /dev/null