Backup (PBS) storage source woes

[peanut42]

I don't know where to start, but I hope someone can point me in the right direction.

When trying to manage or look through backups, I go to the VM > Backup > "Storage" drop down menu, which has two options. The first one is a silly old FreeNAS SMB share (cifs), and then second option is a proxmox backup server (pbs).



When selecting the SMB shared backups I almost get an instant listing of the files. However when selecting the proxmox backup server entry, 8 out of 10 times I get a time out. I need to reselect the 2nd option several times before it even lists the backups / responds.

Both storage sources live in different LANs and I also use two different dedicated NICs from the VM server perspective. The SMB LAN is so much more noiser than the backup LAN. The backup LAN has only one purpose where the proxmox VM server and the proxmox backup server talk to each other directly.

Where can I start to look for pbs related errors?

 

[kellogs]

if you go to shell and ping to the pbs ip any packet loss?

also what is the mtu size and interface duplex setting?

 

[peanut42]

if you go to shell and ping to the pbs ip any packet loss?

also what is the mtu size and interface duplex setting?

No packet loss for the last five minutes and the MTU is 1500 (the default I think). I never messed with he MTU.

 

[kellogs]

ok. since they are on different vlans, is there any firewall in between which could be blocking the ports? Maybe a network trace would be ideal

 

[peanut42]

ok. since they are on different vlans, is there any firewall in between which could be blocking the ports? Maybe a network trace would be ideal

The PBS and VM server are on the same LAN, only a switch in between. There is only one hop.

 

[UdoB]

However when selecting the proxmox backup server entry, 8 out of 10 times I get a time out. I need to reselect the 2nd option several times before it even lists the backups / responds.

Did you setup the PBS datastore on SSDs? https://pbs.proxmox.com/docs/installation.html#recommended-server-system-requirements

Anecdote, possibly describing your problem: when I had setup my first PBS I did that on rotating rust (only). This worked. At first. With the amount of backups stored the number of chunks increased. After several weeks(!) I had the effect you are describing: the first click got a timeout. When I clicked a second time the most of the backups listing was already in cache and now this second click worked.

The solution is easy: kick out rotating rust. Yeah, that would've been expensive.

So my solution (better: workaround) was to introduce fast meta data storage. This approach is possible for ZFS (with a fast "Special Device" added before filling in much data) and also (for example) for brtfs on a Synology.

For me the "lesson learned" was no surprise as it is documented everywhere: PBS needs IOPS!

 

[peanut42]

Did you setup the PBS datastore on SSDs? https://pbs.proxmox.com/docs/installation.html#recommended-server-system-requirements

Anecdote, possibly describing your problem: when I had setup my first PBS I did that on rotating rust (only). This worked. At first. With the amount of backups stored the number of chunks increased. After several weeks(!) I had the effect you are describing: the first click got a timeout. When I clicked a second time the most of the backups listing was already in cache and now this second click worked.

The solution is easy: kick out rotating rust. Yeah, that would've been expensive.

So my solution (better: workaround) was to introduce fast meta data storage. This approach is possible for ZFS (with a fast "Special Device" added before filling in much data) and also (for example) for brtfs on a Synology.

For me the "lesson learned" was no surprise as it is documented everywhere: PBS needs IOPS!

Very good point Udo! Thanks for sharing your experience.

I will build out a new PBS as you suggested and use it as the new primary backup location, and then compare. It will take some time, but I will share what I have learned.