I Think I Get It

[Stereoscope]

I think I finally figured out what Proxmox Backup Server is. It backs up your VMs/CTs that are running on Proxmox. This means that if you have a file server VM running on Proxmox which stores all your data, then this is all part of the VM and will be backed up to Proxmox Backup Server.

For the longest time I thought it just housed copies of your VMs but my file server data was somehow not part of that. I guess since the file server had a pass through HBA card, it somehow made me think it wasn't part of Proxmox. I was going to back this up separately in a Linux box with ZFS separate to Proxmox VE and PBS.

Boy do I feel dumb right now.

I was wondering, how important is it to have metadata cache? I'll be using spinning rust for the backup pool.

 

[Stereoscope]

To answer my own question - https://forum.proxmox.com/threads/metadata-cache-on-zfs-how-much-does-it-help-for-pbs.145904/ so basically don't worry about it. At least for me.

 

[UdoB]

I'll be using spinning rust for the backup pool.

so basically don't worry about it. At least for me.

I am not convinced. In my local universe (aka homelab) I am using rotating rust too. With this a good Special Device is essential.

Anecdote, told multiple times: my very first PBS was classic HDD only. It worked. But after some time, after storing some amount of data, it got slow. It was not even possible to list those backups in the Web Gui. The solution in my specific case was to add some fast metadata storage.

Disclaimer: your-mileage-may-very, you may just try it out

 

[Stereoscope]

How much data did you have before it got slow do you think? I'm using spinning rust as well.

 

[UdoB]

How much data did you have before it got slow do you think?

Single digit TB.

It was just ok to use it for several weeks. Then it got too slow to work with...

Remember: for each and every chunk (below 4 MB of data) metadata has to be read (or written, which is worse). For Gigabytes of data this means a lot of physical head movements. Yes, a lot of metadata is being cached. But a SD eliminates this slow aspect completely. (But HDD-based storage will never "fly"!)

 

[Stereoscope]

Hmm, so doesn't take much it sounds. My other problem is that I plan on running PBS as dual boot, so I'm not sure how much a metadata cache would help between restarts. Is it possible to persist the cache to SSD? Or does it persist by default?

EDIT: Found the answer: "For viewing backup lists it shouldn't matter as long as you never reboot the PBS (because the readcache will be dropped unless you enable persisent L2ARC). For GC special vdevs should be faster as this would also speed up the "touch" operations of the GC." https://forum.proxmox.com/threads/backup-slow-listings.134151/

 

[UdoB]

I'm not sure how much a metadata cache would help between restarts. Is it possible to persist the cache to SSD?

A "Special Device" is NOT a cache. It contains the actual (meta-) data. (What made you think it is???)

If you lose the SD all data in that pool is gone.

You see, it is -of course- persistent

 

[Stereoscope]

Got it, thanks. But looks like I need to look much deeper into it. Lots that I don't understand/know at this point and it's important to warrant some time to learn it.

 

[Johannes S]

For the longest time I thought it just housed copies of your VMs but my file server data was somehow not part of that. I guess since the file server had a pass through HBA card, it somehow made me think it wasn't part of Proxmox. I was going to back this up separately in a Linux box with ZFS separate to Proxmox VE and PBS.

Please note that regular backups won't contain the data on the Discs attached to the HBA. For that you need to run a backup tool ( e.g. proxmox backup Client or restic) inside the NAS VM.

 

[Stereoscope]

Still have some research ahead of me. I've not bumped into the Proxmox Backup Clinet or Restic, so not entirely sure how it all works yet.