USB Offsite backup

[devaux]

Hey,
Is there any way to use USB drives for offsite backups for pve VM backups?

The plan is to do normal backups from the VMs to the pbs local storage, but have additional rotating USB drives that always have at least the previous day's data.
People would plug out/in the usb disks manually every day.

 

[Chris]

Hi,
there is ongoing work to introduce removable datastores in a future version of Proxmox Backup Server, see https://lists.proxmox.com/pipermail/pbs-devel/2024-April/009029.html. Unfortunately I cannot give you a ETA for this.

 

[tcabernoch]

I want a slow tier of storage. Stuff should age out of fast storage and offload to cheaper.
And I hate S3, but its a reasonable option for offsite backups. It would be nice to have that as a slow storage option.

 

[Dunuin]

I want a slow tier of storage. Stuff should age out of fast storage and offload to cheaper.
And I hate S3, but its a reasonable option for offsite backups. It would be nice to have that as a slow storage option.

Problem is that the old backups on the slow storage would have the same IOPS requirements as the new backups on the fast storage as you need to do the same expensive maintainance tasks there too.

Whats already possible is to use multiple datastores (one on fast and one on slow storage), do all the backups to the fast datastore and then use local sync jobs to sync backup snapshots from the fast to the slow one. Both datastores then could have different retentions (fast one keeps backups for a week, slow one for years) and both could use different re-verify/GC intervals (less often maintainance tasks on the slow datastore).

 

[tcabernoch]

Ok. I can try that. Maybe on one of my virtualized PBS's.
Just add another datastore and setup some sync jobs. Pretend its slow storage, see how it all works out.

So if that works like i think ... I'm struggling to find a real-world use case.
What I really want it for is clearing space off my baremetal PBS, but (obviously) retaining access to the backups for longer.
That probably looks like a sync job to a NAS.
From some recent testing I've seen, writing to a NAS share mounted in PBS is about the dumbest thing you could setup.

Well, I might find a use for this in the virtual PBS's.
(EDIT ... Maybe ISCSI? This guy is doing it with a virtual PBS. https://forum.proxmox.com/threads/playground-time-virtual-pbs-with-iscsi-backend.88395/ )

I knew all this already, but hadn't connected the pieces. Thanks.

 

[Dunuin]

Best case would be you get another (maybe slower) bare metal PBS offsite for long-term retention and then let the slow offsite PBS pull the backup snapshots from the fast onsite PBS via sync jobs over a VPN tunnel via internet. That way you will be able to add protection against ransomware and human error, you won't be screwed if a chunk corrupts or when you lose the whole onsite datacenter. And you will be able to do fast live restores of the latest backup snapshots from the fast local PBS and you could directly but slowly restore those old backups from the slow remote PBS if you ever need them again in a worst case scenario.

 

[tcabernoch]

Right. I haven't tried this yet, but I get your point about the primary PBS being slowed down by maintenance operations on the slow data.
A secondary sync partner would break that linkage.
Its a bit convoluted, but the real answer is buy bigger SSDs for the baremetal PBS, so convoluted (and almost free) might be ok.

 

[Chris]

I want a slow tier of storage. Stuff should age out of fast storage and offload to cheaper.
And I hate S3, but its a reasonable option for offsite backups. It would be nice to have that as a slow storage option.

This feature has been requested before, you might want to subscribe to the bugtracker issue to get status updates https://bugzilla.proxmox.com/show_bug.cgi?id=2943

Unfortunately nobody is actively working on this feature from our side at the moment of writing, so this most likely will not be implemented in the near future.