Support for sync to "removable" backup disks

[crazycookie]

Hi there.

I'm experimenting a bit with setting up a 3-2-1 strategy for my home lab. The setup is pretty much the following:

• a tiny PVE cluster with some nodes and one shared bigger (>20TB) storage node
• a tiny PBS node with large primary backup storage (zfs, >20TB) and a hotswap HDD used for local sync from the primary backup storage
• the single sync backup disk is installed as a directory storage

The idea is to sync the latest snapshots to the single backup disk about once a month and then unplug it and move it physically offsite. (No, backblaze or similar cloud services is NOT an option). As the HDD is plugged in at a ordinary backplane hotswaps shouldn't be an issue.

However, as soon I remove the disk the proxmox-backup-proxy process runs at about 80% CPU and is bombing the logs with errors. Restarting it via the UI does not help (its actually not restarting at all). Only way to solve this is to run a systemctl stop and systemctl start explicitly (via ssh of course).

So I wonder if "offsite" backups are only possible for tape backups, not for other removable media? I know there's a "tape simulator" out there, but it would be nice to be able to directly use any removable media (HDD are much cheaper than LTO tapes).

BTW: A workaround for a "clean" procedure like removing the backup datastore first, than removing the directory storage to finally being able to unplug the disk is not an option either. Creating new datastore setups every month is no fun at all.

 

[OsvaldoP]

Think unlock datastore in offline mode is currently in progress. Last activity in dev mailing list last week

More info see here, maybe you explain why this feature is important to you

https://bugzilla.proxmox.com/show_bug.cgi?id=3156

 

[crazycookie]

Thanks, I will have a look at that. I thought the 3-2-1 would explain enough as "1" means offsite and usually even "offline" when it comes to ransomware. Thus, any solution connected to the network (specifically the internet) is not an option.

 

[OsvaldoP]

sorry for the misunderstanding

I meant, leave your consideration in bugtracker, so all devs can see that this feature is important for many users

 

[crazycookie]

Well, I did mention it in bugtracker (no reaction there) but found this one in the mailing list:

https://lists.proxmox.com/pipermail/pbs-devel/2024-March/007965.html

Not sure, but I guess its related and could solve the problem the proxmox-backup-proxystill keeps the datastore busy while holding the .lockfile.

I guess the main problem is the sematic of "offline". Right now it seems "offline" means you cannot reach the datastore via the API, but the processes still keeps the datastore internally "online".

However, for removable media "offline" should really mean "offline" even internally, ideally even "powered off" for save removal.

 

[OsvaldoP]

for your info,
patch is in "no subscription repo"