Sync to two external USB HDDs with weekly rotation

[umeko]

I'm considering to sync backups to a set of two external USB HDDs, where one HDDs is connected and the other one is stored off-site. Several forum threads suggest that syncing to an external HDD should be possible.

One question remains:

Do I need to create two different mount points/datastores for the two disks, or can I use the same mount point/datastore for both disks?
In other words: does PBS keep a kind of state, which becomes inconsistent when disks are swapped, as they will contain different backup snapshots, or is all state (if any) of the datastore kept inside the datastore itself, so it gets refreshed when the disks are swapped?

 

[fireon]

There is a ready-made project for this @The_Invisible: https://hadler.me/cc/tibackup/

You can install it over this Repository: https://apt.iteas.at/

 

[umeko]

There is a ready-made project for this @The_Invisible: https://hadler.me/cc/tibackup/

You can install it over this Repository: https://apt.iteas.at/

@fireon thanks for your comment, but I'm not looking for an 3rd-party solution; since I'm using PBS I want to use a well-integrated PBS on-board solution. Almost all the required features seem to be present in PBS, except I don't know for sure if I can use two external hard-disks in rotation with the same PBS directory storage/datastore config without running into problems.

 

[fireon]

@fireon thanks for your comment, but I'm not looking for an 3rd-party solution; since I'm using PBS I want to use a well-integrated PBS on-board solution. Almost all the required features seem to be present in PBS, except I don't know for sure if I can use two external hard-disks in rotation with the same PBS directory storage/datastore config without running into problems.

The project was developed precisely for this purpose. It is not available from Proxmox itself. USB backup is not supported for enterprise. Another option is to program yourself something that does what you need.

From the Proxmox VE page there are hookscripts that can be applied to backup jobs.
https://pve.proxmox.com/pve-docs/pve-admin-guide.html#_hook_scripts

 

[VictorSTS]

All the information regarding which backups are in a datastore is stored in the datastore itself. You could use the same datastore for both disks and swap them, but I would use one datastore for each drive. Then use sync tasks on each on even weeks for one drive, odd weeks in the other. The host stores in /etc/proxmox-backup the configuration of where is each datastore, which jobs and when to run them, etc.

If you want to automate things, you will need to use UDEV rules to mount the drive in some path and enable the datastore when you plug the drive. Sync jobs will run once the datastore is enabled. Before unplugging the drive you must set the datastore offline first and wait some minutes so proxmox-backup-server closes all file handles. For that to work no sync/gc/purge/verify tasks must be running on the datastore. Once datastore is offline, you can unplug the drive. Maybe a cron job or similar may automate the unplug process by checking if tasks are running before setting the

Did some tests with this some time ago, but there was a bug that made proxmox-backup-server not to close the handles in the datastore and the USB drive couldn't be unlugged cleanly without restarting proxmox-backup-server service and doing so all running tasks got aborted, which wasn't acceptable in my use case.

 

[umeko]

Thanks @VictorSTS ! Good to hear that all the information is in the datastore itself. I don't know yet if I'll first try with one or two datastores. I think with one it would be easier, as swapping is manual, and it could happen that one week the disks would not be swapped, and handling that in the sync tasks could be a little more difficult than just using one datastore for syncing.

Thanks for the hints fot automounting/offline mode, I will take that into account.

 

[fireon]

There is a ready-made project for this @The_Invisible: https://hadler.me/cc/tibackup/

You can install it over this Repository: https://apt.iteas.at/

I have just seen that the page is not up to date. Because there is now a full integration for Proxmox Backup Server. The software also recognizes all the VMs in the backup stores.