Exporting datastores or backups to external hdd/sdd

[Fathi]

Hi,
I am facing the need to export backups off-site offline. From what i have understood, and to simplify, bakups are stored in (fixed size for vms, variable size for containers) files called chunks. And that the set of chunks that makes a backup is stored in indexes, thus allowing chunks to be shared between several backups, achieving deduplication.
This idea is something related to need i described in this post https://forum.proxmox.com/threads/archiving-to-external-hdd-and-deduplication.133122/
I know, i could achieve this with a second pbs off-site and declaring the first pbs as a remote to the second pbs. But, this needs a second site and a good internet connection. In my case the only second site that is available is connected to the internet with 4Mb ADSL link (and this is the maximum speed available in that region, FO internet still being very expensive here).
I have two solutions:
1) Periodically ( ~ once a week ) plug an usb hdd and do an rsync of the pbs datastore to that external hdd,
2) "Export" latest backup(s) from pbs to that external usb plugged hdd. As there is no such feature availbale from the GUI. IS there a cli command that could achieve this ?
3) This function is not implemented in pbs, in that case, do you think this is an interesting feature that we could request to the pbs team ?
TIA.

 

[Dunuin]

3) This function is not implemented in pbs, in that case, do you think this is an interesting feature that we could request to the pbs team ?

Backup to external disks in adddition to LTO for archival is already a planned feature.

1) Periodically ( ~ once a week ) plug an usb hdd and do an rsync of the pbs datastore to that external hdd,

Possible but not sure how well that will work without setting the datastore to read-only maintainance mode first if you care about data integrity of your backups.
With a datastore backed by ZFS I would replicate from local storage to external ZFS formated disk by doing an incremental "zfs send | zfs recv".
Another option would be to use the external Disk as another datastore and then make use of the PBS sync jobs. Something like:

1. plugin disk
2. Mount filesystem
3. disable maintainance mode
4. Run sync job to localhost as target from local datastore to external datastore to sync the latest X backup snapshots
5. run verify job on external datastore
6. run prune on external datastore
7. Wait 24 hours and 5 minutes
8. run GC on external datastore
9. enable maintainance mode
10. Unmount filesystem
11. Unplug disk

2) "Export" latest backup(s) from pbs to that external usb plugged hdd. As there is no such feature availbale from the GUI. IS there a cli command that could achieve this ?

No feature yet. Closest would be a local sync as described above.

 

[Fathi]

Backup to external disks in adddition to LTO for archival is already a planned feature.


Possible but not sure how well that will work without setting the datastore to read-only maintainance mode first if you care about data integrity of your backups.
With a datastore backed by ZFS I would replicate from local storage to external ZFS formated disk by doing an incremental "zfs send | zfs recv".
Another option would be to use the external Disk as another datastore and then make use of the PBS sync jobs. Something like:

1. plugin disk
2. Mount filesystem
3. disable maintainance mode
4. Run sync job to localhost as target from local datastore to external datastore to sync the latest X backup snapshots
5. run verify job on external datastore
6. run prune on external datastore
7. Wait 24 hours and 5 minutes
8. run GC on external datastore
9. enable maintainance mode
10. Unmount filesystem
11. Unplug disk

No feature yet. Closest would be a local sync as described above.

Thank you @Dunuin . Missed the sync to another local datastore option that could emulate that export function.
Also very interresting the zfs send | zfs receive option.
Thank you.