Exporting datastores or backups to external hdd/sdd

Fathi

Well-Known Member
May 13, 2016
123
2
58
52
Tunis, Tunisia
Hi,
I am facing the need to export backups off-site offline. From what i have understood, and to simplify, bakups are stored in (fixed size for vms, variable size for containers) files called chunks. And that the set of chunks that makes a backup is stored in indexes, thus allowing chunks to be shared between several backups, achieving deduplication.
This idea is something related to need i described in this post https://forum.proxmox.com/threads/archiving-to-external-hdd-and-deduplication.133122/
I know, i could achieve this with a second pbs off-site and declaring the first pbs as a remote to the second pbs. But, this needs a second site and a good internet connection. In my case the only second site that is available is connected to the internet with 4Mb ADSL link (and this is the maximum speed available in that region, FO internet still being very expensive here).
I have two solutions:
1) Periodically ( ~ once a week ) plug an usb hdd and do an rsync of the pbs datastore to that external hdd,
2) "Export" latest backup(s) from pbs to that external usb plugged hdd. As there is no such feature availbale from the GUI. IS there a cli command that could achieve this ?
3) This function is not implemented in pbs, in that case, do you think this is an interesting feature that we could request to the pbs team ?
TIA.
 
3) This function is not implemented in pbs, in that case, do you think this is an interesting feature that we could request to the pbs team ?
Backup to external disks in adddition to LTO for archival is already a planned feature.

1) Periodically ( ~ once a week ) plug an usb hdd and do an rsync of the pbs datastore to that external hdd,
Possible but not sure how well that will work without setting the datastore to read-only maintainance mode first if you care about data integrity of your backups.
With a datastore backed by ZFS I would replicate from local storage to external ZFS formated disk by doing an incremental "zfs send | zfs recv".
Another option would be to use the external Disk as another datastore and then make use of the PBS sync jobs. Something like:
  1. plugin disk
  2. Mount filesystem
  3. disable maintainance mode
  4. Run sync job to localhost as target from local datastore to external datastore to sync the latest X backup snapshots
  5. run verify job on external datastore
  6. run prune on external datastore
  7. Wait 24 hours and 5 minutes
  8. run GC on external datastore
  9. enable maintainance mode
  10. Unmount filesystem
  11. Unplug disk

2) "Export" latest backup(s) from pbs to that external usb plugged hdd. As there is no such feature availbale from the GUI. IS there a cli command that could achieve this ?
No feature yet. Closest would be a local sync as described above.
 
  • Like
Reactions: hartupee and Fathi
Backup to external disks in adddition to LTO for archival is already a planned feature.


Possible but not sure how well that will work without setting the datastore to read-only maintainance mode first if you care about data integrity of your backups.
With a datastore backed by ZFS I would replicate from local storage to external ZFS formated disk by doing an incremental "zfs send | zfs recv".
Another option would be to use the external Disk as another datastore and then make use of the PBS sync jobs. Something like:
  1. plugin disk
  2. Mount filesystem
  3. disable maintainance mode
  4. Run sync job to localhost as target from local datastore to external datastore to sync the latest X backup snapshots
  5. run verify job on external datastore
  6. run prune on external datastore
  7. Wait 24 hours and 5 minutes
  8. run GC on external datastore
  9. enable maintainance mode
  10. Unmount filesystem
  11. Unplug disk


No feature yet. Closest would be a local sync as described above.
Thank you @Dunuin . Missed the sync to another local datastore option that could emulate that export function.
Also very interresting the zfs send | zfs receive option.
Thank you.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!