S3 Backup for proxmox backup server

[tiny6996]

I am wondering if it is possible to restore a datastore from S3 storage. If had lost my proxmox backup server, but backed up the datastore to S3. I am thinking that if I install a tool like restic on my pbs and set the backup destination to wasabi. I could setup a new proxmox backup server, install restic, pull down the lastest backup and all the snapshots would appear in the datastore and in proxmox VE. is this the case or is there a different method for send me datastore offsite without having to setup a pbs in a different location?

 

[dcsapak]

i mean yeah you could do that (be careful to also backup the hidden files, e.g. the '.chunks' folder) the end result might not be consistent if it syncs during a garbage collect/backup/prune (so you backup maybe more chunks than necessary)

 

[Ivan Dimitrov]

I have done a small test like that - create PBS repo --> backup the repo with restic to Backblaze --> delete the repo --> mount the restic backup --> recreate the repo --> restore a VM.
It worked OK the benefit of mounting the restic backup is that you do not have to download the whole repo but the speed is limited to the speed of the cloud provider. As long as PBS doesn't support cloud storage backends I think it is an OK solution.

 

[donatello]

I have done a small test like that - create PBS repo --> backup the repo with restic to Backblaze --> delete the repo --> mount the restic backup --> recreate the repo --> restore a VM.
It worked OK the benefit of mounting the restic backup is that you do not have to download the whole repo but the speed is limited to the speed of the cloud provider. As long as PBS doesn't support cloud storage backends I think it is an OK solution.

How do you mount the restic backup?
I tried but I can't.
The only thing is to restore the entire backup.

 

[Ivan Dimitrov]

How do you mount the restic backup?
I tried but I can't.
The only thing is to restore the entire backup.

Hi, In the restic client there is mount option. After the storage is mounted you have to point a PBS data store to this folder. I have done this by editing the /etc/proxmox-backup/datastore.cfg as PBS doesn't have an option to import existing datastore, yet.

$ restic mount --help

The "mount" command mounts the repository via fuse to a directory. This is a
read-only mount.

Snapshot Directories
====================

If you need a different template for all directories that contain snapshots,
you can pass a template via --snapshot-template. Example without colons:

    --snapshot-template "2006-01-02_15-04-05"

You need to specify a sample format for exactly the following timestamp:

    Mon Jan 2 15:04:05 -0700 MST 2006

For details please see the documentation for time.Format() at:
  https://godoc.org/time#Time.Format

EXIT STATUS
===========

Exit status is 0 if the command was successful, and non-zero if there was any error.

Usage:
  restic mount [flags] mountpoint

Flags:
      --allow-other                  allow other users to access the data in the mounted directory
  -h, --help                         help for mount
  -H, --host stringArray             only consider snapshots for this host (can be specified multiple times)
      --no-default-permissions       for 'allow-other', ignore Unix permissions and allow users to read all snapshot files
      --owner-root                   use 'root' as the owner of files and dirs
      --path path                    only consider snapshots which include this (absolute) path
      --snapshot-template template   set template to use for snapshot dirs (default "2006-01-02T15:04:05Z07:00")
      --tag taglist                  only consider snapshots which include this taglist (default [])

Global Flags:
      --cacert file                file to load root certificates from (default: use system certificates)
      --cache-dir directory        set the cache directory. (default: use system default cache directory)
      --cleanup-cache              auto remove old cache directories
      --json                       set output mode to JSON for commands that support it
      --key-hint key               key ID of key to try decrypting first (default: $RESTIC_KEY_HINT)
      --limit-download int         limits downloads to a maximum rate in KiB/s. (default: unlimited)
      --limit-upload int           limits uploads to a maximum rate in KiB/s. (default: unlimited)
      --no-cache                   do not use a local cache
      --no-lock                    do not lock the repository, this allows some operations on read-only repositories
  -o, --option key=value           set extended option (key=value, can be specified multiple times)
      --password-command command   shell command to obtain the repository password from (default: $RESTIC_PASSWORD_COMMAND)
  -p, --password-file file         file to read the repository password from (default: $RESTIC_PASSWORD_FILE)
  -q, --quiet                      do not output comprehensive progress report
  -r, --repo repository            repository to backup to or restore from (default: $RESTIC_REPOSITORY)
      --repository-file file       file to read the repository location from (default: $RESTIC_REPOSITORY_FILE)
      --tls-client-cert file       path to a file containing PEM encoded TLS client certificate and private key
  -v, --verbose n                  be verbose (specify multiple times or a level using --verbose=n, max level/times is 3)

 

[donatello]

I tried to mount and change the "datastore.cfg", but it gave me the following error:
unable to open chunk store 'Wasabi' at "/mnt/s3backup/snapshots/2021-11-17T15:33:18+01:00/wasabi-local/.chunks" - Permission denied (os error 13)

I realized that by changing the file and directory permissions:
chown backup:backup .chunks ct vm .locks -R
or more simply:
chown backup:backup * -R

but it only works if I restore a backup and not mount it.