[TUTORIAL] RELEASE: [proxmox-vzbackup-rclone] proxmox vzbackup rclone integration (automated backup of vm's/ct's to google drive or other cloud storage)

[DemiNe0]

https://github.com/TheRealAlexV/proxmox-vzbackup-rclone

This is a vzbackup hook script that backups up your proxmox vms, containers and pve configs to remote storage such as google drive using proxmox's native vzbackup tool and rclone.

rclone is a command line tool that allows you to sync files from your local disk, to a cloud storage device. RClone is most popular with Google Drive but it can be used for other cloud providers. RClone is based off another tool called RSync but with RClone you get so much more functionality built in such as encryption. See https://rclone.org/.

Backups are stored in the rclone remote and organized into YEAR/MONTH/DAY directories to ease the management of the backup files. The backup script also prunes local backups after a configurable amount of days. The script DOES NOT prune backups stored on the remote. You will need to manage that separately if you do not have unlimited space. There is also an easy to use script to pull old backups from the remote so that you can restore them like you normally would through the webui or vzdump tool.

This was built and tested with Google Drive only, however it should work with other providers as well. I recommend Google Drive because you can get a Google Apps business account for $12 a month which gets you Unlimited drive space.

A quickstart guide can be found over at the projects official github repo: https://github.com/TheRealAlexV/proxmox-vzbackup-rclone
Pull requests and issues are always welcome!

 

[H4R0]

Im also working on a backup hook script but using duplicity.

Uses rsa 4096bit keys for encryption.

Only 111mb for a incremental backup of 8gb src. (running vm, moving data)

INFO: status: 89% (30604591104/34359869440), sparse 65% (22447919104), duration 99, read/write 790/0 MB/s
INFO: status: 95% (32937213952/34359869440), sparse 72% (24780247040), duration 102, read/write 777/0 MB/s
INFO: status: 100% (34359869440/34359869440), sparse 76% (26202796032), duration 104, read/write 711/0 MB/s
INFO: transferred 34359 MB in 104 seconds (330 MB/s)
INFO: archive file size: 7.60GB
INFO: delete old backup '/mnt/cryptcache/backup/dump/vzdump-qemu-100-2020_06_01-22_27_03.vma'
INFO: ---- Offsite Backup of firewall1 ---- Wed Jun  3 00:01:47 UTC 2020
INFO: Local and Remote metadata are synchronized, no sync needed.
INFO: Last full backup date: Mon Jun  1 22:28:38 2020
INFO: --------------[ Backup Statistics ]--------------
INFO: StartTime 1591142510.22 (Wed Jun  3 00:01:50 2020)
INFO: EndTime 1591143182.45 (Wed Jun  3 00:13:02 2020)
INFO: ElapsedTime 672.23 (11 minutes 12.23 seconds)
INFO: SourceFiles 2
INFO: SourceFileSize 8161715716 (7.60 GB)
INFO: NewFiles 1
INFO: NewFileSize 4 (4 bytes)
INFO: DeletedFiles 0
INFO: ChangedFiles 1
INFO: ChangedFileSize 8161715712 (7.60 GB)
INFO: ChangedDeltaSize 0 (0 bytes)
INFO: DeltaEntries 2
INFO: RawDeltaSize 639709280 (610 MB)
INFO: TotalDestinationSizeChange 116314495 (111 MB)
INFO: Errors 0
INFO: -------------------------------------------------
INFO: Local and Remote metadata are synchronized, no sync needed.
INFO: Last full backup date: Mon Jun  1 22:28:38 2020
INFO: No old backup sets found, nothing deleted.
INFO: ---- Finished Backup of firewall1 ---- Wed Jun  3 00:14:02 UTC 2020
INFO: Finished Backup of VM 100 (00:14:00)

 

[DemiNe0]

Oh wow. I never even heard of duplicity. Would love to hear more about your setup.

 

[mir]

https://github.com/TheRealAlexV/proxmox-vzbackup-rclone

This is a vzbackup hook script that backups up your proxmox vms, containers and pve configs to remote storage such as google drive using proxmox's native vzbackup tool and rclone.

rclone is a command line tool that allows you to sync files from your local disk, to a cloud storage device. RClone is most popular with Google Drive but it can be used for other cloud providers. RClone is based off another tool called RSync but with RClone you get so much more functionality built in such as encryption. See https://rclone.org/.

Backups are stored in the rclone remote and organized into YEAR/MONTH/DAY directories to ease the management of the backup files. The backup script also prunes local backups after a configurable amount of days. The script DOES NOT prune backups stored on the remote. You will need to manage that separately if you do not have unlimited space. There is also an easy to use script to pull old backups from the remote so that you can restore them like you normally would through the webui or vzdump tool.

This was built and tested with Google Drive only, however it should work with other providers as well. I recommend Google Drive because you can get a Google Apps business account for $12 a month which gets you Unlimited drive space.

A quickstart guide can be found over at the projects official github repo: https://github.com/TheRealAlexV/proxmox-vzbackup-rclone
Pull requests and issues are always welcome!

Maybe you should consider using restic on-top of rclone to enforce security?
https://github.com/restic/restic

 

[wifi75]

hello sorry but after send command ./vzbackup-rclone.sh
nothing happens...
my configuration is like this:


############# / START CONFIG
dumpdir = "/ var / lib / vz / dump /" # Set this to where your vzdump files are stored
MAX_AGE = 3 # This is the age in days to keep local backup copies. Local backups older than this are deleted.
############# / END CONFIG

do i need to add more?

 

[UdoB]

Good morning,

dumpdir = "/ var / lib / vz / dump /"

is that an copy-n-paste error? There can not be spaces in a pathname. Try "/var/lib/vz/dump" without spaces...

Good luck

 

[wifi75]

Good morning,

is that an copy-n-paste error? There can not be spaces in a pathname. Try "/var/lib/vz/dump" without spaces...

Good luck

yes it is correct on my script....




this below my backup folder

 

[alamoudi.moh]

Tried it with onedrive, the backed up files are in KBs?!! Approximately, 850KBs?

 

[Chiller]

I used the pull-request from Marko298 (https://github.com/TheRealAlexV/proxmox-vzbackup-rclone/pull/14)

But with every code I got a error. But its upload to GDrive, encrypted.

############ /START CONFIG
drive="gdrive_crypt"
dumpdir="/var/lib/vz/dump" # Set this to where your vzdump files are stored
MAX_AGE=3 # This is the age in days to keep local backup copies. Local backups older than this are deleted.
MAX_CLOUD_AGE=90 # This is the age in days to keep cloud backup copies. Cloud backups older than this are deleted
############ /END CONFIG

INFO: Backing up /var/lib/vz/dump/vzdump-lxc-115-2023_05_08-11_42_07.tar.zst to remote storage
INFO: rcloning /var/lib/vz/dump/rclone/2023-05-08
INFO: 2023/05/08 11:45:22 INFO  : 
INFO: Transferred:         338.333 MiB / 418.505 MiB, 81%, 5.834 MiB/s, ETA 13s
INFO: Transferred:            0 / 1, 0%
INFO: Elapsed time:       1m2.6s
INFO: 2023/05/08 11:09:52 ERROR : : error listing: directory not found
INFO: 2023/05/08 11:09:52 ERROR : Attempt 1/3 failed with 2 errors and: directory not found
INFO: 2023/05/08 11:09:52 ERROR : : error listing: directory not found
INFO: 2023/05/08 11:09:52 ERROR : Attempt 2/3 failed with 2 errors and: directory not found
INFO: 2023/05/08 11:09:52 ERROR : : error listing: directory not found
INFO: 2023/05/08 11:09:52 ERROR : Attempt 3/3 failed with 2 errors and: directory not found
INFO: 2023/05/08 11:09:52 Failed to delete with 2 errors: last error was: directory not found

Think its a Problem with the Time-Date-Folder
Anyone a Idea?