Backup to iCloud

[bbuster]

I have a proxmox backup server but i want to backup my backup server. (in case something happens with the backup server). I have enough iCloud storage and would like to backup the proxmox backup vm's (all of it with incremental backups) to iCloud.

Will something like this be possible?

 

[fireon]

That could be a bit complex. iCloud is not officially supported by any Linux distribution that I know of. I found this projects. You could try it out.
https://github.com/cross-platform/icloud-for-linux
https://rclone.org/iclouddrive/

But the safest way will be to share iCloud on a MacOS with Samba as a fileshare and then mount it on PBS. After that, you can configure a sync job.
https://pbs.proxmox.com/docs/managing-remotes.html

I highly recommend testing this setup thoroughly!
It is better to rent a Proxmox backup server from a provider and synchronize it there.

 

[MagicFab]

I haven't tested this yet but you could use rclone to sync to iCloud storage, see :
https://rclone.org/iclouddrive/

 

[Johannes S]

I haven't tested this yet but you could use rclone to sync to iCloud storage, see :
https://rclone.org/iclouddrive/

Bad idea, rclone is known to break backups stored in a PBS datastore.
https://forum.proxmox.com/threads/datastore-synced-with-rclone-broken.154709/
https://forum.proxmox.com/threads/pbs-appears-not-to-write-to-disk.157751/

The reason is that PBS splits the data in a lot of small files for his deduplication magic (the space savings are insane)., To ensure that they are complete and consistent PBS expect them to be synced in a certain manner (at least according to the explainations done by Proxmox stuff here in the forum and in the manual, up to now I' m neither motivated nor skilled enough to check the source code to determine how everything is working). Since the existing options were not sufficient for this suecase the Proxmox developers developed their own sync mechanism for syncing between different PBS instances. It's a pity really: With native rclone support PBS would be able to do backups to all cloud storage providers but at the moment that's not a viable option. There are some hacks which utilize stuff like cifs mounts or fuser filesystems for s3 or ssh access but they are slow at best and potentially loosing data at worse too.

Now which options do you have? I will quote myself (from ( (https://forum.proxmox.com/threads/b...r-with-incremental-chunks.157965/#post-723568 ):

You have following options:

• Using a PBS cloud provider like this one: https://www.tuxis.nl/de/proxmox-backup-server/ Tuxis don't sell to private customers outside the Netherlands or in general to customers outside the EU but they might have resellers (I know of at least one German reseller). And they have a free tier for 150 GB so that might be enough for you.
• Get a Debian vserver from a hoster of your choice (Hetzner, netcup, catando, whatever) and setup PBS on it. This is surprising affordable I'm paying under ten Euro at the moment for a vserver at Netcup + additional storage for the PBS datastore.
• Ditch PBS all together and do the regular vzdump backups of Proxmox. Backup them with restic, borgbackup, rclone or some other simmiliar tool to the cloud provider of your choice. Note: In this case you might be better off in splitting up your vms/lxc (and thus their backups):
  • The VM/LXC OS lives on one virtual disk and is backuped with Proxmox vzdump or PBS
  • The data lives on another virtual disk and is backuped to a cloud storage with restic, borgbackup

The last option might be of interest for you since restic has a rclone backend enabling it to backup to everything supported by rclone.

I myself do something mixed: I backup my vms/lxcs with a local pbs and sync them to a PBS on a netcup vserver. For the bulk data of my NAS VM and notebook I use restic to backup them to a cloud storage. Additionally I do weekly backups of my most important vms with the vzdump option and sync them to the same cloud storage.

Another option might be to buy some external disks and use them as removable datastores with your existing PBS. You could store one of it outside of your place (e.G. in your office or at your mums flat you visit every sunday for tea) and swap between them on a regular schedule. This would be a cheap, although manual and thus error-prone way to have an offsite backup. Of course you could also put a second PBS to your mums place if you just visit her once a month Or if you are doing this as part of your work and your company has more than one location, put one PBS in each of your companies locations and sync between them.

 

[MagicFab]

Interesting, thanks !

I think "rclone is known to break backups stored in a PBS datastore" is misleading, though. As you mention PBS file structure is not designed to properly restore from direct sync with other tools. For sure hosting a PBS on another host to sync an existing PBS setup looks like the only reliable and supported option for now.

Perhaps with the recent introduction of support for external backup providers in PVE 8.4 this will change and give more choice to users.

I am thinking another option is to use PVE (not PBS) backups to "local" storage presented by an existing rclone attached drive (including iCloud). No incremental backups but better than nothing.

 

[Johannes S]

Interesting, thanks !

I think "rclone is known to break backups stored in a PBS datastore" is misleading, though.

It's not because that's exactly what happened in the linked thread: Somebody stored his PBS datastore on a cloud storage with rclone and when he tried to restored it it wasn't possible due to broken data. I don't want to be a nitpicker but I really want people not to do potential disastrous things with their backups. For that reason I' m always copy/pasting the same wall of text everytime somebody mentions PBS and rclone together
If I ever have some time I will finish my more generic writeup on backup options for ProxmoxVE and post it as a tutorial/faq here.

Perhaps with the recent introduction of support for external backup providers in PVE 8.4 this will change and give more choice to users.

At least it's makes implementing such a feature easier, whether somebody will actually do it, is a different story though. At the moment I wouldn't hold my breath. And as said: A native rclone support would be a great thing.

I am thinking another option is to use PVE (not PBS) backups to "local" storage presented by an existing rclone attached drive (including iCloud). No incremental backups but better than nothing.

Yes, that's what I do myself for my weekly additional backups with the native backup of PVE. One need to be cautious though to not use up all the space of the local storage.

 

[bbuster]

After thinking about it i am not going to use iCloud for it, i will setup a different server with proxmox backup on a different location. This will work like proxmox intentioned it so i think this will be the safest option.

Thanks for the replies