[SOLVED] Remote Offsite Backup options

[Jarvar]

Hello Everybody,

Great news about the PVE Backup Server.
Are there any suggestions for backing up remotely?
I am assisting to setup in a small office that does not necessarily have the room for additional backup scenarios like many mentioned on these forums.

In another location, we setup a Synology NAS box and used NFS to backup our Proxmox VMs there. Then we sync those files via Rsync or Synology Drive Sync to another offsite location.

However, at this place right now, I have put in a SFF server ZFS Raid 1 for Boot OF and Storage together since it only has room for 2 drives.
I have 2 USB Drives attached.

Is there any way fo this server to backup remotely? Or better to back up to attached USB drive and than maybe pve-zsync, sshfs, zfs send?

Very excited about the PVE Backup Server and testing it out though.
Thank you.

 

[Alwin]

You can use the Proxmox Backup Server directly to sync the datastore to a remote.
https://pbs.proxmox.com/docs/administration-guide.html#remote

 

[t.lamprecht]

I am assisting to setup in a small office that does not necessarily have the room for additional backup scenarios like many mentioned on these forums.

We recommend for now having the initial backup server locally, but it's still possible to backup directly to a WAN server. Due to the incremental nature often only the first backup is rather slow, but if the workload doesn't generate lots of new data the following ones should be quite quick.

Is there any way fo this server to backup remotely?

What do you mean with safe? Security wise it should be already OK. The traffic is covered by TLS between client and server and you could also client-encrypt it so that even the server has no access to the real data without the client key.

Besides that it may depend on the workload, if the VM "dirties" a lot of data between backup you may run into hiccups here and there, just like you would with an overloaded local backup target.

 

[Jarvar]

@Alwin @t.lamprecht
Thank you so much for your quick reply.
I am quite new at this. I will take a look at this further when I am in front of a computer and server.
I managed to make sure our backup images were down to about 60GB. Over the last couple months it has ballooned up to 100GB which is still manageable, but I would appreciate the incremental backups as not much really changes between each backup.
I also use Duplicati to backup the actual data that changes more regularly than the whole VM. Just figure the benefits of Proxmox is that I can restore the VM to another piece of hardware and be up and running in the case of fire, theft, disaster etc.
Just completed my first local backup to PVE PBS VM of 32GB, Somehow the VM was compressed to fit into a 32 GB VM as well.
Thank you so much, I will look into it.
I will probably make a local backup and transfer that offsite ASAP so atleast I have something while I figure this stuff out.

 

[Jarvar]

I'm sure I am doing something incorrectly.
I port forwarded 8007 tcp and udp to my Proxmox Backup Server for Remote connections.
However I get this error:

create storage failed: error during cfs-locked 'file-storage_cfg' operation: proxmox-backup-client failed: Error: error trying to connect: dns error: failed to lookup address information: Temporary failure in name resolution (500)

I used the current ip address instead of a dynamic dns service and it added. I don't know how long that will last though, but it worked. Maybe I will try a different service as well.

 

[Jarvar]

quick question. I had to go onsite to terminate more cat6 cables. I downloaded a backup to usb drive. Can i carry on incremental backups if I upload it to my pve backup server? Also can these backups be transferred or replicated to other backup servers down the road? Thanks so much.
Oh yeah the format is ZSTD snapshots.

 

[t.lamprecht]

failed to lookup address information: Temporary failure in name resolution

This is an OS error, and means normally that the system really cannot resolve the DNS name - you effectively should get a similar error if you use ping domain.whatever
Free dynamic DNS services are often quite brittle, and you place quite some trust in them. If you can, I'd get a cheap domain from a provider with an API and let a script run a few times a day to update the A record of a subdomain and use that (I'm doing this here and works very good). But just a suggestion, for quick tests that may be overkill.

I downloaded a backup to usb drive.

How did you do that, making a "normal" vzdump archive backup? Currently, it's not directly possible to load those backups into the server, the probably easiest way to do so now, would be restoring it, and doing a stop mode backup into the PBS.
May see if how hard it would be to provide an old to new importer which could do this (semi-) automatically.

 

[Jarvar]

Free dynamic DNS services are often quite brittle, and you place quite some trust in them. If you can, I'd get a cheap domain from a provider with an API and let a script run a few times a day to update the A record of a subdomain and use that (I'm doing this here and works very good). But just a suggestion, for quick tests that may be overkill.

To Clarify, do I need to open the ports in the router or Firewall for remote backups to work? I Thanks.

How did you do that, making a "normal" vzdump archive backup? Currently, it's not directly possible to load those backups into the server, the probably easiest way to do so now, would be restoring it, and doing a stop mode backup into the PBS.
May see if how hard it would be to provide an old to new importer which could do this (semi-) automatically.

Yes pretty much that is what I did. I guess I can do a stop backup over a time when they a longer period where their office is not operating. I ran out of disk space yesterday.
How do I increase the datastore size? I increased the size of the VM and it's reflecting in disk size. Could be that the store has no more room left to complete any functions, or do I have to delete the store and create a new one?
Thank you again. I'll look into the API functions ou mentioned as well.

 

[Jarvar]

I wanted to update that I got PVE Backup Server working remotely with DNS. It lookses like the dns name resolution was having and issue because the PVE node I built before putting in was showing the old DNS of the offsite locaton, but not the gateway of the current location. After I updated that, the dynamic dns name was working when adding names vs. ip addresses. My old storage broke when th IP Adddress changed.

However I did find that when I closed the port forwarding for 8007 TCP and UDP that the storage broke. Can anybody confirm this? Are there any other ports that should be open? Or am I doing this corrrectly?
Thanks.

 

[t.lamprecht]

However I did find that when I closed the port forwarding for 8007 TCP and UDP that the storage broke. Can anybody confirm this? Are there any other ports that should be open? Or am I doing this corrrectly?

Yes, port 8007 TCP is required for this to work, nothing more though. Only that single port is used with HTTP/2 to access the API and upload/download backup data..

 

[Jarvar]

We recommend for now having the initial backup server locally, but it's still possible to backup directly to a WAN server. Due to the incremental nature often only the first backup is rather slow, but if the workload doesn't generate lots of new data the following ones should be quite quick.



What do you mean with safe? Security wise it should be already OK. The traffic is covered by TLS between client and server and you could also client-encrypt it so that even the server has no access to the real data without the client key.

Besides that it may depend on the workload, if the VM "dirties" a lot of data between backup you may run into hiccups here and there, just like you would with an overloaded local backup target.

Sorry for the late reply. I am just coming around to this idea.
Am I able to do a local backup. Restore it on a remote server, back it up to PBS and then continue my backups from the original server to the PBS backup image made from the local backup to remote server?
Essentially
VM1->LocalBackup1
LocalBackup1(transfer offsite)--->Restore offsite to Restore1
Backup Restore1--->PBS1
VM1 backup to -->Remote PBS1

Does that make sense or is it too confusing?
Im getting errors in my initial VM1 to remote PBS1 initial backup that results in an incorrect blob size error.

 

[fabian]

yes, that should work. could you post the complete backup log (client and PBS task) and any PBS-side logs from that time?

 

[Jarvar]

Thank you @fabian

This was the latest output from a verification I just ran.

verify store1:vm/107/2020-08-08T22:45:02Z
2020-08-11T05:54:07-04:00:   check qemu-server.conf.blob
2020-08-11T05:54:07-04:00:   check drive-efidisk0.img.fidx
2020-08-11T05:54:07-04:00:   check drive-scsi0.img.fidx
2020-08-11T05:56:28-04:00: verify store1:vm/107/2020-08-08T22:45:02Z/drive-scsi0.img.fidx failed: blob too small (0 bytes).
2020-08-11T05:56:28-04:00:   check drive-scsi1.img.fidx
2020-08-11T06:28:51-04:00: verify store1:vm/107/2020-08-08T22:45:02Z/drive-scsi1.img.fidx failed: blob too small (0 bytes).
2020-08-11T06:28:51-04:00: TASK ERROR: verfication failed - please check the log for details

 

[fabian]

do you still have the logs from making that backup? could you post them as well? what kind of storage are you using for your store1 datastore?

 

[Jarvar]

@fabian
This is the logs from my email notification after the backup was complete.
The PBS is running as a VM on PVE 6.2-10 and the store1 is a vmdisk ext4 I believe. I have another vm that backed up to that store and the restore was successful. Albeit the VM was only 32GB total about.

I am quite new to this. I don't seem to be able to post the output from the logs. I received one in my email, but when trying to submit there is a limit of 10000 characters.