Questions of a Newbie about virtualization of PBS

[exaveal]

HI all,

thanks so much for this community here. I started to change my homelab-server-landscape to virtualize it with Proxmox. I love it.
Currently I am still figuring out how to finalize my setup and backup strategy.

On my PVE I have a virtualized NAS. The NAS is just a Ubuntu-Server LXC with 12 TB ZFS data storage.
Next to automatic snapshots of all LXC's and VM's, I also want to have a weekly backup to another location.

I have a Raspberry PI 4 connected to a 4 drive RAID. It's running Ubuntu Server and the ZFS RAID offers NFS storage.

My first idea was to create backups from PVE to the Rapsberry PI NFS storage. But only full backups are possible - That means I would have to transfer 12 TB each week.

To massivle reduce the amount of data to transfer each time I need incremental backups. This seems only possible with Proxmox Backup Server.
Would it be a solution to virtualize PBS in my PVE and let PBS save the backups on the NFS storage of the Raspberry PI? Does it work? Are incremental backups working this way? Does the NFS storage from the Raspberry PI must use ZFS filesystem to support incremental backups?

I know there is an inofficial version of PBS for Raspberry PI 4 but I am not so familiar with Linux allowing me to "play" with my backups. So I would really prefer to not use an inofficial PBS version.

Thank you so much!

 

[DerekG]

To massivle reduce the amount of data to transfer each time I need incremental backups. This seems only possible with Proxmox Backup Server.
Would it be a solution to virtualize PBS in my PVE and let PBS save the backups on the NFS storage of the Raspberry PI? Does it work? Are incremental backups working this way? Does the NFS storage from the Raspberry PI must use ZFS filesystem to support incremental backups?

Hi exaveal,

The question should not be, is it possible, but how would you recover should the host die when it is hosting both the PVS & PBS.???
I'm assuming that you are not backing up the full 12TB of Proxmox VM's/LXC's along with the media data? The backup of the media data should be a separate backup routine from the Proxmox infrastructure.

PBS uses it's own storage system, you cannot access the data which has been backed-up by any other method other than a functional PBS instance.
In addition to those issues, PBS is extremely IO intensive, it performs incremental backups not on a file basis but by dividing the source disk into small 'chunks' and comparing those chunks to what it already has stored in the backup target, and then only uploading the chunks which have changed. So the optimal method of storage is to a local SSD store on the local device.

For the reasons stated, I would suggest a dedicated device for the PBS itself. I do have a secondary PBS as a VM which uses zfs ISCSI as a backup store, but that unit does not perform the PVE backups, it instead pulls replicas from the primary PBS and is operational over a 10Gbe link.

I hope that provides some basic understanding of how the system functions, and that will at least point you in the right direction.

 

[exaveal]

Hi & much thanks for your feedback. I really appreciate it.

Running PBS as a VM:
I am aware of the point to use PBS virtualized in the PVE. What I think about regarding MY use case: If my PVE crashes, than I have no problem, because everything is saved on the external Server (in my case the Raspberry Pi 4 including the NAS).
So what I will do, if my PVE crashes: I will set up a new PVE, install PBS again as a VM, connect it to the Raspberry PI 44 and restore my LXC's and VMs. Is there anything wrong in my understand? I don't see any "breakpoint" in this procedure. And it fullfills everything I need.
No question, having an own PBS hardware would of course better - but I do not have it. So I can only work with what I have. I have a NAS and a Raspberry PI 4. As PBS is not available for ARM, I cannot use it. So my only idea is what I wrote in my starting post: Install PBS where I can (only a VM in PVE) and use the NFS storage via NFS provided by the Raspberry PI 4.
It's a hobby homelab - and I have just the option between no backup or backup via a virtualized PBS with bare metal NFS storage. So I wanted to reach to have at least a backup. Also I know it's not the "right" way.

Backing up data (12TB):
PBS is not made for backing up 12 TB. Why not? I really thought it's what PBS is also made for. Incremental backups, that's it. My old solution was rsync backups, but I wanted to make a step forward - not backwards Just from what I read: I do not understand, why PBS is not good for this use case. It saves the data in smal chunks - as you worte. That's perfect. So after the first transmission of 12 TB it should only transfer several MB with each incremental backup.

Would you suggestion be to try out (inofficial) PBS on the Raspberry PI?

 

[DerekG]

So what I will do, if my PVE crashes: I will set up a new PVE, install PBS again as a VM, connect it to the Raspberry PI 44 and restore my LXC's and VMs. Is there anything wrong in my understand? I don't see any "breakpoint" in this procedure. And it fullfills everything I need.

Reconnecting a PBS to a backup datastore, is not a simple process, it's not like connecting to an SMB/NFS share, believe me, I've done it, and even 1st time connecting to an external datastore is not a simple process because the PBS needs complete ownership of the store.! If you do go down that route, especially if using encrypted backups, make sure that you document everything you've done.

It's a hobby homelab - and I have just the option between no backup or backup via a virtualized PBS with bare metal NFS storage. So I wanted to reach to have at least a backup. Also I know it's not the "right" way.

I understand your pain, but really as PBS requires so little resources, I would at least investigate the purchase of a used mini / tiny office PC. My PBS runs on a 2nd Gen Core i3 CPU / 4GB RAM, with a 1TB SSD drive.

PBS is not made for backing up 12 TB. Why not? I really thought it's what PBS is also made for.

No, PBS is designed to backup the VM's and CT's on your Proxmox host(s), even backing the Proxmox host requires the use of the cli. I have tried to explain how PBS works, and I've just gone through the process of solving the backup of a CT which contained a 2TB sparse file. The backup of that CT was taking 2 hours even though the final backup size is less than 10GB. PBS does not read the files system, so on backup of your 12TB it will trawl through the whole thing every backup. By my calculations & from what I've seen, your media backup will take at least 12 hours on a 1Gbe network connection. Find another solution to backup your media files (I'm investigating UrBackup), that can be run in a CT on Proxmox.

Would you suggestion be to try out (inofficial) PBS on the Raspberry PI?

Personally I have zero experience with the Pi or the unofficial PBS build, so I can't advise in that case. I gave my suggestion earlier, get onto eBay if you can, and search for the cheapest mini PC, they will all support SATA drives. After all, you understand that the backup is an important part of your Home lab system.

 

[athompson3117]

I started using PVE a little over a month ago to replace my VMware homelab. I wanted to use my QNAP NAS for backups, especially since it has native support to push files to the cloud. I also wanted to use PBS for the deduplication capabilities to minimize the space used in the cloud. Since this is a homelab, I also didn't want to dedicate another machine to the PBS which would have just been one more thing to tie into the UPS, backup separately, etc. I found this video which lays out exactly how to create a PBS server as a VM with a NAS NFS mount as the storage location. The author is very clear this setup is only for testing/homelab. All the caveats mentioned in above posts apply, but since this is a homelab, I am accepting that level of risk in trade for simplicity.

Now, to mitigate those risks I take additional steps.

1) It should go without saying but the PBS VM is not backed up using PBS. It's on a separate schedule which runs nightly and I keep 4 days of backups sent directly to the NAS under a different share than what is used by PBS (though both targets are in the same parent folder on the NAS). The PBS VM itself is so small that I'm unconcerned with space savings.
2) About every 3 months I take direct backups of the VMs outside of PBS. I also take a backup of a VM if there are major changes to the OS or software on it. I only keep 1 each of these.

This gives me several fallbacks depending on what has failed and whether or not restores go according to plan. I've done several restores for testing and it works just fine as far as I can tell.

Lastly, the NFS performance is poor for initial backups, but after that it gets better. That said, we're talking homelab here, so its assumed VMs are relatively small, and any large files are kept on the NAS and made available to VMs via CIFS/NFS.

 

[DerekG]

Hi athompson3117,

I have done a similar setup for my secondary PBS, but I've used ISCSI as the target, which is presented to the PBS as a block device, just like a drive.
That does improve the performance but in regular use I use the sync feature to pull the backups from the primary PBS.

 

[athompson3117]

@DerekG

Right, and that's precisely what I'd expect using a block-based LUN. My big reason for going NFS is because it allows me to take the backup files right to Backblaze B2 using the native QNAP HBS software. Again, simplicity is the guiding star for the setup.

 

[Johannes S]

My big reason for going NFS is because it allows me to take the backup files right to Backblaze B2 using the native QNAP HBS software. Again, simplicity is the guiding star for the setup.

Did you test that restore actually works?

Z

Thread 'Datastore synced with Rclone broken?'

Sep 20, 2024

I'm trying to set up a remote sync to a friend's RPI for my Proxmox Backup Server. I've been using this script that uses Rclone with an encrypted remote (Crypt) on a schedule:

#!/bin/bash

# A simple script to back up PBS to Crypt remote

_lock_file=/root/.tertiary_backup.lock

_schedule_time=1:30
_max_duration="7.5h"
_cutoff_mode="soft"

_datastore_dir_name=Primary_Backup
_datastore_dir_path=/mnt/datastore/$_datastore_dir_name
_config_dir=$_datastore_dir_path/host_backup

_remote_user=user
_remote_ip=tailscale-ip
_remote_port=22
_remote_name=Tertiary_Backup_Crypt...

• Zeash
• pbs rclone remote backup
• Replies: 11
• Forum: Proxmox Backup: Installation and configuration

 

[athompson3117]

@Johannes S

Yes. QNAP HBS pulled the files back from Backblaze B2 to their original location on the NAS, and PBS picked them up from there. No issues during restore. To be more clear, I'm not putting the backups right into B2 with PBS (if there was some confusion about that). PBS stores the backups on the NAS via the mounted NFS, and then HBS moves the files to B2. If I needed to start from scratch, I could restore the PBS VM from its non-PBS backup (also on the NAS, pushed to B2), then perform restores on the other VMs using PBS.

 

[Johannes S]

To be more clear, I'm not putting the backups right into B2 with PBS (if there was some confusion about that

This is the reason I asked, people ( see linked thread ) broke their backups by implementing their own mechanism for offsite backup instead of PBS native sync features

 

[athompson3117]

That's a good call-out and a definite concern. HBS is pretty good with restores. It saves a lot of metadata so when it restores the files they go back just as they were, including a bunch of host-level attributes.



The setup seems to work, but that said, I don't bank on it exclusively. In a pinch I can use the break-glass NFS backups which exist outside of PBS. There'll be data loss, but faster than having to stand everything back up from scratch. I'd encourage anyone taking this route to do the same.

The homelab has layers of redundancy. PVE is installed with a RAID 1 config onto a couple M.2 NVMe SSDs. The NAS shares are on 8 HDDs running in RAID-Z 2, with the NAS OS/apps on another pair of M.2 NVMe SSDs running in RAID 1. Everything is tied into a UPS. Important files go from the NAS to BB B2 each night.