Questions of a Newbie about virtualization of PBS

[exaveal]

HI all,

thanks so much for this community here. I started to change my homelab-server-landscape to virtualize it with Proxmox. I love it.
Currently I am still figuring out how to finalize my setup and backup strategy.

On my PVE I have a virtualized NAS. The NAS is just a Ubuntu-Server LXC with 12 TB ZFS data storage.
Next to automatic snapshots of all LXC's and VM's, I also want to have a weekly backup to another location.

I have a Raspberry PI 4 connected to a 4 drive RAID. It's running Ubuntu Server and the ZFS RAID offers NFS storage.

My first idea was to create backups from PVE to the Rapsberry PI NFS storage. But only full backups are possible - That means I would have to transfer 12 TB each week.

To massivle reduce the amount of data to transfer each time I need incremental backups. This seems only possible with Proxmox Backup Server.
Would it be a solution to virtualize PBS in my PVE and let PBS save the backups on the NFS storage of the Raspberry PI? Does it work? Are incremental backups working this way? Does the NFS storage from the Raspberry PI must use ZFS filesystem to support incremental backups?

I know there is an inofficial version of PBS for Raspberry PI 4 but I am not so familiar with Linux allowing me to "play" with my backups. So I would really prefer to not use an inofficial PBS version.

Thank you so much!

 

[DerekG]

To massivle reduce the amount of data to transfer each time I need incremental backups. This seems only possible with Proxmox Backup Server.
Would it be a solution to virtualize PBS in my PVE and let PBS save the backups on the NFS storage of the Raspberry PI? Does it work? Are incremental backups working this way? Does the NFS storage from the Raspberry PI must use ZFS filesystem to support incremental backups?

Hi exaveal,

The question should not be, is it possible, but how would you recover should the host die when it is hosting both the PVS & PBS.???
I'm assuming that you are not backing up the full 12TB of Proxmox VM's/LXC's along with the media data? The backup of the media data should be a separate backup routine from the Proxmox infrastructure.

PBS uses it's own storage system, you cannot access the data which has been backed-up by any other method other than a functional PBS instance.
In addition to those issues, PBS is extremely IO intensive, it performs incremental backups not on a file basis but by dividing the source disk into small 'chunks' and comparing those chunks to what it already has stored in the backup target, and then only uploading the chunks which have changed. So the optimal method of storage is to a local SSD store on the local device.

For the reasons stated, I would suggest a dedicated device for the PBS itself. I do have a secondary PBS as a VM which uses zfs ISCSI as a backup store, but that unit does not perform the PVE backups, it instead pulls replicas from the primary PBS and is operational over a 10Gbe link.

I hope that provides some basic understanding of how the system functions, and that will at least point you in the right direction.

 

[exaveal]

Hi & much thanks for your feedback. I really appreciate it.

Running PBS as a VM:
I am aware of the point to use PBS virtualized in the PVE. What I think about regarding MY use case: If my PVE crashes, than I have no problem, because everything is saved on the external Server (in my case the Raspberry Pi 4 including the NAS).
So what I will do, if my PVE crashes: I will set up a new PVE, install PBS again as a VM, connect it to the Raspberry PI 44 and restore my LXC's and VMs. Is there anything wrong in my understand? I don't see any "breakpoint" in this procedure. And it fullfills everything I need.
No question, having an own PBS hardware would of course better - but I do not have it. So I can only work with what I have. I have a NAS and a Raspberry PI 4. As PBS is not available for ARM, I cannot use it. So my only idea is what I wrote in my starting post: Install PBS where I can (only a VM in PVE) and use the NFS storage via NFS provided by the Raspberry PI 4.
It's a hobby homelab - and I have just the option between no backup or backup via a virtualized PBS with bare metal NFS storage. So I wanted to reach to have at least a backup. Also I know it's not the "right" way.

Backing up data (12TB):
PBS is not made for backing up 12 TB. Why not? I really thought it's what PBS is also made for. Incremental backups, that's it. My old solution was rsync backups, but I wanted to make a step forward - not backwards Just from what I read: I do not understand, why PBS is not good for this use case. It saves the data in smal chunks - as you worte. That's perfect. So after the first transmission of 12 TB it should only transfer several MB with each incremental backup.

Would you suggestion be to try out (inofficial) PBS on the Raspberry PI?

 

[DerekG]

So what I will do, if my PVE crashes: I will set up a new PVE, install PBS again as a VM, connect it to the Raspberry PI 44 and restore my LXC's and VMs. Is there anything wrong in my understand? I don't see any "breakpoint" in this procedure. And it fullfills everything I need.

Reconnecting a PBS to a backup datastore, is not a simple process, it's not like connecting to an SMB/NFS share, believe me, I've done it, and even 1st time connecting to an external datastore is not a simple process because the PBS needs complete ownership of the store.! If you do go down that route, especially if using encrypted backups, make sure that you document everything you've done.

It's a hobby homelab - and I have just the option between no backup or backup via a virtualized PBS with bare metal NFS storage. So I wanted to reach to have at least a backup. Also I know it's not the "right" way.

I understand your pain, but really as PBS requires so little resources, I would at least investigate the purchase of a used mini / tiny office PC. My PBS runs on a 2nd Gen Core i3 CPU / 4GB RAM, with a 1TB SSD drive.

PBS is not made for backing up 12 TB. Why not? I really thought it's what PBS is also made for.

No, PBS is designed to backup the VM's and CT's on your Proxmox host(s), even backing the Proxmox host requires the use of the cli. I have tried to explain how PBS works, and I've just gone through the process of solving the backup of a CT which contained a 2TB sparse file. The backup of that CT was taking 2 hours even though the final backup size is less than 10GB. PBS does not read the files system, so on backup of your 12TB it will trawl through the whole thing every backup. By my calculations & from what I've seen, your media backup will take at least 12 hours on a 1Gbe network connection. Find another solution to backup your media files (I'm investigating UrBackup), that can be run in a CT on Proxmox.

Would you suggestion be to try out (inofficial) PBS on the Raspberry PI?

Personally I have zero experience with the Pi or the unofficial PBS build, so I can't advise in that case. I gave my suggestion earlier, get onto eBay if you can, and search for the cheapest mini PC, they will all support SATA drives. After all, you understand that the backup is an important part of your Home lab system.