Advice on backup solution for home use, is PBS the right fit for me?

[krby]

Hi all, I'm relatively new to Proxmox, but an experienced Linux admin. I just built a small proxmox server for home. It's a 1TB ZFS mirror for root and most VM/CT images (I let the pve installer partition this) and a 4TB ZFS mirror for bulk data. I'm using that for files I share out using a fileserver container and for the data directory for a Nextcloud VM.

For background: I have used Duplicati for years to do file-level backups of my old NAS and desktops to an S3 clone service (Wasabi). My mental model for backups is shaped by that. Incremental, retention policy that lets me do X days, X weeks, etc, encrypted on the client side so I don't have to tryst the remote. I'm looking for advice on how to do a backup of my PVE install, and I don't think PBS is what I want, but maybe I'm wrong? What I would like:

• Incremental backup "the entire" PVE to a remote location. No local backup needed or wanted. I think this is just backup up both pools? If I had to start over, I'd re-install on a new box and then restore.
• Encrypted on the client side. I don't have encrypted ZFS pools locally, but would like the backups encrypted at rest on the remote destination. Like Duplicati, I want the encryption to be done before sending.
• Flexible retention policy like keep 7 dailys, 4 weeklys, etc.
• Avoid backup agents in each VM and container.
• Ok with various kinds of "remote storage". Been using an S3 service, but rsync.net or similar is fine. Right now, I'm using about 1TB on Wasabi and it's not growing much, so that kind of ballparks my needs. Wasabi has been ridiculously cheap for this ~$7. I'm ok paying more, but ideally less than about $20 for up to 2TB of capacity (this includes full and incremental backups)

Things I have looked at:

• PBS: From what I can tell with the PBS route, I'd need a separate physical host, which I'd like to avoid. It looks like I might be able to have a remote only repository on something like rsync.net, but not sure
• Sanoid or some zfs send/receive solution to something like rsync.net. This seems about perfect and not too expensive, except I don't think it can do client-side only encryption. I either have encrypted pools locally and use raw sending or I have an encrypted pool mounted on the remote (which means the key is on the remote side)
• Borg backup? I've heard a lot about this, but never used it, unclear to me if it backs up files or how I backup and restore would work for PVE, including VMs and CT images.

Thanks for any advice. Also, I'll say that I ok running the backup software directly on the PVE host if it isn't big or complex.

 

[leesteken]

• PBS: From what I can tell with the PBS route, I'd need a separate physical host, which I'd like to avoid. It looks like I might be able to have a remote only repository on something like rsync.net, but not sure

You can run PBS as a container on PVE (and if you use snapshot mode, then it can even backup itself). But general good practice is to (also) keep (copies of) backups on a different system in a different location.
PS: There is also a whole sub-forum about PBS: https://forum.proxmox.com/forums/proxmox-backup-installation-and-configuration.24/

 

[krby]

You can run PBS as a container on PVE (and if you use snapshot mode, then it can even backup itself). But general good practice is to (also) keep (copies of) backups on a different system in a different location.
PS: There is also a whole sub-forum about PBS: https://forum.proxmox.com/forums/proxmox-backup-installation-and-configuration.24/

Thanks for the quick reply! I didn't post to the PBS subforum because I thought my question was more about "I think I need to use something besides PBS". But, if I'm wrong, that would be great! The PBS integration with PVE seems really good.

I'll post a PBS-focused subset of my question in that subforum.

 

[Johannes S]

Hi all, I'm relatively new to Proxmox, but an experienced Linux admin. I just built a small proxmox server for home. It's a 1TB ZFS mirror for root and most VM/CT images (I let the pve installer partition this) and a 4TB ZFS mirror for bulk data. I'm using that for files I share out using a fileserver container and for the data directory for a Nextcloud VM.

For background: I have used Duplicati for years to do file-level backups of my old NAS and desktops to an S3 clone service (Wasabi). My mental model for backups is shaped by that. Incremental, retention policy that lets me do X days, X weeks, etc, encrypted on the client side so I don't have to tryst the remote. I'm looking for advice on how to do a backup of my PVE install, and I don't think PBS is what I want, but maybe I'm wrong? What I would like:

• Incremental backup "the entire" PVE to a remote location. No local backup needed or wanted. I think this is just backup up both pools? If I had to start over, I'd re-install on a new box and then restore.
• Encrypted on the client side. I don't have encrypted ZFS pools locally, but would like the backups encrypted at rest on the remote destination. Like Duplicati, I want the encryption to be done before sending.
• Flexible retention policy like keep 7 dailys, 4 weeklys, etc.
• Avoid backup agents in each VM and container.

• PBS backups are always full backups AND incremental: Every snapshot allows to restore the complete vm/container but since all data is split in so called chunks (small files) only new data needs to be added. This results in relative small amount of needed disk space and transferred data.
  For example my main datastore uses around 170 GB disk space but all restored snapshots would have 42,732 TB together. Another example: One of my VMs has a 32GB disk. However at the last backup run the system noticed that only around 2 GB had changed so only 2 GB were actually uploaded to the PBS. Concerning the "entire" PVE: The GUI only includes the possibility to backup lxcs and vms, to also backup the PVE host you would use the proxmox-backup-client on the CLI. Of course you could also use another backup tool for this (so you don't need PBS to restore the host OS).
• Client-side encryption is possible, of course you would need to save the key in some secure place so you can still restore after a disc failure.
• Play around with this simulator and you should be able to notice whether it fit's your needs: https://pbs.proxmox.com/docs/prune-simulator/ I think you should be fine.
• Concerning agents: For nextcloud and a fileserver you should be fine with PBS. One caveat though: Some applications (e.G. databases) data should be backuped with dedicated tools for this purpose to ensure their consistency ( (eg mysqldump, veeam agent or another backup tool with application aware backups). Keep this in mind for the future.
• I personally would seperate between VM/CT operating system/application and actual data though.

• Ok with various kinds of "remote storage". Been using an S3 service, but rsync.net or similar is fine. Right now, I'm using about 1TB on Wasabi and it's not growing much, so that kind of ballparks my needs. Wasabi has been ridiculously cheap for this ~$7. I'm ok paying more, but ideally less than about $20 for up to 2TB of capacity (this includes full and incremental backups)

• I pay around 10-15 Euro per month for my vserver which has around 512 GB of storage space (which I use just for the lxcs and vms os/application install data) and around 13 Euro für my 5TB Hetzner storagebox (which holds the backups for my NAS disks and of my notebook).

Things I have looked at:

• PBS: From what I can tell with the PBS route, I'd need a separate physical host, which I'd like to avoid. It looks like I might be able to have a remote only repository on something like rsync.net, but not sure

Sadly PBS doesn't support backing up to S3 yet (it's on the roadmap for several years). It supports external storage (eg. USB disc) as "removable datastore" though. For having an offsite backup you could do this: You could also setup a small vserver on a cheap cloud provider (like hetzner, netcup, ovh etc) and install PBS on it. You would have a PBS VM on your PVE and run a sync job on the vserver PBS to pull the backups on your PBS VM to the vserver PBS. There are some providers of "cloud PBS storage" like https://tuxis.nl or https://cloud-pbs.com/ so you don't need to run your own PBS.
But running both has quite a benefit: You could setup both PBS that they are "append-only" meaning that even if ransomware or an offender takes over your local infrastructure he couldn't delete your remote backups (because you could setup the remote PBS with append-only permissions and setup the firewall that the remote PBS can pull backups from the local PBS but not vice versa). The PBS manual is a good source how to approach such a thing:

• https://pbs.proxmox.com/docs/storage.html#ransomware-protection-recovery
• https://pbs.proxmox.com/docs/managing-remotes.html

• Sanoid or some zfs send/receive solution to something like rsync.net. This seems about perfect and not too expensive, except I don't think it can do client-side only encryption. I either have encrypted pools locally and use raw sending or I have an encrypted pool mounted on the remote (which means the key is on the remote side)
• Borg backup? I've heard a lot about this, but never used it, unclear to me if it backs up files or how I backup and restore would work for PVE, including VMs and CT images.

I personally prefer restic (don't need a server on the remote, s3 or sftp is enough), to use it with PVE you would use PVEs native vzdump backups. They don't need a PBS to be restored but needs more space (since each snapshot is a full although compressed backup archive file). restics or borgs deduplication should still help to save some space though. For VMs and LXCs I prefer PBS though (way more comfortable) but using restic for raw data.

Just to give you some ideas and to quote myself from an earlier thread (this subject is discussed regulary you really should use the search function ):

I myself doing something of a mixed approach: My notebook, my NAS (which is a VM on Proxmox VE) data and the Proxmox hosts are backuped with restic to a hetzner storagebox and a external disk drive. My VMs and containers are backuped to a local PBS which is synced to the remote PBS on Netcup. At the moment I don't split between the OS and data of my VMs (except the NAS) and containers so it's less complicated to oversee everything. But I might if I run out of space at the vserver (storagebox is cheaper). The benefit is, that I can grow the storagespace on both places in small steps (grow as you go) if I need more space without breaking the bank. If at some point the storagebox and netcup together (even with a split between VMs OS and data) cost more than a dedicated server I will ditch them and replace with a dedicated server. If my future budget woudn't allow this I would propably ask some friends whether I might put a low-power-Server at their place and use that for my backups.

T

Post in thread 'Backup PBS to cloud provider with incremental chunks'

Nov 26, 2024

You could and should still have a more frequent backup to PBS due to deduplication you can have a lot of snapshots without needing much space.

Trying to save some power with not having the backup server running 24/7

My VMs and containers are backuped to a local PBS which is synced to the remote PBS on Netcup.

Can you share how this setup works exactly? Do you run that PBS yourself on a VPS?

• thimplicity

So my approach is to use restic (or another backup program, if duplicati is working for you you don't need to switch it!) for the bulk data and the Proxmox VE host operating system and PBS for VMs and lxcs so I can easily restore them in case of an error. I'm also doing a vzdump of my PBS VM and backup it with restic on my disk and storagebox so I can restore it even If I would have to reinstall my ProxmoxVE system from scratch. The procedure would be like this:

• Reinstall ProxmoxVE OS
• Get PBS VM vzdump from restic snapshot, restore to the ProxmoxVE. Add the remote PBS and sync my offsite backups to the local PBS VM
• Restore all VMs and LXCs from the local PBS VM (of course I could also restore from my remote PBS)
• Restore any bulk data with restic from the storagebox

HTH

 

[krby]

Thanks for the detailed info. Before I saw this, I posted in the PBS subforum as leesteken suggested. Repeating a bit to make sure I understand.

* I could run PBS in a container on the PVE host. leesteken suggested with snapshot mode I could even backup this container. Or, I could use another tool to snapshot and back it up to a remote
* Does PBS have to talk to another PBS instance for remote destinations? Or can it be anything I can ssh to or do `zfs send` to? rsync.net would cost 24USD/mo for 2TB.
* I'm moving to all my bulk data living in VMs and containers, so I think just backing those up with PBS would cover the things I care about.
* proxmox-backup-client would back up the PVE config. Which I would then have to scp or something to a remote location


Do I have this right?

 

[leesteken]

* I could run PBS in a container on the PVE host. leesteken suggested with snapshot mode I could even backup this container. Or, I could use another tool to snapshot and back it up to a remote

Yes you could use another tool, if you want to.

* Does PBS have to talk to another PBS instance for remote destinations? Or can it be anything I can ssh to or do `zfs send` to? rsync.net would cost 24USD/mo for 2TB.

It does not "have to" but you can easily pull from one PBS to another regularly (and I think the latest version can push as well). Maybe you could use something else, if you really wanted to.

* I'm moving to all my bulk data living in VMs and containers, so I think just backing those up with PBS would cover the things I care about.

I'm running all my systems (with all of my data) as VMs (with GPU and USB passthrough to "desktop VMs") and instead of using ssh/scp/rsync/rsnapshot (which I used hourly in the past), I now use PBS to backup everything automatically to multiple systems in multiple locations (using the sync/pull functionality).

* proxmox-backup-client would back up the PVE config. Which I would then have to scp or something to a remote location

If you use the backup-client to backup some folders on PVE (like /etc and /etc/pve), they will also easily sync with other PBS,

Do I have this right?

Maybe lookup the 3-2-1 backup principle and you'll see that a few PBS can fullfill those requirements easily (while deduplicating and reducing bandwidth at the same time).

 

[krby]

It does not "have to" but you can easily pull from one PBS to another regularly (and I think the latest version can push as well). Maybe you could use something else, if you really wanted to.

Thanks for the followup!

When you say: "[PBS] does not have to [push to another PBS]" This is what I'm trying to understand. What are the requirements at the remote end of a PBS backup? Assume I'm running a local PBS in a container on PVE. What do I need on the remote end? ssh? rsync? LVM, ZFS, or a filesystem that supports snapshots?

Where can I find info about this? That will let me figure out my options and costs for the remote storage.

 

[leesteken]

When you say: "[PBS] does not have to [push to another PBS]" This is what I'm trying to understand.

That's not what I'm saying...

What are the requirements at the remote end of a PBS backup? Assume I'm running a local PBS in a container on PVE. What do I need on the remote end? ssh? rsync? LVM, ZFS, or a filesystem that supports snapshots?

One PBS can pull from another PBS to keep you backups safe from a power surge or fire (if the remote PBS is in another location).

Where can I find info about this? That will let me figure out my options and costs for the remote storage.

Maybe read the manual: https://pbs.proxmox.com/docs/ . Or read the forum: https://forum.proxmox.com/forums/proxmox-backup-installation-and-configuration.24/ .

 

[krby]

Ok, took a quick look at the manual. It looks to me like the PBS option for remote-only backups are:

• Anything I can make appear as a local posix-y directory. sshfs, s3fs, etc might work. But I don't know how well they will in practice.
• A remote PBS instance. I think this implies a VPS-type service where I can run a PBS instance, so not just a remote storage service.

Neither of these are what I was imagining, I was imagining more like a rsync, borg, etc type model.

Or, I guess I could suck it up and use a local PBS datasotre, encrypt that on backup and then rsync or borg it somewhere else .

 

[leesteken]

Neither of these are what I was imagining, I was imagining more like a rsync, borg, etc type model.

Use whatever you are most comfortable with. I'm sure people can answer concrete questions in your other thread.