PBS vs Freenas + BackupPC (review)

[garbled]

So I've installed PBS on a new server (physical), because the idea seems sound to me. This is basically a comparison, of my current setup, vs switching to PBS. I haven't decided what I'm going to do yet here...

Current Setup:
FreeNas with a NFS export for proxmox to backup into, and then running BackupPC in a jail.

Thoughts on the new setup:

1. Trying to get freenas to do zfs replication into PBS is a nightmare. I had to spend hours fiddling with different datastore setups, options in freenas, etc etc.
2. The proxmox-backup-client is, super icky. Setting up a backup practically requires a custom script per host. There is no way in the GUI to restore TO a host. There is no way to even configure what is backed up in the GUI, it's all done locally per-box.
3. There are no zabbix templates available for PBS.

Pros/Cons of PBS

• The deduplication is super nice. I can see saving tons of space like that for my VM backups.
• The integration with proxmox is pretty good.
• The physical client is an abomination.
• ZFS handling is all command line. If I have to swap a disk, it's going to be completely manual.
• No zabbix templates exist, I'll have to roll my own monitoring from scratch.
• Backups seem to be quick, no complaints on speed.
• Trying to setup zfs replication from FreeNAS to PBS is... a disaster. If you use a single datastore, once your syncs are done, the GC jobs for PBS all fail, because some directories aren't writable by backup. Additionally, because you have to run the zfs sync as root, it sets the base of the tree to be owned by root, and pbs gets upset because it doesn't own it anymore. Eventually I had to create multiple ds's and put the PBS controlled backups in their own location, so they could run GC without hitting the zfs replicas. I'm still having problems where it refuses to mount the resulting volumes.. argh...
• Importing disks initially into PBS was difficult. I used some old disks, and it refused to reinitialize them. Eventually I had to dd the heads off of them and manually apply gpt labels.
• Doing a restore on a phys is.. pain inducing.
• I have no idea how I'm going to backup my databases. I guess I have to manually export some NFS?
• I tried to setup physical backups with a user other than root@pam, but I absolutely could not get it to work. The docs are not helpful here either (just create a user and assign it permissions, ok, well, WHAT permissions?)
• Because it doesn't grok what is going on with the ZFS replication, it just sees that the space on the datastore is magically getting smaller. So all the graphs are useless.

Pros/Cons of BackupPC + FreeNAS

• The GUI for backuppc is atrocious
• Running things in jails is a total hack, and it's really hard to admin the jail.
• The client setup for backuppc is super easy. Add the ssh key, install rsync, do all the config in the gui, done.
• No dedupe. vm backups take up an enormous amount of space.
• Retention is a pain for vm backups. (but great for backuppc)
• Replication of ZFS just works.
• BackupPC can backup windows. And Raspberry Pi's. (yes, I know there is an unofficial rpi client, haven't tried yet)
• disk/zfs/export management in freenas is a billion times better.
• BackupPC and FreeNAS both have zabbix templates. And they just work.

The dedupe is huge. So is the proxmox integration. But this really feels like a "backup proxmox and nothing else" solution right now. The client is just so bad, I can't even express it in words. I thought for sure it would be a daemon and I would be mildly annoyed at having to run it, but this is just... bad. The complete lack of ZFS handling in the gui is pretty bad too.

I'm really going to have to play with it more. I really want to switch over to this, but if I have to maintain a bucket of hacks to make it work, I'm not going to be happy.

 

[sterzy]

Hi,

The proxmox-backup-client is, super icky. Setting up a backup practically requires a custom script per host. There is no way in the GUI to restore TO a host. There is no way to even configure what is backed up in the GUI, it's all done locally per-box.

pretty sure this should be improved once we implement host backups. It's on our roadmap [1].

Trying to setup zfs replication from FreeNAS to PBS is... a disaster. If you use a single datastore, once your syncs are done, the GC jobs for PBS all fail, because some directories aren't writable by backup. Additionally, because you have to run the zfs sync as root, it sets the base of the tree to be owned by root, and pbs gets upset because it doesn't own it anymore. Eventually I had to create multiple ds's and put the PBS controlled backups in their own location, so they could run GC without hitting the zfs replicas. I'm still having problems where it refuses to mount the resulting volumes.. argh...

Just wondering, what your use-case is here? Like why do you want to do that? Sorry if this sounds judgemental, just genuinely curious. If the point is to maintain multiple copies of backups in several locations you could use PBS' sync feature, once you make the switch to using "just" PBS. That should be a lot more robust/flexible.

I tried to setup physical backups with a user other than root@pam, but I absolutely could not get it to work. The docs are not helpful here either (just create a user and assign it permissions, ok, well, WHAT permissions?)

Admittedly documentation could be improved here. Although if all you want to do is create backups it should be enough to assign the DatastoreBackup role. The manual does have an overview of all the roles and what their capabilities [2].

[1]: https://pbs.proxmox.com/wiki/index.php/Roadmap
[2]: https://pbs.proxmox.com/docs/user-management.html#access-roles

 

[garbled]

Hi,

pretty sure this should be improved once we implement host backups. It's on our roadmap [1].

I would greatly welcome that. I like the pbs-server side of how the backups are stored. I like being able to download the individual files and sort them into namespaces. The server side is just great. The client side makes me weep.

Just wondering, what your use-case is here? Like why do you want to do that? Sorry if this sounds judgemental, just genuinely curious. If the point is to maintain multiple copies of backups in several locations you could use PBS' sync feature, once you make the switch to using "just" PBS. That should be a lot more robust/flexible.

So there are 4 main things I want to backup:

1. I have a massive NAS, of just general "data" running FreeNAS. I need backups of all of that. The NAS also currently stores all my other backups, the idea is I want to split the backups of all my things off the NAS, onto a dedicated backup server, so I have a physically separate backup of the NAS data (15+TB) and all the things listed below.
2. I have a proxmox cluster (5-node) with VM/CT's, pbs is obviously the right choice here. (the cluster runs ceph)
3. I have physicals. pfsense, workstations, raspberry pi's. Currently backed up by BackupPC
4. I have 3 databases, postgres, mysql and influxdb, which I want separate backups of just the data.

I want to replicate the NAS, because realistically, of all the data I have, that's the most important. The VM's just need to be backed up, but the NAS is where basically everything I own lives. Right now I replicate internally on the nas from one ds to another. What I want is to be able to replicate from the NAS to a pbs datastore, just so I have that second copy. Think of the NAS as "all the stuff that isn't actually on the VM's", not just the NAS exists purely for backups of the VM's.

Admittedly documentation could be improved here. Although if all you want to do is create backups it should be enough to assign the DatastoreBackup role. The manual does have an overview of all the roles and what their capabilities [2].

[1]: https://pbs.proxmox.com/wiki/index.php/Roadmap
[2]: https://pbs.proxmox.com/docs/user-management.html#access-roles

I should have saved the error. I added that one role, and even tried DatastoreAdmin, but it kept coming back and saying it couldn't set some flag or property, and I think I hunted down it needed some Sys.role, but I couldn't find that to assign other than as a child of the Admin role.

I *really* want to use pbs. It looks like the right solution, but the #1 thing I have to backup is the NAS itself, and I would much rather ZFS replicate than do a pbs-client style backup of it. Right now, I'm testing TrueNas Scale as the backup server, and then running pbs as a VM inside that, to handle the VM backups. This seems to be working, though I find it to be non-ideal.


Thanks for your time!

 

[Dunuin]

You can also install PVE bare metal next to a PBS and virtualize a TrueNAS. And its also possible to virtualize PBS inside a bhyve VM on TrueNAS (thats what I'm doing right now but will change that to TrueNAS virtualized on PVE later). Do that to two servers and you could use the replication task of TrueNAS to backup your network shares and to use a PBS sync job to pull a datastore from another PBS. That way you get two replicated/synced PBS/TrueNAS that will be able to run independently in case one of the two servers fails.

BTW:
Did you get S.M.A.R.T monitoring for your TrueNAS disks working with zabbix? That would be what I'm most interested in monitoring my TrueNAS beside the pools state, but the zabbix template only gets its data using SNMP and SNMP won't include SMART attributes. Tried to create a jail with access to /dev to install the zabbix agent into but never got that working.

 

[garbled]

To clarify, I don't need 2 copies of my backups. Right now I have one monstrous NAS, that has all my data, AND does backups (of itself, and of my VMs). My goal is a separate backup server (for all the obvious reasons) where I move all the backup functions the NAS is currently doing to the new backup server. One of those, and perhaps the most important, is the backup of all the data on the NAS.

Right now my setup is just internal replication on the nas to a second pool, which does not help me if the entire nas explodes in a fireball or something. I'm not at the point of redundancy yet, where I can afford multiple backup servers, or have backups of backups.