Backup tool for standalone ZFS datasets to Proxmox Backup Server

[NickyDoes]

PBS handles VM and LXC backups well, but it doesn't have a native answer for ZFS datasets that live outside VMs and containers.

If you run a NAS with ZFS datasets—photos, videos, media libraries, ISOs—exported over NFS or SMB and mounted into your VMs, that data lives on the hypervisor or a storage server, not inside the guest. PBS doesn't touch it.

I wrote zpbs-backup to fill this gap. Configuration lives entirely in ZFS properties. `zpbs-backup` manages those properties.

zpbs-backup set zpbs:backup=true tank/media

zpbs-backup set zpbs:schedule=weekly tank/media/movies

zpbs-backup set zpbs:retention=30d,12w,12m,3y tank/photos

zpbs-backup set zpbs:priority=10 tank/documents

zpbs-backup set zpbs:backup=false tank/media/downloads

The tool discovers marked datasets and backs them up to PBS.

zpbs-backup

Features:

• Works with any ZFS dataset (VM NAS, Samba, External NAS)
• Auto-discovery via ZFS properties—no config files
• Inheritance through the dataset hierarchy (enable on parent, children follow)
• Per-dataset schedules (daily/weekly/monthly)
• Per-dataset retention policies Priority ordering for critical data
• Dry-run and audit modes Systemd timer for unattended operation
• email and syslog notification options

 

[Johannes S]

What does this do, what you can't achieve with the native ptoxmox-backup-Client?

 

[NickyDoes]

I've already written what it does, and am not about to go on the full defensive. Reiterating succinctly: This is a wrapper for PBC. It adds inheritable backup config from within the dataset properties (vs. remembering to modify your PBC scripts). Other features as previously described.

I'm happy to defend or defer if I've missed something, but you've got to ask a more cogent question first.

 

[victorclaessen]

Dear NickyDoes, thank you for this. This was exactly what I was looking for .

I had some trouble getting it to run (partly because I did not read the documentation well enough).
Some observations:
* `pip install .` didn't work on proxmox ve. I had to make a virtualenv and point the systemd service to the executable in the virtualenv directory;
* even though I gave the token full permissions, it still did not manage to create its automatically derived namespace. It worked when I suggested a namespace myself, though;
* I'm still not having any luck pruning (Error: permission check failed - missing Datastore.Modify|Datastore.Prune -> even though the token has these permissions). If I find the time I'll probably post an issue in your repo if that's allright with you.

But other than that, this is really useful for us. Thanks again!

Best regards,

Victor

 

[NickyDoes]

I have some simplifications to features and docs queued up for a push - nothing overly breaking, but it's worth reading/checking. I'm expecting to get time later today (US East Coast / GMT -5) to make this a properly installable package. Go ahead and either post or put up a PR in the meantime. I'm happy to have others go through my code, docs, etc.

Thanks for the help on this.

 

[NickyDoes]

I've made many modifications and improvements to zpbs-backup. If you tried it already, and were disappointed or confused, I apologize. Please give v0.5.0 or later a try. Expect most changes to be breaking, aside from the dataset properties you already might have set.

Here's what's been improved:

• the tool is fully installable. Download the appropriate package from github releases and install with something like `sudo apt install ./zpbs-backup_0.5.0_amd64.deb`.
• proxmox-backup-server config can be via environment variables or at `/etc/zpbs-backup/pbs.conf`.
• added a pbs connectivity check. This also announces which config source zpbs-backup is using to connect to pbs.
• installs as a service. Runs at ~2 am. Run now, or run now with -bg to run now in the background

This tool solves a long overdue problem for me. I hope you find it useful as well.