Best way to do efficient local & offsite backup (w/ ZFS)?

[lpulley]

I manage a PVE node from my home running stuff like a personal cloud, streaming, etc. A friend of mine also manages a home PVE node for similar purposes. We both had been lacking off-site backup, so we each installed PBS on our PVE hosts. We each run a scheduled PVE backup to our own PBS instances, and each PBS instance has a sync job to pull the other site's backups to itself. (The sites are linked via Wireguard configured on each host.)

This setup satisfies us to some extent. We both finally have incremental and browsable backups both on-machine (for "whoops, I messed this up") and offsite (for "whoops, the server caught fire"). However, there are some serious drawbacks to this strategy that we've since realized, first and foremost of which is the fact that our ZFS datasets are being stored on-machine twice: in the live dataset itself and as incremental backups in PBS. This is a frustrating use of space.

It seems to me that it would make more sense to use ZFS snapshots as on-machine backup for ZFS datasets and keep an offsite copy of each ZFS dataset up to date, maybe using zfs send/zfs recv. However, this seems like the kind of thing I'd be setting up custom cron jobs for and mucking around with manually every time I add a new mount point. I think I'd miss the elegance of PVE/PBS backup and restore. I'm loving the easy backup browsing and easy restoring with this system, but using 2x the storage space for all my mounts is not sustainable.

I'm looking into PVE storage replication, but I'm not sure that we really want a cluster. Is there an elegant way to do both on-machine and offsite backups including ZFS mount points that I should consider in this scenario?

 

[fiona]

Hi,
you might be interested in pve-zsync which works without clustering.

 

[guletz]

Hi,

And pve-zfs it is relly fast!

Good luck /Bafta !

 

[lpulley]

you might be interested in pve-zsync which works without clustering.

Ah yes, I did see this! This would work well for synchronization without clustering.

Is there a similarly nice way to have PVE use ZFS snapshots during scheduled backup of ZFS datasets, rather than temporarily snapshotting it and copying everything into an archive?

 

[fiona]

Ah yes, I did see this! This would work well for synchronization without clustering.

Is there a similarly nice way to have PVE use ZFS snapshots during scheduled backup of ZFS datasets, rather than temporarily snapshotting it and copying everything into an archive?

Because vzdump is designed to be agnostic towards the underlying storage, there is no such feature. It creates an archive that can be restored on every other type of storage as well.

 

[lpulley]

Because vzdump is designed to be agnostic towards the underlying storage,

Alright, this actually makes a lot of sense.

I'll look into scheduling ZFS backups, then. Maybe I'll have PVE make regular rootfs dumps to a ZFS dataset storage alongside my mount point datasets, and then sync all of them offsite with pve-zsync.