Sync ZFS to remote ZFS datastore

H

harmonyp

Member
Nov 26, 2020
195
4
23
46
I would like to sync my ZFS datastore (root@prox1) to my remote storage server (root@storage) preferably every 15 minutes. It must only sync new data just like zfs replication (which is local only???)

Code: 
root@prox1:~# zfs list
NAME                    USED  AVAIL     REFER  MOUNTPOINT
zfs                    3.09T  1.98T       96K  /zfs
zfs/base-1000-disk-0   12.3G  1.98T     12.3G  -
-
-
-
-

My stroage server also runs PBS that all virtual machines will backup to

Code: 
root@storage:~# zfs list
NAME                   USED  AVAIL     REFER  MOUNTPOINT
zfs                    917G  49.9T       96K  /zfs
zfs/PBS                870G  49.9T      870G  /zfs/PBS
zfs/base-100-disk-0   24.1G  49.9T     1.19G  -
-
-
-
-

I have come across some scripts like https://blog.daveeddy.com/2015/12/05/automatic-zfs-snapshots-and-backups/ that appear to do this but the issue is every time it sends a full snapshot as I want to keep only 1 backup to save space it's not an option to send all virtual machines data every time it runs.

I've come across https://pve.proxmox.com/wiki/PVE-zsync but I don't really understand how to set it up to do this for every virtual machine without manually entering the VMID

This looks like what I want done but it's for PBS to PBS only https://pbs.proxmox.com/docs/managing-remotes.html#sync-jobs
 
What exactlty do you want to replicate? The whole pool, just the PBS dataset or the zvols?
harmonyp said:
I have come across some scripts like https://blog.daveeddy.com/2015/12/05/automatic-zfs-snapshots-and-backups/ that appear to do this but the issue is every time it sends a full snapshot as I want to keep only 1 backup to save space it's not an option to send all virtual machines data every time it runs.
Click to expand...
That's how ZFS replication works. For a incremental replication it is important that both store the same snapshots, as it will only send the difference between the old and the new snapshot and if one of them is missing, it can't know what to refer to and has to send everything again. So you need to keep at least 2 snapshots.

harmonyp said:
I've come across https://pve.proxmox.com/wiki/PVE-zsync but I don't really understand how to set it up to do this for every virtual machine without manually entering the VMID
Click to expand...
Didn't used that yet. But as far as I understand you would need to create a pve-zsync job for each VM/LXC.
 
Dunuin said:
What exactlty do you want to replicate? The whole pool, just the PBS dataset or the zvols?

That's how ZFS replication works. For a incremental replication it is important that both store the same snapshots, as it will only send the difference between the old and the new snapshot and if one of them is missing, it can't know what to refer to and has to send everything again. So you need to keep at least 2 snapshots.


Didn't used that yet. But as far as I understand you would need to create a pve-zsync job for each VM/LXC.
Click to expand...
I want to replicate the whole zfs pool. ZFS replication would be perfect but the issue is it does not support external servers unless they are in the cluster? I have two different clusters I want to do this for so that wont really work.

Ideally my nodes ZFS would have an additional copy for example on the storage server

/zfs/node1
/zfs/node2
/zfs/node3

From looking at https://pve.proxmox.com/wiki/PVE-zsync again it seems it requires them to be part of a cluster too?
 
Last edited:
Are there any further updates on replicating the VMs between clusters using ZFS? I am looking into replicating the VMs from the cluster at the data center via WAN over to corporate office. I know this can be done using ZFS send commands but hoping in more scripted or WebGUI way of doing this as VMs could move between hosts which may break it.

It's cool to see the replication between hosts in the same cluster works very well and now wondering between clusters.
 
Dunuin said:
There was cross-cluster migration using ZFS replication as a feature preview added with PVE7.4. See the "qm remote-migrate" command.
Click to expand...
I saw that command which is cool but wanted to simply replicate the live VMs over to the remote cluster and then leave the VMs in it's current state until I manually switch them over.

I suggested the command qm remote-replicate for that. And then qm remote-migrate to actually migrate them over.

Or qm remote-migrate replicate-only

Something along those lines.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back