VM Replication, Cluster Quorum and Node via unstable Network Link

[FelixJ]

Good Morning everyone!
I would like to use the VM replication feature.
The replication target node is within a LAN-Segment, that is connected via a Outdoor-WIFI P2P Link only.
Although the average throughput of that link is about 30 MB/s, I consider it as "unstable" as the link is, for one used for other stuff as well, and then, as it is Outdoor, it's quality depends on the weather as well.
So, as the replication feature requires the target-node to be part of the PVE Cluster, I was wondering, what happens to the cluster, if that WIFI Link starts bouncing on/off or breaks down at all.
I don't want to jeopardize the sanity of the cluster due to that.

I was thinking, why is a cluster-membership necessarily, to replicate a vm? Couldn't that be done similar to a PBS replication job? There, the participating replicating PBS Servers are not clustered at all.

regards,
Felix

 

[fiona]

Hi,
replication is only possible between cluster nodes, because it ensures guest IDs and storage configurations match. If you want to replicate to external nodes, there also is pve-zsync available.

 

[FelixJ]

Hi,
thank you for the hint with pve-zsync:
I tried that today, non successful:
pve-zsync sync --source 120 --dest 192.168.10.71:120 --Verbose --maxsnap 2 --limit 512
ERROR Message:
ERROR: in path

The source VM lies on a ceph storage, the destination storage is a "normal" path.
What I was not able to figure out is, if ZFS is mandatory for this to work.
Thanks for your help!
regards,
Felix

 

[fiona]

Yes, it only works for ZFS, but the replication feature in PVE currently has the same limitation, so adding the external node to the cluster wouldn't help either.

Maybe your best bet is to keep backups for the VMs on the external node (or somewhere it can reach them) and restore when necessary?

 

[FelixJ]

Hi,thanks for clearing that up. May I ask, why is ZFS the limitation? Isn't there an other way to sync online? Like creating a vm snapshot as one does it during backup? One could even dirty bitmaps-feature as used with PBS but instead of adding chunks, updating a vm disk...
Maybe PBS could integrate a "re-assembly" - feature, which updates vm-files from the chunks-store...
However, for the time being, I will continue to restore as required the vms from the chunks-store that replicates to my co-location - how ever, the downside is, it takes a while, nothing for the impatient...

 

[fiona]

Hi,thanks for clearing that up. May I ask, why is ZFS the limitation? Isn't there an other way to sync online? Like creating a vm snapshot as one does it during backup? One could even dirty bitmaps-feature as used with PBS but instead of adding chunks, updating a vm disk...
Maybe PBS could integrate a "re-assembly" - feature, which updates vm-files from the chunks-store...
However, for the time being, I will continue to restore as required the vms from the chunks-store that replicates to my co-location - how ever, the downside is, it takes a while, nothing for the impatient...

Mostly because it hasn't been implemented yet. Also, with ZFS you have storage-level support for incremental snapshots. What you are suggesting would be at one level above the storage layer (at qemu's abstracted storage layer), and one would need to work around certain things (like making dirty bitmaps persistent when the VM is shut down, drive is moved etc.).