pve-zsync snapshots with multiple disks problem

[dendi]

Hello,

I'm trying pve-zsync and noticed a strange behavior.


zfs list -t snapshot -o name,creation
NAME CREATION
rpool/data/vm-100-disk-1@rep_test1-100_2017-08-27_19:46:36 Sun Aug 27 19:46 2017
rpool/data/vm-100-disk-2@rep_test1-100_2017-08-27_19:46:36 Sun Aug 27 19:49 2017

snapshots are created in different times and I think it should create problems when, for example, I have a VM with more disks on the same volume with lvm.

Do you think this could be a bug?

I will test this with replication inside a cluster in a few days.

 

[wolfgang]

Hi,

pve-zsync is a standalone tool and work on the storage level only.
You are correct it makes the replication disk by disk.

Storage replication works different
and make the shapshot of multiple disks at once.
Also it freeze the fs of the VM before the snapshot is made if guest-agent is installed.
FS freeze will flush the VM cache and write all dirty pages to the disk.

 

[dendi]

Thank you wolfgang.

Do you think is it possible to change pve-zsync script and make all snapshot before any send, and then begin sending data to remote node?

Anyway, I'm planning to use replication inside a cluster, with 4 nodes replicating to node 5.
From node 5 I'll use pve-zsync to remote node.
Then I should have always consistency, because the VM snapshot is made by replication.
What do you think?

 

[dendi]

Hello again,
I can't replicate with pve-zsync the VM 100 from node 5 to a remote zfs.
VM 100 is active on node 6 and replicated by cluster to node 5.
pve-zsync doesn't find it in the local node and says: VM 100 doesn't exist.
Is there a common way to make it work?
Thank you

 

[dendi]

Ok, I just learned that I can't replicate from a replicated resource, it's a zfs behavior

So now the problem is that pve-zsync doesn't create consistent VM snapshot with 2 or more disks.

Will you implement this feature in next releases?

Thank you again

 

[guletz]

Ok, I just learned that I can't replicate from a replicated resource, it's a zfs behavior

Not true. You can do it from zfs command line, or using various zfs replication tools.

So now the problem is that pve-zsync doesn't create consistent VM snapshot with 2 or more disks.

I guess you can, if you will use a dedicated dataset for this Vm(rpool/data/vm-with-2-vdisk). Add this dataset in your storage pool, and use this for only this Vm. Then do the pve-zsync for this dataset. Any snapshot of this dataset Wil be done for this 2 vdisk at the same time.

Good luck

 

[wolfgang]

Ok, I just learned that I can't replicate from a replicated resource, it's a zfs behavior

The Problem is you have no config on this node and so it will not find the VM.
But you can use the VM images as source.

Will you implement this feature in next releases?

We will discuss if we will change.

 

[dendi]

Thank you gulets for the dataset tip! I's a good workaround for now!

About the zfs send from replicated dataset, I tried with pve-zsync --source vm-image and with zfs send without success because, as I understand, zfs needs that the receiving last snapshot is the same of the source.
From Oracle documentation https://docs.oracle.com/cd/E18752_01/html/819-5461/gbchx.html

If you make changes to the file system and you do not roll back the receiving file system to receive the new incremental snapshot or you do not use the -F option, you see a message similar to the following:

host1# zfs send -i tank/dana@snap4 tank/dana@snap5 | ssh host2 zfs recv newtank/dana
cannot receive: destination has been modified since most recent snapshot

I think that pve cluster replication uses -F option, as it deletes snaphots after its last snaphot

You are welcome with other solutions, thank you!

 

[guletz]

Try to use zrep. Many guys that use zfs are use this tool for asynchronous replication. Others are using syncoid/sanoid.

Another option, very interesting:

https://serverfault.com/questions/8...-continuous-backups-of-zfs-pool/842740#842740