PVE 5.1 does not not finish ZFS Replication

[cpzengel]

hi,

fresh installation does not complete any ssh replication.
first tested on pve 3.4 as a targen, then I updated and used a fresh created zpool as a target
after first replication the system does not finish the job
canceling the replication is working and freezes the target to busy the dataset and keeps the zfs recv running.
only rebooting target releases the dataset

formerly we had this from freenas 9.10 > pve4/5

a local replication from one tank to another is working!

this is extremely necessary to us, so please help!

action i took with an almost empty snapshot was

zfs send -pv rpool2/data/HIS-KASSEN@zfs-auto-snap_hourly-2017-10-31-1617 | ssh root@10.0.0.9 zfs recv -dvF Raid10




proxmox-ve: 5.1-25 (running kernel: 4.13.4-1-pve)

pve-manager: 5.1-35 (running version: 5.1-35/722cc488)

pve-kernel-4.13.4-1-pve: 4.13.4-25

libpve-http-server-perl: 2.0-6

lvm2: 2.02.168-pve6

corosync: 2.4.2-pve3

libqb0: 1.0.1-1

pve-cluster: 5.0-15

qemu-server: 5.0-17

pve-firmware: 2.0-3

libpve-common-perl: 5.0-20

libpve-guest-common-perl: 2.0-13

libpve-access-control: 5.0-7

libpve-storage-perl: 5.0-16

pve-libspice-server1: 0.12.8-3

vncterm: 1.5-2

pve-docs: 5.1-12

pve-qemu-kvm: 2.9.1-2

pve-container: 2.0-17

pve-firewall: 3.0-3

pve-ha-manager: 2.0-3

ksm-control-daemon: 1.2-2

glusterfs-client: 3.8.8-1

lxc-pve: 2.1.0-2

lxcfs: 2.0.7-pve4

criu: 2.11.1-1~bpo90

novnc-pve: 0.6-4

smartmontools: 6.5+svn4324-1

zfsutils-linux: 0.7.2-pve1~bpo90

 

[guletz]

Try to use pve-zsync. It work very well.

 

[cpzengel]

It only does one snapshot at a time
So if a manual transfer does not work, an automatic should have problems as well

 

[guletz]

I am not sure that I understand what you want to say

 

[cpzengel]

pve-zsync is based on ssh zfs send, so I don’t think it’s working on 5.1
Aonyone uses that combo on 5.1?

 

[cpzengel]

so i tested it with a cluster. fresh install on both machines with

PVE Storage Replication via Web GUI

replication takes forever for 64kb dataset


()
Replication Log
2017-11-01 18:29:00 101-0: start replication job
2017-11-01 18:29:00 101-0: guest => VM 101, running => 0
2017-11-01 18:29:00 101-0: volumes => local-zfs:vm-101-disk-1
2017-11-01 18:29:01 101-0: create snapshot '__replicate_101-0_1509557340__' on local-zfs:vm-101-disk-1
2017-11-01 18:29:01 101-0: full sync 'local-zfs:vm-101-disk-1' (__replicate_101-0_1509557340__)

and not continuing

dataset showing up on remote side

rpool/data/vm-101-disk-1 64K 2.63T 64K -


receive processes do not finish

root 5895 0.3 0.4 294812 65692 ? Ss 18:29 0:00 /usr/bin/perl /usr/sbin/pvesm import local-zfs:vm-101-disk-1 zfs - -with-snapshots 1

root 5901 99.8 0.0 33600 3188 ? R 18:29 3:36 zfs recv -F -- rpool/data/vm-101-disk-1

 

[cpzengel]

https://bugzilla.proxmox.com/show_bug.cgi?id=1550

 

[guletz]

pve-zsync is based on ssh zfs send, so I don’t think it’s working on 5.1
Aonyone uses that combo on 5.1?

Me. And is working very good!

 

[cpzengel]

Me. And is working very good!

Same pveversion on both sides?
I am not the only one with those problems

 

[guletz]

Same pveversion on both sides?

Yes. And I use pve-zsync in 3 different clusters. On all cases I have no problem with this. A day ago, on this Forum another guy was doing the same and he was able to do it also in his case! And in one case is a special one, because is a 2 nodes cluster , and one node is online one time in a week, only to do the replication.

See here the post:
https://forum.proxmox.com/threads/m...des-to-single-storage-node.37749/#post-186271

 

[cpzengel]

I have fresh install and none ssh replication is running, neighter manual nor by gui

 

[wolfgang]

zfs 0.7.x is incompatible with all zfs < 0.6.5.11

see
https://github.com/zfsonlinux/zfs/issues/5699

Your problems with fresh installed PVE5.1 can't be reproduced.
Please send the error output of your manual send/receive.

 

[cpzengel]

That is the output.after finishing the first snapshot replication, both getting stucked

The sending site can be stopped without effect
The receiving side hast to be rebooted

 

[wolfgang]

To be sure you try manual send/receive from PVE 5.1(send) to PVE 5.1(receive)?

 

[cpzengel]

yes i am a zfs youtube podcaster and have tested as described above!

zfs send -pv rpool2/data/HIS-KASSEN@zfs-auto-snap_hourly-2017-10-31-1617 | ssh root@10.0.0.9 zfs recv -dvF Raid10

after transmiting the snapshot it takes forever and only can be interupted by ctrl+c and remote side crashing

 

[wolfgang]

Is the target storage working proper.
I mean can you use it local?

 

[cpzengel]

yes, i tested an old upgraded zfs on target and created a new one with pve 5.1

 

[wolfgang]

can you send me a "zpool status " and "zpool get all" to show me your pool setting form the target.
Also "zfs get all rpool2/data/HIS-KASSEN" would be helpful to see from the source.

 

[cpzengel]

F*****
I took a 5.0 Stick to install on Target side!!!!!!!!!
I´ll try 5.1

 

[jeffwadsworth]

This is why it is crucial for people to pveversion -v every node.