ZFS Storage Replication...

G

Gilberto Ferreira

Renowned Member
Aug 21, 2014
389
51
93
Joinville, Santa Catarina, Brazil, Brazil
Hi... I have here just one PowerEdge Dell 2950 as a lab server...
With this server, I enable nested virtulization, in order to create a 3 node cluster.
I have deploy 3 VM with Proxmox 5, inside Proxmox 5 server...
I also try Storage Replication.
Offline, the migration is ok.
But online, I always get this error:

qm migrate 100 prox02 --online --with-local-disks
2017-08-26 21:31:57 starting migration of VM 100 to node 'prox02' (10.1.1.20)
2017-08-26 21:31:58 found local disk 'stg-zfs:vm-100-disk-1' (in current VM config)
2017-08-26 21:31:58 copying disk images
2017-08-26 21:31:58 starting VM 100 on remote node 'prox02'
22017-08-26 21:32:05 start remote tunnel
2017-08-26 21:32:06 ssh tunnel ver 1
2017-08-26 21:32:06 starting storage migration
2017-08-26 21:32:06 scsi0: start migration to to nbd:10.1.1.20:60000:exportname=drive-scsi0
drive mirror is starting for drive-scsi0
drive-scsi0: transferred: 0 bytes remaining: 34359738368 bytes total: 34359738368 bytes progression: 0.00 % busy: 1 ready: 0
drive-scsi0: transferred: 102760448 bytes remaining: 34256977920 bytes total: 34359738368 bytes progression: 0.30 % busy: 1 ready: 0
drive-scsi0: transferred: 205520896 bytes remaining: 34154217472 bytes total: 34359738368 bytes progression: 0.60 % busy: 1 ready: 0
drive-scsi0: transferred: 373293056 bytes remaining: 33986445312 bytes total: 34359738368 bytes progression: 1.09 % busy: 1 ready: 0
drive-scsi0: transferred: 525336576 bytes remaining: 33834401792 bytes total: 34359738368 bytes progression: 1.53 % busy: 1 ready: 0
drive-scsi0: transferred: 649068544 bytes remaining: 33710669824 bytes total: 34359738368 bytes progression: 1.89 % busy: 1 ready: 0
drive-scsi0: transferred: 693108736 bytes remaining: 33666629632 bytes total: 34359738368 bytes progression: 2.02 % busy: 1 ready: 0
drive-scsi0: transferred: 703594496 bytes remaining: 33656143872 bytes total: 34359738368 bytes progression: 2.05 % busy: 1 ready: 0
drive-scsi0: transferred: 716177408 bytes remaining: 33643560960 bytes total: 34359738368 bytes progression: 2.08 % busy: 1 ready: 0
drive-scsi0: transferred: 728760320 bytes remaining: 33630978048 bytes total: 34359738368 bytes progression: 2.12 % busy: 1 ready: 0
drive-scsi0: transferred: 736100352 bytes remaining: 33623638016 bytes total: 34359738368 bytes progression: 2.14 % busy: 1 ready: 0
drive-scsi0: transferred: 751828992 bytes remaining: 33607909376 bytes total: 34359738368 bytes progression: 2.19 % busy: 1 ready: 0
drive-scsi0: transferred: 760217600 bytes remaining: 33599520768 bytes total: 34359738368 bytes progression: 2.21 % busy: 1 ready: 0
drive-scsi0: transferred: 775946240 bytes remaining: 33583792128 bytes total: 34359738368 bytes progression: 2.26 % busy: 1 ready: 0
drive-scsi0: transferred: 789577728 bytes remaining: 33570160640 bytes total: 34359738368 bytes progression: 2.30 % busy: 1 ready: 0
drive-scsi0: transferred: 810549248 bytes remaining: 33549189120 bytes total: 34359738368 bytes progression: 2.36 % busy: 1 ready: 0
drive-scsi0: transferred: 818937856 bytes remaining: 33540800512 bytes total: 34359738368 bytes progression: 2.38 % busy: 1 ready: 0
drive-scsi0: transferred: 835715072 bytes remaining: 33524023296 bytes total: 34359738368 bytes progression: 2.43 % busy: 1 ready: 0
drive-scsi0: transferred: 854589440 bytes remaining: 33505148928 bytes total: 34359738368 bytes progression: 2.49 % busy: 1 ready: 0
drive-scsi0: transferred: 867172352 bytes remaining: 33492566016 bytes total: 34359738368 bytes progression: 2.52 % busy: 1 ready: 0
drive-scsi0: transferred: 882900992 bytes remaining: 33476837376 bytes total: 34359738368 bytes progression: 2.57 % busy: 1 ready: 0
drive-scsi0: transferred: 893386752 bytes remaining: 33466351616 bytes total: 34359738368 bytes progression: 2.60 % busy: 1 ready: 0
drive-scsi0: transferred: 902823936 bytes remaining: 33456914432 bytes total: 34359738368 bytes progression: 2.63 % busy: 1 ready: 0
drive-scsi0: transferred: 915406848 bytes remaining: 33444331520 bytes total: 34359738368 bytes progression: 2.66 % busy: 1 ready: 0
drive-scsi0: transferred: 921698304 bytes remaining: 33438040064 bytes total: 34359738368 bytes progression: 2.68 % busy: 1 ready: 0
drive-scsi0: transferred: 932184064 bytes remaining: 33427554304 bytes total: 34359738368 bytes progression: 2.71 % busy: 1 ready: 0
drive-scsi0: transferred: 942669824 bytes remaining: 33417068544 bytes total: 34359738368 bytes progression: 2.74 % busy: 1 ready: 0
drive-scsi0: transferred: 973078528 bytes remaining: 33386659840 bytes total: 34359738368 bytes progression: 2.83 % busy: 1 ready: 0
drive-scsi0: transferred: 985661440 bytes remaining: 33374076928 bytes total: 34359738368 bytes progression: 2.87 % busy: 1 ready: 0
drive-scsi0: transferred: 1001390080 bytes remaining: 33358348288 bytes total: 34359738368 bytes progression: 2.91 % busy: 1 ready: 0
drive-scsi0: transferred: 1009778688 bytes remaining: 33349959680 bytes total: 34359738368 bytes progression: 2.94 % busy: 1 ready: 0
drive-scsi0: transferred: 1019215872 bytes remaining: 33340522496 bytes total: 34359738368 bytes progression: 2.97 % busy: 1 ready: 0
drive-scsi0: transferred: 1029701632 bytes remaining: 33330036736 bytes total: 34359738368 bytes progression: 3.00 % busy: 1 ready: 0
drive-scsi0: transferred: 1048576000 bytes remaining: 33311162368 bytes total: 34359738368 bytes progression: 3.05 % busy: 1 ready: 0
drive-scsi0: transferred: 1068498944 bytes remaining: 33291239424 bytes total: 34359738368 bytes progression: 3.11 % busy: 1 ready: 0
drive-scsi0: transferred: 1081081856 bytes remaining: 33278656512 bytes total: 34359738368 bytes progression: 3.15 % busy: 1 ready: 0
drive-scsi0: transferred: 1093664768 bytes remaining: 33266073600 bytes total: 34359738368 bytes progression: 3.18 % busy: 1 ready: 0
drive-scsi0: transferred: 1105199104 bytes remaining: 33254539264 bytes total: 34359738368 bytes progression: 3.22 % busy: 1 ready: 0
drive-scsi0: transferred: 1113587712 bytes remaining: 33246150656 bytes total: 34359738368 bytes progression: 3.24 % busy: 1 ready: 0
drive-scsi0: transferred: 1124073472 bytes remaining: 33235664896 bytes total: 34359738368 bytes progression: 3.27 % busy: 1 ready: 0
drive-scsi0: transferred: 1146093568 bytes remaining: 33213644800 bytes total: 34359738368 bytes progression: 3.34 % busy: 1 ready: 0
drive-scsi0: transferred: 1163919360 bytes remaining: 33195819008 bytes total: 34359738368 bytes progression: 3.39 % busy: 1 ready: 0
drive-scsi0: transferred: 1175453696 bytes remaining: 33184284672 bytes total: 34359738368 bytes progression: 3.42 % busy: 1 ready: 0
drive-scsi0: transferred: 1189085184 bytes remaining: 33170653184 bytes total: 34359738368 bytes progression: 3.46 % busy: 1 ready: 0
drive-scsi0: transferred: 1198522368 bytes remaining: 33161216000 bytes total: 34359738368 bytes progression: 3.49 % busy: 1 ready: 0
drive-scsi0: transferred: 1217396736 bytes remaining: 33142341632 bytes total: 34359738368 bytes progression: 3.54 % busy: 1 ready: 0
drive-scsi0: transferred: 1228931072 bytes remaining: 33130807296 bytes total: 34359738368 bytes progression: 3.58 % busy: 1 ready: 0
drive-scsi0: transferred: 1247805440 bytes remaining: 33111932928 bytes total: 34359738368 bytes progression: 3.63 % busy: 1 ready: 0
drive-scsi0: transferred: 1262485504 bytes remaining: 33097252864 bytes total: 34359738368 bytes progression: 3.67 % busy: 1 ready: 0
drive-scsi0: transferred: 1271922688 bytes remaining: 33087815680 bytes total: 34359738368 bytes progression: 3.70 % busy: 1 ready: 0
drive-scsi0: transferred: 1284505600 bytes remaining: 33075232768 bytes total: 34359738368 bytes progression: 3.74 % busy: 1 ready: 0
drive-scsi0: transferred: 1299185664 bytes remaining: 33060552704 bytes total: 34359738368 bytes progression: 3.78 % busy: 1 ready: 0
drive-scsi0: transferred: 1317011456 bytes remaining: 33042726912 bytes total: 34359738368 bytes progression: 3.83 % busy: 1 ready: 0
drive-scsi0: transferred: 1323302912 bytes remaining: 33036435456 bytes total: 34359738368 bytes progression: 3.85 % busy: 1 ready: 0
drive-scsi0: transferred: 1341128704 bytes remaining: 33018609664 bytes total: 34359738368 bytes progression: 3.90 % busy: 1 ready: 0
drive-scsi0: transferred: 1388314624 bytes remaining: 32971423744 bytes total: 34359738368 bytes progression: 4.04 % busy: 1 ready: 0
drive-scsi0: transferred: 1479540736 bytes remaining: 32880197632 bytes total: 34359738368 bytes progression: 4.31 % busy: 1 ready: 0
drive-scsi0: transferred: 1485832192 bytes remaining: 32873906176 bytes total: 34359738368 bytes progression: 4.32 % busy: 1 ready: 0
drive-scsi0: transferred: 1485832192 bytes remaining: 32873906176 bytes total: 34359738368 bytes progression: 4.32 % busy: 1 ready: 0
drive-scsi0: transferred: 1485832192 bytes remaining: 32873906176 bytes total: 34359738368 bytes progression: 4.32 % busy: 1 ready: 0
drive-scsi0: transferred: 1485832192 bytes remaining: 32873906176 bytes total: 34359738368 bytes progression: 4.32 % busy: 1 ready: 0
drive-scsi0: transferred: 1485832192 bytes remaining: 32873906176 bytes total: 34359738368 bytes progression: 4.32 % busy: 1 ready: 0
drive-scsi0: transferred: 1485832192 bytes remaining: 32873906176 bytes total: 34359738368 bytes progression: 4.32 % busy: 1 ready: 0
drive-scsi0: transferred: 1485832192 bytes remaining: 32873906176 bytes total: 34359738368 bytes progression: 4.32 % busy: 1 ready: 0
drive-scsi0: transferred: 1485832192 bytes remaining: 32873906176 bytes total: 34359738368 bytes progression: 4.32 % busy: 1 ready: 0
drive-scsi0: transferred: 1485832192 bytes remaining: 32873906176 bytes total: 34359738368 bytes progression: 4.32 % busy: 1 ready: 0
drive-scsi0: transferred: 1485832192 bytes remaining: 32873906176 bytes total: 34359738368 bytes progression: 4.32 % busy: 1 ready: 0
drive-scsi0: transferred: 1485832192 bytes remaining: 32873906176 bytes total: 34359738368 bytes progression: 4.32 % busy: 1 ready: 0
drive-scsi0: transferred: 1485832192 bytes remaining: 32873906176 bytes total: 34359738368 bytes progression: 4.32 % busy: 1 ready: 0
drive-scsi0: transferred: 1485832192 bytes remaining: 32873906176 bytes total: 34359738368 bytes progression: 4.32 % busy: 1 ready: 0
drive-scsi0: transferred: 1485832192 bytes remaining: 32873906176 bytes total: 34359738368 bytes progression: 4.32 % busy: 1 ready: 0
drive-scsi0: transferred: 1485832192 bytes remaining: 32873906176 bytes total: 34359738368 bytes progression: 4.32 % busy: 1 ready: 0
drive-scsi0: transferred: 1485832192 bytes remaining: 32873906176 bytes total: 34359738368 bytes progression: 4.32 % busy: 1 ready: 0
drive-scsi0: transferred: 1485832192 bytes remaining: 32873906176 bytes total: 34359738368 bytes progression: 4.32 % busy: 1 ready: 0
drive-scsi0: transferred: 1485832192 bytes remaining: 32873906176 bytes total: 34359738368 bytes progression: 4.32 % busy: 1 ready: 0
drive-scsi0: transferred: 1485832192 bytes remaining: 32873906176 bytes total: 34359738368 bytes progression: 4.32 % busy: 1 ready: 0
drive-scsi0: transferred: 1485832192 bytes remaining: 32873906176 bytes total: 34359738368 bytes progression: 4.32 % busy: 1 ready: 0
drive-scsi0: transferred: 1485832192 bytes remaining: 32873906176 bytes total: 34359738368 bytes progression: 4.32 % busy: 1 ready: 0
drive-scsi0: transferred: 1485832192 bytes remaining: 32873906176 bytes total: 34359738368 bytes progression: 4.32 % busy: 1 ready: 0
drive-scsi0: transferred: 1485832192 bytes remaining: 32873906176 bytes total: 34359738368 bytes progression: 4.32 % busy: 1 ready: 0
drive-scsi0: transferred: 1485832192 bytes remaining: 32873906176 bytes total: 34359738368 bytes progression: 4.32 % busy: 1 ready: 0
drive-scsi0: transferred: 1485832192 bytes remaining: 32873906176 bytes total: 34359738368 bytes progression: 4.32 % busy: 1 ready: 0
drive-scsi0: transferred: 1485832192 bytes remaining: 32873906176 bytes total: 34359738368 bytes progression: 4.32 % busy: 1 ready: 0
drive-scsi0: transferred: 1485832192 bytes remaining: 32873906176 bytes total: 34359738368 bytes progression: 4.32 % busy: 1 ready: 0
drive-scsi0: transferred: 1485832192 bytes remaining: 32873906176 bytes total: 34359738368 bytes progression: 4.32 % busy: 1 ready: 0
drive-scsi0: transferred: 1485832192 bytes remaining: 32873906176 bytes total: 34359738368 bytes progression: 4.32 % busy: 1 ready: 0
drive-scsi0: Cancelling block job
2017-08-26 21:33:52 ERROR: online migrate failure - mirroring error: VM 100 not running
2017-08-26 21:33:52 aborting phase 2 - cleanup resources
2017-08-26 21:33:52 migrate_cancel
2017-08-26 21:33:52 migrate_cancel error: VM 100 not running
drive-scsi0: Cancelling block job
2017-08-26 21:33:52 ERROR: VM 100 not running
2017-08-26 21:34:09 ERROR: command '/usr/bin/ssh -o 'BatchMode=yes' -o 'HostKeyAlias=prox02' root@10.1.1.20 pvesm free stg-zfs:vm-100-disk-1' failed: exit code 4
2017-08-26 21:34:12 ERROR: migration finished with problems (duration 00:02:15)
migration problems

I have ran the first syn replication and finish with success!
I do not have any sync schedule between the servers.

What it's wrong??
 
UPDATE!
When I try to migrate the offline VM to PROX01 to PROX02, now I get this error:

qm migrate 100 prox02 --with-local-disks
2017-08-26 21:41:30 starting migration of VM 100 to node 'prox02' (10.1.1.20)
2017-08-26 21:41:30 found local disk 'stg-zfs:vm-100-disk-1' (in current VM config)
2017-08-26 21:41:30 copying disk images
send from @ to ZFS/vm-100-disk-1@__migration__ estimated size is 6.89G
total estimated size is 6.89G
TIME SENT SNAPSHOT
ZFS/vm-100-disk-1 name ZFS/vm-100-disk-1 -
volume 'ZFS/vm-100-disk-1' already exists
command 'zfs send -Rpv -- ZFS/vm-100-disk-1@__migration__' failed: got signal 13
send/receive failed, cleaning up snapshot(s)..
2017-08-26 21:41:32 ERROR: Failed to sync data - command 'set -o pipefail && pvesm export stg-zfs:vm-100-disk-1 zfs - -with-snapshots 0 -snapshot __migration__ | /usr/bin/ssh -o 'BatchMode=yes' -o 'HostKeyAlias=prox02' root@10.1.1.20 -- pvesm import stg-zfs:vm-100-disk-1 zfs - -with-snapshots 0 -delete-snapshot __migration__' failed: exit code 255
2017-08-26 21:41:32 aborting phase 1 - cleanup resources
2017-08-26 21:41:32 ERROR: found stale volume copy 'stg-zfs:vm-100-disk-1' on node 'prox02'
2017-08-26 21:41:32 ERROR: migration aborted (duration 00:00:03): Failed to sync data - command 'set -o pipefail && pvesm export stg-zfs:vm-100-disk-1 zfs - -with-snapshots 0 -snapshot __migration__ | /usr/bin/ssh -o 'BatchMode=yes' -o 'HostKeyAlias=prox02' root@10.1.1.20 -- pvesm import stg-zfs:vm-100-disk-1 zfs - -with-snapshots 0 -delete-snapshot __migration__' failed: exit code 255
migration aborted
 
You must log in or register to reply here.
Top Bottom