VMs Migration Fails

[Antonino89]

Hi Guys,

i'm quite done with setting into my proxmox environment.
Actually i can't migrate VM's, neither CTs, but let's focus to VMs actually...

If i try to migrate VM from Server1 to Server3 i get this type of error:

task started by HA resource agent
2017-08-22 11:26:35 starting migration of VM 101 to node 'Server3' (192.168.100.13)
2017-08-22 11:26:35 found local disk 'LVM1:vm-101-disk-1' (via storage)
2017-08-22 11:26:35 copying disk images
volume lvm1/vm-101-disk-1 already exists
command 'dd 'if=/dev/lvm1/vm-101-disk-1' 'bs=64k'' failed: got signal 13
send/receive failed, cleaning up snapshot(s)..
2017-08-22 11:26:37 ERROR: Failed to sync data - command 'set -o pipefail && pvesm export LVM1:vm-101-disk-1 raw+size - -with-snapshots 0 | /usr/bin/ssh -o 'BatchMode=yes' -o 'HostKeyAlias=Server3' root@192.168.100.13 -- pvesm import LVM1:vm-101-disk-1 raw+size - -with-snapshots 0' failed: exit code 255
2017-08-22 11:26:37 aborting phase 1 - cleanup resources
2017-08-22 11:26:37 ERROR: found stale volume copy 'LVM1:vm-101-disk-1' on node 'Server3'
2017-08-22 11:26:37 ERROR: migration aborted (duration 00:00:05): Failed to sync data - command 'set -o pipefail && pvesm export LVM1:vm-101-disk-1 raw+size - -with-snapshots 0 | /usr/bin/ssh -o 'BatchMode=yes' -o 'HostKeyAlias=Server3' root@192.168.100.13 -- pvesm import LVM1:vm-101-disk-1 raw+size - -with-snapshots 0' failed: exit code 255
TASK ERROR: migration aborted

HA is configured. This is my storage configuration:

root@Server1:~# more /etc/pve/storage.cfg

dir: local

path /var/lib/vz

content iso,backup,vztmpl


lvmthin: local-lvm

thinpool data

vgname pve

content rootdir,images


iscsi: Shared0

portal 192.168.100.14

target iqn.2012-06.com.example:target0

content none


lvm: LVM0

vgname lvm0

base Shared0:0.0.0.scsi-1FREEBSD_MYDEVID_0

content rootdir,images

shared 1


iscsi: Shared1

portal 192.168.100.14

target iqn.2012-06.com.example1:target0

content none


lvm: LVM1

vgname lvm1

base Shared1:0.0.1.scsi-1FREEBSD_MYDEVID_1

content images,rootdir

shared 0


LVM0 Is used for VMs and LVM1 is used for CTs

See also images attached, any help?

 

[wolfgang]

Hi,

if your LVM1 is available on all Nodes you have to mark it as shared.
If not this can't work, because HA will only work with shared/distributed storage.

 

[Antonino89]

Hi,

if your LVM1 is available on all Nodes you have to mark it as shared.
If not this can't work, because HA will only work with shared/distributed storage.

Okay changed...

pvesm set LVM0 -shared 1
pvesm set LVM1 -shared 1

Now migrations regarding VMs work. But not for CTs they pass to the other server butthey fails during restarting...

Can i post here the error output or i have to open another thread?

Thanks

 

[wolfgang]

Yes you can, because this thread is not a long thread and has not much users.

 

[Antonino89]

Yes you can, because this thread is not a long thread and has not much users.

Ok, so... i tried to migrate a CT from Server2 to Server3:

"Job for lxc@103.service failed because the control process exited with error code.
See "systemctl status lxc@103.service" and "journalctl -xe" for details.
TASK ERROR: command 'systemctl start lxc@103' failed: exit code 1"

"
● lxc@103.service - LXC Container: 103

Loaded: loaded (/lib/systemd/system/lxc@.service; disabled; vendor preset: enabled)

Drop-In: /usr/lib/systemd/system/lxc@.service.d

└─pve-reboot.conf

Active: failed (Result: exit-code) since Tue 2017-08-22 12:51:32 CEST; 1min 41s ago

Docs: man:lxc-start

man:lxc

Process: 4789 ExecStart=/usr/bin/lxc-start -n 103 (code=exited, status=1/FAILURE)


Aug 22 12:51:27 Server3 systemd[1]: Starting LXC Container: 103...

Aug 22 12:51:32 Server3 lxc-start[4789]: lxc-start: tools/lxc_start.c: main: 366 The container failed to

Aug 22 12:51:32 Server3 lxc-start[4789]: lxc-start: tools/lxc_start.c: main: 368 To get more details, ru

Aug 22 12:51:32 Server3 lxc-start[4789]: lxc-start: tools/lxc_start.c: main: 370 Additional information

Aug 22 12:51:32 Server3 systemd[1]: lxc@103.service: Control process exited, code=exited status=1

Aug 22 12:51:32 Server3 systemd[1]: Failed to start LXC Container: 103.

Aug 22 12:51:32 Server3 systemd[1]: lxc@103.service: Unit entered failed state.

Aug 22 12:51:32 Server3 systemd[1]: lxc@103.service: Failed with result 'exit-code'."


root@Server3:~# lxc-start -n 103 -F -l DEBUG -o /tmp/lxc-103.log

can't activate LV '/dev/lvm1/vm-103-disk-1': Failed to find logical volume "lvm1/vm-103-disk-1"

lxc-start: conf.c: run_buffer: 464 Script exited with status 5.

lxc-start: start.c: lxc_init: 450 Failed to run lxc.hook.pre-start for container "103".

lxc-start: start.c: __lxc_start: 1337 Failed to initialize container "103".

lxc-start: tools/lxc_start.c: main: 366 The container failed to start.

lxc-start: tools/lxc_start.c: main: 370 Additional information can be obtained by setting the --logfile and --logpriority options.

root@Server3:~# lxc-start -n 102 -F -l DEBUG -o /tmp/lxc-103.log

lxc-start: tools/lxc_start.c: main: 301 Container is already running.


CT number 2 was migrated... i do not understand why for some containers it doesn't works...

Thanks

 

[wolfgang]

Can you send the output of

lvs -v

 

[Antonino89]

Can you send the output of

lvs -v

Here we go.

root@Server3:~# lvs -v

LV VG #Seg Attr LSize Maj Min KMaj KMin Pool Origin Data% Meta% Move Cpy%Sync Log Convert LV UUID LProfile

vm-100-disk-1 lvm0 1 -wi-a----- 10.00g -1 -1 253 9 WUbHO5-qTnx-D7cF-HYaF-afI7-8Aia-zt7OGj

vm-101-disk-1 lvm0 1 -wi-a----- 10.00g -1 -1 253 8 2q7oXu-JFqH-Y3WI-dotG-4cDt-36dG-5Yfwrp

vm-101-disk-1 lvm1 1 -wi-a----- 5.00g -1 -1 253 6 wcb9XC-vkzv-rqLZ-jUes-bI60-kioN-sojORl

vm-102-disk-1 lvm1 1 -wi------- 10.00g -1 -1 -1 -1 nojo3G-4Wdq-TKZc-78Fe-34i0-QuUm-YO93Hr

data pve 1 twi-a-tz-- 73.12g -1 -1 253 5 0.00 0.43 oQsYW7-A8BS-FXjn-8Uo5-gU15-PXGI-9j4iJb

root pve 1 -wi-ao---- 31.75g -1 -1 253 1 yGc1YI-aN4m-7Vuf-MPOY-i82q-jOgA-3SEA8s

swap pve 1 -wi-ao---- 7.00g -1 -1 253 0 fNP96d-fRSX-NjoJ-b0Kp-XdmZ-8S6b-etSUgh

vm-103-disk-1 pve 1 -wi-a----- 5.00g -1 -1 253 2 5Zy66e-wi5y-6AxK-Fguw-db4Z-ck49-IOl1iy

 

[wolfgang]

I don't know why but your disk from CT 103 is loaded on local-lvm,
but your config say it is on lvm1.

 

[Antonino89]

I don't know why but your disk from CT 103 is loaded on local-lvm,
but your config say it is on lvm1.

Uhm, let me try to delete it and create it again...

 

[Antonino89]

Container 3 is not add to the HA. This could be the root problem?

 

[wolfgang]

Container 3 is not add to the HA. This could be the root problem?

No