can not migrate VMs

Z

zunder1990

New Member
Mar 10, 2023
2
0
1
I have a 3 node cluster all running pve-manager/7.3-6/723bb6ec (running kernel: 5.15.85-1-pve). All nodes use local non shared storage. When I try to migrate vms from two of the nodes to node called rackserver it all fails at step right after, drive mirror is starting for drive-scsi0. I can migrate between the other two nodes with no errors. The started about the same time I updated the whole cluster to 7.3 from 7.1 and removed a node on a really old version and added it back to cluster after a wipe a fresh install.

Code: 
2023-03-10 13:59:11 starting migration of VM 108 to node 'rackserver' (10.10.0.13)
2023-03-10 13:59:11 found local disk 'ssd-vmhost2:108/vm-108-disk-0.raw' (in current VM config)
2023-03-10 13:59:11 starting VM 108 on remote node 'rackserver'
2023-03-10 13:59:12 [rackserver] Use of uninitialized value $PVE::JSONSchema::PVE_TAG_RE in concatenation (.) or string at /usr/share/perl5/PVE/DataCenterConfig.pm line 161.
2023-03-10 13:59:12 [rackserver] Use of uninitialized value $PVE::JSONSchema::PVE_TAG_RE in concatenation (.) or string at /usr/share/perl5/PVE/DataCenterConfig.pm line 196.
2023-03-10 13:59:12 [rackserver] Use of uninitialized value $PVE::JSONSchema::PVE_TAG_RE in concatenation (.) or string at /usr/share/perl5/PVE/DataCenterConfig.pm line 196.
2023-03-10 13:59:12 [rackserver] Use of uninitialized value $PVE::JSONSchema::PVE_TAG_RE in concatenation (.) or string at /usr/share/perl5/PVE/DataCenterConfig.pm line 222.
2023-03-10 13:59:12 [rackserver] Use of uninitialized value $PVE::JSONSchema::PVE_TAG_RE in concatenation (.) or string at /usr/share/perl5/PVE/DataCenterConfig.pm line 222.
2023-03-10 13:59:15 [rackserver] Use of uninitialized value $PVE::JSONSchema::PVE_TAG_RE in concatenation (.) or string at /usr/share/perl5/PVE/DataCenterConfig.pm line 161.
2023-03-10 13:59:15 [rackserver] Use of uninitialized value $PVE::JSONSchema::PVE_TAG_RE in concatenation (.) or string at /usr/share/perl5/PVE/DataCenterConfig.pm line 196.
2023-03-10 13:59:15 [rackserver] Use of uninitialized value $PVE::JSONSchema::PVE_TAG_RE in concatenation (.) or string at /usr/share/perl5/PVE/DataCenterConfig.pm line 196.
2023-03-10 13:59:15 [rackserver] Use of uninitialized value $PVE::JSONSchema::PVE_TAG_RE in concatenation (.) or string at /usr/share/perl5/PVE/DataCenterConfig.pm line 222.
2023-03-10 13:59:15 [rackserver] Use of uninitialized value $PVE::JSONSchema::PVE_TAG_RE in concatenation (.) or string at /usr/share/perl5/PVE/DataCenterConfig.pm line 222.
2023-03-10 13:59:15 volume 'ssd-vmhost2:108/vm-108-disk-0.raw' is 'ssd1:vm-108-disk-0' on the target
2023-03-10 13:59:15 start remote tunnel
Use of uninitialized value $PVE::JSONSchema::PVE_TAG_RE in concatenation (.) or string at /usr/share/perl5/PVE/DataCenterConfig.pm line 161.
Use of uninitialized value $PVE::JSONSchema::PVE_TAG_RE in concatenation (.) or string at /usr/share/perl5/PVE/DataCenterConfig.pm line 196.
Use of uninitialized value $PVE::JSONSchema::PVE_TAG_RE in concatenation (.) or string at /usr/share/perl5/PVE/DataCenterConfig.pm line 196.
Use of uninitialized value $PVE::JSONSchema::PVE_TAG_RE in concatenation (.) or string at /usr/share/perl5/PVE/DataCenterConfig.pm line 222.
Use of uninitialized value $PVE::JSONSchema::PVE_TAG_RE in concatenation (.) or string at /usr/share/perl5/PVE/DataCenterConfig.pm line 222.
2023-03-10 13:59:17 ssh tunnel ver 1
2023-03-10 13:59:17 starting storage migration
2023-03-10 13:59:17 scsi0: start migration to nbd:unix:/run/qemu-server/108_nbd.migrate:exportname=drive-scsi0
drive mirror is starting for drive-scsi0
drive-scsi0: Cancelling block job
drive-scsi0: Done.
2023-03-10 14:00:17 ERROR: online migrate failure - block job (mirror) error: interrupted by signal
2023-03-10 14:00:17 aborting phase 2 - cleanup resources
2023-03-10 14:00:17 migrate_cancel
same error
Code: 
root@rackserver:/etc/pve# pvesm status
Use of uninitialized value $PVE::JSONSchema::PVE_TAG_RE in concatenation (.) or string at /usr/share/perl5/PVE/DataCenterConfig.pm line 161, <DATA> line 960.
Use of uninitialized value $PVE::JSONSchema::PVE_TAG_RE in concatenation (.) or string at /usr/share/perl5/PVE/DataCenterConfig.pm line 196, <DATA> line 960.
Use of uninitialized value $PVE::JSONSchema::PVE_TAG_RE in concatenation (.) or string at /usr/share/perl5/PVE/DataCenterConfig.pm line 196, <DATA> line 960.
Use of uninitialized value $PVE::JSONSchema::PVE_TAG_RE in concatenation (.) or string at /usr/share/perl5/PVE/DataCenterConfig.pm line 222, <DATA> line 960.
Use of uninitialized value $PVE::JSONSchema::PVE_TAG_RE in concatenation (.) or string at /usr/share/perl5/PVE/DataCenterConfig.pm line 222, <DATA> line 960.
Name                   Type     Status           Total            Used       Available        %
bigdata-backup          dir     active    129872072896     78615405440     51256667456   60.53%
hdd-vmhost2             dir   disabled               0               0               0      N/A
local                   dir     active        98497780        17755408        75692824   18.03%
local-lvm           lvmthin     active       594092032               0       594092032    0.00%
ssd-mirror          zfspool     active      1804599296       194697718      1609901577   10.79%
ssd-mirror-dell     zfspool   disabled               0               0               0      N/A
ssd-vmhost2             dir   disabled               0               0               0      N/A
ssd1                zfspool     active       377946112       170849508       207096604   45.20%
same error
Code: 
root@rackserver:/etc/pve# pveversion
Use of uninitialized value $PVE::JSONSchema::PVE_TAG_RE in concatenation (.) or string at /usr/share/perl5/PVE/DataCenterConfig.pm line 161.
Use of uninitialized value $PVE::JSONSchema::PVE_TAG_RE in concatenation (.) or string at /usr/share/perl5/PVE/DataCenterConfig.pm line 196.
Use of uninitialized value $PVE::JSONSchema::PVE_TAG_RE in concatenation (.) or string at /usr/share/perl5/PVE/DataCenterConfig.pm line 196.
Use of uninitialized value $PVE::JSONSchema::PVE_TAG_RE in concatenation (.) or string at /usr/share/perl5/PVE/DataCenterConfig.pm line 222.
Use of uninitialized value $PVE::JSONSchema::PVE_TAG_RE in concatenation (.) or string at /usr/share/perl5/PVE/DataCenterConfig.pm line 222.
pve-manager/7.2-3/c743d6c1 (running kernel: 5.15.30-2-pve)

And in syslog
Code: 
Mar 10 14:17:40 rackserver qmeventd[3876]: read: Connection reset by peer
Mar 10 14:17:40 rackserver systemd[1]: 108.scope: Succeeded.
Mar 10 14:17:40 rackserver systemd[1]: 108.scope: Consumed 3.706s CPU time.
Mar 10 14:17:40 rackserver qmeventd[943189]: Use of uninitialized value $PVE::JSONSchema::PVE_TAG_RE in concatenation (.) or string at /usr/share/perl5/PVE/DataCenterConfig.pm line 161.
Mar 10 14:17:40 rackserver qmeventd[943189]: Use of uninitialized value $PVE::JSONSchema::PVE_TAG_RE in concatenation (.) or string at /usr/share/perl5/PVE/DataCenterConfig.pm line 196.
Mar 10 14:17:40 rackserver qmeventd[943189]: Use of uninitialized value $PVE::JSONSchema::PVE_TAG_RE in concatenation (.) or string at /usr/share/perl5/PVE/DataCenterConfig.pm line 196.
Mar 10 14:17:40 rackserver qmeventd[943189]: Use of uninitialized value $PVE::JSONSchema::PVE_TAG_RE in concatenation (.) or string at /usr/share/perl5/PVE/DataCenterConfig.pm line 222.
Mar 10 14:17:40 rackserver qmeventd[943189]: Use of uninitialized value $PVE::JSONSchema::PVE_TAG_RE in concatenation (.) or string at /usr/share/perl5/PVE/DataCenterConfig.pm line 222.
 
zunder1990 said:
I have a 3 node cluster all running pve-manager/7.3-6/723bb6ec (running kernel: 5.15.85-1-pve).
Click to expand...

Looks not so:
zunder1990 said:
pve-manager/7.2-3/c743d6c1 (running kernel: 5.15.30-2-pve)
Click to expand...

Properly update all nodes and reboot them one after the other afterwards; so that they are all on the same versions and booted kernel.

If your nodes do not have active subscriptions, see here how to:
https://forum.proxmox.com/threads/reboot-occurs-randomly.119840/post-520320
 
I thought that the host was update to already but I was wrong. most have gotten mixed up since the host was also doing other stuff too. Host is fully patched and rebooted. errors have gone away during the live migration
 
You must log in or register to reply here.
Top Bottom