I have a VM live migration issue in PVE 8.2
I searched very hard on the forum and I was able to find that the issue was present in older version of Proxmox VE.
https://forum.proxmox.com/threads/ha-live-migration-with-zfs-replication-on-pve-6-2.69708/
I am running PVE 8.2 and live migration still errors for those HA VMs with zfs replication.
Is there anyone else having the same issues??
Is this not allowed by design? or It is a bug?
I was not able to find answer to this question even after hours of searching on the forum.
Here is my output of pveversion -v
I searched very hard on the forum and I was able to find that the issue was present in older version of Proxmox VE.
https://forum.proxmox.com/threads/ha-live-migration-with-zfs-replication-on-pve-6-2.69708/
I am running PVE 8.2 and live migration still errors for those HA VMs with zfs replication.
Is there anyone else having the same issues??
Is this not allowed by design? or It is a bug?
I was not able to find answer to this question even after hours of searching on the forum.
Code:
task started by HA resource agent
2024-08-09 15:03:24 use dedicated network address for sending migration traffic (10.21.250.106)
2024-08-09 15:03:25 starting migration of VM 104 to node 'ozone-set00-j09-svr06' (10.21.250.106)
2024-08-09 15:03:25 found local, replicated disk 'hdd-2m-data:vm-104-disk-0' (attached)
2024-08-09 15:03:25 found generated disk 'local-zfs:vm-104-cloudinit' (in current VM config)
2024-08-09 15:03:25 virtio0: start tracking writes using block-dirty-bitmap 'repl_virtio0'
2024-08-09 15:03:25 replicating disk images
2024-08-09 15:03:25 start replication job
QEMU Guest Agent is not running - VM 104 qmp command 'guest-ping' failed - got timeout
2024-08-09 15:03:28 guest => VM 104, running => 1749892
2024-08-09 15:03:28 volumes => hdd-2m-data:vm-104-disk-0
2024-08-09 15:03:30 create snapshot '__replicate_104-0_1723183405__' on hdd-2m-data:vm-104-disk-0
2024-08-09 15:03:30 using secure transmission, rate limit: none
2024-08-09 15:03:30 incremental sync 'hdd-2m-data:vm-104-disk-0' (__replicate_104-0_1723183203__ => __replicate_104-0_1723183405__)
2024-08-09 15:03:32 send from @__replicate_104-0_1723183203__ to hdd-2m-data/vm-104-disk-0@__replicate_104-0_1723183405__ estimated size is 170K
2024-08-09 15:03:32 total estimated size is 170K
2024-08-09 15:03:32 TIME SENT SNAPSHOT hdd-2m-data/vm-104-disk-0@__replicate_104-0_1723183405__
2024-08-09 15:03:33 successfully imported 'hdd-2m-data:vm-104-disk-0'
2024-08-09 15:03:33 delete previous replication snapshot '__replicate_104-0_1723183203__' on hdd-2m-data:vm-104-disk-0
2024-08-09 15:03:34 (remote_finalize_local_job) delete stale replication snapshot '__replicate_104-0_1723183203__' on hdd-2m-data:vm-104-disk-0
2024-08-09 15:03:35 end replication job
2024-08-09 15:03:35 copying local disk images
2024-08-09 15:03:36 full send of rpool/data/vm-104-cloudinit@__migration__ estimated size is 65.3K
2024-08-09 15:03:36 total estimated size is 65.3K
2024-08-09 15:03:36 TIME SENT SNAPSHOT rpool/data/vm-104-cloudinit@__migration__
2024-08-09 15:03:36 successfully imported 'local-zfs:vm-104-cloudinit'
2024-08-09 15:03:36 volume 'local-zfs:vm-104-cloudinit' is 'local-zfs:vm-104-cloudinit' on the target
2024-08-09 15:03:36 starting VM 104 on remote node 'ozone-set00-j09-svr06'
2024-08-09 15:03:39 volume 'hdd-2m-data:vm-104-disk-0' is 'hdd-2m-data:vm-104-disk-0' on the target
2024-08-09 15:03:39 start remote tunnel
2024-08-09 15:03:40 ssh tunnel ver 1
2024-08-09 15:03:40 starting storage migration
2024-08-09 15:03:40 virtio0: start migration to nbd:unix:/run/qemu-server/104_nbd.migrate:exportname=drive-virtio0
drive mirror re-using dirty bitmap 'repl_virtio0'
drive mirror is starting for drive-virtio0
channel 3: open failed: connect failed: open failed
drive-virtio0: Cancelling block job
drive-virtio0: Done.
2024-08-09 15:03:40 ERROR: online migrate failure - mirroring error: VM 104 qmp command 'drive-mirror' failed - Failed to read initial magic: Unexpected end-of-file before all data were read
2024-08-09 15:03:40 aborting phase 2 - cleanup resources
2024-08-09 15:03:40 migrate_cancelHere is my output of pveversion -v
Code:
root@ozone-set00-j09-svr06:~/.ssh# pveversion -v
proxmox-ve: 8.2.0 (running kernel: 6.8.4-2-pve)
pve-manager: 8.2.2 (running version: 8.2.2/9355359cd7afbae4)
proxmox-kernel-helper: 8.1.0
proxmox-kernel-6.8: 6.8.4-2
proxmox-kernel-6.8.4-2-pve-signed: 6.8.4-2
ceph-fuse: 17.2.7-pve3
corosync: 3.1.7-pve3
criu: 3.17.1-2
glusterfs-client: 10.3-5
ifupdown2: 3.2.0-1+pmx8
ksm-control-daemon: 1.5-1
libjs-extjs: 7.0.0-4
libknet1: 1.28-pve1
libproxmox-acme-perl: 1.5.0
libproxmox-backup-qemu0: 1.4.1
libproxmox-rs-perl: 0.3.3
libpve-access-control: 8.1.4
libpve-apiclient-perl: 3.3.2
libpve-cluster-api-perl: 8.0.6
libpve-cluster-perl: 8.0.6
libpve-common-perl: 8.2.1
libpve-guest-common-perl: 5.1.1
libpve-http-server-perl: 5.1.0
libpve-network-perl: 0.9.8
libpve-rs-perl: 0.8.8
libpve-storage-perl: 8.2.1
libspice-server1: 0.15.1-1
lvm2: 2.03.16-2
lxc-pve: 6.0.0-1
lxcfs: 6.0.0-pve2
novnc-pve: 1.4.0-3
proxmox-backup-client: 3.2.0-1
proxmox-backup-file-restore: 3.2.0-1
proxmox-kernel-helper: 8.1.0
proxmox-mail-forward: 0.2.3
proxmox-mini-journalreader: 1.4.0
proxmox-offline-mirror-helper: 0.6.6
proxmox-widget-toolkit: 4.2.1
pve-cluster: 8.0.6
pve-container: 5.0.10
pve-docs: 8.2.1
pve-edk2-firmware: 4.2023.08-4
pve-esxi-import-tools: 0.7.0
pve-firewall: 5.0.5
pve-firmware: 3.11-1
pve-ha-manager: 4.0.4
pve-i18n: 3.2.2
pve-qemu-kvm: 8.1.5-5
pve-xtermjs: 5.3.0-3
qemu-server: 8.2.1
smartmontools: 7.3-pve1
spiceterm: 3.3.0
swtpm: 0.8.0+pve1
vncterm: 1.8.0
zfsutils-linux: 2.2.3-pve2
root@ozone-set00-j09-svr06:~/.ssh# pveversion -v
