Livemigration bricht ab und VM stopt

[konabi]

Hallo,
ich habe einen Cluster bestehend aus 3 PVE Nodes.
Jeder Node hat seinen eignen zfs Storage, also eigene Platten und kein Shared Storage.
Es ist mir jetzt schon mehrmals passiert dass die LiveMigration von einen Server auf den anderen abbricht.
Ich vermute ein problem bei der Übername des Arbeitsspeichers auf den neuen Node.
Offline Migration funktioniert.
Was könnte das problem sein?

Proxmox
Virtual Environment 6.4-13
Virtual Machine 111 (srv-dc01) on node 'pve03'
Logs
()
2021-11-04 15:07:37 use dedicated network address for sending migration traffic (192.168.111.203)
2021-11-04 15:07:38 starting migration of VM 111 to node 'pve03' (192.168.111.203)
2021-11-04 15:07:38 found local disk 'zfs-storage:vm-111-disk-0' (in current VM config)
2021-11-04 15:07:38 starting VM 111 on remote node 'pve03'
2021-11-04 15:07:40 volume 'zfs-storage:vm-111-disk-0' is 'zfs-storage:vm-111-disk-0' on the target
2021-11-04 15:07:40 start remote tunnel
2021-11-04 15:07:41 ssh tunnel ver 1
2021-11-04 15:07:41 starting storage migration
2021-11-04 15:07:41 virtio0: start migration to nbd:192.168.111.203:60001:exportname=drive-virtio0
drive mirror is starting for drive-virtio0
drive-virtio0: transferred 5.0 MiB of 50.0 GiB (0.01%) in 0s
drive-virtio0: transferred 885.0 MiB of 50.0 GiB (1.73%) in 1s
drive-virtio0: transferred 1.6 GiB of 50.0 GiB (3.22%) in 2s
drive-virtio0: transferred 2.4 GiB of 50.0 GiB (4.73%) in 3s
drive-virtio0: transferred 3.1 GiB of 50.0 GiB (6.22%) in 4s
drive-virtio0: transferred 3.9 GiB of 50.0 GiB (7.73%) in 5s
drive-virtio0: transferred 4.6 GiB of 50.0 GiB (9.19%) in 6s
drive-virtio0: transferred 5.3 GiB of 50.0 GiB (10.68%) in 7s
drive-virtio0: transferred 6.1 GiB of 50.0 GiB (12.15%) in 8s
drive-virtio0: transferred 6.8 GiB of 50.0 GiB (13.64%) in 9s
drive-virtio0: transferred 7.6 GiB of 50.0 GiB (15.11%) in 10s
drive-virtio0: transferred 8.3 GiB of 50.0 GiB (16.56%) in 11s
drive-virtio0: transferred 9.0 GiB of 50.0 GiB (18.00%) in 12s
drive-virtio0: transferred 9.7 GiB of 50.0 GiB (19.47%) in 13s
drive-virtio0: transferred 10.4 GiB of 50.0 GiB (20.88%) in 14s
drive-virtio0: transferred 11.0 GiB of 50.0 GiB (22.01%) in 15s
drive-virtio0: transferred 11.8 GiB of 50.0 GiB (23.56%) in 16s
drive-virtio0: transferred 12.5 GiB of 50.0 GiB (25.03%) in 17s
drive-virtio0: transferred 13.2 GiB of 50.0 GiB (26.49%) in 18s
drive-virtio0: transferred 14.0 GiB of 50.0 GiB (27.94%) in 19s
drive-virtio0: transferred 14.7 GiB of 50.0 GiB (29.41%) in 20s
drive-virtio0: transferred 15.4 GiB of 50.0 GiB (30.88%) in 21s
drive-virtio0: transferred 16.1 GiB of 50.0 GiB (32.29%) in 22s
drive-virtio0: transferred 16.9 GiB of 50.0 GiB (33.72%) in 23s
drive-virtio0: transferred 17.6 GiB of 50.0 GiB (35.22%) in 24s
drive-virtio0: transferred 18.3 GiB of 50.0 GiB (36.68%) in 25s
drive-virtio0: transferred 19.0 GiB of 50.0 GiB (38.10%) in 26s
drive-virtio0: transferred 19.8 GiB of 50.0 GiB (39.53%) in 27s
drive-virtio0: transferred 20.5 GiB of 50.0 GiB (40.90%) in 28s
drive-virtio0: transferred 21.1 GiB of 50.0 GiB (42.26%) in 29s
drive-virtio0: transferred 21.8 GiB of 50.0 GiB (43.65%) in 30s
drive-virtio0: transferred 22.5 GiB of 50.0 GiB (45.10%) in 31s
drive-virtio0: transferred 23.2 GiB of 50.0 GiB (46.42%) in 32s
drive-virtio0: transferred 24.0 GiB of 50.0 GiB (47.92%) in 33s
drive-virtio0: transferred 24.7 GiB of 50.0 GiB (49.41%) in 34s
drive-virtio0: transferred 25.4 GiB of 50.0 GiB (50.85%) in 35s
drive-virtio0: transferred 26.2 GiB of 50.0 GiB (52.37%) in 36s
drive-virtio0: transferred 26.9 GiB of 50.0 GiB (53.87%) in 37s
drive-virtio0: transferred 27.7 GiB of 50.0 GiB (55.31%) in 38s
drive-virtio0: transferred 28.4 GiB of 50.0 GiB (56.79%) in 39s
drive-virtio0: transferred 29.1 GiB of 50.0 GiB (58.21%) in 40s
drive-virtio0: transferred 29.8 GiB of 50.0 GiB (59.67%) in 41s
drive-virtio0: transferred 30.6 GiB of 50.0 GiB (61.13%) in 42s
drive-virtio0: transferred 31.3 GiB of 50.0 GiB (62.60%) in 43s
drive-virtio0: transferred 32.0 GiB of 50.0 GiB (64.06%) in 44s
drive-virtio0: transferred 32.8 GiB of 50.0 GiB (65.55%) in 45s
drive-virtio0: transferred 33.4 GiB of 50.0 GiB (66.77%) in 46s
drive-virtio0: transferred 34.1 GiB of 50.0 GiB (68.20%) in 47s
drive-virtio0: transferred 34.8 GiB of 50.0 GiB (69.63%) in 48s
drive-virtio0: transferred 35.5 GiB of 50.0 GiB (71.00%) in 49s
drive-virtio0: transferred 36.2 GiB of 50.0 GiB (72.31%) in 50s
drive-virtio0: transferred 36.8 GiB of 50.0 GiB (73.67%) in 51s
drive-virtio0: transferred 37.5 GiB of 50.0 GiB (75.07%) in 52s
drive-virtio0: transferred 38.2 GiB of 50.0 GiB (76.49%) in 53s
drive-virtio0: transferred 39.0 GiB of 50.0 GiB (77.92%) in 54s
drive-virtio0: transferred 39.6 GiB of 50.0 GiB (79.27%) in 55s
drive-virtio0: transferred 40.3 GiB of 50.0 GiB (80.63%) in 56s
drive-virtio0: transferred 41.0 GiB of 50.0 GiB (81.92%) in 57s
drive-virtio0: transferred 41.6 GiB of 50.0 GiB (83.27%) in 58s
drive-virtio0: transferred 42.3 GiB of 50.0 GiB (84.58%) in 59s
drive-virtio0: transferred 43.0 GiB of 50.0 GiB (85.99%) in 1m
drive-virtio0: transferred 43.7 GiB of 50.0 GiB (87.37%) in 1m 1s
drive-virtio0: transferred 44.4 GiB of 50.0 GiB (88.81%) in 1m 2s
drive-virtio0: transferred 45.1 GiB of 50.0 GiB (90.13%) in 1m 3s
drive-virtio0: transferred 45.8 GiB of 50.0 GiB (91.56%) in 1m 4s
drive-virtio0: transferred 46.5 GiB of 50.0 GiB (92.97%) in 1m 5s
drive-virtio0: transferred 47.2 GiB of 50.0 GiB (94.37%) in 1m 6s
drive-virtio0: transferred 47.9 GiB of 50.0 GiB (95.75%) in 1m 7s
drive-virtio0: transferred 48.5 GiB of 50.0 GiB (97.02%) in 1m 8s
drive-virtio0: transferred 49.2 GiB of 50.0 GiB (98.34%) in 1m 9s
drive-virtio0: transferred 49.9 GiB of 50.0 GiB (99.79%) in 1m 10s
drive-virtio0: transferred 50.0 GiB of 50.0 GiB (100.00%) in 1m 11s, ready
all 'mirror' jobs are ready
2021-11-04 15:08:52 starting online/live migration on tcp:192.168.111.203:60000
2021-11-04 15:08:52 set migration capabilities
2021-11-04 15:08:52 migration downtime limit: 100 ms
2021-11-04 15:08:52 migration cachesize: 1.0 GiB
2021-11-04 15:08:52 set migration parameters
2021-11-04 15:08:52 start migrate command to tcp:192.168.111.203:60000
2021-11-04 15:08:53 migration active, transferred 696.9 MiB of 8.0 GiB VM-state, 1018.8 MiB/s
2021-11-04 15:08:54 migration active, transferred 1.8 GiB of 8.0 GiB VM-state, 1.1 GiB/s
2021-11-04 15:08:55 migration active, transferred 2.8 GiB of 8.0 GiB VM-state, 1.1 GiB/s
2021-11-04 15:08:56 migration active, transferred 3.9 GiB of 8.0 GiB VM-state, 1.2 GiB/s
query migrate failed: VM 111 qmp command 'query-migrate' failed - client closed connection

2021-11-04 15:08:58 query migrate failed: VM 111 qmp command 'query-migrate' failed - client closed connection
query migrate failed: VM 111 not running

2021-11-04 15:09:00 query migrate failed: VM 111 not running
query migrate failed: VM 111 not running

2021-11-04 15:09:02 query migrate failed: VM 111 not running
query migrate failed: VM 111 not running

2021-11-04 15:09:04 query migrate failed: VM 111 not running
query migrate failed: VM 111 not running

2021-11-04 15:09:06 query migrate failed: VM 111 not running
query migrate failed: VM 111 not running

2021-11-04 15:09:08 query migrate failed: VM 111 not running
2021-11-04 15:09:08 ERROR: online migrate failure - too many query migrate failures - aborting
2021-11-04 15:09:08 aborting phase 2 - cleanup resources
2021-11-04 15:09:08 migrate_cancel
2021-11-04 15:09:08 migrate_cancel error: VM 111 not running
drive-virtio0: Cancelling block job
2021-11-04 15:09:08 ERROR: VM 111 not running
2021-11-04 15:09:12 ERROR: migration finished with problems (duration 00:01:35)
TASK ERROR: migration problems

Vieln Dank für Eure Hilfe.

 

[mira]

Bitte den Output von pveversion -v posten.
Zusätzlich bitte auch die VM Config mittels qm config <VMID>.

 

[fabian]

und auf beiden seiten im journal schauen ob rund um den abbruch auffaellige eintraege sind..