VM and CT migration

[calvinstreeting]

for some reason every time i try and migrate a CT/VM it hanges and never finishes.. any ideas ?

https://192.168.0.11:8006/#

Task viewer: CT 101 - Migrate (pragueabsentdream ---> tokyo)

OutputStatus

Stop

Download
2024-06-15 14:07:57 starting migration of CT 101 to node 'tokyo' (192.168.0.11)
2024-06-15 14:07:57 found local volume 'local-lvm:vm-101-disk-0' (in current VM config)
2024-06-15 14:07:59 Logical volume "vm-101-disk-0" created.
2024-06-15 14:08:02 253427712 bytes (253 MB, 242 MiB) copied, 3 s, 84.5 MB/s
2024-06-15 14:08:05 603389952 bytes (603 MB, 575 MiB) copied, 6 s, 101 MB/s
2024-06-15 14:08:08 953483264 bytes (953 MB, 909 MiB) copied, 9 s, 106 MB/s
2024-06-15 14:08:11 1303576576 bytes (1.3 GB, 1.2 GiB) copied, 12 s, 109 MB/s
2024-06-15 14:08:14 1653538816 bytes (1.7 GB, 1.5 GiB) copied, 15 s, 110 MB/s
2024-06-15 14:08:17 2003632128 bytes (2.0 GB, 1.9 GiB) copied, 18 s, 111 MB/s
2024-06-15 14:08:20 2353725440 bytes (2.4 GB, 2.2 GiB) copied, 21 s, 112 MB/s
2024-06-15 14:08:23 2703687680 bytes (2.7 GB, 2.5 GiB) copied, 24 s, 113 MB/s
2024-06-15 14:08:26 3053780992 bytes (3.1 GB, 2.8 GiB) copied, 27 s, 113 MB/s
2024-06-15 14:08:29 3403874304 bytes (3.4 GB, 3.2 GiB) copied, 30 s, 113 MB/s
2024-06-15 14:08:32 3753836544 bytes (3.8 GB, 3.5 GiB) copied, 33 s, 114 MB/s
2024-06-15 14:08:35 4103929856 bytes (4.1 GB, 3.8 GiB) copied, 36 s, 114 MB/s
2024-06-15 14:08:38 4454023168 bytes (4.5 GB, 4.1 GiB) copied, 39 s, 114 MB/s
2024-06-15 14:08:41 4803985408 bytes (4.8 GB, 4.5 GiB) copied, 42 s, 114 MB/s
2024-06-15 14:08:44 5154078720 bytes (5.2 GB, 4.8 GiB) copied, 45 s, 115 MB/s
2024-06-15 14:08:47 5504172032 bytes (5.5 GB, 5.1 GiB) copied, 48 s, 115 MB/s
2024-06-15 14:08:50 5854134272 bytes (5.9 GB, 5.5 GiB) copied, 51 s, 115 MB/s
2024-06-15 14:08:53 6204227584 bytes (6.2 GB, 5.8 GiB) copied, 54 s, 115 MB/s
2024-06-15 14:08:56 6554320896 bytes (6.6 GB, 6.1 GiB) copied, 57 s, 115 MB/s
2024-06-15 14:08:59 6904283136 bytes (6.9 GB, 6.4 GiB) copied, 60 s, 115 MB/s
2024-06-15 14:09:09 8071086080 bytes (8.1 GB, 7.5 GiB) copied, 70 s, 115 MB/s
2024-06-15 14:09:13 131072+0 records in
2024-06-15 14:09:13 131072+0 records out
2024-06-15 14:09:13 8589934592 bytes (8.6 GB, 8.0 GiB) copied, 74.7063 s, 115 MB/s

 

[calvinstreeting]

ok.. so after leacing it for 1hr

024-06-15 14:09:13 131072+0 records out
2024-06-15 14:09:13 8589934592 bytes (8.6 GB, 8.0 GiB) copied, 74.7063 s, 115 MB/s
2024-06-15 15:01:45 successfully imported 'local-lvm:vm-101-disk-0'

 

[UdoB]

Just guessing: "Buffer-Bloat" and write-caches.

Your destination is able to write only ~2 MB/s. The question of course is: why? I have not used LVM for more than a decade or so and never in PVE context, so I have no experience here.

Is the destiantion busy, so that some processes fight for IO-time? Check the destination LVM. Is it healthy? Is an mdraid below it? Is it degraded? Are the SMART-values of all drives ok? Rotating rust or SSD? Topology?

Again: just pure guessing, to give you some hints...

 

[leesteken]

Does the destination drive use QLC flash memory or SMR? Then is can take a long time for the writes to actually finish.

 

[calvinstreeting]

Just guessing: "Buffer-Bloat" and write-caches.

Your destination is able to write only ~2 MB/s. The question of course is: why? I have not used LVM for more than a decade or so and never in PVE context, so I have no experience here.

Is the destiantion busy, so that some processes fight for IO-time? Check the destination LVM. Is it healthy? Is an mdraid below it? Is it degraded? Are the SMART-values of all drives ok? Rotating rust or SSD? Topology?

Again: just pure guessing, to give you some hints...

destination dell r-630... bu souce is older hardware (as migrating, bakingup for home server swap..) so could be hops and lan ... etc.... alot between A-B.. but it did get there in the end.. while i did other stuffs

 

[gfngfn256]

while i did other stuffs

Other stuff on node "Tokyo". That would easily explain what's going on.

However as UdoB points out - you don't really provide much HW info about the storage target.