disk_move shovels much more data than disk contains, finalizing repeats and takes a very long time

IZT

New Member
Jul 13, 2024
5
0
1
Hi, I did not find something similar via search engine: We lately have problems when moving disks, the process continues for more than 100% of data, moves way more data than the disk size accounts for, then the finishing seems to repeat certain steps all over again and again, like so:

Code:
qm move_disk 116 virtio0 iztserver2nvme --delete

the disk in question is 32GB big, but the log says:

Code:
create full clone of drive virtio0 (iztserver3nvme:116/vm-116-disk-0.qcow2)
Formatting '/mnt/pve/iztserver2nvme/images/116/vm-116-disk-0.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off preallocation=metadata compression_type=zlib size=34359738368 lazy_refcounts=off refcount_bits=16
drive mirror is starting for drive-virtio0
drive-virtio0: transferred 0.0 B of 32.0 GiB (0.00%) in 0s
drive-virtio0: transferred 807.2 MiB of 32.0 GiB (2.46%) in 1s

[... 46 similar lines deleted ...]

drive-virtio0: transferred 31.8 GiB of 32.0 GiB (99.38%) in 49s
drive-virtio0: transferred 32.1 GiB of 32.1 GiB (100.00%) in 50s
drive-virtio0: transferred 32.2 GiB of 32.2 GiB (100.00%) in 51s
drive-virtio0: transferred 32.3 GiB of 32.3 GiB (100.00%) in 52s
drive-virtio0: transferred 32.4 GiB of 32.4 GiB (100.00%) in 53s
drive-virtio0: transferred 32.5 GiB of 32.5 GiB (100.00%) in 54s
drive-virtio0: transferred 32.6 GiB of 32.6 GiB (100.00%) in 55s
drive-virtio0: transferred 32.6 GiB of 32.6 GiB (100.00%) in 56s
drive-virtio0: transferred 32.7 GiB of 32.7 GiB (100.00%) in 57s
drive-virtio0: transferred 32.8 GiB of 32.8 GiB (100.00%) in 58s

[so far so good, but then...]

drive-virtio0: transferred 32.9 GiB of 32.9 GiB (99.99%) in 59s
drive-virtio0: transferred 33.0 GiB of 33.0 GiB (100.00%) in 1m

[... 127 similar lines deleted ...]

drive-virtio0: transferred 45.0 GiB of 45.1 GiB (100.00%) in 3m 9s
drive-virtio0: transferred 45.1 GiB of 45.1 GiB (100.00%) in 3m 10s
drive-virtio0: transferred 45.2 GiB of 45.2 GiB (100.00%) in 3m 11s, still busy
all 'mirror' jobs are ready
drive-virtio0: Completing block job...
drive-virtio0: Completed successfully.
all 'mirror' jobs are ready

[... 236 similar lines deleted ...]

all 'mirror' jobs are ready
drive-virtio0: Completing block job...
drive-virtio0: Completed successfully.
drive-virtio0: mirror-job finished
TASK OK

In one case it took more time than a coffee break to complete such a small disk_move.

Is this a known phenomenon? Anything we can do about?

Thanks, Gregor
 
Hi,
that means that the guest is writing new data more quickly to the disk than can be transferred to the target. Try to reduce IO inside the guest or move the disk while the guest is shut down to mitigate the issue.