Move Disk hangs forever in version 8.2.4

BigBrodeur

New Member
Sep 1, 2024
8
2
3
When i say forever, it means many many hours, nothing happens.

I inadvertently created a couple of VMs using my NAS network drive instead of local-lvm that I would like to move these back to local-LVM

No go through the GUI (hangs forever) and no go through command line either using:
qm move_disk VMID source_disk target_storage

which results in command alot like the one that is launched by the GUI:
'/usr/bin/qemu-img convert -p -n -f qcow2 -O raw /mnt/pve/ProxMoxShare/images/110/vm-110-disk-0.qcow2 zeroinit:/dev/pve/vm-110-disk-0'

I observed that while this command tries to run (and do nothing, just sits there forever) there is no activity on this network leg at all on my managed switch and also by doing a pvesm status, my Network Share becomes inactive. Come back active when i stop the command from the gui.
Network share is on a Qnap that has a capacity of 800-900MB/s transfer speed so this is not the bottleneck.

I tried to fool the system (during one full day) trying to bring my image to "the box"
1. adding a USB key and format it with proxmox filing system
2. upload a qcow2 image (by going around the GUI that won't accept anything else than iso file so temporarily renaming it to iso)
3. Added a new lvm disk on the VM
4. back to cmd line, rename iso to qcow2 and initiated the above command (convert) over that disk -> It worked but the result Won't boot
5. It tried to go back to my initial network share, no boot anymore. I would guess that by mounting it again, it was too stoopid and created an empty share instead of reusing the one already there :)
6. Fortunately i had backed up the original qcow2 image, so i copy it over the network share, this won't boot again :)
7. I still have that iso renamed to qcow2 into /mnt/pve/USB/template/iso

but regardless what i try, it doesn't seem to go. I don't understand why it works for other ppl.

I read a thread saying that the default option for network shares was put to async_io uring and that could cause the hang-up. But i also tried to change that successively to 'native' and 'threads' - the result is the same. GUI will hand forever.

Remember when GUI hangs, pvesm status becomes inactive and as soon as i stop the GUI it comes back.
Also it seems that if i try to access this network share from any other node (i have 4 total) when it is in this state, it is "kaput" ! untill i stop the initiated move in the origin GUI.

I will go out on a limb here and say "maybe" this function need refinement. It is bizarre that it simply hangs up forever without giving any error.
But i saw some ppl on youtube having it work. Me it's a no go. I think i will re-create all the VMs from scratch but it is very limiting to be 'stucked forever' where you initially put the disk, becaus 1 error and you are doomed.
 
I tried more tests today. In all cases i did not check the 'move' checkbox.
1. Moving storage from local-lvm (NvME RAW) to USB (Qcow2) -> Works, VM boots.
2. Moving storage from USB (Qcow) -> NAS (Qcow2) Works, VM boots.
3. Moving storage from NAS (Qcow2) to local-lvm (again, RAM) , transfer never starts, window stay stuck there just like i reported above.
 
Last edited:
I tried a couple more tests and check the 'move' checkbox.
1. Moving storage from local-lvm (NvME RAW) to USB (Qcow2) -> Works, VM boots.
2. Moving storage from USB (Qcow) -> local-lvm (NvME RAW) Works, VM boots.

So it seems the problem really is moving from NAS shared storage to anywhere else...
 
I also tried to shutdown all my noded except the one on wich the VM i am trying to move disk storage is on, to see if this could be related to the fact that this is shared storage.
1. Shut down all nodes except the one
2. Set cluster expectations to be able to test / start this vm:
pvecm expected 1
3. Start / stop the VM with storage attached to local-lvm -> OK
4. Move VM storage to ProxMox Share on NAS, OK, VM boots and shutdown
5. Trying to move the storage back from the NAS to loval lvm - no go. Process will never start.
 
Hey, did you ever find a solution for that? Because I'm just stuck with a pretty similar issue. After exactly 4.2TB the transfer seemingly stops. Going from 400mb/s from the NAS NFS share to 0.
 
Hey, did you ever find a solution for that? Because I'm just stuck with a pretty similar issue. After exactly 4.2TB the transfer seemingly stops. Going from 400mb/s from the NAS NFS share to 0.
ask your nas again and check big filetransver from your desktop pc to your nas.
Maybe you have a big video file?

i use e.g. rsync to my zfs nas based an Proxmox VE and store there all my raw video files.
 
Last edited:
  • Like
Reactions: news
On this NAS are multiple VMDK disks over 4TB. Moved there with VMware. That makes me confident that the NAS does not have an issue with files over 4TB. And the PURE-Fibre-Thick-LVM volumes also have multiple disks over 4TB.
It is a bit curious that it stops after 4TB. But its after 4.2, so he is already over the "limit". The same volume mounted to VMware shows a 16TB file limit. Which is also what QNAP says: https://www.qnap.com/en/how-to/faq/...size-of-a-single-file-i-can-save-onto-the-qts

Edit: Actually the thick-LVM does NOT have disks over 4TB. It has VMs with an overall size over 4TB, but the invidivual disks are below 4TB. So that could be a reason, couldn't find the limitations of Thick-LVM yet.
Edit2: On the other hand the LVM volumes are all 15TB, only the VMs on the volumes are not. So I would assume that if you can create a volume with 15TB, you can put a 15TB VM on it.
 
Last edited:
it stops after 4TB. But its after 4.2, so he is already over the "limit".
I am not convinced!

4 binary TB (correct: 4 TiB) is 4398046511104 Bytes. And it is 4294967296 KILOBytes. The last one is possibly used to show wrongly 4.2 TB...
 
Is it possible that LVM does not take VMDK's? Just tried to create a 1GB vmdk on a test VM and it says:

failed to update VM 108: unsupported format 'vmdk' at /usr/share/perl5/PVE/Storage/LVMPlugin.pm line 697. (500)

Now the question is, if I convert during the disk migration, will that make a difference. Does he read from the source and write qcow to the destination. Or does he copy first and then convert.
 
Last edited:
it will convert on the fly.. please provide more details about your storage and a full task log.
 
Hi,
Is it possible that LVM does not take VMDK's? Just tried to create a 1GB vmdk on a test VM and it says:

failed to update VM 108: unsupported format 'vmdk' at /usr/share/perl5/PVE/Storage/LVMPlugin.pm line 697. (500)
no, vmdk is not supported on top of LVM. Only raw, and if using snapshots-as-volume-chain, qcow2.