Can't migrate running VM which uses mapped devices, but works with USB port direct

[alpharesearch]

Hello,

I just upgraded to V8.0.3 to start using the new resource mappings function. At least it sounded like something useful for my situation. But it's not working like when I just set USB ports directly on the VM. Maybe I don't understand the feature completely but it didn't work out as expected.

I have two identical NUCs with identical USB devices plugged in, the first node is dia and the second is io. My VM with ID 200 is a home assistant installation. When I use USB ports (don't use resource mappings function) I'm able to migrate the VM. Not via GUI, but it works via shell with the force option:

From shell on io:
qm migrate 200 dia --online --force --with-local-disks

From shell on dia:
qm migrate 200 io --online --force --with-local-disks

I wasn't aware that resource mappings vs USB port direct was not the same and I still had the resource mappings fuction and HA active. Normally the 200 VM runs on io, but this morning I found it on dia. Not sure what happened, but I guess the HA must have moved it for some reasons unknown to me.

But I was not able to move it back with the commands that used to work. Shutdonw of VM and offline moving worked. Here is what I get with the resource mapping enabled:

root@io:~# qm migrate 200 dia --online --force --with-local-disks
Use of uninitialized value in pattern match (m//) at /usr/share/perl5/PVE/QemuServer.pm line 2687.
Use of uninitialized value in pattern match (m//) at /usr/share/perl5/PVE/QemuServer.pm line 2687.
2023-06-30 08:32:30 ERROR: migration aborted (duration 00:00:01): can't migrate running VM which uses mapped devices: usb1, usb0
migration aborted

After this happened I set the two USB ports back the old way and now I can freely migrate the VM with the command again, but how was HA able to move the VM with resource mappings active?

Would be good to get this functionality with resource mappings working or was this functionality deliberately held back or do I miss some parameter on my command?

Thanks,
Markus

 

[dcsapak]

hi,

i don't think live migration with passed through usb devices was deliberate, this cannot work in general (e.g. each device has an internal state that is not synced, but e.g. the guest driver relies upon..., imagine a usb drive with different filesystems on it, what would happen on the target if it's migrated in the middle of a write?)

so we left that out deliberately for mapped usb-devices until we have some solution to make live-migration working properly (e.g. with unplugging before and plugging after the migration again)

in any case, could you post the vm config?

as for ha, if a node fenced itself, ha can 'steal' a vm, which is just moving the config file and restarting on the node. This is intended to work with the mappings too, since it does not involve live migration

EDIT: in any case, you can open a bugreport here: https://bugzilla.proxmox.com
we might want to introduce a flag to allow live-migrating (with --force) in that case when the admin knows what he's doing...