[SOLVED] remote-migration & cloudinit

[erathia]

So i have two proxmox clusters, both have the access to the RBD storage of my first cluster, now i want to migrate a VM to test the possibility of migrating the whole thing to my new cluster...

After researching a litttle, i found i cant live-migrate to a remote host, via '"qm remote-migrate", i tried to migrate a basic windows VM and it worked like a charm. But my problems started when i tried to move a linux VM that has a cloudinit..

What i tried:

qm remote-migrate 120 120 'apitoken=PVEAPIToken=user@pam!migration=azerty654,host=10.197.241.57,fingerprint=XX:XX' --target-bridge vmbr0 --target-storage rbd --online 1

Here's what i got in return:

Establishing API connection with remote at 'XX.XX.XX.XX'
2024-06-10 20:08:26 remote: started tunnel worker 'UPID:px4:00152FD6:123006DF:6667411A:qmtunnel:1020:user@pam!migration:'
tunnel: -> sending command "version" to remote
tunnel: <- got reply
2024-06-10 20:08:26 local WS tunnel version: 2
2024-06-10 20:08:26 remote WS tunnel version: 2
2024-06-10 20:08:26 minimum required WS tunnel version: 2
websocket tunnel started
2024-06-10 20:08:26 starting migration of VM 120 to node 'px4' (XX.XX.XX.XX)
tunnel: -> sending command "bwlimit" to remote
tunnel: <- got reply
tunnel: -> sending command "bwlimit" to remote
tunnel: <- got reply
2024-06-10 20:08:27 found generated disk 'rbd:vm-120-cloudinit' (in current VM config)
2024-06-10 20:08:27 found local disk 'rbd:vm-120-disk-0' (attached)
2024-06-10 20:08:27 copying local disk images
2024-06-10 20:08:27 ERROR: no export formats for 'rbd:vm-120-cloudinit' - check storage plugin support!
2024-06-10 20:08:27 aborting phase 1 - cleanup resources
tunnel: -> sending command "quit" to remote
tunnel: <- got reply
2024-06-10 20:08:28 ERROR: migration aborted (duration 00:00:02): no export formats for 'rbd:vm-120-cloudinit' - check storage plugin support!
migration aborted

The exact same method worked in a VM that doesnt have a cloudinit, is there another way to do this without removing the cloudinit?
Also i dont understand why the VM copies the disk, since the disk stays on the same storage and location..

 

[fiona]

Hi,

2024-06-10 20:08:28 ERROR: migration aborted (duration 00:00:02): no export formats for 'rbd:vm-120-cloudinit' - check storage plugin support!
migration aborted

the cloudinit disk would need to be copied via the storage layer (other disk are live-migrated via QEMU's drive-mirror), but unfortunately, there is no export/import method implemented yet for the RBD plugin. It was not needed in the past, before remote-migration was introduced.

The exact same method worked in a VM that doesnt have a cloudinit, is there another way to do this without removing the cloudinit?
Also i dont understand why the VM copies the disk, since the disk stays on the same storage and location..

It might be the same underlying storage, but each cluster has it's own storage configuration for it. See also: https://bugzilla.proxmox.com/show_bug.cgi?id=4928

 

[static302]

Same happens with cloudinit drives located on zfs storages.
Do you have any plans for implementing cloudinit drives live migration?f

 

[fiona]

Hi,

Same happens with cloudinit drives located on zfs storages.
Do you have any plans for implementing cloudinit drives live migration?

are you specifying a different target storage? It should work if you have ZFS as both source and target. It's true that ZFS export/import in our storage layer could be extended, it currently only can export/import to/from other ZFS type storages.

 

[static302]

Hi,

are you specifying a different target storage? It should work if you have ZFS as both source and target. It's true that ZFS export/import in our storage layer could be extended, it currently only can export/import to/from other ZFS type storages.

It doesn’t matter whether it’s a ZFS-to-Ceph or ZFS-to-ZFS migration. However, our storages are encrypted, so if you rely on ZFS functionality for the migration, it won’t work with encrypted storages, afaik.

 

[fiona]

It doesn’t matter whether it’s a ZFS-to-Ceph or ZFS-to-ZFS migration. However, our storages are encrypted, so if you rely on ZFS functionality for the migration, it won’t work with encrypted storages, afaik.

That's a different issue tracked in https://bugzilla.proxmox.com/show_bug.cgi?id=2350