qm remote-migrate: failed to handle 'config' command - only root can set 'args' config

[Derock]

I am building up a new proxmox cluster and would like to migrate one of my MacOS VMs to the new cluster.

qm remote-migrate 129 129 'apitoken=PVEAPIToken=root@pam!migrate=<redacted>,host=10.0.0.23,fingerprint=<...>' --target-storage zfs-vms --target-bridge vmbr0 --online

I am using the `--online` directive because I am migrating to a server that uses ZFS and the old cluster did not use ZFS, so I get an error when doing offline migrations.

Upon running the command, migration errors on the following:

2023-12-04 20:40:08 ERROR: migration aborted (duration 00:00:04): error - tunnel command '{"cmd":"config","conf":"agent: 1\nargs: -device isa-applesmc,osk=\"ourhardworkbythesewordsguardedpleasedontsteal(c)AppleComputerInc\" -smbios type=2 -device usb-kbd,bus=ehci.0,port=2 -global nec-usb-xhci.msi=off -global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=off -cpu host,vendor=GenuineIntel,+invtsc,+hypervisor,kvm=on,vmware-cpuid-freq=on\nballoon: 0\nbios: ovmf\nboot: order=virtio0;net0\ncores: 4\ncpu: Haswell\ncpuunits: 1024\nefidisk0: zfs-vms:vm-129-disk-1,efitype=4m,format=raw,size=528K\nlock: migrate\nmachine: q35\nmemory: 8192\nmeta: creation-qemu=7.2.0,ctime=1685063312\nname: ventura\nnet0: virtio=<cool mac address>,bridge=vmbr0,firewall=1\nnuma: 0\nonboot: 1\nostype: other\nscsihw: virtio-scsi-pci\nsmbios1: uuid=<uuid>\nsockets: 1\nvga: vmware\nvirtio0: zfs-vms:vm-129-disk-0,cache=unsafe,discard=on,format=raw,iothread=1,size=64G\nvmgenid: <uuid>\n","firewall-config":null}' failed - failed to handle 'config' command - only root can set 'args' config
migration aborted

Is there a solution to this?
The API key has Administrator permissions on the new cluster.

 

[fiona]

Hi,
I think you'll need to temporarily remove the custom args from the VM configuration. You can set the Freeze CPU on startup in the VM's options to prevent it from actually booting if booting wouldn't work without the args.

Since one can do anything with args, it's restricted to root@pam only (IIRC even tokens for root@pam can't use it). There is no superuser privilege currently, once the following feature request is implemented, you could also use that: https://bugzilla.proxmox.com/show_bug.cgi?id=2582

 

[TOC]

Hello, any news here already? I have the same issue with the "config" command.

Input:
qm remote-migrate 3023 3023 \
'host=192.168.xxx.xxx,apitoken=PVEAPIToken=root@pam!transfer=<redacted>,fingerprint=<redacted>' \
--target-bridge=vmbr0 --target-storage SSD8-Storage-PRX036:SSD8-Storage-PRX027

Output:
2024-01-23 14:23:53 ERROR: migration aborted (duration 00:18:13): error - tunnel command '{"cmd":"config","conf":"agent: 1\nballoon: 0\nboot: cdn\nbootdisk: virtio0\ncores: 2\ncpu: host\nide2: none,media=cdrom\nlock: migrate\nmachine: pc-i440fx-7.2\nmemory: 32768\nname: xxx-xxxx023\nnet0: virtio=B2:78:5C:xx:xx:xx,bridge=vmbr0\nonboot: 1\nostype: win10\nrunningmachine: pc-i440fx-4.0\nscsihw: virtio-scsi-single\nsmbios1: uuid=a5b7f4c8-5f7a-40a8-9e00-ea65d3a54874\nsockets: 2\nstartup: up=30\nvirtio0: SSD8-Storage-PRX027:vm-3023-disk-0,cache=writethrough,discard=on,iothread=1,size=150G\nvirtio1: SSD8-Storage-PRX027:vm-3023-disk-1,cache=writethrough,discard=on,iothread=1,size=350G\nvmgenid: 66d4efea-52c1-4c98-985b-59327a4e97a1\n","firewall-config":null}' failed - failed to handle 'config' command - only root can set 'runningmachine' config
migration aborted

Proxmox Version on both Hosts: 7.4-16

 

[fiona]

Hi,

Hello, any news here already? I have the same issue with the "config" command.

Input:
qm remote-migrate 3023 3023 \
'host=192.168.xxx.xxx,apitoken=PVEAPIToken=root@pam!transfer=<redacted>,fingerprint=<redacted>' \
--target-bridge=vmbr0 --target-storage SSD8-Storage-PRX036:SSD8-Storage-PRX027

Output:
2024-01-23 14:23:53 ERROR: migration aborted (duration 00:18:13): error - tunnel command '{"cmd":"config","conf":"agent: 1\nballoon: 0\nboot: cdn\nbootdisk: virtio0\ncores: 2\ncpu: host\nide2: none,media=cdrom\nlock: migrate\nmachine: pc-i440fx-7.2\nmemory: 32768\nname: xxx-xxxx023\nnet0: virtio=B2:78:5C:xx:xx:xx,bridge=vmbr0\nonboot: 1\nostype: win10\nrunningmachine: pc-i440fx-4.0\nscsihw: virtio-scsi-single\nsmbios1: uuid=a5b7f4c8-5f7a-40a8-9e00-ea65d3a54874\nsockets: 2\nstartup: up=30\nvirtio0: SSD8-Storage-PRX027:vm-3023-disk-0,cache=writethrough,discard=on,iothread=1,size=150G\nvirtio1: SSD8-Storage-PRX027:vm-3023-disk-1,cache=writethrough,discard=on,iothread=1,size=350G\nvmgenid: 66d4efea-52c1-4c98-985b-59327a4e97a1\n","firewall-config":null}' failed - failed to handle 'config' command - only root can set 'runningmachine' config
migration aborted

Proxmox Version on both Hosts: 7.4-16

can you share the VM configuration cat /etc/pve/qemu-server/3023.conf? I don't think the runningmachine option should be set in the configuration if the machine isn't currently supsended (in which case migration is not supported/implemented). Did you ever remove the suspended state volume manually or similar?

 

[TOC]

Hi,

can you share the VM configuration cat /etc/pve/qemu-server/3023.conf? I don't think the runningmachine option should be set in the configuration if the machine isn't currently supsended (in which case migration is not supported/implemented). Did you ever remove the suspended state volume manually or similar?

Hello,

here are the config for the 3023 machine:

agent: 1
balloon: 0
boot: cdn
bootdisk: virtio0
cores: 2
cpu: host
hostpci0: 0000:af:00
ide2: none,media=cdrom
machine: pc-i440fx-7.2
memory: 32768
name: xxx-xxxx
net0: virtio=xx:xx:xx:xx:xx:xx,bridge=vmbr0
numa: 1
onboot: 1
ostype: win10
runningmachine: pc-i440fx-4.0
scsihw: virtio-scsi-single
smbios1: uuid=a5b7f4c8-5f7a-40a8-9e00-ea65d3a54874
sockets: 2
startup: up=30
virtio0: SSD8-Storage-xxxx:vm-3023-disk-0,cache=writethrough,discard=on,iothread=1,size=150G
virtio1: SSD8-Storage-xxxx:vm-3023-disk-1,cache=writethrough,discard=on,iothread=1,size=350G
vmgenid: a7007ed6-9bab-4ca1-a1d7-4d48b8684786

I shutdown the machine before i tried to move it with qm remote-migrate. I also removed the hostpci0 mapping.

As far as i know, i never tried to remove a suspended state volume manually or similar. In the past the systems where on Proxmox 6.4. Could this be the reason?

I took over the environment from my predecessor. What is this "runningmachine" doing? Should i just remove it and try it again?

Thank you in advance.

 

[fiona]

I took over the environment from my predecessor. What is this "runningmachine" doing? Should i just remove it and try it again?

Yes. It is used for snapshots and when hibernating to remember which machine version (because different ones are not compatible see section Machine Version) was used by the VM when the VM state was saved. But for migration, the current machine version is sent to the target machine differently, not saved in the config: (forcemachine in https://git.proxmox.com/?p=qemu-ser...cd4c107f23fe8c42e186f501a83e7c3;hb=HEAD#l1070 )

 

[TOC]

Yes. It is used for snapshots and when hibernating to remember which machine version (because different ones are not compatible see section Machine Version) was used by the VM when the VM state was saved. But for migration, the current machine version is sent to the target machine differently, not saved in the config: (forcemachine in https://git.proxmox.com/?p=qemu-ser...cd4c107f23fe8c42e186f501a83e7c3;hb=HEAD#l1070 )

Hello,

after i removed the "runningmachine" parameter from the qemu config the migration works without any problems.

Thank you Fiona for the quick help.