ERROR: online migrate failure - VM xx qmp command 'migrate' failed - Nested VMX virtualization does not support live migration yet

[dan.ger]

Hello,

we migrated a ceph Proxmox cluster from 5.4.1 to 6.0 with 3 nodes. Everything works fine, but live migration no longer works with nested virtualization activated and VMs with CPU type host. The three nodes are the same physical machines, nvmes, cpus, ram and so on.

When I migrate a vm live with cpu type host with nested virtualization turned on for every node, I got following error message:

ERROR: online migrate failure - VM xxx qmp command 'migrate' failed - Nested VMX virtualization does not support live migration yet

This error seems to belong to qemu-kvm package, I search a little bit and found a ticket for redhat that fixes the issue:
https://bugzilla.redhat.com/show_bug.cgi?id=1689216

The only workaround I found was:
1. Deactivate nested virtualization on physical nodes (that's not possible for me)
2. Change cpu type to kvm64 (Just a temporary workaround, cause I want to use all cpu features in a vm)

I think it depends on debian buster packages, is that right?

Kind regards,
Daniel

 

[spirit]

you can also choose the right vm cpu model = to your host cpu model. (According to bugzilla bugreport, the vmx flag blocking the live migration, only occur with host model).

host cpu model should be used only for specific case (like nested virtualization for example)

 

[dan.ger]

Thanks, but i just want to pass the Intel Xeon Gold 5138 into the VMs. Is ther way to diasble the vmx for vms like an additional parameter Cpu=host -vmx or something else?

 

[spirit]

Thanks, but i just want to pass the Intel Xeon Gold 5138 into the VMs.

Why not simply use Skylake cpu model . (it's xeon gold 6138 ?) ?
They are good reason to not use Host, as sometime some cpu flags are not supported/bugged /filtrered by qemu when choosing the correct cpu model vs host model.

Is ther way to diasble the vmx for vms like an additional parameter Cpu=host -vmx or something else?

with last proxmox versions, where you have differents cpu flags for spectre/meltdown in gui cpu advanced section, I think you should be able to add your own flags, editing /etc/pve/qemu-server/<vmid>.conf

 

[dan.ger]

Hello,

thanks for the hint, but following line will cause an error:

cpu: host,flags=-vmx

Error:
flags: value does not match the regex pattern vm xxx - unable to parse value of 'cpu' - format error

Also the ui does not show all cpu types, as the command:

qemu-system-x86_64 -cpu ?

So I ended up to set all cpus to Skylake-Server, only for nested virtualized VMs I set cpu to host.

It seems that are only defined flags are allowed. Following comand ends with error:

qm set 100 -cpu cputype=host,flags=-vmx

400 Parameter verification failed.
cpu: invalid format - format error
cpu.flags: value does not match the regex pattern