ERROR: online migrate failure - VM xx qmp command 'migrate' failed - Nested VMX virtualization does not support live migration yet

dan.ger

Active Member
May 13, 2019
82
7
28
Hello,

we migrated a ceph Proxmox cluster from 5.4.1 to 6.0 with 3 nodes. Everything works fine, but live migration no longer works with nested virtualization activated and VMs with CPU type host. The three nodes are the same physical machines, nvmes, cpus, ram and so on.

When I migrate a vm live with cpu type host with nested virtualization turned on for every node, I got following error message:

ERROR: online migrate failure - VM xxx qmp command 'migrate' failed - Nested VMX virtualization does not support live migration yet

This error seems to belong to qemu-kvm package, I search a little bit and found a ticket for redhat that fixes the issue:
https://bugzilla.redhat.com/show_bug.cgi?id=1689216

The only workaround I found was:
1. Deactivate nested virtualization on physical nodes (that's not possible for me)
2. Change cpu type to kvm64 (Just a temporary workaround, cause I want to use all cpu features in a vm)

I think it depends on debian buster packages, is that right?

Kind regards,
Daniel
 
Last edited:
you can also choose the right vm cpu model = to your host cpu model. (According to bugzilla bugreport, the vmx flag blocking the live migration, only occur with host model).

host cpu model should be used only for specific case (like nested virtualization for example)
 
Thanks, but i just want to pass the Intel Xeon Gold 5138 into the VMs. Is ther way to diasble the vmx for vms like an additional parameter Cpu=host -vmx or something else?
 
Thanks, but i just want to pass the Intel Xeon Gold 5138 into the VMs.

Why not simply use Skylake cpu model . (it's xeon gold 6138 ?) ?
They are good reason to not use Host, as sometime some cpu flags are not supported/bugged /filtrered by qemu when choosing the correct cpu model vs host model.

Is ther way to diasble the vmx for vms like an additional parameter Cpu=host -vmx or something else?

with last proxmox versions, where you have differents cpu flags for spectre/meltdown in gui cpu advanced section, I think you should be able to add your own flags, editing /etc/pve/qemu-server/<vmid>.conf
 
  • Like
Reactions: ramrot
Hello,

thanks for the hint, but following line will cause an error:

cpu: host,flags=-vmx

Error:
flags: value does not match the regex pattern vm xxx - unable to parse value of 'cpu' - format error

Also the ui does not show all cpu types, as the command:
Code:
qemu-system-x86_64 -cpu ?

So I ended up to set all cpus to Skylake-Server, only for nested virtualized VMs I set cpu to host.

It seems that are only defined flags are allowed. Following comand ends with error:
Code:
qm set 100 -cpu cputype=host,flags=-vmx

400 Parameter verification failed.
cpu: invalid format - format error
cpu.flags: value does not match the regex pattern
 
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!