VMs crash on migration to host with downgraded CPU

alyarb

Active Member
Feb 11, 2020
137
20
38
37
We are introducing newer servers to our environment with Xeon Gold 6148 in combination with our E5-2697v2 hosts. So now we have a mix of Ivy Bridge-EP and Skylake-IP.

I expected the requirement was to configure the VM's CPU to "IvyBridge" to fix the CPU flags, instruction set extensions, etc to a set that is compatible with both the older and newer hardware.

With the vCPU configured to "IvyBridge," the guest operating system correctly identifies the virtual CPU as "IvyBridge," and I can live migrate a VM from the Ivy Bridge Host to the Skylake host without any trouble. However, migrating BACK from a Skylake host to an Ivy Bridge host causes the VM to freeze, and needs to be forced down before it can be run again on that host.

Out of curiosity, I tried the same exercise with the vCPU configured as kvm64, and the behavior is the same. Even with the downgraded CPU flags I cannot live migrate to from newer hardware to older.

Please let me know what I'm doing wrong.
 
https://forum.proxmox.com/threads/l...ween-higher-to-lower-frequencies-cpus.120766/
https://forum.proxmox.com/threads/s...e-migration-between-2-different-hosts.128244/

There are a few more similar threads.

There is no formal testing of live migration in heterogeneous environment. PVE developers highly recommend homogeneous for stable operations. If you have to use dis-similar hosts, make sure you use latest software, including kernel - likely an opt-in version over default.


Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
 
after apt update; apt install pve-kernel-6.2 and rebooting, I have 2 test nodes (of different architecture) running this kernel:
Linux 6.2.16-4-bpo11-pve #1 SMP PREEMPT_DYNAMIC PVE 6.2.16-4~bpo11+1

and pinning the virtual CPU to "IvyBridge" and they are able to migrate VMs successfully back and forth. Thank you.
 
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!