VMs crash on migration to host with downgraded CPU

[alyarb]

We are introducing newer servers to our environment with Xeon Gold 6148 in combination with our E5-2697v2 hosts. So now we have a mix of Ivy Bridge-EP and Skylake-IP.

I expected the requirement was to configure the VM's CPU to "IvyBridge" to fix the CPU flags, instruction set extensions, etc to a set that is compatible with both the older and newer hardware.

With the vCPU configured to "IvyBridge," the guest operating system correctly identifies the virtual CPU as "IvyBridge," and I can live migrate a VM from the Ivy Bridge Host to the Skylake host without any trouble. However, migrating BACK from a Skylake host to an Ivy Bridge host causes the VM to freeze, and needs to be forced down before it can be run again on that host.

Out of curiosity, I tried the same exercise with the vCPU configured as kvm64, and the behavior is the same. Even with the downgraded CPU flags I cannot live migrate to from newer hardware to older.

Please let me know what I'm doing wrong.

 

[bbgeek17]

https://forum.proxmox.com/threads/l...ween-higher-to-lower-frequencies-cpus.120766/
https://forum.proxmox.com/threads/s...e-migration-between-2-different-hosts.128244/

There are a few more similar threads.

There is no formal testing of live migration in heterogeneous environment. PVE developers highly recommend homogeneous for stable operations. If you have to use dis-similar hosts, make sure you use latest software, including kernel - likely an opt-in version over default.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[alyarb]

after apt update; apt install pve-kernel-6.2 and rebooting, I have 2 test nodes (of different architecture) running this kernel:
Linux 6.2.16-4-bpo11-pve #1 SMP PREEMPT_DYNAMIC PVE 6.2.16-4~bpo11+1

and pinning the virtual CPU to "IvyBridge" and they are able to migrate VMs successfully back and forth. Thank you.