[SOLVED] Different CPUs in a Cluster (AMD/CPU)

inno-forum

Member
Oct 28, 2021
60
3
13
30
We planned to expand our Cluster with a Host with Intel CPU.
And I have many questions.

1) If we use as CPU "kvm64" live migration will work, right? (Documentation "CPU Type")

2) If we have Host (AMD) failure and on the VM CPU "HOST" was set, the VM will be migrated to the new Host(Intel) will the VM works?
On Failure, the VM will be stopped and started, right? So the CPU type should be irrelevant.

Thanks :)
 
1) If we use as CPU "kvm64" live migration will work, right? (Documentation "CPU Type")
Yes it should, “kvm64” acts as a “lowest common denominator”.

2) If we have Host (AMD) failure and on the VM CPU "HOST" was set, the VM will be migrated to the new Host(Intel) will the VM works?
On Failure, the VM will be stopped and started, right? So the CPU type should be irrelevant.
Well if you migrate a VM between an AMD and an Intel host and the VMs CPU type is “Host” it may freeze or crash. The behavior he is not really predictable from the provided information here. We would not recommend relying on live migration between AMD and Intel hosts as this is generally not tested too much and often comes with the above-mentioned drawbacks. Please be aware that even if this works for a while, it could easily break with newer kernel or QEMU versions.
 
  • Like
Reactions: inno-forum
Yes it should, “kvm64” acts as a “lowest common denominator”.


Well if you migrate a VM between an AMD and an Intel host and the VMs CPU type is “Host” it may freeze or crash. The behavior he is not really predictable from the provided information here. We would not recommend relying on live migration between AMD and Intel hosts as this is generally not tested too much and often comes with the above-mentioned drawbacks. Please be aware that even if this works for a while, it could easily break with newer kernel or QEMU versions.
1)
The Documentation say
"If you care about live migration without security, or have mixed Intel/AMD cluster, choose the lowest compatible virtual QEMU CPU type."

We have an AMD EPYC 7302 and an Intel(R) Xeon(R) Gold 6336Y.
I tried the live migration with kvm64 and x86-64-v2-AES.
An Ubuntu Server will crash or will be not available (sometime directly after the migration, sometime after some Hour)
With Windows we had no Problem.

The behaviour with linux it is a bug? Do you know this Problem?

2)
If I use a QEMU CPU which flags should I activate to mitigate the Meltdown / Spectre vulnerability?

Have the x86-64-v2-AES the flags per default?

Thanks a lot :)
 
The behaviour with linux it is a bug? Do you know this Problem?
Going off of the provided information, I can't really say much. Can you please provide the VMs configuration as well as the output of pveversion --verbose.

If I use a QEMU CPU which flags should I activate to mitigate the Meltdown / Spectre vulnerability?
These bugs affect the actual CPUs that your system uses. The issues arise from modern CPU's usage of “speculative execution”. This does not impact QEMU or the virtual CPU type directly. So you will need to activate them on the host. There they should be activated by default. You may check which mitigations are active with lscpu. You can also check if you are impacted by any issue with spectre-meltdown-checker, you will need to install it with apt install spectre-meltdown-checker first.
 
  • Like
Reactions: inno-forum
Going off of the provided information, I can't really say much. Can you please provide the VMs configuration as well as the output of pveversion --verbose.
Hier the Information
These bugs affect the actual CPUs that your system uses. The issues arise from modern CPU's usage of “speculative execution”. This does not impact QEMU or the virtual CPU type directly. So you will need to activate them on the host. There they should be activated by default. You may check which mitigations are active with lscpu. You can also check if you are impacted by any issue with spectre-meltdown-checker, you will need to install it with apt install spectre-meltdown-checker first.

So I used the check.
  • Host AMD
    Everything is fine

  • Host Intel
    Everything is fine

  • VM on AMD Host with CPU Type "Host"
    Just one vulnerability (Photo1)

  • VM on AMD Host with CPU Type "x86-64-v2-AES"
    Just one vulnerability (Photo1----if I set the recommend flag for the vulnerability(PVE Doc) nothing change)

  • VM on Intel Host with CPU Type "Host"
    Everything is fine

  • VM on Intel Host with CPU Type "x86-64-v2-AES"
    8 vulnerabilities (if I set the recommend flag for the vulnerability(PVE Doc) the count will be 6)


    How can I solve it?
 

Attachments

  • pveversion_amd.txt
    2.5 KB · Views: 1
  • pveversion_intel.txt
    1.4 KB · Views: 0
  • vm.txt
    2.1 KB · Views: 2
  • AMD_x86-64-v2-AES.png
    AMD_x86-64-v2-AES.png
    40.2 KB · Views: 11
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!