HA Cluster and CPU requirements

[muekno]

Documentation says CPUs in HA cluster even cluster has to be the same vendor to migrate and run VMs without problems.
Question, is it enough to be the same vendor meaning all CPUS are Intel or are AMD or shouldn't it be the same CPU or at a minimum the same CPU family.

Thanks for clearing

Regards

Rainer

 

[adresner]

I can't say one way or another but if they are all AMD in the same class ie: Epyc and you choose a virtualized version of that CPU (not host), it seems logical that it would work?

 

[kellogs]

Documentation says CPUs in HA cluster even cluster has to be the same vendor to migrate and run VMs without problems.
Question, is it enough to be the same vendor meaning all CPUS are Intel or are AMD or shouldn't it be the same CPU or at a minimum the same CPU family.

Thanks for clearing

Regards

Rainer

I am using kvm64 for all my vm processor setting and so far I don’t have any problem migrating around the node.

 

[Ramalama]

You should simply select a cpu profile that all cpu's in your cluster supports in the VM's you wanna able to migrate.
If they have all the same cpu, or slightly different, like just more/less core versions, you can use "host" as Cpu Profile in your vm-settings, which will simply passthrough the cpu cores (with all features that the cpu supports)

Thats all.

 

[muekno]

So if I use "host" and configure my VM that their requirements are not above the weakest CPU everthing should work fine.

 

[Ramalama]

So if I use "host" and configure my VM that their requirements are not above the weakest CPU everthing should work fine.

Whatever you said, it doesn't makes sense.

Host = enables all cpu features, that the cpu supports on the Hypervisor the VM is running on.

If you set Host in vm-settings and migrate the VM to another host with a different CPU your VM will freeze.
If you reboot the VM, it will work again, but passthrough the cores of the new host, where you just migrated the VM to.
Live-Migration between different CPU's with "Host" as CPU-Profile in the VM-Settings will always freeze the VM.
It's like you take your Computer and Swap a completely different CPU, while its running.
It only works if you swap to the same CPU (Or a slightly different model from the same Plattform/Generation).

Slightly different only works with as example:
Genoa 9374F with Genoa 9474F or Genoa 9274F.
Or Ryzen 5800x with Ryzen 5900x.
But will freeze on live migration, if its a different cpu, even the same plattform/generation but for example one has IGPU and the other Don't.

So if you have different CPU's, you can use for example the x86_64_v2_aes Profile, if your CPUs are halfway actual.
You can use x86_64_v3 or V4 Profile, if all cpus in your cluster supports that profile.

If you find the Profile which all CPU's support with the most instructions, you get the most performance usually inside your VM and you can Live-Migrate.

I don't know how to simplify this.

You can use this script for example:
https://github.com/HenrikBengtsson/x86-64-level

It will tell you which Profile your cpu supports, if server 1 supports x86-64-v3, server 2 supports x86-64-v1 and server3 supports x86-64-v4, you have to use x86-64-v1.
Because all the higher Profiles support all instructions from the lower Profiles too.

Proxmox use x86-64-v2+aes as default, which is a good default to cover the most CPU's (old and new) and still have a relatively good instruction support.
But if you can use a higher/better Profile like v3/v4, you should, if all you cpus in your servers support it.

Cheers

 

[muekno]

Tanks for clearing

 

[VictorSTS]

Documentation says CPUs in HA cluster even cluster has to be the same vendor to migrate and run VMs without problems.

A small clarification: CPU type is important for Live Migration, not HA (except for HA's shutdown policy migrate, that uses Live Migration [1]). When HA kicks in, the VM is already off (the host that had it running is either off or unreachable and HA has already fenced it) and the VM will need to be started in the new host, making the CPU setting irrelevant.

I am using kvm64 for all my vm processor setting and so far I don’t have any problem migrating around the node.

Consider using another type compatible with your host(s) to get more CPU instructions available for your VMs, specially AES which is missing from kvm CPU types.

You can use this script for example:
https://github.com/HenrikBengtsson/x86-64-level

Thanks for this!

[1] https://pve.proxmox.com/wiki/High_Availability#ha_manager_shutdown_policy

 

[kellogs]

A small clarification: CPU type is important for Live Migration, not HA (except for HA's shutdown policy migrate, that uses Live Migration [1]). When HA kicks in, the VM is already off (the host that had it running is either off or unreachable and HA has already fenced it) and the VM will need to be started in the new host, making the CPU setting irrelevant.


Consider using another type compatible with your host(s) to get more CPU instructions available for your VMs, specially AES which is missing from kvm CPU types.


Thanks for this!

[1] https://pve.proxmox.com/wiki/High_Availability#ha_manager_shutdown_policy

Hello Victor,

Thank you for the information. Do you know if there is any tool which i could find out the lowest denominator which i could use as CPU and able to perform migration without any issue?

Thanks!

 

[VictorSTS]

Do you know if there is any tool which i could find out the lowest denominator which i could use as CPU and able to perform migration without any issue?

@Ramalama posted a script that seems useful for that very question. OTOH, just know your hardware, use the same microcode level [1] in all servers and test.

[1] https://pve.proxmox.com/wiki/Firmware_Updates

 

[kellogs]

@Ramalama posted a script that seems useful for that very question. OTOH, just know your hardware, use the same microcode level [1] in all servers and test.

[1] https://pve.proxmox.com/wiki/Firmware_Updates

Hello Victor,

I have managed to determine SandyBridge is good as vm CPU and only 3 nodes are throwing this error

host doesn't support requested feature: cpuid.01h:ecx.aes [bit 25] when doing migration to it.

I am going to try to update BIOS on these 3 nodes and see if it makes any difference or not.

Thank you

 

[kellogs]

I have upraded the motherboard BIOS and now these 3 nodes accept VMs which are created with CPU SandyBridge