[SOLVED] Clustered node hardware: how similar is good enough?

[CZappe]

I'm looking to build out a set of custom Supermicro servers that we procured a couple of years ago with additional servers to create a new Proxmox cluster. Right now I'm getting quotes from different vendors for the new hardware. What I'm finding, perhaps unsurprisingly, is that the hardware we purchased a couple years ago (CPU, RAM, NICs) has since undergone incremental changes, and that having a server built today means that a CPU from a specific vendor/line will be usually be a different generation. Here's an example:

CPU (ca. 2022): AMD EPYC (Rome) 7302, 3.0GHz, 16C/32T, 155W​

CPU (ca. 2024): AMD EPYC (Genoa) 9124, 3.0GHz, 16C/32T, 200W​

​

These CPUs are from the same vendor and product line but aren't really the same. Similarly, our original servers are running DDR4-3200 RAM and the new systems are getting configured with DDR5-4800, since the newer CPUs support it. That's great and all, but my sense is that there's really no advantage to integrating a newer CPU or faster RAM into a cluster that is limited by older equipment. Also, we'd be paying more for the upgrade but I'm wary of potentially creating headaches for ourselves in terms of interoperability between the clustered nodes.

My question is, how identical do clustered nodes need to be to each other, and how much variance can there be between nodes before things start getting squirrely? Should I press for server specifications that match the specs of our existing hardware?

 

[cheiss]

Hi,

My question is, how identical do clustered nodes need to be to each other, and how much variance can there be between nodes before things start getting squirrely?

Depends on your usage. If you intend to use live-migration/HA, you must take care to choose the correct CPU type for VM.
See also the CPU type section in our admin guide, where all of this is explained in detail.

In your case, there are different EPYC-Rome-v* models available by default - the correct one depends on the exact CPU model. If you choose Rome as CPU type, that also cover compatibility with the newer Genoa models.

Other than that, there are no arbitrary restrictions w.r.t. to hardware in a cluster. So with a bit of care, there should be no problems.

 

[CZappe]

Thanks for the clarification! Indeed, we are interested in the live-migration/HA capabilities of Proxmox clustering. The ability to specify a "least common denominator" for CPU model is exactly what I was looking for. I appreciate it!

 

[D 0 N M 3 G A]

If you are faced with a situation where you cannot use matching hardware across your nodes, will using the generic CPU modes work? If you use those generic modes, how much performance loss is normal to expect?