Question: Experience Using AMD EPYC 7313 CPU for a Production Proxmox System?

[wayfarerdev]

Hello,

Does anyone have experience using AMD EPYC Milan-family CPUs, specifically the 7313, with Proxmox in a production system?

One system I am considering is an ASUS ESC-4000A-E11. This is listed as compatible via the Ubuntu compatibility list.

The ServeTheHome site noted an excellent experience with EPYC CPUs and Proxmox several years ago. However, I have encountered some forum posts (here, here, and here) noting more recent problems with Proxmox and EPYC CPUs.

My prior experience with Proxmox is exclusively on Intel Xeon servers.

Thank you in advance for any guidance and feedback!

 

[ness1602]

I have one Epyc server right now,and it works really great, so my recommendation is to use it of course. What type of storage do you plan with it ?

 

[wayfarerdev]

@ness1602,

Thank you for the feedback. Right now, I'm looking at using a mix of flash and traditional spinning disks with paired drives in ZFS mirrors.

 

[cwt]

We‘re running several Epyc PVE systems (Milan and Genoa) without any problems.

Spinning disks? You should reconsider this and think about Enterprise SSDs or NVME. Normal hdds perform terrible on ZFS, especially as storage for virtual OS disks. They are ok for pure „data grave“ storage but nothing more.

 

[UdoB]

My production cluster has two machines with dual EPYC 7443 from Dell. Works great.

Mixing with legacy Intel is/was a problem though. Live migration between those architectures did not work (half a year ago), even with "kvm64" CPU type. Actually I am unsure about the current state of this topic with Kernel 6.2.

As I do have two EPYC and two Xeon nodes I keep my VMs in these halfs of the cluster and only migrate offline if required.

Best regards

 

[wayfarerdev]

We‘re running several Epyc PVE systems (Milan and Genoa) without any problems.

Spinning disks? You should reconsider this and think about Enterprise SSDs or NVME. Normal hdds perform terrible on ZFS, especially as storage for virtual OS disks. They are ok for pure „data grave“ storage but nothing more.

@cwt Thank you for this feedback. I expect most of the storage in this configuration would be enterprise SSDs, but we may have some spinning disks as well to supplement.

 

[wayfarerdev]

My production cluster has two machines with dual EPYC 7443 from Dell. Works great.

Mixing with legacy Intel is/was a problem though. Live migration between those architectures did not work (half a year ago), even with "kvm64" CPU type. Actually I am unsure about the current state of this topic with Kernel 6.2.

As I do have two EPYC and two Xeon nodes I keep my VMs in these halfs of the cluster and only migrate offline if required.

Best regards

@UdoB Thank you for this feedback; it is good to know your strategy for dealing with a mixed Intel and AMD environment. Our use case rarely requires live migration, so I will likely follow a similar approach of offline migration as needed.