Request for Hardware Recommendations

[thomas-hn]

Hello,

at the moment I am thinking about shifting my Proxmox VE to a new server.

First some information about my system running at the moment:

• Mainboard: GA-MA69G-S3H
• CPU: Intel Core i5-4690S (old CPU model from the year 2014)
• The system runs with 7 VMs and 4 LXCs at around 40% CPU load.
• The 4 cores (only 4C/4T) are clocked with 3.2 GHz.
• 32 GB RAM (almost completely used)

Requirements for the new system:

• Power Consumption
  • The overall system power consumption should be at a maximum of around 50 - 60 Watt. This is important because of the very high energy costs in Germany.
  • A) Can we reach this power consumption range with my requirements?
• CPU
  • Definitely a newer CPU generation than my current one.
  • At least 8 cores.
  • B) Will Hyperthreading be beneficial for virtualization? If so, maybe 8C/16T?
  • Some more L2/L3 Cache for the virtualization.
  • C) Is it possible to handle the load of the old i5-4690S CPU with a newer CPU model in almost Idle state and very low power consumption?
  • I have really problems in getting a feeling on how the system load of a new CPU will be if I apply the same VMs and LXCs of my current system to the new system.
  • A server-grade CPU if possible.
  • D) Which CPU do you recommend?
• Mainboard
  • A board from Supermicro or Asrock Rack.
  • IPMI
  • Expansion Slots:
    • At least one PCIe x1 for later extensions of a TV card.
  • E) Which mainboard do you recommend?
• RAM
  • 128 GB ECC
• Storage
  • 2-4 Enterprise SSDs (NVMe or SATA (with hotswap), using ZFS)
• Network:
  • F) Can we get SFP+ with my power restrictions (onboard or separate NIC)?
• Chassis
  • Maybe some small Supermicro tower
  • Fans: 2-3
• Power Supply
  • Some 80+ efficiency

I prefer on buying new hardware instead of used parts (just as a hint, especially, regarding mainboard and CPU).

Thanks for any recommendations, hints, and discussions.

Thomas

 

[leesteken]

ASRock rack has an X570 (good for passthrough) consumer Ryzen but enterprise motherboard and consumer Ryzen supports ECC unofficially (I use Crucial) and AM4 is currently cheap. Unless you want to go full (second-hand) enterprise platform, it might be something to consider. I doubt you'll find something with less than 60W total unless you go laptop (or NUC-like); most "efficient" x86 CPUs use more than that but you also get (much) more performance.

 

[sb-jw]

What do you think of the base Supermicro X13SAE-F and an Intel i5-13600T?
There may also be a few guides on how to undervolt the CPU or limit its power consumption without losing too much performance.

If it really needs to be server hardware, it often costs significantly more than workstation or consumer hardware. In addition, it often consumes more power and is potentially louder overall (case fan, etc.). Server boards are usually in EATX format and often do not fit into standard chassis.

 

[thomas-hn]

ASRock rack has an X570 (good for passthrough) consumer Ryzen but enterprise motherboard and consumer Ryzen supports ECC unofficially (I use Crucial) and AM4 is currently cheap. Unless you want to go full (second-hand) enterprise platform, it might be something to consider. I doubt you'll find something with less than 60W total unless you go laptop (or NUC-like); most "efficient" x86 CPUs use more than that but you also get (much) more performance.

I think the 570 chipset is more power hungry than the 470.

What do you think of the base Supermicro X13SAE-F and an Intel i5-13600T?
There may also be a few guides on how to undervolt the CPU or limit its power consumption without losing too much performance.

If it really needs to be server hardware, it often costs significantly more than workstation or consumer hardware. In addition, it often consumes more power and is potentially louder overall (case fan, etc.). Server boards are usually in EATX format and often do not fit into standard chassis.

Oh, I was not aware that the newer Core i5 are capable of ECC memory.
Beside the X13SAE-F there is also the X13SAZ-F (R680E chipset) which fits better in server chassis (the RAM is better located in the air flow).
Has someone here experience with the i5-13600T and its power consumption in real life?

What do you think about the Xeon D or Xeon W (not server-grade) processors, could they also be an option?

 

[thomas-hn]

Any further hardware recommendations from other users?
I cannot believe that ASRock Rack with an X570 and Ryzen or the Supermicro X13SAE-F with an Intel i5-13600T are the only recommendations.

 

[LnxBil]

Any further hardware recommendations from other users?

Not sure if this 100% fits you, but I'm running with an ODROID H3+, 64 GB non-ECC and two Enterprise SATA SSDs with Optane NVMe as SLOG with about 9-11W idle.

 

[jdancer]

May want to look at embedded CPU motherboards using an Intel Xeon-D or Atom. Optionally comes with 10GbE (fiber and/or copper) and/or SAS controller.


Supermicro and ASRock Rack makes them. I use the Supermicro configurator at wiredzone.com

 

[xnd]

Not sure if this 100% fits you, but I'm running with an ODROID H3+, 64 GB non-ECC and two Enterprise SATA SSDs with Optane NVMe as SLOG with about 9-11W idle.

9-11 W when things are running? Or idle?
Because with newest BIOS the Odroid H3/H3+ and some tweaks (powertop) it is able to achieve only 1.5 - 2 W idle (when only Debian running) - for more info visit forum.odroid.com

 

[LnxBil]

9-11 W when things are running? Or idle?

"My idle", so running my whole infrastructure on two enterprise SSDs and one optane, but nothing CPU intensive.

Because with newest BIOS the Odroid H3/H3+ and some tweaks (powertop) it is able to achieve only 1.5 - 2 W idle (when only Debian running) - for more info visit forum.odroid.com

Thank you for pointing that out! I'll have a look.