pointers/advice for server refresh

[Binary Bandit]

Hello everyone,

It's time for a hardware refresh for our three-node cluster. I appreciate any advice, links to relevant threads about hardware issues, and so on.

Here's what we're considering x 3:

1. Latest Proxmox 8.4
2. Dell PowerEdge R6525 10x2.5" NVMe
3. No TPM (I don't see a reason for this?)
4. 1 x AMD EPYC 73F3 CPU
5. 8 x 64GB DDR4 RDIMM 3200MT/s
6. CEPH storage - 2 x Dell 2.5 U.2 NVMe 7.68TB Solid State Drive
   It seems that two drives per storage node are okay now? I recall the recommended minimum count being four per node in the past.
7. boot storage - Dell 15G PowerEdge BOSS-S2 Controller Card with 2x 240GB M.2 SSDs
8. VM LAN communication - 1 x OCP Dell Mellanox ConnectX-6 Lx Dual Port 10/25GbE SFP28 OCP Network Card
   These will be bonded and connected to 2 x MLAG switch ports.
   In digging through the forums and Googleing, I see issues with Mellanox cards in the past, but this appears not to be an issue with Proxmox 8.x.
9. CEPH communication - 2 x PCI Dell Mellanox ConnectX-6 Single Port HDR100 QSFP56 Network Adapter
   Same comments as item 8, above.
10. We'll be backing up and restoring VMs to move from our existing Intel-based cluster.

best to all,

James

 

[LnxBil]

We sold a small CEPH cluster to a customer last year with similar specs, cheapness in mind and he's very happy with it. Install was straight forward and we encountered no problems:

• Dell R6615, 10x2.5'' NVMe
• EPYC 9124
• 2x 64 GB 5600
• 2x 960 U2 PCIe 4.0
• 2x BOSS 480 GB
• Intel OCP 10 GBE Base-T