Proxmox installation on SuperMicro server with 4node cluster

[jsengupta]

Hi,

Is anyone there who could confirm that the following server hardware from SuperMicro would support the latest (as of now) proxmox version 6.1? If it supports, then we shall have 4 of these servers in production environment and look to configure CEPH as storage solution

Model	SUPERMICRO SSG-6029P-E1CR12T​
Processor	2 x 5120 2.1Ghz 14-Core Xeon Gold Processor Dual Socket P (LGA 3647)
Motherboard	Super X11DPH-T
Chipset	Intel® C622
Memory	6 x 32GB DDR4 2666 ECC Registered; 16 DIMM slots
Expansion Slots	3 x PCI-E 3.0 x16, 4 x PCI-E 3.0 x8 (All Low Profile)
Onboard LAN	2 x 10GBase-T LAN ports with Intel X722 + PHY Intel X557 2 x AOC-SGP-i2 Dual Gigabit LAN Card
Onboard VGA	ASPEED AST2500 BMC
Storage Controller	Broadcom 3108 SAS3 (12Gbps) controller 2GB 1866MHz DDR3 on-card cache
RAID Support	HW RAID 0, 1, 5, 6, 10, 50, 60
Storage	1 x 240GB SATA SSD S4510 2 x 240GB SATA SSD S4510 1x 600GB SAS 10K RPM 2.5” 8 x 2TB SAS 7200RPM 3.5” HDD
Drive Bays	12 x Hot-swap 3.5" SAS3/SATA3 drive bays 2 x Hot-swap 2.5" SATA3 drive bays (rear)
Power Supply	1200W Redundant Power Supplies Titanium Level (96%)
Cooling System	3 x Heavy duty 8cm PWM fans
Form Factor	2U Rackmount
Others	USB Keyboard and Mouse

Thanks,
Joy Sengupta

 

[Stoiko Ivanov]

While I don't have experience with that particular Server line or system - so cannot guarantee that everything will work without any problems:
* most/all of the components should be supported by a recent linux kernel (PVE is based on the latest Ubuntu kernel)
* Intel NICs (both the 10G (i40e) and the AOC-SGP-i2 (igb)) should work fine especially if you keep the firmware up to date - every now and then some users report problems with a particular nic - which can be addressed by turning off certain offloading features

The one thing you should really consider is the Storage Controller - Broadcom 3108 SAS3 looks like a hardware raid-controller - and those are not supported for use with CEPH-OSDs (or ZFS) - since those storage systems need access to the raw disk devices (and we have seen quite many cases where problems (performance or functional) were related to a HW-Raid controller)


I hope this helps!

 

[jsengupta]

Hi,

Thank you for your reply.

Do you suggest, we should go for the storage controller which does not support RAID? Could you please provide us some example storage controller from SuperMicro for Ceph?

Thanks

 

[Stoiko Ivanov]

Sadly I'm not too familiar with the current lineup of Supermicro Servers - but looking at https://www.supermicro.com/en/products/storage/cards
I would suggest a compatible card, which is labeled as 'HBA only' or 'IT/HBA Mode' for Ceph or ZFS

I hope this helps!

 

[jsengupta]

Thank you very much for your reply!

One more question. Suppose we discard Broadcom 3108 SAS3 and we plug-in a JBOD expansion slot, will it help?

Thanks,

 

[jsengupta]

Hi,

Could you please suggest, which among the below 3 CPUs will best fit for CEPH with 4 node:

1) SKL-SP 4116 12C/24T 2.1G 16.5M 9.6GT UPI
2) 5120 2.1Ghz 14-Core Xeon Gold Processor Dual Socket P (LGA 3647)
3) Intel® Xeon® Gold 5220 Processor


Thanks,