3nodes Ceph with 2xSSD in Raid1 for Proxmox + Journal, 24x1.2TB for OSD

Icefire

Member
Mar 20, 2022
18
0
6
43
Pretty new to Ceph and even Proxmox.
I'm currently getting my 3rd server identical as my 2 others (2x28core+HT, 512gb ram) with 2 SSD + 24 1.2Tb SAS drive.

How would you configure that? I'm running a few VM, mostly not data intensive (got 4 Tb of enterprise data currently)
I want the servers to be clusterd/HA and plan for SSD/HD/server failure.

Would making a software Raid1 on the SSD for Proxmox than using the rest as Journal for Ceph a good idea?
a quick read was that Raid 1 for journal not good and loosing a disk when you loose have the journals a catastrophic failure..

having 3 nodes make it 6 SSD for journals. Do they rebuild if 1 fail?
 
12 HDD OSDs on one SSD (how large are they?) is a bit too much as failure domain and IOPS stress on the SSD.
And you should not use these SSDs for the Proxmox installation if they are to be used as RocksDB partitions (there is no journal any more for Ceph OSDs). HDD-OSDs always need their RocksDB and WAL on SSD, otherwise the performance will be unusable.
 
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!