We run a small IT managed services business, we're looking to replace our SAN / ESXi environment. Ceph / Proxmox looks to be a good fit so we've come up with the following straw-man for day 1
We'd welcome any feedback / recommendations on this setup ... all comments are welcome
Given we have only about 12U total rack space and limited budget to play with we've decided to build a 4 node combined Ceph / Prod VM's cluster. Ceph Monitors would also therefore be cluster VM's
4 node Ceph / Proxmox cluster
Each node
2 x 72GB 10K SAS RAID 1 for the OS
1 x 500GB SSD for Journal
5 x 300GB 10K SAS JBOD for OSD's
We therefore calculate total OSD to be approx 6 TB and IOPS of 2640 (based on the formula of 20 x 150 x .88 ... being total OSD HDD count x HDD IOPS x .88 penalty)
We require a total of 2TB and we're looking to run a replication ratio of x2 so require 4TB minimum
We calculate the total 6TB will allow us sufficient short term growth whilst catering for a node failure / down for maintenance without cluster disruption
Similarly, we calculate that CPU calculations will allow enough combined compute to run Ceph, customer VM's and ensure we have enough CPU headroom to cater for a host failure / maintenance. We've used a calculation of 1.5GHz required per OSD. Ceph will account for 28% of the total CPU pool (based on total combined GHz only – no hyper threading allowance made)
Servers are 2 x DL360 G6's and 2 x DL385 G6's with a spare 360 G6 as cold standby / additional future Ceph node
For Networks we're looking at 1 Ceph public vLAN with each host dual homed via 1G NIC's
1 Ceph data vLAN with each host dual homed via 10G SFP+
We intend on merging Ceph monitors / management traffic with the production OSD data over the 10G Ceph data vLAN
We're struggling to get our heads around the pools and placement groups so any advice on those would also be appreciated
Looking forward to the next few weeks of building, testing HA and getting a good look under the covers
Thanks
We'd welcome any feedback / recommendations on this setup ... all comments are welcome
Given we have only about 12U total rack space and limited budget to play with we've decided to build a 4 node combined Ceph / Prod VM's cluster. Ceph Monitors would also therefore be cluster VM's
4 node Ceph / Proxmox cluster
Each node
2 x 72GB 10K SAS RAID 1 for the OS
1 x 500GB SSD for Journal
5 x 300GB 10K SAS JBOD for OSD's
We therefore calculate total OSD to be approx 6 TB and IOPS of 2640 (based on the formula of 20 x 150 x .88 ... being total OSD HDD count x HDD IOPS x .88 penalty)
We require a total of 2TB and we're looking to run a replication ratio of x2 so require 4TB minimum
We calculate the total 6TB will allow us sufficient short term growth whilst catering for a node failure / down for maintenance without cluster disruption
Similarly, we calculate that CPU calculations will allow enough combined compute to run Ceph, customer VM's and ensure we have enough CPU headroom to cater for a host failure / maintenance. We've used a calculation of 1.5GHz required per OSD. Ceph will account for 28% of the total CPU pool (based on total combined GHz only – no hyper threading allowance made)
Servers are 2 x DL360 G6's and 2 x DL385 G6's with a spare 360 G6 as cold standby / additional future Ceph node
For Networks we're looking at 1 Ceph public vLAN with each host dual homed via 1G NIC's
1 Ceph data vLAN with each host dual homed via 10G SFP+
We intend on merging Ceph monitors / management traffic with the production OSD data over the 10G Ceph data vLAN
We're struggling to get our heads around the pools and placement groups so any advice on those would also be appreciated
Looking forward to the next few weeks of building, testing HA and getting a good look under the covers
Thanks