Hey all; I've made use of Proxmox for a while and I'm looking to take my first few steps into ceph -- I have 3x QCT QuantaGrid SD1Q-1ULH that I will be making use of.
Each of these nodes have a Xeon D-1541 with an OCP Mezz 10Gb SFP+ (QCT Intel® 82599ES dual-port 10G SFP+ OCP mezzanine) and Intel® X552 Dual Ports 10Gb SFP+ for VM and ceph networks (will be making use of a broadcast network for this); system storage configuration is as follows:
(12) 3.5"/2.5" Hot-Swappable 12Gb/s SAS HDD/SSD or 6Gb/s SATA HDD/SSD
(4) 2.5" Hot-Swappable 7mm 6Gb/s SATA SSD (note: these front slots support 12Gb/s SAS as well)
(1) PCIe 2280 M.2 SSD
(2) 2.5" 7mm 6Gb/s SATA SSD for OS
I'm still in the process of narrowing down what size HDDs do I want to start populating these with; I'd like to make use of the onboard NVMe option (journal?) while the OS is installed in a ZFS Mirror.
I'm looking to set up a balanced system that can scale for high capacity as I add spinners to this but as this supports SSDs I was hoping to make use of this as well.
Has anyone worked with a similar layout to the above? What would be the ideal combination to make the best use of this equipment?
If I'm looking to start small (think 1 HDD per node, 2x SSD for OS per node and 1 NVMe drive per node) would the above still be a good fit?
Is ceph a good choice for this or should I consider GlusterFS as well?
I'm looking to learn more as I go but rather not waste funds if I can avoid that -- not looking to rely on these just yet until I'm more comfortable with ceph so destroying the cluster for the sake of learning is fine by me as well so suggest away!
Each of these nodes have a Xeon D-1541 with an OCP Mezz 10Gb SFP+ (QCT Intel® 82599ES dual-port 10G SFP+ OCP mezzanine) and Intel® X552 Dual Ports 10Gb SFP+ for VM and ceph networks (will be making use of a broadcast network for this); system storage configuration is as follows:
(12) 3.5"/2.5" Hot-Swappable 12Gb/s SAS HDD/SSD or 6Gb/s SATA HDD/SSD
(4) 2.5" Hot-Swappable 7mm 6Gb/s SATA SSD (note: these front slots support 12Gb/s SAS as well)
(1) PCIe 2280 M.2 SSD
(2) 2.5" 7mm 6Gb/s SATA SSD for OS
I'm still in the process of narrowing down what size HDDs do I want to start populating these with; I'd like to make use of the onboard NVMe option (journal?) while the OS is installed in a ZFS Mirror.
I'm looking to set up a balanced system that can scale for high capacity as I add spinners to this but as this supports SSDs I was hoping to make use of this as well.
Has anyone worked with a similar layout to the above? What would be the ideal combination to make the best use of this equipment?
If I'm looking to start small (think 1 HDD per node, 2x SSD for OS per node and 1 NVMe drive per node) would the above still be a good fit?
Is ceph a good choice for this or should I consider GlusterFS as well?
I'm looking to learn more as I go but rather not waste funds if I can avoid that -- not looking to rely on these just yet until I'm more comfortable with ceph so destroying the cluster for the sake of learning is fine by me as well so suggest away!