Best use of hardware - Ceph

[anthony]

Alright I am switching from a 2 node ZFS "cluster" to a proper 3 node ceph cluster. I am trying to figure out the best use of the hardware I have. I have 3 Identical servers, each has 1 500gb NVME, 3 2TB SSDs, and 5 2TB HDDs. I have seen where you can have the primary drive be an SSD with a replica on the HDD which would be great. Id like to have it on atleast 3 different drives across atleast 2 servers, primarily read/writing from/to the SSDs. Is this doable? Im trying to wrap my head around how.


Is that the best use of the hardware? I am drawing a blank as to how the rule would best be written for this.

 

[Alwin]

Best go through our and Ceph's docs to see the hardware recommendations. Maybe it sheds more light.
https://pve.proxmox.com/pve-docs/chapter-pveceph.html
https://docs.ceph.com/en/nautilus/start/hardware-recommendations/

 

[anthony]

What I ended up doing is setting up the primary affinity to 1 for all the SSDs, and 0 for all the HDDs, than doing a pool with a simple host replication rule, a pool size of 3 with a min of 2. That doesnt 100% guarantee that a secondary is written to an HDD, but it does make sure the primary is on an SSD. I used the NVME as A WAL for the HDDs