Using small SSDs with Ceph

S

SteveITS

Active Member
Proxmox Subscriber
Feb 6, 2025
451
176
43
Coming from Virtuozzo to Proxmox we had some storage HDDs that were using 200 GB SSDs for read and write caching. With Ceph we can use them for DB+WAL of course. Eventually I expect we'll replace the remaining HDDs with 1 or 2 TB SSDs. At that point, is there a downside to using an SSD that small as an OSD? The "weight" would account for its size...

Thanks.
 
If you have OSDs with very different sizes in the cluster, it gets a bit more complicated.
Performance wise, the larger OSDs will see more load as they store more data.
What happens if you have many small and a few large OSDs? What happens if a large OSD fails? Can the remaining OSDs/hosts accomodate the data to get back to full redundancy without running out of space?
 
Hi, thanks for the reply. Let's assume Ceph has lots of available storage, four or more 1+ TB drives per node, and rebalancing isn't an issue.

Basically I'm asking if it's better to repurpose one or two 200 GB drives as extra/unnecessary OSDs, or remove them completely and use them for something else.
 
If you have enough large SSDs and have other use-cases for the small SSDs, then I would use them with the other use-cases.

Keep in mind, that the number of OSDs / host is important in smaller clusters. Especially when the number of nodes is equal to the number of replicas (size). In most situations this means 3-node clusters. Because if a single OSD fails, Ceph cannot recover to another node as the other nodes already have a replica. Therefore it can only recover to other OSDs in the same node.
 
You must log in or register to reply here.
Top Bottom