CEPH guide, best practice, best config proposal

J

Janus006

New Member
Aug 25, 2024
1
0
1
Hi all,Few month ago, I switched to Proxmox from VMWare. To celebrate this, I added a 3rd nodes to be sure to be able to have a real cluster and also use CEPH. I tried to use CEPH but i'm realy not sure my setup was optimised.
I recently reinstalled all my Proxmox cluster but I want to learn more the best practices about CEPH before re-implementing it.
Here is my hardware setup:
Dell R720 with:
- local storage
- 4x 1Gb interfaces
- 2x 10 GB SFP+ interfaces
- 2x 1TB nvme disk
Dell R730 with:
- local storage
- 2x 1Gb interfaces
- 2x 10 GB SFP+ interfaces
- 2x 1TB nvme disk
small Thinkcenter (i7 10th gen CPU) machine with:
- 1x 1Gb interfaces
- 2x 10 GB SFP+ interfaces
- 1x 500GB nvmedisk for the OS and local storage
- 1x 1TB nvme disk
All my SFP+ are connected via a 10GB swtich

As my setup have 5 nvme disk and my machines linked via 10GB, i'm not sure of the best configuration to adopt.

Have you any ideas ? And/or have you some good complete guides for me ?

Thank you,
 
Ceph is great as it scales into infinity. For only three nodes you may find that there are some pitfalls. You may go that road, but you should know them before starting that journey:
UdoB

[TUTORIAL] Thread 'FabU: can I use Ceph in a _very_ small cluster?'

Ceph is great, but it needs some resources above the theoretical minimum to work reliably. My assumptions for the following text:
  • you want to use Ceph because... why not?
  • you want to use High Availability - which requires Shared Storage (note that a complete solution needs more things like a redundant network stack and power supplies)
  • you want to start as small (and cheap) as possible, because this is... “only” a Homelab
You plan for three Nodes. Each node has s single dedicated disk for use as an “OSD”. This is the documented...
  • Like
 
  • Like
Reactions: Johannes S
You must log in or register to reply here.
Top Bottom