Ceph cluster with one node having no hard drives for specific pool.

[brucexx]

I have 5 node Ceph cluster run under Proxmox. I am moving VMs from an old PVE/Ceph clusters to a new one PVE/Ceph clusters. During this time my new 5 node Ceph cluster will not have all the drives installed. I will have only 8 SSDs, planning to put 2 per server which will cover 4 servers, the 5th server will not have any SSDs installed (this is just temporary for about 2 Weeks).

Questions:
1. Will the Ceph and or PVE have an issue with it ?
2. When adding storage to Proxmox would I add the ip addres of the 5th node or jsut the ip addresses of 4 nodes which hold the drives ?

Alternatively, I could not install the 5th server for now and run that on 4 servers then when I have all SSDs I remove move everything to local drives , remove the pool , add the 5th node and create a new storage pool which would evenly distribute all the drives.

Let me know what you think.

Thank you

 

[brucexx]

anybody on this ?

 

[Moayad]

Hi,

1. Will the Ceph and or PVE have an issue with it ?

No, Ceph will work as long as the MONs and OSDs on the 4 nodes are functional and maintain quorum. However, maybe the performance will be limited to the 4 nodes with SSDs. Additionally, I would ensure that you set noout flag during maintenance to prevent unexpected rebalancing.

2. When adding storage to Proxmox would I add the ip addres of the 5th node or jsut the ip addresses of 4 nodes which hold the drives ?

Only include the IPs of nodes that have active OSDs. The 5th node can join the cluster once its SSDs are added aand OSDs are deployed.

Alternatively, I could not install the 5th server for now and run that on 4 servers then when I have all SSDs I remove move everything to local drives , remove the pool , add the 5th node and create a new storage pool which would evenly distribute all the drives.

Running 4 nodes as start is a practical choice, once the SSDs are available, add the 5th node, create its OSDs, and rebalance...

 

[brucexx]

Thank you