Adding 2 more nodes to 3 node Proxmox Ceph Cluster

[ositedw]

Hello i have a 3 node Proxmox 8,.3 Ceph Cluster with 5 OSDs in each node.
I want to add another 2 nodes with each have 5 OSDs.
Network is 100GB.

Is this the right way:
1) Install Proxmox on node 4 and 5
2) Join Node 4 and 5 to the cluster
3) Install Ceph on node 4 and 5
4) set global ceph osd flags: nobackfill and norebalance
5) Create the OSDS on node 4 and 5
6) unset global ceph osd flags: nobackfill and norebalance

Is this the right way or did i miss something ?
the Storage pool will be useable during this right and vms keep running ?

So long

Domenic

 

[mr44er]

Is this the right way:
1) Install Proxmox on node 4 and 5
2) Join Node 4 and 5 to the cluster
3) Install Ceph on node 4 and 5
4) set global ceph osd flags: nobackfill and norebalance
5) Create the OSDS on node 4 and 5
6) unset global ceph osd flags: nobackfill and norebalance

Suggestion (maybe better, there is no best way):
1) Install Proxmox on node 4 and 5
2) set global ceph osd flags: nobackfill and norebalance
3) Join Node 4
4)Install Ceph on node 4
5)Create the OSDS on node 4 (osd15,osd16,osd17,osd18,osd19)
6)Join Node 5
7)Install Ceph on node 5
8)Create the OSDS on node 5 (osd20,osd21,osd22,osd23,osd24)
9)unset global ceph osd flags: nobackfill and norebalance

Is this the right way or did i miss something ?

That works, sure.

the Storage pool will be useable during this right and vms keep running ?

Yes. If your network is properly configured, IP-adresses fit, no loop, no fencing...then yes. I recommend pinging into every network from a new node before joining the cluster.
nobackfill and norebalance on/off is nice during the procedure, but in doubt optional and could be ignored.