Upgrading / Adding extra Ceph OSD Disks

[MarkjT]

Hello,

We have a Proxmox Cluster which has 9 servers. All servers currently have one o/s disk and 3 OSD disks of size 1TB each.

We are looking to upgrade the 1TB disks to 2TB disks. My first question is if we change one OSD disk at a time to 2TB disks, will Ceph automatically recognise the new space available ?

My next question, 7 servers will take up to 8 disks. If we add extra disks into these servers for OSD but leave the remaining two servers in place but they only have 4 disks (one o/s, 3 OSD) would this allow us to expand the storage but will it cause a problem as the two servers only have 3 OSD disks.

Any advice would be most appreciated.

Thank you in advance,

Mark.

 

[Alwin]

We are looking to upgrade the 1TB disks to 2TB disks. My first question is if we change one OSD disk at a time to 2TB disks, will Ceph automatically recognise the new space available ?

IMHO, if you have space anyway add the 2TB as OSDs first and at the same the set the weight of the old ones to 0. This way, ceph will do a recovery and try to place the PGs from the old OSDs onto the new ones, all on the same host. The other way would be to remove the OSDs first by setting weight 0 and after recovery remove them from the host. Then add the new ones.

My next question, 7 servers will take up to 8 disks. If we add extra disks into these servers for OSD but leave the remaining two servers in place but they only have 4 disks (one o/s, 3 OSD) would this allow us to expand the storage but will it cause a problem as the two servers only have 3 OSD disks.

You will create an imbalance on the hosts, as the servers with less OSDs and in that case also smaller disks, will less participate. Besides that, you need to allocate extra resources for more OSDs on the server. 1 GB RAM per TB of OSD disk space, roughly one core per OSD and added network bandwidth. Distribute your OSDs evenly throughout the cluster.