Hi,
I have a three node home lab PVE cluster. Each node is also a CEPH node and has two OSDs (one being assigned to a "fast" pool for apps and one being assigned to a "slow" pool for data) (I know that's fewer than recommended and I am contemplating adding more OSDs but this is a home lab...) and I have a 3/2 rule. So I understand that a copy of each bit should reside on each node.
Now I am thinking about adding a fourth PVE node to the PVE cluster which, however, would not be up 24/7 but only for special compute tasks. While I would like to have the VMs that should run on this node in the CEPH app pool and the data it needs in the CEPH data pool, I probably want to avoid putting in OSDs (because otherwise there is no guarantee that a copy of every bit will continue to sit on each of the original three nodes - right?).
Can the fourth node be a CEPH node anyway (i.e. without OSDs)? If this works, I am aware that my network would become a bottle neck when loading apps (data probably not so much, as they are coming from slow disks anyway). But can I do that or is this going to create some unexpected issues down the road?
Bonus questions: A 4/2 rule would probably accomplish the same (but require more OSDs), right? But if I turn on the fourth node only every once in a while, syncing all the missing stuff would probably congest my network...
Any thoughts?
Thanks!
I have a three node home lab PVE cluster. Each node is also a CEPH node and has two OSDs (one being assigned to a "fast" pool for apps and one being assigned to a "slow" pool for data) (I know that's fewer than recommended and I am contemplating adding more OSDs but this is a home lab...) and I have a 3/2 rule. So I understand that a copy of each bit should reside on each node.
Now I am thinking about adding a fourth PVE node to the PVE cluster which, however, would not be up 24/7 but only for special compute tasks. While I would like to have the VMs that should run on this node in the CEPH app pool and the data it needs in the CEPH data pool, I probably want to avoid putting in OSDs (because otherwise there is no guarantee that a copy of every bit will continue to sit on each of the original three nodes - right?).
Can the fourth node be a CEPH node anyway (i.e. without OSDs)? If this works, I am aware that my network would become a bottle neck when loading apps (data probably not so much, as they are coming from slow disks anyway). But can I do that or is this going to create some unexpected issues down the road?
Bonus questions: A 4/2 rule would probably accomplish the same (but require more OSDs), right? But if I turn on the fourth node only every once in a while, syncing all the missing stuff would probably congest my network...
Any thoughts?
Thanks!
Last edited:
