Proxmox & Ceph

[liszca]

I am planning to get in touch with Ceph on Proxmox.

Let me first introduce my hardware I am planning to use before having my questions.

Here is the motherboard with CPU, and it got one downside more of that in a second:
https://geizhals.de/biostar-a10n-8800e-v6-1-a2280308.html

The downside of this board is it is only possible to use 2x S-ATA or 1x M.2 & 1x S-ATA.

Thinking about a 3-Node-Configuration I am not sure how many disks/SSDs are need at a minimum and what is the minimum useful disk configuration in case one disk fails.

Reading documentation around Ceph I figured out its actually necessary to have 4 nodes but that's my target for the future for me.

After having Proxmox install on a USB-Stick I could give it two SSDs for having replicas in case one SSD dies.

Is that the node minimum (to have some redundency on a node level), or can I just use one SSD for the OSD and thats it?

The Board mentioned above proved to be super reliable in a ZFS Mirror configuration and I want it to use on my Ceph journey.

Please let me know if you need more details to help me out.

 

[ness1602]

First rule,don't install proxmox to usb sticks or similar, you will get your node fried in 7 days or so.

 

[liszca]

even with no VMs/Containers running there?

 

[Darkk]

even with no VMs/Containers running there?

Because of the amount of logs it creates.

 

[liszca]

Because of the amount of logs it creates.

so I have only one disk for ceph and three nodes sounds like the fourth node is needed to keep up with a traditional zfs cluster of three nodes in case of a disk failure?

 

[Dunuin]

You could run PVE of a USB SSD. That would be fine. You just need the durability of a real SSD with its wear leveling, caching and higher quality NAND and controller. That mainboard got an internal USB 3.1 gen 1 header. You could get a adapter like this one...

...and a USB to SATA cable like this one...

...and use that to for one or two additional internal 2.5" SSDs if your case got enough 2.5" slots.

Just keep in mind that USB and the cheap SATA controllers used there aren't that reliable.

Should be around 70€ for 2x 120GB SSDs + 2x USB3-to-SATA-cable + USB3-Header-to-dual-USB-type-A-adapter. So not that bad for a mirrored PVE boot/system disk.