Proxmox Ceph - ITX Mainboard 2x SATA

[liszca]

Somebody gave to me 5 ITX mainboards with 2x SATA each.

Can I build Proxmox with Ceph on them where a total of 2 hosts can fail?
Do I really need 2x SATA ports? Isn't one enough?

The nice thing with these boards they all come equiped with 32 GB of RAM and an Old AMD Bulldozer CPU. Two of them already proofed to be 100% reliable over 2 years while only consuming 20 watts.

I am asking because its not sure if it is worth investing in 2.5 Gb Ethernet.

 

[LnxBil]

Can I build Proxmox with Ceph on them where a total of 2 hosts can fail?

Yes, 5 nodes need at least 3 running nodes, so 2 hosts can fail.

Do I really need 2x SATA ports? Isn't one enough?

You would normally have dedicated disks for ceph, so that the OS and CEPH has at least one disk.

In this minimalstic setup, you could also run only with one, that's correct. The only would should be an enterprise SSD in order to get decent performance. Everything will also run with GBE if you already have it onboard. In general is this more a "waste-reclycle-built" and totally fine but I am not able to say which part is the bottleneck, because everything is sub-optimal and slow, yet it is totally doable.

 

[liszca]

In general is this more a "waste-reclycle-built" and totally fine but I am not able to say which part is the bottleneck, because everything is sub-optimal and slow, yet it is totally doable.

I bet the old CPU is the bottleneck, network i'll fix with USB 2.5 Gb Ethernet:
https://www.biostar-europe.com/app/de/mb/introduction.php?S_ID=950

In general I have to be energy sensitive. My PV can only compensate when the sun is shining.

 

[LnxBil]

I bet the old CPU is the bottleneck, network i'll fix with USB 2.5 Gb Ethernet:
https://www.biostar-europe.com/app/de/mb/introduction.php?S_ID=950

CPU would not be on my list. I bet your internal pcie ethernet will have a much better rtt than your usb solution. Also try to have jumbo frames enabled in your switch.

In general I have to be energy sensitive. My PV can only compensate when the sun is shining.

Then don't go with a cluster. 3 nodes is more energy consuming than a single-node system with power-on-demand-replication.

 

[liszca]

Then don't go with a cluster. 3 nodes is more energy consuming than a single-node system with power-on-demand-replication.

my biggest problem with a single node is what if a hypervisor update goes wrong.

on the other hand the 2 node solution doesn't feel like a solution to the problem. For example I didn't figure out how to enable pvecm expect 1 before reboot of the other node.

 

[LnxBil]

my biggest problem with a single node is what if a hypervisor update goes wrong.

Yes, a possibility, but a very small one. You could go with ZFS and snapshot before so that you have better chances. Normally it won't go wrong (talkin from experience updating hundreds of machines).

on the other hand the 2 node solution doesn't feel like a solution to the problem. For example I didn't figure out how to enable pvecm expect 1 before reboot of the other node.

Yes, two nodes is a hassle...

 

[liszca]

You could go with ZFS and snapshot before so that you have better chances.

Last time I had to go back to snapshot there was something wrong with the identity of that host. But its to long ago to remember correctly. How are you doing your snapshot just create a snap of "rpool" is there something special on later re-using that snap.
I am only regularly using snapshot from the web interface.

 

[LnxBil]

How are you doing your snapshot just create a snap of "rpool" is there something special on later re-using that snap.
I am only regularly using snapshot from the web interface.

Yes, regular snapshotting the whole pool. It's also good for storing as backup off-site. On-site and off-site, I have different snapshot cleanup routines to thin out snapshots after different times. Until now, I never needed a snapshot, yet I have them with a timespan of years.