3 node setup, homelab?

[Stefan Pettersson]

Ok, I have for a long time had two servers running proxmox and I recently got hold of a Thinkserver TS140 cheap. XEON E3-1245v3 I think and 16 gig of ram. The other two servers are supermicro boards with E3-1241v2 (I think that was the modell?) and 24 gigs of ram.

I have also gotten a hold of some infiniband adapters, one dual port adapter and two single port adapters, 10Gb

I also have two 250gb SSD samsung, 3 OCZ 60Gb SSD and four 2 TB SATS drives.

How wold you set this up? Should i go with threee proxmox nodes and ceph? I only have two 250Gb SSD:s
Is it possible to use one node with proxmox installed as a storage node and centralise storage?

Or should i install freenas on the thinkserver and share it like SCSI??

What do you recommend??

 

[Stefan Pettersson]

Ok, 229 views and not one opinion...

Would be nice if someone atleast gave a recommendation?

This is for homelab only, not production.

*EDIT*

I could also get a hold of a third 250Gb Samsung SSD, would that help to get a recommendation?

 

[tom]

There is no common answer here. We do not know your planned workload or if you need HA.

For Ceph, your SSDs are to slow, so you will see not the best performance and also the SSDs will die soon (I expect these are consumer or prosumer SSDs).

If you just want to play and learn, Ceph is of course possible, also with HA.

If you want to run real workload for home use, think of using ZFS, but make sure you limit the RAM usage for ZFS.

 

[Stefan Pettersson]

Thanks Tom Really appriciate it!

The disks i am using are Samsung 850 Pro Series MZ-7KE256BW, I guess you count them as prosumer disks. The load will not be high on the disks.

So, ZFS on the TS140 with freenas and share the disks as ISCSI target maybe? I think freenas has support for infiniband. I would like to centralise storage to one node if possible?

Play and learn are ofcourse the goal but preferably not ending upp breaking the hardware, are ceph that write intense??

 

[tom]

yes, a samsung 850 pro is a "prosumer" SSD.

check the well known blog from sebastian han and look after the performance on the samsung 850 pro in a ceph environment.
https://www.sebastien-han.fr/blog/2...-if-your-ssd-is-suitable-as-a-journal-device/

as I have limited experience with freenas, I cannot answer the freenas question.