Homelab cluster

[merox]

Hello guys,

So I just recieved 3x mini HP PCs (ProDesk 600 G3) with 16GB Ram and 480GB ssd.

I want to create a proxmox cluster and I want to ask you which will be the best configuration for storage with these 3 nodes ?

I’m curious about ceph too, but not sure yet if this configuration is enough or will be too overloaded in the future.

Any suggest ? Thank you!

 

[merox]

Up

 

[UdoB]

You know probably https://pve.proxmox.com/pve-docs/pve-admin-guide.html#chapter_pveceph ?

For learning it might be okay. For actual usage... it depends on your requirements and your expectation.

Do not forget: three nodes is the absolute minimum. You will get into trouble if one node fails.

And the complexity rises drastically if you rely on Ceph for your VMs. You will experience problems you didn't know they could exist. Been there, done that. (In a definitive test cluster.)

So again: for learning things your three node approach might be good...

Just my 2 €¢ - have fun!

 

[Dunuin]

You know probably https://pve.proxmox.com/pve-docs/pve-admin-guide.html#chapter_pveceph ?

For learning it might be okay. For actual usage... it depends on your requirements and your expectation.

Do not forget: three nodes is the absolute minimum. You will get into trouble if one node fails.

And the complexity rises drastically if you rely on Ceph for your VMs. You will experience problems you didn't know they could exist. Been there, done that. (In a definitive test cluster.)

So again: for learning things your three node approach might be good...

Just my 2 €¢ - have fun!

I would think MiniPCs would be a terrible choice for ceph for anything running in production, as these are missing the expansion capabilities? I'm not that into ceph, but from what I heard so far you want multiple OSDs (=disks) per node and multiple NICs with at least one being fast (atleast 10Gbit). If you don't need a real shared storage (so you are ok losing some seconds of data when using HA and a node fails) ZFS with replication might be an alternative.

 

[merox]

I would think MiniPCs would be a terrible choice for ceph for anything running in production, as these are missing the expansion capabilities? I'm not that into ceph, but from what I heard so far you want multiple OSDs (=disks) per node and multiple NICs with at least one being fast (atleast 10Gbit). If you don't need a real shared storage (so you are ok losing some seconds of data when using HA and a node fails) ZFS with replication might be an alternative.

but will be ok (zfs with replication )if I have only 1 disks (ssd) per node ? Or should I buy a second one only for proxmox installation? Thank tou

 

[Dunuin]

One should work. But keep in mind that ZFS got a lot of overhead. It would be recommended to use enterprise grade SSDs and ECC RAM for brst data integrity, stability and life expectation.

 

[merox]

One should work. But keep in mind that ZFS got a lot of overhead. It would be recommended to use enterprise grade SSDs and ECC RAM for brst data integrity, stability and life expectation.

If need ECC ram maybe will not be the best choice to use zfs replication for MiniPCs, as I said, I have 3x ( Hp ProDesk 600 G3 ), then which good alternative I have for this setup? Thank you so much!

 

[Dunuin]

If need ECC ram maybe will not be the best choice to use zfs replication for MiniPCs, as I said, I have 3x ( Hp ProDesk 600 G3 ), then which good alternative I have for this setup? Thank you so much!

It will work without ECC RAM, but you can never trust the bit rot protection. So not that great for data integrity.