Good Idea? cluster with or without ceph

[Heptan]

Intention
We have to replace 3 aging Hyper-V Servers, hosting total of 45 virtual machines, most of them not very busy. Most not very important, except for DC/ADFS and stuff.

New Hardware available
2x Dell R650xs, 1x CPU Intel(R) Xeon(R) Silver 4310, 12 cores, 192GB RAM, PERC H755 Front (Embedded) Controller
2x HP DL360 Gen10, 1x CPU Intel(R) Xeon(R) Silver 4208 CPU, 8 cores, 192GB RAM, HPE P408i-a Controller
Each server has one 480GB SSD, two 2TB SSDs. The all have a dual-port 10GBE card.
Possibility to add more RAM/CPU/Disk is here.

Idea
Would it make sense to create a proxmox cluster with ceph? Add all the 2TB SSDs for ceph?
Or rather go for local ZFS storage? The old Hyper-V servers had local storage only and it was ok. And for one migration a month I can live with a few minutes to migrate.

I have no experience with ceph yet, proxmox a little. I know about ZFS, Debian, esxi and similar stuff.
I wonder if this will get to complicated, as it is not my main job, I just happen to be the guy that has an idea..

I read about having a minimum of 7 nodes for ceph, others running production on 3 nodes.. What would you recommend in this case?

 

[LnxBil]

Would it make sense to create a proxmox cluster with ceph?

Yes, absolutely. I'd recommend to have three identical nodes if possible. 20 GBE (LACP bonded) is also OK for the lower end hardware and "only" two OSDs. I would consider switching out the CPU to a model with higher base clock. You don't need 12C on each node for your light work VMs.

 

[virtus223]

is there any real need for ceph? with your setup, i would just use zfs replication.

 

[LnxBil]

is there any real need for ceph? with your setup, i would just use zfs replication.

If you want zero dataloss and a real cluster with fast live migration ... CEPH is the way to go with this hardware and MUCH MUCH easier to work with in day-to-day VM creation/deletion.

 

[Heptan]

Thank you for your insight. I will have to do some more reading on ceph
Should I add another server to get to 5 nodes?

 

[LnxBil]

Should I add another server to get to 5 nodes?

Or remove one.

 

[Heptan]

How would removing one be better?

 

[LnxBil]

How would removing one be better?

You need an odd number for quorum and start out with an even number. So adding or removing one ...