[SOLVED] 3-node cluster

[Heidegeist]

Hello,

I need help regarding the technical decisions for setting up a 3 -node proxmox cluster.
I am new to proxmox and also have no experience with systems like Ceph, ZFS an such.

As for the hardware I have 3 hosts which are not identical but can provide similar performance. Resources:
CPU: 20/24/24 cores - RAM 256 GB each - SSD: 12/12/6+6 TB - HDD: 60+ TB each. Enough for one machine to host all current VMs.
One of these hosts ist brandnew and empty, only with the proxmox default installation. The other two are in production in a 3-node ganeti cluster.

I have 20 VMs, mostly WinServer.
19 VMs can live entirely on SSD storage. 1 VM can have its system and DB also on SSD but needs plenty (~50TB) of additional HDD storage available for medical images.

So my first question is how to organize storage replication/redundancy.

I need complete availability during work hours of course (6-20).
Live migration is not mandatory but would be nice, allowing me to do maintenance during work hours without downtime.
Automatic failover is even less mandatory - but is being asked for by my (non IT) superiors every once in a while.

Thanks in advance
Andreas

 

[SOLTECSIS - Carles Munyoz]

Hi,
I think that a Proxmox cluster using Ceph as the distributed filesystem is a perfect option for you.

With this setup you can have all your VMs replicated in all hosts using a Ceph pool and in the event of a cluster node failure you will be able to start the virtual machines in the other nodes meanwhile you repair the fault node.

You can benefit too from the live migration for make updates in one of the cluster nodes without service interruption. For example, if you have to update one of the cluster nodes, you can live migrate its VMs to other node and make the update without service interruption and avoiding disturb the users.

A 10Gbps switch stack is advisable for the Ceph network and 20Gpbs trunks from each host to this stack switch for grant that the network will not be a bootle neck in the I/O system.

Greetings.

 

[Heidegeist]

Thanks for pointing me in the right direction. Ceph was actually what I was expecting, too.
However, I read somewhere that Ceph with HDD is not advisable. The data on my HDD will rather be archived data which are expected to be online but will not be accessed heavily. I hope this doesn't affect the overall performance of the cluster, does it?

Next question:
Will it be possible to get the ONE pve/ceph node up and running first with all VMs and add the other nodes later?
If not, I could provide two empty nodes at most but not all 3 nodes at once. My VMs need to live somewhere and work until I can migrate them to the new cluster.

Regards
Andreas

 

[SOLTECSIS - Carles Munyoz]

I don't see any problem using HDD for Ceph, but have in mind that the I/O performance would be better if you use SSD or NVMe disks instead.

If you have different kinds of disks (for example, HDD and SSD) you can create pools that use only HDD disks and pools that use only SSD disks. This way you can have the VM that require less I/O performance in the HDD pools and the ones with more I/O requirements in the SDD pools.

You can create a two nodes Ceph cluster, move to it the VMs and then add the third node for complete the cluster.

 

[Heidegeist]

Thanks again - I'll make it so.

 

[SOLTECSIS - Carles Munyoz]

Creat!
Good luck and if you need more help do not hesitate to contact me again.