Cluster and HA in 4.x - would love to learn more

[Kei]

Hello, for a home lab I would like to setup a Proxmox VE 4.2 cluster with local storage for the VMs, mirrored presumably with DRBD9. However, I learned that I cannot implement HA without having a third node. I tryed learning why is that, and I stumbled upon concepts like quorum, split brain and such... I would like to learn more about this, expecially what I can or can't do with two nodes (or maybe two nodes plus one just for the quorum) but all the documentation (that I can find or understand at least) seems to be for lower versions and not for 4.x.
I would love to purchase Mastering Proxmox - Second Edition, too bad it's due in august and I would really like to learn as soon as possible. So where can I find the most comprehensive documentation avout CLusters and HA in Proxmox VE 4.x ?

 

[wosp]

You need to have at least 3 nodes for a real cluster because you need to have a odd number of nodes (not really true since you can also have 4, 6, 8 etc nodes, but for example a 4 node cluster may still lose only 1 node (like a 3 node cluster), while a 5 node cluster may lose 2 nodes. A 6 node cluster also may only lose 2 nodes, while a 7 node cluster may lose 3 nodes etc. So I prefer to always have a odd number of nodes), and since 1 node isn't a cluster, 3 is the minimum. This is because there needs to be a majority (quorum) to take decisions (each node has 1 vote, 50% of all votes +1 wins). If one node is offline, the remaining (at least 2) nodes can both conclude they can reach each other, so they are online and working and the missing node has a problem ("let's take over te VM's that were running on the missing node"). If you have a "cluster" of only 2 nodes, they both can THINK the other node is gone, while in fact this isn't the case but they only can't see each other for some reason: a split brain.

Lots of information can be found at the PVE wiki (https://pve.proxmox.com/wiki/Main_Page) and on this forum. Don't hesitate to post any question you may have to this forum.

BTW, a HA-setup with local storage is not very useful. Have a look at Ceph. With 3 nodes and at least 1 HDD for storage in each node you can setup Ceph and have shared storage.

 

[Kei]

Thank you wosp, that was very clear. What do you think of (in a home set-up) using a raapberry pi or other low power server to be the third node of the cluster? Is It feasible? Of course I should set the cluster to NOT be used to host any VM but just as quorum partecipator.

 

[wosp]

Thank you wosp, that was very clear. What do you think of (in a home set-up) using a raapberry pi or other low power server to be the third node of the cluster? Is It feasible? Of course I should set the cluster to NOT be used to host any VM but just as quorum partecipator.

You can simply define HA groups to achieve the thirth node isn't used for VM's. But, as far as I know PVE will only work on AMD64 and/or EMT64, not on ARM. So I don't think a RPi will work.

On non-production envoirement a low end /power server will do the job and if one HDD is large enough for storing all your data, even Ceph will work with only 2 nodes hosting a HDD (OSD) and the thirth (low power) node is only a monitoring node. But also DRBD9 is a good alternative. Please see https://www.proxmox.com/en/proxmox-ve/requirements for hardware requirements.

 

[mir]

But, as far as I know PVE will only work on AMD64 and/or EMT64, not on ARM. So I don't think a RPi will work.

The third node running on RPi does not need to be running any PVE packages at all. All it needs to run is Corosync2 which it is perfectly capable of. Besides, porting is going on for pve-cluster to ARM so in a not so far distance integration of a Pi for quorum purposes only should be manageable by the pvecm tool.

 

[Kei]

The third node running on RPi does not need to be running any PVE packages at all. All it needs to run is Corosync2 which it is perfectly capable of

Thank you for your intervention, mir. Do you know if there is any walkthrough for this setup on the net for a complete newbie that I am?
As far as I understand, I need to install Debian and then the package Corosync2 and that's it. Buyt how do I configure it to interact properly with the other two nodes?

 

[mir]

I already have a Pi running in a lab setup (virtualized) which I need to test on a psychical Pi. I have received the Pi but I haven't have time to test it yet. As soon as I have confirmation that it works on a psychical Pi I will write a step guide on the wiki.

 

[Kei]

I already have a Pi running in a lab setup (virtualized) which I need to test on a psychical Pi. I have received the Pi but I haven't have time to test it yet. As soon as I have confirmation that it works on a psychical Pi I will write a step guide on the wiki.

Looking forward to it!!!