Proxmox 4 : Evolve from 1 server to HA on 2 servers ?

[leodom]

Iaorana !

Last year I installed Proxmox 3 on a cheap dedicated server and
played a bit with hosting small websites on containers: I love it

I feel ready to go a step beyond and setup a 'sleep well' solution
involving one more robust server and one for backup/failover/HA

After quite exhausting web hunting, it seems that a good solution
could be using proxmox on 2 nodes servers using DRDB.

1 - Would anyone know if I can have HA using 2 nodes from
dissimilar servers running the same linux and proxmox ?
Or do I need to have the exact same hardware as well ?

2 - Can I mirror an existing proxmox server for HA
or should I start from scratch with both servers ?

Of course I am completely open to any links I could have missed
and other possibilities to have my servers running by themselves
through incidents

I saw some internet pages for proxmox 3 but
haven't found out much for P4 about requisites and configurations
to start ordering my servers...

Thanks for reading,
Dom

 

[manu]

1. Different servers are ok ! No pb there
2. Just go with fresh install. With the proxmox installer you get the stuff done in 10 minutes, so need to go for work arounds here

 

[adamb]

You can't have HA with 2 nodes. You need atleast 3 nodes.

 

[wbumiller]

Please be aware that "cluster" doesn't imply HA.

To be clear: a two-node cluster is entirely possible. But not a two-node HA-cluster. Two nodes simply don't qualify for high availability. If one node goes down, the other one either loses quorum, unless you fix that in which case it becomes a non-cluster. If the two nodes simply lose connection to each other, you're running a split brain unless you "fix" the quorum issue by giving one node 2 and one node 1 votes. But then if the 2-vote node dies the 1-vote node loses quorum...

 

[adamb]

Please be aware that "cluster" doesn't imply HA.

To be clear: a two-node cluster is entirely possible. But not a two-node HA-cluster. Two nodes simply don't qualify for high availability. If one node goes down, the other one either loses quorum, unless you fix that in which case it becomes a non-cluster. If the two nodes simply lose connection to each other, you're running a split brain unless you "fix" the quorum issue by giving one node 2 and one node 1 votes. But then if the 2-vote node dies the 1-vote node loses quorum...

Hence the title of the post.

Proxmox 4 : Evolve from 1 server to HA on 2 servers ?

 

[leodom]

Hi there, I thank you a lot for your replies !

I looked back for definitions for clusters, quorum, HA ; I may have them not very clear yet, since I am quite new to this...

What I really need is having a bunch of small slow paced websites in their due virtual servers (I have that part already running), and not have to rush to fix something that would have broken. Sailing a few days between islands should not produce sweating uneasiness of disaster anticipation

So I researched on how to have 2 servers on different locations that would be synchronized so that if the main one is faulty (no ping for a while, hardware failure) the traffic goes to the second one and business continues as usual.

Then I found this article about Two-Node_High_Availability_Cluster and thought I had encountered a solution to my need.

1 - I understood of it that DRBD can be used for a network raid-like implementation.

2 - So maybe the 2 servers don't need to be in a cluster of 2 nodes, just a node on one server that will be mirrored ?

3 - And as high availibility, I need websites to be available within a reasonable amount of minutes after a fault, but automatically and up-to-date. (even if a manual intervention was needed at some point afterwards to resolve the initial issue, resynchronize etc...)

I now have 1 production server installed with proxmox 3 (and a Debian test server of which I may get rid of soon, but that can be used meanwhile)

Ideally, I would now book an ovh soyoustart server running proxmox 4 beta,
transfer or install all the virtual containers / lxc,
then reuse my smaller server for redondancy.

4 - If understood well, I should better start from scratch to have an easier working mirrored solution, spending more to get 2 new servers, a soyoustart and a smaller one...

Thanks again for your clarifications and to bear with my imprecisions

 

[adamb]

Hi there, I thank you a lot for your replies !

I looked back for definitions for clusters, quorum, HA ; I may have them not very clear yet, since I am quite new to this...

What I really need is having a bunch of small slow paced websites in their due virtual servers (I have that part already running), and not have to rush to fix something that would have broken. Sailing a few days between islands should not produce sweating uneasiness of disaster anticipation

So I researched on how to have 2 servers on different locations that would be synchronized so that if the main one is faulty (no ping for a while, hardware failure) the traffic goes to the second one and business continues as usual.

Then I found this article about Two-Node_High_Availability_Cluster and thought I had encountered a solution to my need.

1 - I understood of it that DRBD can be used for a network raid-like implementation.

2 - So maybe the 2 servers don't need to be in a cluster of 2 nodes, just a node on one server that will be mirrored ?

3 - And as high availibility, I need websites to be available within a reasonable amount of minutes after a fault, but automatically and up-to-date. (even if a manual intervention was needed at some point afterwards to resolve the initial issue, resynchronize etc...)

I now have 1 production server installed with proxmox 3 (and a Debian test server of which I may get rid of soon, but that can be used meanwhile)

Ideally, I would now book an ovh soyoustart server running proxmox 4 beta,
transfer or install all the virtual containers / lxc,
then reuse my smaller server for redondancy.

4 - If understood well, I should better start from scratch to have an easier working mirrored solution, spending more to get 2 new servers, a soyoustart and a smaller one...

Thanks again for your clarifications and to bear with my imprecisions

If you just want a cluster with no automatic failover you can do that with two nodes and drbd. It will require your intervention everytime there is an issue. If you want HA, you need at the very least 3 nodes.

 

[leodom]

Thanks adamb for taking the time to write back, although your post tend to leave me more puzzled than informed...

Should I infer from it that the reply to my questions 1-4 is: 'Build a 3 servers cluster and then you can have HA, that is, your websites will still be accessible even if one of the server fails' ? Meaning that I should order 3 servers to make it work ?

If not, my focus is not on the concepts of clusters, ha and quorum disks, but really on finding a solution to my expressed need, solution that may or may not make use of those concepts... (Like 2- is what I need a cluster, or is there another name for 'mirrored server'?)

And should I discard the Two-Node_High_Availability_Cluster as more or less a proof of concept, something that is not really safe to implement in real environment ?

Well, off to read more about failover and synchronization...

 

[adamb]

Thanks adamb for taking the time to write back, although your post tend to leave me more puzzled than informed...

Should I infer from it that the reply to my questions 1-4 is: 'Build a 3 servers cluster and then you can have HA, that is, your websites will still be accessible even if one of the server fails' ? Meaning that I should order 3 servers to make it work ?

If not, my focus is not on the concepts of clusters, ha and quorum disks, but really on finding a solution to my expressed need, solution that may or may not make use of those concepts... (Like 2- is what I need a cluster, or is there another name for 'mirrored server'?)

And should I discard the Two-Node_High_Availability_Cluster as more or less a proof of concept, something that is not really safe to implement in real environment ?

Well, off to read more about failover and synchronization...

Yep if you want an automated failover then you will need 3 nodes.

 

[PigLover]

No rule says the third-node needs to be very capable - all it has to do is hold the quorum. Why not get a low-scale machine that can support Proxmox and add it to the cluster as a third node. You don't actually have to run any VMs on it (or you could even run trivial VMs like a PBX server). You let the other two nodes play HA buddies and the little one just serves as a referee.

As a test I've loaded Proxmox on a bay-trail NUC...worked perfect. Get them used for about $100 and throw in a small SSD and you've got your quorum server. You could do something "fully professional" without spending more than $200 using a small 1U server case and a low-scale motherboard.

 

[leodom]

Great, adamb, piglover ; got it !

So the best is to get 2 decent servers and a small one...

A - Is it ok to use servers on 2 distant (big) data-centers ? Could latency be a problem for drbd synchronization ?

B - Can I use an existing proxmox dedicated server to serve as quorum ? Does it have to be the same Proxmox 4 version ?

C - What would happen if 3rd referee server crashes ?

Have ordered the first server and have to wait availability to order a similar 2nd one...

 

[udo]

Great, adamb, piglover ; got it !

So the best is to get 2 decent servers and a small one...

A - Is it ok to use servers on 2 distant (big) data-centers ? Could latency be a problem for drbd synchronization ?

Hi,
1. yes - latency is an problem and can be a very big performance killer.
2. with pve3 all cluster member must be in one network (multicast) - you can build an openvpn-network (some people run such config). Don't know how this work with pve4.

apropos drbd - in you first post you write about containers. drbd don't work with openvz-containers (I haven't tested lxc-containers with drbd yet).

B - Can I use an existing proxmox dedicated server to serve as quorum ? Does it have to be the same Proxmox 4 version ?

pve4 use a different cluster communication - so all cluster member need to be pve4

C - What would happen if 3rd referee server crashes ?

the remaining two work like before (because they have quorum). You are still able to migrate VMs form node 1 to node 2.

Udo

 

[leodom]

Well I now have my 3 servers installed with Proxmox 4, ready to set this all up... wish me luck!

1. yes - latency is an problem and can be a very big performance killer.

OK, as long as it is only performance killer but not breaking things, it should be ok with the small websites

2. with pve3 all cluster member must be in one network (multicast) - you can build an openvpn-network (some people run such config). Don't know how this work with pve4.

Thanks for pointing that out, I found a link I will check to set it up : http://www.nedproductions.biz/wiki/...-2.x-cluster-running-over-an-openvpn-intranet

apropos drbd - in you first post you write about containers. drbd don't work with openvz-containers (I haven't tested lxc-containers with drbd yet).

Well, unless there is an easy openvz to lxc converter, I may have to stick to just transfer the websites the old way, once all is done and working it will save time in the future anyway (I already picture myself sipping some Maitai in complete technoblivion - and please don't ruin the mood with sarcastic truths about illusory IT peace, that image may keep me going through installs )