Questions on creating my first cluster

[cosmos]

I want to implement a cluster using low-end hardware. No raid, duplicate PSUs etc. Just 2 PCs, having VT-capable Core I5 cpus. AFAIK, implementing a cluster implies having storage as a separate box. That is, at least two CPU boxes, plus one to host the VM storage of both.

Is it possible to avoid having a separate storage box? That is, have only a couple of servers running? I've seen this kind of setup on some Hyper-V systems, whereas the "slave" server includes the storage. I believe that the slave is constantly replicating the state of the master?

Is this possible on pve and, if so, would you recommend against it?

 

[t.lamprecht]

AFAIK, implementing a cluster implies having storage as a separate box. That is, at least two CPU boxes, plus one to host the VM storage of both.

That depends on what you need, if you do not need live migration of VMs between the node and do not need failover functionality a shared storage is not necessarily needed. (You still can move the VMs offline around, but its a little hard on the network and not that user friendly).

Is it possible to avoid having a separate storage box? That is, have only a couple of servers running? I've seen this kind of setup on some Hyper-V systems, whereas the "slave" server includes the storage. I believe that the slave is constantly replicating the state of the master?

Give GlusterFS a look, it could be an really good option for your rather small setup. It may work with two nodes and is not that hard to setup, also although it would be a little better to have the storage on separate machines you can also install GLusterFS on the PVE nodes. I'd give it a try if it matches your needs regarding performance and usefulness.

Is this possible on pve and, if so, would you recommend against it?

I would always recommend against a two node system if something like HA or more reliability is needed, that said I understand that there are a lot use cases where two node clusters make sense, and if it's only to control two node over a single interface or doing a manual failover if a node fails.

Also it depends on what and how much you want to run on it.

 

[cosmos]

Thanks for the info, I'll give GlusterFS a look.

I am not looking for real time migrate etc. I want to use these in a scenario whereas a node running 2-3 low-load Windows VMs hardware fails, in which case I would like to be able to start a backup node from a state that was as "near" as possible to the state the main node was in, before crashing.

So, what I might have to do is setup two nodes, in exact the same way. Also setup the exact same VMs in both nodes, having storage for them a GlusterFS volume. Is that correct?

I hope I'll be able to dig some info in this forum, this looks like a very nice project!

And again, thanks for the informative response, much appreciated.

 

[t.lamprecht]

I am not looking for real time migrate etc. I want to use these in a scenario whereas a node running 2-3 low-load Windows VMs hardware fails, in which case I would like to be able to start a backup node from a state that was as "near" as possible to the state the main node was in, before crashing.
So, what I might have to do is setup two nodes, in exact the same way. Also setup the exact same VMs in both nodes, having storage for them a GlusterFS volume. Is that correct?

Hmm, you can also install samba and then GlusterFS may directly export Samba Shares (or how its called in the windows world), with that you could map the samba share in the windows VMs as a drive and use it for the important data/data for applications, you may loose some performance here but you also get reliability. You could run such a VM on each node and if one node goes down you could continue wit the VM running on the other (you would need to simply start the applications needed or activate them).

There would be also another way, move the config to the other node if one goes down and simply start it there, here is no need for any samba mappings or application starting but you have to boot the VM (should not be too much time). You can move the VM fast and easy through the web GUI (migrate button) but for this quorum is needed, if you have two nodes then you need to give one quorum manually if the other fails, do as root over the command line:

pvecm expected 1
# or
pvecm e 1

This tells the remaining node temporary that it should only expected one vote (the vote from itself) and that it is safe to continue operation in the cluster and allows to start VMs again.
After you repaired the other Node you can reset it to 2.

I would prefer the second solution, but possible only because I don't really want to touch windows VMs, see what works better for you and use that.

If I over explained something please just ignore it and if you have question left feel free to ask, I may have caused some confusing with the project specific terminology and listing all solution I could think of at once

 

[cosmos]

Thanks, I'll most likely (try to) follow the second approach.

Which howto (if any) would you recommend for me to start reading?

 

[mmenaz]

cut
There would be also another way, move the config to the other node if one goes down and simply start it there, here is no need for any samba mappings or application starting but you have to boot the VM (should not be too much time). You can move the VM fast and easy through the web GUI (migrate button) but for this quorum is needed, if you have two nodes then you need to give one quorum manually if the other fails, do as root over the command line:

pvecm expected 1
# or
pvecm e 1

This tells the remaining node temporary that it should only expected one vote (the vote from itself) and that it is safe to continue operation in the cluster and allows to start VMs again.
After you repaired the other Node you can reset it to 2.

I don't understand, because my experience with 2 nodes on shared sotrage tells me that you have to manually move the config files after having "unlocked" the shared config directory through quorum assignment. So you can't use the web gui to migrate the vm simply because if you click on the vm on the dead node it timeouts and shows an error.
All the "migration" needs to be done through shell. I've asked multiple times for a more "user friendly" solution in this forum but was always denied.
People that sell VMWare 2 node cluster can have HA, migration and a lot of thing with a consinstent saving in hardware and licenses (Guest licenses too, since they are often related to the number of nodes of the cluster, even if you run it just in one or 2 nodes using the 3° just for quorum).
Any clarification is apreciated

 

[t.lamprecht]

I don't understand, because my experience with 2 nodes on shared sotrage tells me that you have to manually move the config files after having "unlocked" the shared config directory through quorum assignment. So you can't use the web gui to migrate the vm simply because if you click on the vm on the dead node it timeouts and shows an error.

I understand you in this regard, but its not totally easy. Our cluster file system has the base principle that each node owns it VMs configs, and only itself can touch it and thus also move it, this gives us consistency and error safeness guarantees in this regards, we should try to not circumvent that if possible. So we only steal a other nodes config if its managed by HA and was fenced.

So a workaround would be three nodes, or at least two nodes + quorum devices and then you not only can simply to that, but also it gets done automatically if the VM is HA managed.

VMWare 2 node cluster can have HA

Ill bet I can break that "HA" 2 node HA is marketing speech, I said that often and will continue saying that, as it is a fundamental rule of our world.

migration

You can always migrate too, what do I miss here, that its a little more inconvenient if one node is dead is a bummer, I guess, but you can do it.
And further the licensing model from VMWare is less simpler and more expensive, AFAIK, for a third "quorum" only node you just need corosync (the pve one) and pve-cluster, nothing more, no license, sorry I don't see the point here...

That said I'll give a look if we/I can come up with a good idea/solution regarding recovery of VMs thru the node without throwing anything away

Oh, also that is maybe a little off topic here, so I don't know how much vaule it gives to this thread discussing it here

 

[t.lamprecht]

Which howto (if any) would you recommend for me to start reading?

Hmm, for GlusterFS that could worth a read http://www.gluster.org/community/documentation/index.php/QuickStart (Step 4 is not needed for you).
For the PVE Cluster there's the wiki and the pve-docs project which is in work currently, beginning with 4.2 man pages should be more useful.
It depends also on the knowledge/experience you have already with PVE - installing and configuring the cluster should be quite easily doable:
https://pve.proxmox.com/wiki/Proxmox_VE_4.x_Cluster

 

[cosmos]

Thank you for all information!