Large proxmox installations (clusters!). I need some developer's attention

[RRJ]

This thread is now added to favorites.
Great, glad we still have such people, who help and do not go nervous because of such people like us asking same questions million times over and over again.
Will report here as soon, as I start playing with HA.

 

[makje]

RRJ you may be overthinking the fencing part, with 300.000 euro to spend you will probably have many more than 3 nodes.
If 1 or 2 nodes fail, the others recognize those 2 as failed and cast a vote to kick them out. As they have majority they win and those 2 are cast out and whatever VMs were on those 2 nodes get restarted on other nodes.
You simply need to have enough spare nodes in your cluster to handle the redistribution of the VMs.

What you should be concerned about with that much money to spend is expectations of your bosses and data security.

Firstly, those VMs get restarted on another host but that takes a few minutes in which the VM is down. Its not a seamless takeover like VMWares Fault Tolerance!
Secondly fencing protects you from a failure of a node that hosts VMs, it does (as far as i can see) nothing about failures from your SAN.

So your SAN and it's data is your weak spot. Offcourse you have backups on completerly separate media (with these data volumes probably a TB NAS somewehere else) but with your SAN offline you have NO VMs running.
Now you can get a second SAN en have your first SAN replicate everything to your second SAN (often takes a additional license).

But then you only have your second SAN all ready to go, but as far as i know the Proxmox cluster has no idea of this second SAN and won't use it to startup the VMs again?
I reckon you could add the second SAN as storage to the cluster yourself (and remove the original SAN) and then start up all VMs again, but i don't see it happening automatically?

disclaimer: i am still researching the specifics on SAN myself in combination with Proxmox (or VMware, have got to keep an open mind).
I only have a few years experience with Proxmox in a 1 or 2 node cluster with local storage and no HA.

I don't want to make things more difficult for you, i simply have much the same questions as you do and have done some researching (was going to open a topic myself when i found yours)
I'm hoping there is a way to make the SAN redundant in Proxmox as i like this product so i hope there is someone else with a clear explanation of what is possible and how to do it

 

[makje]

RRJ you may be overthinking the fencing part, with 300.000 euro to spend you will probably have many more than 3 nodes.
If 1 or 2 nodes fail, the others recognize those 2 as failed and cast a vote to kick them out. As they have majority they win and those 2 are cast out and whatever VMs were on those 2 nodes get restarted on other nodes.
You simply need to have enough spare nodes in your cluster to handle the redistribution of the VMs.

What you should be concerned about with that much money to spend is expectations of your bosses and data security.

Firstly, those VMs get restarted on another host but that takes a few minutes in which the VM is down. Its not a seamless takeover like VMWares Fault Tolerance!
Secondly fencing protects you from a failure of a node that hosts VMs, it does (as far as i can see) nothing about failures from your SAN.

So your SAN and it's data is your weak spot. Offcourse you have backups on completerly separate media (with these data volumes probably a TB NAS somewehere else) but with your SAN offline you have NO VMs running.
Now you can get a second SAN en have your first SAN replicate everything to your second SAN (often takes a additional license).

But then you only have your second SAN all ready to go, but as far as i know the Proxmox cluster has no idea of this second SAN and won't use it to startup the VMs again?
I reckon you could add the second SAN as storage to the cluster yourself (and remove the original SAN) and then start up all VMs again, but i don't see it happening automatically?

disclaimer: i am still researching the specifics on SAN myself in combination with Proxmox (or VMware, have got to keep an open mind).
I only have a few years experience with Proxmox in a 1 or 2 node cluster with local storage and no HA.

I don't want to make things more difficult for you, i simply have much the same questions as you do and have done some researching (was going to open a topic myself when i found yours)
I'm hoping there is a way to make the SAN redundant in Proxmox as i like this product so i hope there is someone else with a clear explanation of what is possible and how to do it

 

[dietmar]

But then you only have your second SAN all ready to go, but as far as i know the Proxmox cluster has no idea of this second SAN and won't use it to startup the VMs again?

This is usually solved by using multipath. But this depends on the SAN.