Clustering pros vs. cons

[showiproute]

Hello everyone,

I am currently migrating from ESXi6.7 to Proxmox.

As I have two servers which host different VMs on different datastores I would like to know if there are any pros as well as cons if I would link both Proxmox servers to a cluster?!

Both servers are not equal when it comes to hardware (mostly different storage sizes).

 

[Stefan_R]

Not really any downsides, if you don't enable any of the advanced features (HA, replication, etc...) it just allows you to migrate VMs and control both servers from one interface. If you have a network connection between the two nodes, you should consider clustering them.

See also our documentation on the topic: https://pve.proxmox.com/pve-docs/chapter-pvecm.html

 

[showiproute]

Hi Stefan,

I have ordered another 10 Gbit NIC - so both server are connected via two 10 GBASE-T connections to a switch.
What about clustering a very remote side with limited internet connection?
Yesterday I (unfortunately) clustered my existing backup server at a remote place where "Fuchs und Hase" still says "Good Night" to each other () and it barely destroyed the existing configuration on the remote server - I had to rebuild the VMs and attach the existing disks.

Is there a way to say my existing cluster: This is a new one, just recognize it, show some stats but do not do any HA, Backup, .... to it?

 

[showiproute]

I also want not to have any issues with quorum as during the winter a power outage is quite common in this rural area.

 

[Stefan_R]

Clustering has a latency requirement, and should only be done with local machines. High latency can cause issues with corosync quorum, as you mention yourself.

Is there a way to say my existing cluster: This is a new one, just recognize it, show some stats but do not do any HA, Backup, .... to it?

Not sure what you mean? Removing a node from the cluster can be done according to the documentation I linked earlier...

 

[showiproute]

Not sure what you mean? Removing a node from the cluster can be done according to the documentation I linked earlier...

I was able to separate the "lost" server from my main cluster but at the lost machine I was not able to leave the cluster as it was the only existing machine inside this cluster.

Regarding documentation: I have read that already - but what I do not get: Why is the pveproxy-certificate being replaced?
Is there a possiblity to replace the cluster CA certificate?

My plan would be: Create a CSR on Proxmox, let it signed by my AD-DC and use it as intermediate CA - is that possible?

 

[oguz]

I was able to separate the "lost" server from my main cluster but at the lost machine I was not able to leave the cluster as it was the only existing machine inside this cluster.

have you followed the instructions here? [0]

once you remove the corosync configuration files and restart pve-cluster it should be cleaned (you can check with pvecm status, it should error saying that node isn't a part of cluster)

[0]: https://pve.proxmox.com/wiki/Cluster_Manager#pvecm_separate_node_without_reinstall

 

[showiproute]

Yes, more or less I did exactly that - but unfortunately my backup server removed the whole VM configuration and I had to rebuild it manually after the un-clustering.

 

[oguz]

Ensure all shared resources are cleanly separated! Otherwise you will run into conflicts and problems.

 

[oguz]

Why is the pveproxy-certificate being replaced?
Is there a possiblity to replace the cluster CA certificate?

My plan would be: Create a CSR on Proxmox, let it signed by my AD-DC and use it as intermediate CA - is that possible?

pveproxy cert is being replaced so that the cluster nodes can talk to each other via the API as well.
the certificate can be replaced by you. see here [0]

[0]: https://pve.proxmox.com/wiki/Certificate_Management

 

[showiproute]

I have already replaced the pveproxy certificate.
My intention was to replace the CA certificate.

 

[oguz]

in that case read here[0].

it's stored in /etc/pve/priv/pve-root-ca.key


[0]: https://pve.proxmox.com/wiki/Proxmox_Cluster_File_System_(pmxcfs)

 

[showiproute]

Also done that but according to the wiki topic about Certificates it's mentioned that playing around with CAs in the filesystem is not a good idea:

Do not replace or manually modify the automatically generated node certificate files in /etc/pve/local/pve-ssl.pem and /etc/pve/local/pve-ssl.key or the cluster CA files in /etc/pve/pve-root-ca.pem and /etc/pve/priv/pve-root-ca.key.

 

[oguz]

Also done that but according to the wiki topic about Certificates it's mentioned that playing around with CAs in the filesystem is not a good idea:

yes it's not a good idea if you don't know what you're doing because if done incorrectly it will mess with the cluster communication.

but you said you wanted to do it so i'm telling you

 

[showiproute]

Yeah and I am also thankful for your advise!
I just thought that there is kind of a similar process as replacing the pveproxy certificate via the WebGUI.