[SOLVED] Streched unbalanced cluster

M

ManuS

New Member
Jan 9, 2025
4
0
1
Hello,
Is it supported to create a stretched Proxmox cluster between two data centers with a different number of nodes in each?
The storage would not be Ceph but based on Metro replication of two storage arrays with active/active access on Luns (iSCSI) from both datacenters.
The desired configuration would be a stretched cluster with 4 Proxmox servers in one data center and 3 in the other.
Thank you for your help.
 
Hi @ManuS , welcome to the forum.

The supported/recommended configuration is to have equal number of nodes in each site with additional node in a third site.
With your proposed configuration, if SiteA(4node) fails, the SiteB(3node) will not have a majority/quorum of cluster members (3 out of 7). The services will not run without manual intervention.

Cheers.

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
 
  • Like
Reactions: floh8, ManuS and Johannes S
Hi @bbgeek17,
Many thanks for your fast and clear answer.
I'll find another server to have equal number of nodes in each Datacenter.
But I'll not have another server for the third site.
Can this proxmox server in third site can run in a virtual machine as it will not run any VM or container on it ? Is it supported ?
Thanks again
 
ManuS said:
Can this proxmox server in third site can run in a virtual machine as it will not run any VM or container on it ?
Click to expand...
It can. But that's not the goal here. If you want to run VMs there it needs to be part of the cluster. Then the whole connectivity - including all VLANs/Subnets - needs to be available with low latency, right?

For PVE Quorum, which has nothing to do with Ceph: if you just put a Quorum Device on the third location then the requirements are massively lower. It can be located in any network and the connection is allowed to be (relatively) slow and it also allows high latency.

Disclaimer: I've never setup such a construct - just replying with my "base knowledge"...
 
  • Like
Reactions: Johannes S
ManuS said:
Can this proxmox server in third site can run in a virtual machine as it will not run any VM or container on it ?
Click to expand...
Yes, the PVE server can run as a VM in another hypervisor (PVE, ESXi, HyperV, etc). It does not even need to be a PVE server. It can be a Debian VM, and has very low specs. The QDevice is described here: https://pve.proxmox.com/wiki/Cluster_Manager#_corosync_external_vote_support

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
 
  • Like
Reactions: ManuS and Johannes S
bbgeek17 said:
Yes, the PVE server can run as a VM in another hypervisor (PVE, ESXi, HyperV, etc). It does not even need to be a PVE server. It can be a Debian VM, and has very low specs. The QDevice is described here: https://pve.proxmox.com/wiki/Cluster_Manager#_corosync_external_vote_support

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
Click to expand...
Many thanks again for your answer.
We'll take a look to run this Proxmox server for third site in a VM
Regards
 
alexskysilk said:
The number of nodes isnt the issue; understand that since all data is replicated between site a and site b, you can only use the capacity of the smaller side no matter what you do- there is no benefit to having more OSD space on one side.
Click to expand...
We'll not use Ceph for storage, but Aray based synchronus metro replication as written in my first message
 
alexskysilk said:
I dont understand what that means. if you're not using ceph for storage, why are you using it at all?
Click to expand...
I don't think Op ever said they would use Ceph. Its the opposite - they specifically said that they are not using Ceph, probably because a common advice in general (especially outside of this forum) is to use Ceph.

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
 
You must log in or register to reply here.
Top Bottom
Back