Proxmox 7.X biggest number of nodes in cluster !

[Dave.r]

Dear Proxmox team ,

A quick question regarding the number of possible nodes within proxmox 7.x cluster .
What is the limitation in terms of numbers of nodes in one cluster ?
We simply would like to understand how big the cluster can be ?
is it possible to have 100+ nodes in one cluster or simply need to split the nodes in 32 nodes per groups?
Also we would very much appreciate it if you kindly attach links/articles/pdfs ( if exist of course ) of a very large implementations of proxmox cluster limitations in terms of number of nodes .
Thank you very much for your help in advance .

Best Wishes
David

 

[Hannes Laimer]

Hey,

there is no hard limit, it mostly depends on your underlying hardware and network, some discussion about that already happened in [1] and [2]. You can also take a look at [3][4].

[1] https://forum.proxmox.com/threads/very-large-proxmox-cluster-possibility.55439/
[2] https://forum.proxmox.com/threads/max-cluster-nodes-with-pve6.55277/
[3] https://pve.proxmox.com/wiki/Cluster_Manager
[4] https://pve.proxmox.com/pve-docs/pve-admin-guide.html#_cluster_network

 

[spirit]

100 nodes, definitively not. (and If I remember , they are really a software limite around 90-100, but maybe this have changed).
But even corosync developper don't support 100 nodes cluster.

32 nodes with good network, low latency switch, no packet loss, (but really need to be tested) and maybe some tuning.

20 nodes, easy, without tuning.

 

[Tmanok]

I liked the idea mentioned in the second link, a cluster of clusters aka a business continuity cluster. I've often wondered how larger cloud enterprises handled such growth, I think they start turning to solutions such as OpenNebula or Cloudstack, personally I always find those types of platforms so much less featureful and lacking in "personality" compared to PVE, XCP, or ESXi.

The second link also mentioned only about 1Mbps of actual traffic, so the packets per second and latency are really what screw you over. I find that especially interesting, because you could throw tuned hardware at the problem (e.g. high freq CPU, high freq RAM, dedicated NIC, dedicated datacentre quality switching, fast IOPS root storage) but it sounds like the problem is ultimately software needing to scale better?

Also why was multicast removed? Sounds silly.