How many nodes can be in one cluster?

e100

Renowned Member
Nov 6, 2010
1,267
40
68
Columbus, Ohio
ulbuilder.wordpress.com
The wiki says "A Proxmox VE Cluster consists of several nodes (up to 16 physical nodes, probably more)."
http://pve.proxmox.com/wiki/Proxmox_VE_2.0_Cluster

I currently have 15 nodes in one of our clusters and plan to add two more for a total of 17.
In the future I could see us reaching 20 servers in this cluster.

Is there a limit to the number of nodes, if so, what is the limit?
Is there anything that can be adjusted to increase the limit?
 

dietmar

Proxmox Staff Member
Staff member
Apr 28, 2005
17,137
532
133
Austria
www.proxmox.com
Is there a limit to the number of nodes, if so, what is the limit?
Is there anything that can be adjusted to increase the limit?

The used corosync settings are optimized for less than 16 nodes. If you run more, you may need to
change totem protocol timeouts (man cororync.conf).
 

e100

Renowned Member
Nov 6, 2010
1,267
40
68
Columbus, Ohio
ulbuilder.wordpress.com
The used corosync settings are optimized for less than 16 nodes. If you run more, you may need to
change totem protocol timeouts (man cororync.conf).

I am not finding much information about this but here is what I found so far.
The man page does refer to some values that might need changed for > 32 and > 64 nodes.

join, token and consensus are three timeouts that might need changed based on what I have read on various mailing lists.

token is 1000ms by default
consensus is 1200ms by default
join is 50ms by default

This mail thread seems to have some good info on large corosync clusters:
http://www.mail-archive.com/openais@lists.linux-foundation.org/msg04232.html

One person said that token=1s and consensus=5s worked for his 48 node cluster.
Based on that I assume that if I slightly increased consensus my 17 node cluster would be fine, might even be ok without making any changes.
 

dietmar

Proxmox Staff Member
Staff member
Apr 28, 2005
17,137
532
133
Austria
www.proxmox.com
One person said that token=1s and consensus=5s worked for his 48 node cluster.
Based on that I assume that if I slightly increased consensus my 17 node cluster would be fine, might even be ok without making any changes.

Yes, I also think it will work.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!