We have a 16 node cluster and are preparing to expand to 32 nodes.
Because of the hemmed in architecture of the Dell blades and the M1000e modular chassis, we had some interesting choices to make as far as the physical networking. Each server has 6x 10 GbE ports and 8x 1 GbE ports.
The 10 GbE ports are split evenly between the Ceph front-end and back-end networks (2 LACP LAGs 30 GbE each). The remaining 1 GbE interfaces are used for the production general purpose networking, host management network, Corosync link0 and link1, etc and they are bonded in active-backup.
I know it's been asked before, but not very recently: what is the official maximum # of nodes that PVE can support, and what is the maximum # of nodes you all have had to maintain in production?
How well do you think a 32 node cluster will do on a dedicated, low-latency 1 GbE network with multiple extra Corosync links?
Would it be advisable to additionally configure Corosync link2 and link3 to share with each 30 Gb Ceph network? And how does PVE and Corosync know which link to prefer when there are multiple Corosync links configured?
Because of the hemmed in architecture of the Dell blades and the M1000e modular chassis, we had some interesting choices to make as far as the physical networking. Each server has 6x 10 GbE ports and 8x 1 GbE ports.
The 10 GbE ports are split evenly between the Ceph front-end and back-end networks (2 LACP LAGs 30 GbE each). The remaining 1 GbE interfaces are used for the production general purpose networking, host management network, Corosync link0 and link1, etc and they are bonded in active-backup.
I know it's been asked before, but not very recently: what is the official maximum # of nodes that PVE can support, and what is the maximum # of nodes you all have had to maintain in production?
How well do you think a 32 node cluster will do on a dedicated, low-latency 1 GbE network with multiple extra Corosync links?
Would it be advisable to additionally configure Corosync link2 and link3 to share with each 30 Gb Ceph network? And how does PVE and Corosync know which link to prefer when there are multiple Corosync links configured?
