Optimal Network Setup for a 13-Node Proxmox Cluster with NFS and 10Gbps/1Gbps NICs

[Extinct0005]

Hello Proxmox Community,

I am in the process of setting up a Proxmox VE cluster and am looking for advice on the optimal network configuration. Here are the details of my HW:

• Cluster Size: 13 Nodes + 1 Backup Server
• Network Interfaces per Node: 2x10Gbps NICs and 4x1Gbps NICs
• Storage: Shared NFS Storage

I am coming from XCP-ng community, and there I would: use 1x10Gbps for NFS/Storage, 1x10Gbps for VM traffic(internet), 1x1Gbps for management.

From what I am reading, the setup for proper Proxmox cluster will be different: I will use another 1x1Gbps for Corosync and I can also chain servers using another nic to have RRP.

I am looking for advice on the best practices of networking setup, including high speed of backups.

I appreciate any insights, experiences, or recommendations you can provide to help me optimize this setup. Thank you in advance for your assistance!

 

[pvps1]

i'd definitly go for bonding the 2 x 10gbs interfaces and run vmmigration etc and storage in vlans within. 2 x 1gbs for corosync, 2 x 1gbs bonding for non speed relevant vlans, uplinks, etc

 

[Extinct0005]

i'd definitly go for bonding the 2 x 10gbs interfaces and run vmmigration etc and storage in vlans within. 2 x 1gbs for corosync, 2 x 1gbs bonding for non speed relevant vlans, uplinks, etc

Thank you. Question - usually there is strong incentive to separate the storage network from anything else to make sure 1 uplink is just for underlying storage access. Bonding 2x10Gbps interfaces are kind of going against that. Is this not concern with the Proxmox?

 

[pvps1]

i'd prefere redundancy. migrations are done rarly, so 95% of the time the bond would be used for storage only.

depends on your situation but we dont do any network without redundant link

 

[Extinct0005]

i'd prefere redundancy. migrations are done rarly, so 95% of the time the bond would be used for storage only.

depends on your situation but we dont do any network without redundant link

So what are you saying is that having 2x10Gbps for storage/backup/internet with 2x1Gbps for corosync is good enough? What about management, is it used for migrations and needs to be beefier?

 

[pvps1]

you can separate management and migration network (and should)