Suggested networking scheme

[tettabeta]

Hello all
we have set up a cluster with three nodes plus one backup server.
All cluster nodes are the same type/memory/storage, while the backup server has more memory and storage.

On each server we have two NiCs: both boards are two ports, one board is 1GbE the other one is a 10GbE,
We have also two switches (1GbE and 10GbE) to connect all the above.
Average load is low/moderate.

We are wondering what is the preferred approach with above available hardware.
I.e.:
How many ports for Cluster storage / backup communications?
How many ports for corosync?
We can get along with just one port from each server for public data access.

Best Regards
Davide

 

[Pierre-Yves]

Hello
We're planning to setup a new cluster with nearly the same setup (3 main node server and 1 backup server).
2 x 10 Gbs for connecting Proxmox hosts to shared storage (iscsi multipath for us) with no Vlan (dedicated to SAN)
1 x 1 Gbs cards for each Proxmox node for admin purpose
3 x 1 Gbs bond/bridge in a LACP bond and VLAN support
With this, all the VMs or containers are connected to the bond/bridge once or several times (several netword cards for each VMs, each network cards are the same except VLAN)
We setup like this not to mix SAN packets and VMs packets.
With a bond supporting Vlans, there is a best effort load for the network trafic for VLAN.

Regards
Pierre-Yves

 

[tettabeta]

Hello Pierre
therefore you use the (bonded?) 10Gb channel to connect the cluster to the PBS, right ?
Thanks
Davide

 

[Pierre-Yves]

Hello
No the 10G bond is dedicated to SAN access.
No other traffic on this bond. We can't afford a network congestion on the SAN network.
Backup will go through the the 3 x 1 Gbs bond

Regards

 

[tettabeta]

Interesting: are you sending corosync traffic on any reserved link ?
Brgds
Davide

 

[Pierre-Yves]

The 1 gbs last network card is dedicated to https/corosync.