Understanding Proxmox Networking

[Digitaldaz]

I'm trying to understand Proxmox networking internally.

I've currently set up a three node cluster with Ceph using the guide which configures 10Gb networking without a switch ie all directly attached using a bond.

I'm now realising that what this probably means is that the regular cluster traffic and any migrations are probably using the LAN network which as well as not being isolated is only 1Gb.

So my question is, can I get additional 10Gb cards and configure them using the same scenario as the Ceph and using another bond? Will this work or does the multicast or anything else cause a problem with this?

If necessary I will buy 10Gb switches but I want to avoid this if possible as I'll need two and also do not have the rackspace in the cab.

TIA
Daz

 

[Alwin]

Depending on how you setup your mesh network, if eg. routed then multicast is not forwarded. If it is broadcast then you can also put cluster and migration traffic on separate bonds. Especially migration and cluster traffic should be separated or cluster traffic through two corosync rings should be configured.

 

[Digitaldaz]

Depending on how you setup your mesh network, if eg. routed then multicast is not forwarded. If it is broadcast then you can also put cluster and migration traffic on separate bonds. Especially migration and cluster traffic should be separated or cluster traffic through two corosync rings should be configured.

Alwin, thanks for that, is there any books, docs that explain this?

 

[Alwin]

So you mean like this wiki page?
https://pve.proxmox.com/wiki/Full_Mesh_Network_for_Ceph_Server

 

[Digitaldaz]

Thanks, I think along with this one, that does it: https://pve.proxmox.com/wiki/Cluster_Manager