Proxmox + CEPH - using same NICs but different VLANs for public/cluster networks

[mgaudette112]

Hi,

I am close to turning on my Ceph setup and would like some advice regarding the cluster network. As it stands, right now I do not have enough NICs for a Ceph cluster network, so I was originally going to go with Ceph having only a public network. I've done that before on another cluster, it worked well enough for my needs.

I will be using (on each host) two NICs ports for Ceph with LACP, with the two ports connected to two different switches.

I was thinking for this newer cluster to setup 2 VLANs for the Ceph (public and cluster) networks, but use the same network ports for both. I understand there aren't a lot of benefits to this, in terms of capacity/bandwidth/redundancy but I feel like it possibly might help with troubleshooting, monitoring and especially with moving the Ceph cluster network to a best-practice setup later. At the cost of a little more complexity.

Any advice against doing this? Or should I just stick to a public Ceph network given my lack of network ports?

 

[VictorSTS]

That's what I usually do: using 2 vlans over the same bond add the benefit that if you ever need to separate the cluster network to a different bond, no Ceph configuration changes will have to be made (i.e. add the cluster network and restart services), making that conversion process trivial by just moving the Ceph Cluster vlan to a different bond.

 

[mgaudette112]

That's what I usually do: using 2 vlans over the same bond add the benefit that if you ever need to separate the cluster network to a different bond, no Ceph configuration changes will have to be made (i.e. add the cluster network and restart services), making that conversion process trivial by just moving the Ceph Cluster vlan to a different bond.

Thank you for confirming my plan works for at least someone else.