What i currently have is 6 node cluster all running on a single 10GB network,
What im looking to setup is the same cluster running on three separate physical networks,
1. PVE Management network
2. Application network (VMs)
3. Ceph
I'm assuming this is pretty standard but i can't seem to get things to work right splitting out the second network for ceph. Each node has 4 1GB nics to test the new network I'm plugging the second port (eno2) into a separate 'dumb' layer 2 switch not connected to any other network and assigning eno2 a new bridge.
After configuring and wiring each node this way i'm still seeing requests go in and out of eno1 on each node to hit the 172.16.3.x subnet.
I feel like there's something basic i'm missing here but i'm not sure as to what that might be. Try as i might i've never really been able to wrap my head around more complex networking. Hoping to avoid setting up VLANs for this as i've tried a few times and never been able to get things to work that way either.
The end goal i'm hoping to achieve is route ceph through the layer 2 switch to remove the load from the router and layer 3 switch that run the rest of the network. In addition i'm hoping to isolate the ceph network so it's also only accessible from inside of proxmox and guests but that's secondary at this point in time.
What im looking to setup is the same cluster running on three separate physical networks,
1. PVE Management network
2. Application network (VMs)
3. Ceph
I'm assuming this is pretty standard but i can't seem to get things to work right splitting out the second network for ceph. Each node has 4 1GB nics to test the new network I'm plugging the second port (eno2) into a separate 'dumb' layer 2 switch not connected to any other network and assigning eno2 a new bridge.
After configuring and wiring each node this way i'm still seeing requests go in and out of eno1 on each node to hit the 172.16.3.x subnet.
I feel like there's something basic i'm missing here but i'm not sure as to what that might be. Try as i might i've never really been able to wrap my head around more complex networking. Hoping to avoid setting up VLANs for this as i've tried a few times and never been able to get things to work that way either.
The end goal i'm hoping to achieve is route ceph through the layer 2 switch to remove the load from the router and layer 3 switch that run the rest of the network. In addition i'm hoping to isolate the ceph network so it's also only accessible from inside of proxmox and guests but that's secondary at this point in time.