Moving from 3-node full-mesh SDN to a 6-node using switches

[traveltrex]

Hi there,

we currently have a 3-node full-meshed SDN using OpenFabric for our ceph backend (used on separate interfaces and networks). Ceph backend uses 25gbit/s FC (we have 2x 2 Port Cards). We want to increase our cluster by 3 nodes but dissolving the SDN in the process, as the setup using a switch with this many nodes is much simpler and less cost-intensive.

What would be the best way to switch away from SDN without downtime?

Thanks in advance!

Patrick

 

[woma]

Hi Patrick,

Because new and old machines are same numbers you will have resources to temporarily run all guests only using new machines, so my safe and boring approach would be:

• build new 3 node cluster connected to switches
• setup up and test virtual network bridges and storage needed by your guests
• add both clusters to Proxmox Datacenter Manager
• PDM: cross cluster live migration of guests from old to new cluster
• shutdown old cluster machines
• rewire networking and connect old machines to switches
• perform fresh install of PVE on old machines
• join old machines to new cluster

If you don't like or have PDM, "qm remote-migrate" on CLI is possible.

Source and target clusters both keep stable configuration and network cabling while doing that transfer.
You can test new cluster before migrating guests to it and you do not touch old known working cluster either.