Changing IPv4 subnet size on existing SDN Fabric

Sep 7, 2025
39
8
8
Is it possible to change the IPv4 subnet size on an existing SDN Fabric? I would like to go from a /24 to a /27, which won't require changing any IP addresses, since the three IP addresses allocated for the Ceph network will still fit. That will allow me to allocate some of the space to a dedicated Corosync network. I cannot just randomly pick a new IP address, as private address spaces are "randomly" allocated and routed internally within the organisation, so if I pick something and it is later allocated by networks, things will start breaking. I could ask for more space, but for reasons I would prefer not to.

However, when I click on edit in the web interface, I cannot change the address prefix to change the network size.:(

I am wondering whether I can just change it in /etc/pve/sdn/fabrics.cfg on each node and then reload the networking on each node in the cluster if "ifreload -a"? The system is not yet in production, nor does it have any running VMs other than test ones that can be shut down during the change. I would rather not redo the Ceph config.
 
Some brute force greping suggests the following files have the /24 in them

/etc/ceph/ceph.conf
/etc/frr/frr.conf
/etc/pve/ceph.conf
/etc/pve/sdn/fabrics.cfg

Easy enough to update them all with sed. I think I could restart the OpenFabric SDN with an "systemctl restart frr", but I am not sure about Ceph?
 
The following worked for me. Obviously, as the cluster is not production yet, turning all the VM's off and shutting down Ceph was an option.

1. update the config

sed -i s#.0/24\$#.0/27# /etc/ceph/ceph.conf
sed -i s#.0/24\$#.0/27# /etc/frr/frr.conf
sed -i s#.0/24\$#.0/27# /etc/pve/ceph.conf
sed -i s#.0/24\$#.0/27# /etc/pve/sdn/fabrics.cfg


2. Shutdown all VMs on every node
3. Set the following flags:

ceph osd set noout
ceph osd set nobackfill
ceph osd set norecover

4. reboot all the nodes
5. Set the following flags:

ceph osd unset noout
ceph osd unset nobackfill
ceph osd unset norecover