Apply changes to SDN configuration on a single node

[mfed]

I have a heavily customized networking configuration and for some reason not yet apparent to me it does not survive the ifupdown2 reload.
I made a change to my SDN configuration (changed one of the VXLAN VTEPs) and need to apply the change. If I do 'Apply' from GUI it will do reload on all servers at the same time, which will break my cluster. Is there a way to apply the configuration to one node at a time, so I can reboot each node one by one?
For now I can manually change the address in /etc/network/interfaces.d/sdn, but not sure if this is all that the SDN 'Apply' button does...

 

[spirit]

you can use the network reload configuration on the host, it will regenerated the sdn config only for this host + ifreload.
or command line, on the node, with "'pvesh set /nodes/<nodename>/network"

(the sdn apply button launch this pvesh command, host by host)

Just curious, do you need special tuning not available in sdn ? I could look to implement them if needed.

also, if ifupdown2 reload don't work fine, it could be great to have a debug log with "ifreload -a -d".

 

[mfed]

The heavy customization part mostly resides within FRR. It's an ECMP routed configuration with each server connecting to two L3 switches, each connection on its own subnet, using /32 addresses on dummy interface for communication between each other and using OSPF to exchange the routing information. The whole intention is to provide redundancy across two switches, and my switches (Mikrotik) were not reliable using an MLAG configuration, but do quite well with the L3 HW offloaded switching.

I tried the command you provided, thank you for that. From what I can see it looks like my FRR configuration stops working completely after I use that command, I did 'systemctl restart frr' from the console and it seemed to help to recover. I am going to use the script to do the pvesh command and then restart frr as a workaround for now...

Thank you again for your tips

 

[spirit]

The heavy customization part mostly resides within FRR. It's an ECMP routed configuration with each server connecting to two L3 switches, each connection on its own subnet, using /32 addresses on dummy interface for communication between each other and using OSPF to exchange the routing information. The whole intention is to provide redundancy across two switches, and my switches (Mikrotik) were not reliable using an MLAG configuration, but do quite well with the L3 HW offloaded switching.

ah ok, with ospf. Currently I'm doing the same with bgp at work (and it's 100% implemented with "bgp controller", ecmp with a loopback). I could look to implement ospf too later. . (I don't known too much how ospf is working, so it'll take some time)

Note that you can overide frr config generation, with a "/etc/frr/frr.conf.local" file on host

I tried the command you provided, thank you for that. From what I can see it looks like my FRR configuration stops working completely after I use that command, I did 'systemctl restart frr' from the console and it seemed to help to recover. I am going to use the script to do the pvesh command and then restart frr as a workaround for now...

Thank you again for your tips

mmm, maybe it's a frr reload bug. (frr reload is called after ifreload, but I known that for some config it can be buggy).
I never have tested ospf deeply.
do you run last packages version ? (8.2.2-1+pve1)

 

[mfed]

ah ok, with ospf. Currently I'm doing the same with bgp at work (and it's 100% implemented with "bgp controller", ecmp with a loopback). I could look to implement ospf too later. . (I don't known too much how ospf is working, so it'll take some time)

Note that you can overide frr config generation, with a "/etc/frr/frr.conf.local" file on host


mmm, maybe it's a frr reload bug. (frr reload is called after ifreload, but I known that for some config it can be buggy).
I never have tested ospf deeply.
do you run last packages version ? (8.2.2-1+pve1)

I tried to do bgp at first, but it didn't work right away and was a little bit steep learning curve, so I decided to do OSPF first. It's definitely on my plate to do my config with bgp later.

Regarding /etc/frr/frr.conf.local can you please elaborate how it might be helpful?

As for the frr reload, I have a suspicion that the issue with reload might be due to my decision to use frr config to assign IP addresses. Maybe the system expects the IPs to be assigned in /etc/network/interfaces. I will test this theory later. And yes, I run everything up-to-day to the no-subscription repository, so the frr is 8.2.2-1+pve1.

And below is my frr config, in case you want to try the OSPF config. The routes with admin distance 254 are supposed to be the last resort routes when OSPF fails for any reason.

ip route 192.168.9.12/32 192.168.251.1 254
ip route 192.168.9.12/32 192.168.252.1 254
ip route 192.168.9.13/32 192.168.251.1 254
ip route 192.168.9.13/32 192.168.252.1 254
ip route 192.168.10.12/32 192.168.251.1 254
ip route 192.168.10.12/32 192.168.252.1 254
ip route 192.168.10.13/32 192.168.251.1 254
ip route 192.168.10.13/32 192.168.252.1 254
!
interface dummy0
 ip address 192.168.10.11/32
 ip address 192.168.9.11/32
 ip ospf area 10
 ip ospf passive
exit
!
interface t1
 ip address 192.168.251.2/30
 ip ospf area 0
 ip ospf dead-interval 4
 ip ospf hello-interval 1
 ip ospf network point-to-point
 ip ospf retransmit-interval 1
exit
!
interface t2
 ip address 192.168.252.2/30
 ip ospf area 0
 ip ospf dead-interval 4
 ip ospf hello-interval 1
 ip ospf network point-to-point
 ip ospf retransmit-interval 1
exit
!
router ospf
 ospf router-id 192.168.9.11
 area 10 stub
exit
!
ip prefix-list PEERS-9 seq 5 permit 192.168.9.0/24 ge 24
ip prefix-list PEERS-10 seq 5 permit 192.168.10.0/24 ge 24
!
route-map PEERS permit 9
 match ip address prefix-list PEERS-9
 set src 192.168.9.11
exit
!
route-map PEERS permit 10
 match ip address prefix-list PEERS-10
 set src 192.168.10.11
exit
!
route-map PEERS permit 99
exit
!
ip protocol static route-map PEERS
!
ip protocol ospf route-map PEERS
!

 

[spirit]

Hi,sorry to be late.

Regarding /etc/frr/frr.conf.local can you please elaborate how it might be helpful?

about frr.conf.local, it's about evpn zone config generation with frr. Not related in your case (if I understand, you are doing ospf underlay + simple vxlan tunnel on top , without evpn ,right ?)

It's more to avoid that sdn config overide frr.conf when using evpn/bgp when you do custom config.

Maybe the system expects the IPs to be assigned in /etc/network/interfaces. I will test this theory later.

It's quite possible than ifreload remove ip address because they are not existing in /etc/network/interfaces.
then a reload/restart of frr reassign them.

(you can do a "ifreload -a -d" to have debug log)


maybe can you try in /etc/network/interfaces

auto t1
iface t1
    address 192.168.251.2/30

auto t2
iface t2
    address 192.168.252.2/30


auto dummy0
iface dummy0
    address 192.168.10.11/32
    address 192.168.9.11/32
    link-type dummy