frr.conf and frr.conf.local are not merging completely

Tekuno-Kage · Thursday at 16:39

I need feedback on what I'm experiencing:

My Goal

Make the FRR merge (SDN-generated /etc/frr/frr.conf + /etc/frr/frr.conf.local) produce the same active configuration I get when I manually apply the changes via vtysh. Currently, that is not the case happening.

Environment (facts)

Proxmox VE 9.x (clean install)
FRRouting 10.3.1
Proxmox SDN (EVPN/VXLAN; OSPF underlay)

Inputs to the merge (facts)

A) SDN-generated /etc/frr/frr.conf :

Code:

frr version 10.3.1
frr defaults datacenter
hostname RvraStgo-ProdPVE-01
log syslog informational
service integrated-vtysh-config
!
!
interface CEPH_Clstr
 access-list pve_ospf_RvraStgo_ips permit 10.22.51.0/24
 ip ospf area 10.22.0.2
!
interface CEPH_Pblc
 access-list pve_ospf_RvraStgo_ips permit 10.22.52.0/24
 ip ospf area 10.22.0.2
!
interface vmbr0
 no ip ospf passive
!
router bgp 65520
 bgp router-id 10.22.0.31
 no bgp hard-administrative-reset
 no bgp default ipv4-unicast
 coalesce-time 1000
 no bgp graceful-restart notification
 neighbor VTEP peer-group
 neighbor VTEP remote-as 65520
 neighbor VTEP bfd
 neighbor VTEP update-source dummy_RvraStgo
 neighbor 10.22.0.32 peer-group VTEP
 neighbor 10.22.0.33 peer-group VTEP
 !
 address-family l2vpn evpn
  neighbor VTEP activate
  neighbor VTEP route-map MAP_VTEP_IN in
  neighbor VTEP route-map MAP_VTEP_OUT out
  advertise-all-vni
 exit-address-family
exit
!
router ospf
 passive interface default
exit
!
route-map MAP_VTEP_IN permit 1
exit
!
route-map MAP_VTEP_OUT permit 1
exit
router ospf
 ospf router-id 10.22.0.31
exit
!
interface dummy_RvraStgo
 ip ospf area 10.22.0.2
 ip ospf passive
exit
!
interface vmbr0
 ip ospf area 10.22.0.2
exit
!
access-list pve_ospf_RvraStgo_ips permit 10.22.0.0/24
!
route-map pve_ospf permit 100
 match ip address pve_ospf_RvraStgo_ips
 set src 10.22.0.31
exit
!
ip protocol ospf route-map pve_ospf
!
!
line vty

B) Local /etc/frr/frr.conf.local (intent is to augment A):

Code:

!
interface vmbr0
 no ip ospf passive
exit
!
interface CEPH_Clstr
 ip ospf area 10.22.0.2
exit
!
interface CEPH_Pblc
 ip ospf area 10.22.0.2
exit
!
access-list pve_ospf_RvraStgo_ips permit 10.22.51.0/24
access-list pve_ospf_RvraStgo_ips permit 10.22.52.0/24
!
router ospf
 passive-interface default
exit
!

Reference Information; Interfaces (facts):

Code:

root@RvraStgo-ProdPVE-01:~# vtysh -c "show int brief"
Interface       Status  VRF             Addresses
---------       ------  ---             ---------
CEPH_Clstr      up      default         10.22.51.31/32
                                        fe80::f48b:c6ff:fe58:3051/64
CEPH_Pblc       up      default         10.22.52.31/32
                                        fe80::2044:b5ff:fe80:c1ac/64
Trnsprt_01      up      default         fe80::42b0:34ff:fef9:43c3/64
dummy_RvraStgo  up      default         10.22.0.31/32
                                        fe80::443c:32ff:fe21:37cb/64
eno1            up      default
lo              up      default
vSwitch         up      default         fe80::ec25:cfff:fe68:af80/64
vmbr0           up      default         10.22.3.31/24
                                        fe80::42b0:34ff:fef9:43c3/64
vxlan_vSwitch   up      default

Result after GUI “Apply” or reboot (facts)

Code:

root@RvraStgo-ProdPVE-01:~# vtysh -c "show run"
Building configuration...

Current configuration:
!
frr version 10.3.1
frr defaults datacenter
hostname RvraStgo-ProdPVE-01
log syslog informational
no ip forwarding
no ipv6 forwarding
service integrated-vtysh-config
!
route-map MAP_VTEP_IN permit 1
exit
!
route-map MAP_VTEP_OUT permit 1
exit
!
route-map pve_ospf permit 100
 match ip address pve_ospf_RvraStgo_ips
 set src 10.22.0.31
exit
!
interface CEPH_Clstr
 ip ospf area 10.22.0.2
exit
!
interface dummy_RvraStgo
 ip ospf area 10.22.0.2
 ip ospf passive
exit
!
interface vmbr0
 ip ospf area 10.22.0.2
 no ip ospf passive
exit
!
router bgp 65520
 bgp router-id 10.22.0.31
 no bgp hard-administrative-reset
 no bgp default ipv4-unicast
 coalesce-time 1000
 no bgp graceful-restart notification
 neighbor VTEP peer-group
 neighbor VTEP remote-as 65520
 neighbor VTEP bfd
 neighbor VTEP update-source dummy_RvraStgo
 neighbor 10.22.0.32 peer-group VTEP
 neighbor 10.22.0.33 peer-group VTEP
 !
 address-family l2vpn evpn
  neighbor VTEP activate
  neighbor VTEP route-map MAP_VTEP_IN in
  neighbor VTEP route-map MAP_VTEP_OUT out
  advertise-all-vni
 exit-address-family
exit
!
router ospf
 ospf router-id 10.22.0.31
 passive-interface default
exit
!
access-list pve_ospf_RvraStgo_ips seq 5 permit 10.22.51.0/24
access-list pve_ospf_RvraStgo_ips seq 10 permit 10.22.52.0/24
access-list pve_ospf_RvraStgo_ips seq 15 permit 10.22.0.0/24
!
ip protocol ospf route-map pve_ospf
!
end

The desired configuration was manually applied by adding it to /etc/frr/frr.conf.local using the vtysh:

Code:

root@RvraStgo-ProdPVE-01:~# vtysh

Hello, this is FRRouting (version 10.3.1).
Copyright 1996-2005 Kunihiro Ishiguro, et al.

RvraStgo-ProdPVE-01# config t
RvraStgo-ProdPVE-01(config)# !
RvraStgo-ProdPVE-01(config)# interface vmbr0
interface CEPH_Pblc
 ip ospf area 10.22.0.2
exit
!
access-list pve_ospf_RvraStgo_ips permit 10.22.51.0/24
access-list pve_ospf_RvraStgo_ips permit 10.22.52.0/24
!
router ospf
 passive-interface default
exit
!RvraStgo-ProdPVE-01(config-if)#  no ip ospf passive
RvraStgo-ProdPVE-01(config-if)# exit
RvraStgo-ProdPVE-01(config)# !
RvraStgo-ProdPVE-01(config)# interface CEPH_Clstr
RvraStgo-ProdPVE-01(config-if)#  ip ospf area 10.22.0.2
RvraStgo-ProdPVE-01(config-if)# exit
RvraStgo-ProdPVE-01(config)# !
RvraStgo-ProdPVE-01(config)# interface CEPH_Pblc
RvraStgo-ProdPVE-01(config-if)#  ip ospf area 10.22.0.2
RvraStgo-ProdPVE-01(config-if)# exit
RvraStgo-ProdPVE-01(config)# !
RvraStgo-ProdPVE-01(config)# access-list pve_ospf_RvraStgo_ips permit 10.22.51.0/24
RvraStgo-ProdPVE-01(config)# access-list pve_ospf_RvraStgo_ips permit 10.22.52.0/24
RvraStgo-ProdPVE-01(config)# !
RvraStgo-ProdPVE-01(config)# router ospf
RvraStgo-ProdPVE-01(config-router)#  passive-interface default
RvraStgo-ProdPVE-01(config-router)# exit
RvraStgo-ProdPVE-01(config)# !
RvraStgo-ProdPVE-01(config)# exit
RvraStgo-ProdPVE-01#
RvraStgo-ProdPVE-01# show run
Building configuration...

Current configuration:
!
frr version 10.3.1
frr defaults datacenter
hostname RvraStgo-ProdPVE-01
log syslog informational
no ip forwarding
no ipv6 forwarding
service integrated-vtysh-config
!
route-map MAP_VTEP_IN permit 1
exit
!
route-map MAP_VTEP_OUT permit 1
exit
!
route-map pve_ospf permit 100
 match ip address pve_ospf_RvraStgo_ips
 set src 10.22.0.31
exit
!
interface CEPH_Clstr
 ip ospf area 10.22.0.2
exit
!
interface CEPH_Pblc
 ip ospf area 10.22.0.2
exit
!
interface dummy_RvraStgo
 ip ospf area 10.22.0.2
 ip ospf passive
exit
!
interface vmbr0
 ip ospf area 10.22.0.2
 no ip ospf passive
exit
!
router bgp 65520
 bgp router-id 10.22.0.31
 no bgp hard-administrative-reset
 no bgp default ipv4-unicast
 coalesce-time 1000
 no bgp graceful-restart notification
 neighbor VTEP peer-group
 neighbor VTEP remote-as 65520
 neighbor VTEP bfd
 neighbor VTEP update-source dummy_RvraStgo
 neighbor 10.22.0.32 peer-group VTEP
 neighbor 10.22.0.33 peer-group VTEP
 !
 address-family l2vpn evpn
  neighbor VTEP activate
  neighbor VTEP route-map MAP_VTEP_IN in
  neighbor VTEP route-map MAP_VTEP_OUT out
  advertise-all-vni
 exit-address-family
exit
!
router ospf
 ospf router-id 10.22.0.31
 passive-interface default
exit
!
access-list pve_ospf_RvraStgo_ips seq 5 permit 10.22.51.0/24
access-list pve_ospf_RvraStgo_ips seq 10 permit 10.22.52.0/24
access-list pve_ospf_RvraStgo_ips seq 15 permit 10.22.0.0/24
!
ip protocol ospf route-map pve_ospf
!
end
RvraStgo-ProdPVE-01#

Observed running config (relevant diffs):

Code:

root@RvraStgo-ProdPVE-01:~# diff -u /etc/frr/running.conf /etc/frr/reboot.conf
--- /etc/frr/running.conf       2025-10-29 13:18:49.369225060 -0400
+++ /etc/frr/reboot.conf        2025-10-29 13:29:13.343840139 -0400
@@ -25,10 +25,6 @@
  ip ospf area 10.22.0.2
 exit
 !
-interface CEPH_Pblc
- ip ospf area 10.22.0.2
-exit
-!
 interface dummy_RvraStgo
  ip ospf area 10.22.0.2
  ip ospf passive
@@ -62,7 +58,6 @@
 !
 router ospf
  ospf router-id 10.22.0.31
- passive-interface default
 exit
 !
 access-list pve_ospf_RvraStgo_ips seq 5 permit 10.22.51.0/24
root@RvraStgo-ProdPVE-01:~#

Could you advise on the steps I need to take, whether related to my part or the SDN code, to ensure that the Proxmox SDN merge produces the same results as the vtysh CLI? If you need any additional information or outputs from me, I am happy to provide them. Thank you very much for your help.

ggoller · Friday at 15:02

Ah, yes - I apologize. Our frr.conf.local merging is very broken, we're hoping to rework this completely in the next few versions.
In this case you can fix it by adding a seq attribute to the access-list.

So change your frr.conf.local to:

Code:

!
interface vmbr0
 no ip ospf passive
exit
!
interface CEPH_Clstr
 ip ospf area 10.22.0.2
exit
!
interface CEPH_Pblc
 ip ospf area 10.22.0.2
exit
!
access-list pve_ospf_RvraStgo_ips seq 10 permit 10.22.51.0/24
access-list pve_ospf_RvraStgo_ips seq 11 permit 10.22.52.0/24
!
router ospf
 passive-interface default
exit
!

This should work.

Tekuno-Kage · Friday at 17:02

ggoller said:
Ah, yes - I apologize. Our frr.conf.local merging is very broken, we're hoping to rework this completely in the next few versions.

Thanks for the answer @ggoller, and for clearly confirming the issue and having an action plan. We really appreciate that!

To clarify, the access lists are not experiencing issues; they have been merged successfully, and the problem is more consistent with the interfaces.

I've found that using the 'passive-interface default' setting in your OSPF protocol configuration can be pretty beneficial. It helps enhance security by minimizing unnecessary OSPF traffic. For the specific links where you need to establish relationships, incorporating the 'no ip ospf passive' command can also be helpful. This approach appears to streamline OSPF management while ensuring that essential connectivity remains intact. Just a thought that might help!

Please let us know when to test the rework so we can assist with validation.

Search

Search

frr.conf and frr.conf.local are not merging completely

Tekuno-Kage

Renowned Member

Inputs to the merge (facts)

ggoller

Proxmox Staff Member

Tekuno-Kage

Renowned Member

We value your privacy

frr.conf and frr.conf.local are not merging completely

Tekuno-Kage

Renowned Member

Inputs to the merge (facts)​

ggoller

Proxmox Staff Member

Tekuno-Kage

Renowned Member

We value your privacy

Inputs to the merge (facts)