Default route advertisement to SDN EVPN

kemeris · Wednesday at 18:50

Hello,

i was able to peer Fortigate VM (acts as VTEP) with Proxmox SND EVPN. Everything is working fine, but i can't see Fortigate as "Exit node" on proxmox cluster.
I suspect i need to announce default route from Fortigate?

I guest i need to add?

Code:

route-map MAP_VTEP_IPV4_IN permit 10
  match ip address prefix-list only_default
exit
router bgp 65000
  address-family ipv4 unicast
    neighbor VTEP activate
    neighbor VTEP route-map MAP_VTEP_IPV4_IN in

Here is my frr.conf

Code:

frr version 8.5.2

frr defaults datacenter

hostname proxmox1-1

log syslog informational

service integrated-vtysh-config

!

!

vrf vrf_z10001

 vni 10001

 ip route 10.0.21.0/24 null0

 ip route 10.0.22.0/24 null0

 ip route 10.0.23.0/24 null0

 ip route 10.0.24.0/24 null0

exit-vrf

!

vrf vrf_z10002

 vni 10002

 ip route 10.0.20.0/24 null0

 ip route 10.0.22.0/24 null0

 ip route 10.0.23.0/24 null0

 ip route 10.0.24.0/24 null0

 ipv6 route 2001:1ab9:f002:2::/118 null0

exit-vrf

!

vrf vrf_z10007

 vni 10007

 ip route 10.0.20.0/24 null0

 ip route 10.0.21.0/24 null0

 ip route 10.0.23.0/24 null0

 ip route 10.0.24.0/24 null0

 ipv6 route 2001:1ab9:f002:2::/118 null0

exit-vrf

!

vrf vrf_z10009

 vni 10009

 ip route 10.0.20.0/24 null0

 ip route 10.0.21.0/24 null0

 ip route 10.0.22.0/24 null0

 ip route 10.0.24.0/24 null0

 ipv6 route 2001:1ab9:f002:2::/118 null0

exit-vrf

!

vrf vrf_z10015

 vni 10015

 ip route 10.0.20.0/24 null0

 ip route 10.0.21.0/24 null0

 ip route 10.0.22.0/24 null0

 ip route 10.0.23.0/24 null0

 ipv6 route 2001:1ab9:f002:2::/118 null0

exit-vrf

!

router bgp 65000

 bgp router-id 10.0.4.1

 no bgp hard-administrative-reset

 no bgp default ipv4-unicast

 coalesce-time 1000

 no bgp graceful-restart notification

 neighbor VTEP peer-group

 neighbor VTEP remote-as 65000

 neighbor VTEP bfd

 neighbor 10.0.4.3 peer-group VTEP

 neighbor 10.0.4.4 peer-group VTEP

 neighbor 10.0.7.2 peer-group VTEP

 !

 address-family ipv4 unicast

  import vrf vrf_z10001

  import vrf vrf_z10002

  import vrf vrf_z10007

  import vrf vrf_z10009

  import vrf vrf_z10015

 exit-address-family

 !

 address-family ipv6 unicast

  import vrf vrf_z10001

  import vrf vrf_z10002

  import vrf vrf_z10007

  import vrf vrf_z10009

  import vrf vrf_z10015

 exit-address-family

 !

 address-family l2vpn evpn

  neighbor VTEP activate

  neighbor VTEP route-map MAP_VTEP_IN in

  neighbor VTEP route-map MAP_VTEP_OUT out

  advertise-all-vni

 exit-address-family

exit

!

router bgp 65000 vrf vrf_z10001

 bgp router-id 10.0.4.1

 no bgp hard-administrative-reset

 no bgp graceful-restart notification

 !

 address-family ipv4 unicast

  redistribute connected

 exit-address-family

 !

 address-family ipv6 unicast

  redistribute connected

 exit-address-family

 !

 address-family l2vpn evpn

  default-originate ipv4

  default-originate ipv6

 exit-address-family

exit

!

router bgp 65000 vrf vrf_z10002

 bgp router-id 10.0.4.1

 no bgp hard-administrative-reset

 no bgp graceful-restart notification

 !

 address-family ipv4 unicast

  redistribute connected

 exit-address-family

 !

 address-family ipv6 unicast

  redistribute connected

 exit-address-family

 !

 address-family l2vpn evpn

  default-originate ipv4

  default-originate ipv6

 exit-address-family

exit

!

router bgp 65000 vrf vrf_z10007

 bgp router-id 10.0.4.1

 no bgp hard-administrative-reset

 no bgp graceful-restart notification

 !

 address-family ipv4 unicast

  redistribute connected

 exit-address-family

 !

 address-family ipv6 unicast

  redistribute connected

 exit-address-family

 !

 address-family l2vpn evpn

  default-originate ipv4

  default-originate ipv6

 exit-address-family

exit

!

router bgp 65000 vrf vrf_z10009

 bgp router-id 10.0.4.1

 no bgp hard-administrative-reset

 no bgp graceful-restart notification

 !

 address-family ipv4 unicast

  redistribute connected

 exit-address-family

 !

 address-family ipv6 unicast

  redistribute connected

 exit-address-family

 !

 address-family l2vpn evpn

  default-originate ipv4

  default-originate ipv6

 exit-address-family

exit

!

router bgp 65000 vrf vrf_z10015

 bgp router-id 10.0.4.1

 no bgp hard-administrative-reset

 no bgp graceful-restart notification

 !

 address-family ipv4 unicast

  redistribute connected

 exit-address-family

 !

 address-family ipv6 unicast

  redistribute connected

 exit-address-family

 !

 address-family l2vpn evpn

  default-originate ipv4

  default-originate ipv6

 exit-address-family

exit

!

ip prefix-list only_default seq 1 permit 0.0.0.0/0

!

ipv6 prefix-list only_default_v6 seq 1 permit ::/0

!

route-map MAP_VTEP_IN deny 1

 match ip address prefix-list only_default

exit

!

route-map MAP_VTEP_IN deny 2

 match ipv6 address prefix-list only_default_v6

exit

!

route-map MAP_VTEP_IN deny 3

 match ip address prefix-list only_default

exit

!

route-map MAP_VTEP_IN deny 4

 match ipv6 address prefix-list only_default_v6

exit

!

route-map MAP_VTEP_IN deny 5

 match ip address prefix-list only_default

exit

!

route-map MAP_VTEP_IN deny 6

 match ipv6 address prefix-list only_default_v6

exit

!

route-map MAP_VTEP_IN deny 7

 match ip address prefix-list only_default

exit

!

route-map MAP_VTEP_IN deny 8

 match ipv6 address prefix-list only_default_v6

exit

!

route-map MAP_VTEP_IN deny 9

 match ip address prefix-list only_default

exit

!

route-map MAP_VTEP_IN deny 10

 match ipv6 address prefix-list only_default_v6

exit

!

route-map MAP_VTEP_IN permit 11

exit

!

route-map MAP_VTEP_OUT permit 1

exit

!

line vty

spirit · Thursday at 08:12

you need to announce a default type5 route from your fortigate,

and it should already by done with your

Code:

address-family l2vpn evpn
    default-originate ipv4
   default-originate ipv6
exit-address-family

"neighbor VTEP activate" need to be in "address-family l2vpn evpn", like in your config

to enable the evpn peering

so, it seem to be correct

you can use : "vtysh -c sh bgp l2vpn evpn", to display current evpn routes on both fortigate && proxmox nodes ?

and "vtysh -c sh ip bgp summary" to display the bgp sessions ?

spirit · Thursday at 08:14

(just to be sure, your frr.conf is on fortigate ? forgate use frr ? )

kemeris · Thursday at 20:42

spirit said:
you need to announce a default type5 route from your fortigate,

Ah, this make sense. I have completely misunderstood other posts on this forum.
In my first post i have submitted frr.conf of my one proxmox cluster node 10.0.4.1/22, not Fortigate 10.0.7.2/22. I doubt fortigate use frr.
On proxmox node i see Fortigate does not advertise routes Type-5, only Type-2,3. I have not found solution yet, waiting for help on fortigate forum.

Code:

Route Distinguisher: 65000:200001

 *>i [2]:[0]:[48]:[92:07:48:a8:6f:7e]

                    10.0.7.2                      100      0 i

                    RT:65000:200001

 *>i [3]:[0]:[32]:[10.0.7.2]

                    10.0.7.2                      100      0 i

                    RT:65000:200001 ET:8

Route Distinguisher: 65000:200002

 *>i [2]:[0]:[48]:[6e:01:80:37:db:71]

                    10.0.7.2                      100      0 i

                    RT:65000:200002

 *>i [3]:[0]:[32]:[10.0.7.2]

                    10.0.7.2                      100      0 i

                    RT:65000:200002 ET:8

Route Distinguisher: 65000:200007

 *>i [2]:[0]:[48]:[36:17:60:72:76:90]

                    10.0.7.2                      100      0 i

                    RT:65000:200007

 *>i [3]:[0]:[32]:[10.0.7.2]

                    10.0.7.2                      100      0 i

                    RT:65000:200007 ET:8

Route Distinguisher: 65000:200009

 *>i [2]:[0]:[48]:[1a:1c:ad:18:ca:38]

                    10.0.7.2                      100      0 i

                    RT:65000:200009

 *>i [3]:[0]:[32]:[10.0.7.2]

                    10.0.7.2                      100      0 i

                    RT:65000:200009 ET:8

Route Distinguisher: 65000:200015

 *>i [2]:[0]:[48]:[52:39:60:6c:83:29]

                    10.0.7.2                      100      0 i

                    RT:65000:200015

 *>i [3]:[0]:[32]:[10.0.7.2]

                    10.0.7.2                      100      0 i

                    RT:65000:200015 ET:8

I also have another strange problem: if I restart frr.service on any Proxmox node, the BGP session reestablishes and everything works fine. However, if I restart BGP on the FortiGate side, the session between proxmox nodes and fortigate never reestablishes until I restart frr on all Proxmox nodes. Any ideas what could be wrong:

Fortigate side :

Code:

vpn # get router info bgp neighbors 10.0.4.1
VRF 0 neighbor table:
BGP neighbor is 10.0.4.1, remote AS 65000, local AS 65000, internal link
 Member of peer-group "proxmox" for session parameters
  BGP version 4, remote router ID 10.0.4.1
  BGP state = Established, up for 09:17:43
  Last read 00:00:02, hold time is 9, keepalive interval is 3 seconds
  Configured hold time is 180, keepalive interval is 60 seconds
  Neighbor capabilities:
    Dynamic: received
    Route refresh: advertised and received (new)
    Graceful restart helper
    Address family IPv4 Unicast: advertised
    Address family IPv6 Unicast: advertised
    Address family L2VPN EVPN: advertised and received
  Received 65311 messages, 1 notifications, 0 in queue
  Sent 72629 messages, 22 notifications, 0 in queue
  Route refresh request: received 28, sent 0
  NLRI treated as withdraw: 0
  Minimum time between advertisement runs is 30 seconds

 For address family: IPv4 Unicast
  BGP table version 1, neighbor version 0
  Index 3, Offset 0, Mask 0x8
  "proxmox" peer-group member
  AF-dependant capabilities:
    Graceful restart: advertised, helper
  Inbound soft reconfiguration allowed
  Community attribute sent to this neighbor (both)
  Default information originate, default not sent
  0 accepted prefixes, 0 prefixes in rib
  0 announced prefixes

 For address family: IPv6 Unicast
  BGP table version 1, neighbor version 0
  Index 0, Offset 0, Mask 0x0
  "proxmox" peer-group member
  Community attribute sent to this neighbor (both)
  0 accepted prefixes, 0 prefixes in rib
  0 announced prefixes

 For address family: L2VPN EVPN
  BGP table version 5, neighbor version 4
  Index 0, Offset 0, Mask 0x0
  "proxmox" peer-group member
  AF-dependant capabilities:
    Graceful restart: advertised, helper
    Additional Path:
      Receive-mode: received
  Inbound soft reconfiguration allowed
  Community attribute sent to this neighbor (both)
  21 accepted prefixes, 21 prefixes in rib
  10 announced prefixes

 Connections established 24; dropped 23
 Graceful-restart Status:
  Remote restart-time is 120 sec
Local host: 10.0.7.2, Local port: 179
Foreign host: 10.0.4.1, Foreign port: 48430
Egress interface: 4
Nexthop: 10.0.7.2
Nexthop interface: port2
Nexthop global: fe80::be24:11ff:fedd:5e9c
Nexthop local: ::
BGP connection: non shared network
Last Reset: 09:17:50, due to BGP Notification sent
Notification Error Message: (CeaseUnspecified Error Subcode)

Proxmox node:

Code:

root@proxmox1-1:/etc# vtysh -c "show bgp neighbor 10.0.7.2"
BGP neighbor is 10.0.7.2, remote AS 65000, local AS 65000, internal link
  Local Role: undefined
  Remote Role: undefined
 Member of peer-group VTEP for session parameters
  BGP version 4, remote router ID 10.0.7.2, local router ID 10.0.4.1
  BGP state = Established, up for 09:20:27
  Last read 00:00:01, Last write 00:00:02
  Hold time is 9 seconds, keepalive interval is 3 seconds
  Configured hold time is 9 seconds, keepalive interval is 3 seconds
  Configured tcp-mss is 0, synced tcp-mss is 1510
  Configured conditional advertisements interval is 60 seconds
  Neighbor capabilities:
    4 Byte AS: advertised and received
    Extended Message: advertised
    AddPath:
      L2VPN EVPN: RX advertised
    Paths-Limit:
      L2VPN EVPN: advertised (0)
    Dynamic: advertised
    Long-lived Graceful Restart: advertised
    Route refresh: advertised and received
    Enhanced Route Refresh: advertised
    Address Family IPv4 Unicast: received
    Address Family IPv6 Unicast: received
    Address Family L2VPN EVPN: advertised and received
    Hostname Capability: advertised (name: proxmox1-1,domain name: n/a) not received
    Version Capability: advertised software version (FRRouting/10.2.2) not received
    Graceful Restart Capability: advertised and received
      Remote Restart timer is 120 seconds
      Address families by peer:
        L2VPN EVPN(not preserved)
  Graceful restart information:
    End-of-RIB send: L2VPN EVPN
    End-of-RIB received: L2VPN EVPN
    Local GR Mode: Helper*
    Remote GR Mode: Helper
    R bit: False
    N bit: False
    Timers:
      Configured Restart Time(sec): 120
      Received Restart Time(sec): 120
      Configured LLGR Stale Path Time(sec): 0
  Message statistics:
    Inq depth is 0
    Outq depth is 0
                         Sent       Rcvd
    Opens:                  1          1
    Notifications:          0          0
    Updates:               81         11
    Keepalives:         11209      12576
    Route Refresh:          1          0
    Capability:             0          0
    Total:              11292      12588
  Minimum time between advertisement runs is 0 seconds

 For address family: L2VPN EVPN
  VTEP peer-group member
  Update group 1, subgroup 1
  Packet Queue length 0
  NEXT_HOP is propagated unchanged to this neighbor
  Community attribute sent to this neighbor(all)
  advertise-all-vni
  Inbound path policy configured
  Outbound path policy configured
  Route map for incoming advertisements is *MAP_VTEP_IN
  Route map for outgoing advertisements is *MAP_VTEP_OUT
  10 accepted prefixes

  Connections established 1; dropped 0
  Last reset 09:20:31,  Waiting for peer OPEN (n/a)
  Internal BGP neighbor may be up to 255 hops away.
Local host: 10.0.4.1, Local port: 48430
Foreign host: 10.0.7.2, Foreign port: 179
Nexthop: 10.0.4.1
Nexthop global: fe80::ee0d:9aff:fe43:fb34
Nexthop local: fe80::ee0d:9aff:fe43:fb34
BGP connection: shared network
BGP Connect Retry Timer in Seconds: 10
Estimated round trip time: 0 ms
Read thread: on  Write thread: on  FD used: 37

  BFD: Type: single hop
  Detect Multiplier: 3, Min Rx interval: 300, Min Tx interval: 300
  Status: Up, Last update: 0:09:20:26

spirit · 2025-05-23T09:45:20+0200

ah ok ! ^_^
for the restart, I really don't known (maybe it's an option of fortigate side), but you can use

"/usr/lib/frr/frr-reload.py /etc/frr/frr.conf --reload"

to reload config without restarting the frr process

kemeris · 2025-05-23T09:54:44+0200

Thank you @spirit, for your help! I'll post my final Fortigate configuration later in case anyone else needs it.

Search

Search

Default route advertisement to SDN EVPN

kemeris

Member

spirit

Distinguished Member

spirit

Distinguished Member

kemeris

Member

spirit

Distinguished Member

kemeris

Member

We value your privacy