[SOLVED] [SDN] Issue with FRR version 7.5.1-99+pve~really7.4 and set src

M

maloddon

New Member
Proxmox Subscriber
Sep 2, 2021
4
0
1
France
Hi !

As I tried to update a node this morning to latest FRR version, i get into a problem related to the latest FRR version (7.5.1-99+pve~really7.4) which has an issue regarding set src route-map instruction. The issue encoutered is exactly this one : https://github.com/FRRouting/frr/issues/9097, and has been fixed by @idryzhov.

Is there any way to bump the build with this patch to get back the set src function ?

We use set src to use loopback IPs for internal communication between the nodes, which influence the "local-tunnelip" used by the SDN plugin and included in /etc/network/interfaces.d/sdn.

Running on : pve-manager/7.0-11/63d82f4e (running kernel: 5.11.22-2-pve), on enterprise repo.

Thanks for your help.
 
Last edited:
Hi,
we have revert the 7.5.1 to 7.4 because of multiple evpn bugs . (that's why it's named (7.5.1-99+pve~really7.4)).

I can try to backport the patch.

Could your share you current frr config && /etc/pve/sdn/* ?

I'm curious to see your usage, because I have already implemented loopback support in bgp controller plugin)
 
Hi Spirit,

Here are the config files :

frr.conf (a lot comes from frr.conf.local, which we handle in another way for now) :

Code: 
log syslog informational
ip forwarding
ipv6 forwarding
frr defaults datacenter
service integrated-vtysh-config
hostname xxx
!
!
vrf vrf_01
 vni 1000
exit-vrf
!
vrf vrf_02
 vni 2000
exit-vrf
!
vrf vrf_03
 vni 3500
exit-vrf
!
vrf vrf_04
 vni 100
exit-vrf
!
vrf vrf_05
 vni 3000
exit-vrf
!
vrf vrf_06
 vni 666
exit-vrf
!
vrf vrf_07
 vni 2500
exit-vrf
!
!
router bgp xxxx
 bgp router-id 10.xxx.xxx.101
 no bgp ebgp-requires-policy
 no bgp default ipv4-unicast
 bgp disable-ebgp-connected-route-check
 coalesce-time 1000
 neighbor 10.xxx.xxx.1 remote-as external
 neighbor 10.xxx.xxx.1 bfd
 neighbor 10.xxx.xxx.1 ebgp-multihop 10
 neighbor 10.xxx.xxx.1 update-source 10.xxx.xxx.101
 neighbor 10.xxx.xxx.2 remote-as external
 neighbor 10.xxx.xxx.2 bfd
 neighbor 10.xxx.xxx.2 ebgp-multihop 10
 neighbor 10.xxx.xxx.2 update-source 10.xxx.xxx.101
 neighbor 10.xxx.xx.1 remote-as external
 neighbor 10.xxx.xx.1 bfd
 neighbor 10.xxx.x.1 remote-as external
 neighbor 10.xxx.x.1 bfd
 !
 address-family ipv4 unicast
  redistribute connected route-map redistribute-loopback
  neighbor 10.xxx.xx.1 activate
  neighbor 10.xxx.xx.1 soft-reconfiguration inbound
  neighbor 10.xxx.xx.1 allowas-in
  neighbor 10.xxx.x.1 activate
  neighbor 10.xxx.x.1 soft-reconfiguration inbound
  neighbor 10.xxx.x.1 allowas-in
 exit-address-family
 !
 address-family l2vpn evpn
  neighbor 10.xxx.xxx.1 activate
  neighbor 10.xxx.xxx.1 soft-reconfiguration inbound
  neighbor 10.xxx.xxx.1 allowas-in
  neighbor 10.xxx.xxx.1 route-map evpn_src out
  neighbor 10.xxx.xxx.2 activate
  neighbor 10.xxx.xxx.2 soft-reconfiguration inbound
  neighbor 10.xxx.xxx.2 allowas-in
  neighbor 10.xxx.xxx.2 route-map evpn_src out
  advertise-all-vni
  advertise ipv4 unicast
  advertise ipv6 unicast
 exit-address-family
!
router bgp 65401 vrf vrf_01
 !
 address-family ipv4 unicast
  redistribute connected
 exit-address-family
 !
 address-family l2vpn evpn
  advertise ipv4 unicast
  advertise ipv6 unicast
 exit-address-family
!
router bgp 65401 vrf vrf_02
 !
 address-family ipv6 unicast
  redistribute connected
 exit-address-family
 !
 address-family l2vpn evpn
  advertise ipv4 unicast
  advertise ipv6 unicast
 exit-address-family
!
router bgp 65401 vrf vrf_05
 !
 address-family ipv6 unicast
  redistribute connected
 exit-address-family
 !
 address-family l2vpn evpn
  advertise ipv4 unicast
  advertise ipv6 unicast
 exit-address-family
!
router bgp 65401 vrf vrf_06
 !
 address-family ipv6 unicast
  redistribute connected
 exit-address-family
 !
 address-family l2vpn evpn
  advertise ipv4 unicast
  advertise ipv6 unicast
 exit-address-family
!
route-map correct_src permit 1
 set src 10.xxx.xxx.101
!
route-map redistribute-loopback permit 1
 match interface lo
!
route-map redistribute-loopback permit 2
 match interface dummy0
!
route-map evpn_src permit 10
 set ip next-hop peer-address
!
ip protocol bgp route-map correct_src
!
!
line vty

sdn file (redacted) :

Code: 
auto VRF1v2
iface VRF1v2
    hwaddress xxxxx
    bridge_ports vxlan_VRF1v2
    bridge_stp off
    bridge_fd 0
    mtu 9001
    vrf vrf_1


auto vrf_1
iface vrf_1
    vrf-table auto
    post-up ip route add vrf vrf_1 unreachable default metric 4278198272


auto vrfbr_1
iface vrfbr_1
    bridge-ports vrfvx_1
    bridge_stp off
    bridge_fd 0
    mtu 9001
    vrf vrf_1


auto vxlan_VRF1v2
iface vxlan_VRF1v2
    vxlan-id 2500
    vxlan-local-tunnelip 10.xxx.xxx.101  # LOOPBACK IP
    bridge-learning off
    bridge-arp-nd-suppress on
    mtu 9001

Some parts were made on previous version of SDN, which was not complete as it is now, so we made some custom config to match our requirements.
 
I have tested the patch, it seem to apply fine with 7.4, I'll try test it and send it soon.

I don't known if it could work for you,
but at work, I'm using bgp-evpn with ebgp (each proxmox host have is own ASN), with a loopback for the undelay. (Underlay is full l3 with 2 ecmp paths with a loopback)

ex for node1:

/etc/pve/sdn/zones.cfg
Code: 
evpn: testzone
        controller evpnctl
        vrf-vxlan 10000

/etc/pve/sdn/controllers.cfg

Code: 
evpn: evpnctl
        asn 65001
        peers X.X.X.X, X.X.X.X    #my centrals evpn route reflector

bgp: bgpnode1
        asn 12345
        node node1
        peers 10.3.98.2,10.3.97.2   #my top of rack bgp routers
        ebgp 1
        ebgp-multihop 3
        loopback dummy0

[CODE]

/etc/network/interfaces

auto eth0
iface eth0 inet static
        address  10.3.98.1/24

auto eth1
iface eth1 inet static
        address  10.3.97.1/24


auto dummy0
iface dummy0 inet static
        address 10.3.99.2/32
        link-type dummy


generate frr config:

Code: 
!
vrf vrf_testzone
 vni 10001
exit-vrf
!
router bgp 12345
 bgp router-id 10.3.99.2
 no bgp default ipv4-unicast
 coalesce-time 1000
 bgp network import-check
 no bgp ebgp-requires-policy
 bgp disable-ebgp-connected-route-check
 neighbor BGP peer-group
 neighbor BGP remote-as external
 neighbor BGP bfd
 neighbor BGP ebgp-multihop 3
 neighbor 10.3.97.2 peer-group BGP
 neighbor 10.3.98.2 peer-group BGP
 neighbor VTEP peer-group
 neighbor VTEP remote-as external
 neighbor VTEP bfd
 neighbor VTEP ebgp-multihop 10
 neighbor VTEP update-source dummy0
 neighbor X.X.X.X peer-group VTEP
 neighbor X.X.X.X peer-group VTEP
 !
 address-family ipv4 unicast
  network 10.3.99.2/32
  neighbor BGP activate
  neighbor BGP soft-reconfiguration inbound
 exit-address-family
 !
 address-family l2vpn evpn
  neighbor VTEP activate
  advertise-all-vni
  autort as 12345
 exit-address-family
!
router bgp 12345 vrf vrf_testzone
 no bgp ebgp-requires-policy
 !
 address-family l2vpn evpn
  route-target import 65001:10001
  route-target export 65001:10001
 exit-address-family
!
router bgp 12345 vrf vrf_customer
 no bgp ebgp-requires-policy
 !
 address-family l2vpn evpn
  route-target import 65001:10000
  route-target export 65001:10000
 exit-address-family
!
line vty
!

/etc/network/interfaces.d/sdn

Code: 
....
auto vxlan_vnetxxxx
iface vxlan_vnetxxxx
        vxlan-id xxxx
        vxlan-local-tunnelip 10.3.99.2
        bridge-learning off
        bridge-arp-nd-suppress on
        mtu 1500

The loopback (10.3.99.2) is announced is the bgp underlay network and used for vxlan-local-tunnelip src.
 
Hi !

I just tried it on the lab and seems to work just fine, the route map is ok and the route have the loopback as src-ip. Also downgraded/upgraded nodes to the really7.4 version and all strange behaviour that we were not able to explain just disappeared (suddent loss of mac, random packet loss, broken arp...). As you said, 7.5 seems to be very unstable for EVPN !
 
maloddon said:
Hi !

I just tried it on the lab and seems to work just fine, the route map is ok and the route have the loopback as src-ip. Also downgraded/upgraded nodes to the really7.4 version and all strange behaviour that we were not able to explain just disappeared (suddent loss of mac, random packet loss, broken arp...). As you said, 7.5 seems to be very unstable for EVPN !
Click to expand...
yes, frr have implement new evpn feature called multihoming, but they are a lot of bugs (some of them fixed in 8.0, but I have see other strange bugs).

I'll send the patch to pve-devel mailing, it should be released soon.

Thanks for the test !
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom