Proxmox EVPN/BGP weighted routing

[chrispage1]

Hi,

I'm trying to configure a Proxmox SDN. We have two sites connected via Layer 2 switching and over that a Layer 3 EVPN

I've configured a controller with two peers which are two VyOS routers. One is in site A and one is in site B

Ideally, I'd like site A traffic to route via the Site A router and site B traffic to route via the site B router.

The problem I'm having is that the EVPN controller doesn't seem to allow weighting so I actually get equal weight routing in both sites. This means some traffic will flow out of Site A and some out of Site B

node0-pmx-thn# show  ip route vrf vrf_EVPN
Codes: K - kernel route, C - connected, L - local, S - static,
       R - RIP, O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR,
       f - OpenFabric, t - Table-Direct,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure

IPv4 unicast VRF vrf_EVPN:
B>* 0.0.0.0/0 [200/0] via 172.16.210.1, vrfbr_EVPN onlink, weight 1, 14:16:40
  *                   via 172.16.210.2, vrfbr_EVPN onlink, weight 1, 14:16:40

Is there a supported way via SDN to apply weighting?

 

[shanreich]

Currently, there's no supported way in SDN itself to apply weighting. If you have distinct subnets per site, then you could additionally announce more specific routes than 0.0.0.0/0 from the routers, depending on how you assigned the subnets to the sites. Then traffic would always flow via a specific router as long as they are announcing the routes, but if one router fails then the more general route would be used.

 

[chrispage1]

Thanks @shanreich

The problem is I want to use EVPN Zones to define the subnets and this is purely for the purposes of transit out. I could weight what BGP is broadcasting but this would have an impact on both zones causing internet traffic to flow across our P2P link

I've tried setting frr.conf.local but FRR is too clever and manually amending the permit. I presume there is no alternate way to modify FRR?

Thanks,
Chris.

 

[shanreich]

I've tried setting frr.conf.local but FRR is too clever and manually amending the permit. I presume there is no alternate way to modify FRR?

Currently there isn't, I'm afraid but we're working on introducing support for configuring custom route-maps soon.

 

[chrispage1]

Thanks - would be great to see soon!

An absolute hack, but may be useful for someone... This can actually be achieved by creating a script that will retrospectively update the path mapping.

We can then use systemd to watch for frr changes and retrospectively update the peering priorities. To achieve this, you need to create a path watcher, a service and an executable.

/etc/frr/fix-bgp-weights.sh

#!/bin/bash
sleep 2

# inject our priority rules
/usr/bin/vtysh -c "conf t" \
  -c "route-map MAP_VTEP_IN permit 1" \
  -c "  match peer 172.16.210.1" \
  -c "  set local-preference 500" \
  -c "exit" \
  -c "route-map MAP_VTEP_IN permit 5" \
  -c "  set local-preference 100" \
  -c "exit"

# force the new routes to be recalculated
/usr/bin/vtysh -c "clear bgp l2vpn evpn * in"

/etc/systemd/system/bgp-weight-watcher.path

[Unit]
Description=Re-apply BGP Weights after SDN update

[Service]
Type=oneshot
ExecStart=/etc/frr/fix-bgp-weights.sh

/etc/systemd/system/bgp-weight-watcher.service

[Unit]
Description=Re-apply BGP Weights after SDN update
After=pve-sdn-commit.service frr.service network-online.target

[Service]
Type=oneshot
ExecStart=/etc/frr/fix-bgp-weights.sh

Ensure the sh file is executable and that the path watcher is enabled:

chmod +x /etc/frr/fix-bgp-weights.sh
systemctl enable --now bgp-weight-watcher.path