EVPN/SDN with BGP External Advertisement UI Limitations

[n0doubt]

Hey everyone,

I am setting up Proxmox to use EVPN Zones, and then advertise routes to these zones to my core router with FRR and BGP, with the goal being a highly available/ECMP routing setup so that I can take cluster members down and etc, but still be able to route into the SDN. I also want to be able to have any new VNETS I create be advertised "automatically" to the rest of my network, so this setup works very very well for me. I can also do North/South SIEM/netflow monitoring this way which is very nice, as well as have another router/cluster over EVPN at another site, advertising those same SDN routes

I have a three node cluster, and all three nodes are on a transit net to talk to my router. This is not a troubleshooting post as everything is working correctly now. This SDN functionality is insanely powerful and I am very happy it exists, and I would like to commend the proxmox team for their excellent implementation here

Sparing everyone the gritty Pf related details (hint, don't use PfSense for this, get an actual core router), there are some small things missing from the proxmox UI still. I am running Virtual Environment 9.1.9.

1. Cross-VRF Route Leaking​

Proxmox SDN strictly isolates tenant networks into their own Virtual Routing and Forwarding (VRF) tables. While the UI makes it easy to peer BGP and advertise routes outward, there is no UI toggle to allow the EVPN VRF to import routes from the hypervisor's global routing table (such as a default internet route or physical LAN subnets).

• I had to manually create and edit the /etc/frr/frr.conf.local file to inject advanced Free Range Routing (FRR) commands on every node.

  • router bgp 65001 vrf vrf_sdnzone
     address-family ipv4 unicast
      import vrf default
      exit
     exit
    end

• Specifically, adding import vrf default inside the VRF's BGP address-family block.

2. Reverse Path Filtering (rp_filter) and ECMP​

When using Equal-Cost Multi-Path (ECMP) routing with BGP, traffic is highly susceptible to asymmetric paths. The Linux kernel's default Reverse Path Filtering (rp_filter) treats VXLAN-encapsulated traffic arriving on VRF bridge interfaces (vrfvx_ and vrfbr_) as "spoofed" if the return path differs from the ingress path, silently dropping the packets

• There is no option in the Proxmox GUI or the SDN zone settings to relax kernel routing parameters for virtual network interfaces.
• I had to manually modify sysctl parameters via the CLI to disable strict filtering. I would have to do this every time I create a Linux VLAN, which is only occasionally, but I have to do it on every node.
• This requires creating a persistent configuration file like /etc/sysctl.d/99-evpn.conf and setting net.ipv4.conf.all.rp_filter=0 and net.ipv4.conf.default.rp_filter=0 to prevent the hypervisor from blackholing returning EVPN traffic.

Cheers,

 

[ggoller]

Hi!
we're working on the VRF support for fabrics and the ability to leak into VRFs from other VRFs.
We thought about adding a GUI for sysctls a while back, maybe you could create a feature request on our bugzilla (https://bugzilla.proxmox.com/)?