EVPN with multiple exitnodes and no primary exit node possible?

[mathlin]

Hello,
We have a three node cluster running 9.1.2 with all three nodes as exit nodes.
VMs inside the cluster can reach outside and back normally, but it is only if we have a primary exit node defined that nodes on the outside can ping and reach nodes inside the cluster.
Seems similar to this https://forum.proxmox.com/threads/evpn-vpls-with-multi-exit-nodes-firewall-drop-packet-with-asymetric-routing.158225/
but i have manually set net.ipv4.conf.all.rp_filter to 0 but it does not solve the issue.
I do not notice any changes in the routing when i designate a primary exit router or remove it.
I do not have a firewall activated (had but deactivated them on two nodes), not the hosts or cluster firewall or the VNET firewall.
Conntrack is still active, is that what is causing me these issues or Is this working as intended?
I would like to avoid using a primary exit node since it could put us in a bad spot if the designated node should fail for some reason.

Thanks
Mathias

 

[shanreich]

Can you post the output of the following command?

head -n10 /proc/sys/net/ipv4/conf/*/rp_filter

I've seen occassions where simply setting net.ipv4.conf.all.rp_filter wasn't sufficient on PVE 9 anymore, so could you double-check that the setting actually persisted?

 

[mathlin]

Sure

root@gn-b1-pve05:~# head -n10 /proc/sys/net/ipv4/conf/*/rp_filter

==> /proc/sys/net/ipv4/conf/all/rp_filter <==

0



==> /proc/sys/net/ipv4/conf/bond0.395/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/bond0/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/bond1.393/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/bond1.394/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/bond1/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/default/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/dummy_THN02/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/eno12399np0/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/eno12409np1/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/eno8303/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/eno8403/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/ens1f0np0/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/ens1f1np1/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/ens3f0np0/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/ens3f1np1/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/fwbr104i0/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/fwbr113i1/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/fwbr113i2/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/fwln104i0/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/fwln113i1/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/fwln113i2/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/fwpr104p0/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/fwpr113p1/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/fwpr113p2/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/lo/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/MGMTDHCP/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/MGMT/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/SQL/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/tap104i0/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/tap107i0/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/tap107i1/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/tap107i2/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/tap113i0/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/tap113i1/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/tap113i2/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/vlan9000/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/vmbr0.4090/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/vmbr0/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/vrfbr_evpnzone/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/vrf_evpnzone/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/vrfvx_evpnzone/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/vxlan_MGMTDHCP/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/vxlan_MGMT/rp_filter <==

2



==> /proc/sys/net/ipv4/conf/vxlan_SQL/rp_filter <==

2

 

[shanreich]

Can you try and set it manually for the uplink interfaces via the following command?

echo 0 > /proc/sys/net/ipv4/conf/<UPLINK>/rp_filter

This won't persist, but for testing purposes it would be interesting.

 

[mathlin]

After setting it on the uplinks only and then rapidly loosing patience and setting it everywhere it works.
Now its just a matter of removing them one by one and finding out what links that really need the setting.
Would a file in /etc/sysctl.d/ of similar be a working workaround, or is there something else to set / do?

Thanks alot