Hello,What do you want to do exactly with your vyos router ? Because you can't route from outside to 1 vrf with 10.0.0.1 and another vrf 10.0.0.1. (they are no magic session tracking).
Multitenancy with same subnets can only live inside their own vrf (or zone in proxmox).
but even with differents subnet, I think your config is wrong. (you should try to enable the exit-node option on proxmox, and look the generated /etc/frr/frr.conf ,as vyos seem to use frr too, it can give your some inspiration).
I could be great to have your promox sdn config too (/etc/pve/sdn/*.cfg)
you should announce the default 0.0.0.0 in each vrf evpn network (and not the real netework), you need an vxlan (l3vni) interface for each vrf (same vxlanid than defined the proxmox zone). (5000 && 5002 from your config)
on proxmox, you can do #vtysh -c "sh ip bgp l2vpn evpn" , you should see 0.0.0.0/0 routes with your vyos ip as gateway.
This evpn default route is not imported in proxmox kernel routing table. (you don't see in #ip route)
From that, vm will send traffic ton his local anycast gateway on the vnet, and the vnet will forward to vyos because of the evpn 0.0.0.0 route.
After that, on Vyos, you can allow traffic going between different vrf (the defaut vrf (the real world - and the tenant vrf for example). You need to a vrf leak (importing/extporting routes between vrfs).
This is done with "import vrf <othervrfname>" in the source vrf router
But this can works of course, if you use differents subnets.
thank you for your reply @spirit .
I want to achieve multi tenancy with overlapping IP addresses. I followed this instructions: https://blog.vyos.io/l3vpn-evpn-creating-a-multi-tenant-service-provider-network (sorry to not shared previously), and adapt them to my proxmox setup.
I realized I don't need the IP addresses on the bridges. Using pseudo-interfaces on vyos, and use them to do SNAT, I can reach outside "real world" (but only from one tenant, and I'm still understanding the NAT rules on vyos if the source addresses of two VMs are the same but belonging to different VRFs).
Today I still got the strange behavior when I announce the default route in each VRF through vyos (see the below output of the 'show ip route vrf all' command):
Code:
VRF vrf_customB:
K>* 0.0.0.0/0 [255/8192] unreachable (ICMP unreachable), 6d23h31m
C>* 10.3.2.0/24 is directly connected, custB9, 6d23h31m
B>* 10.3.2.17/32 [200/0] via 10.2.2.6, vrfbr_customB onlink, weight 1, 00:03:56
VRF vrf_tenantC:
B>* 0.0.0.0/0 [200/0] via 10.2.2.2, vrfbr_tenantC onlink, weight 1, 00:00:09
K * 0.0.0.0/0 [255/8192] unreachable (ICMP unreachable), 1d20h54m
C>* 10.3.2.0/24 is directly connected, tenantC1, 1d20h51m
As you can see, there's no default route on vrf_customB. The default route on vrf_customB disappears once I announce the same route through tenantC VRF.
Sorry for the following long *c,fg files from /etc/pve/sdn/
Code:
cat /etc/pve/sdn/*.cfg
evpn: evpn01
asn 65003
peers 10.2.2.6,10.2.2.4,10.2.2.7,10.2.2.2
bgp: bgppve06
asn 65003
node pve06
peers 10.2.2.1,10.2.2.11,10.2.2.12
bgp-multipath-as-path-relax 0
ebgp 0
subnet: customA-10.3.0.0-24
vnet vnet5
gateway 10.3.0.1
subnet: customA-10.3.1.0-24
vnet MNGSDN
gateway 10.3.1.1
subnet: customA-10.3.2.0-24
vnet backups
gateway 10.3.2.1
subnet: customB-10.3.2.0-24
vnet custB9
gateway 10.3.2.1
subnet: tenantC-10.3.2.0-24
vnet tenantC1
gateway 10.3.2.1
vnet: vnet5
zone customA
tag 4505
vnet: MNGSDN
zone customA
tag 4506
vnet: backups
zone customA
alias backups
tag 4507
vnet: custB9
zone customB
tag 5001
vnet: tenantC1
zone tenantC
tag 5003
evpn: customA
controller evpn01
vrf-vxlan 4504
ipam pve
mac FE:2B:38:68:1F:23
mtu 1500
evpn: customB
controller evpn01
vrf-vxlan 5000
advertise-subnets 1
ipam pve
mac 56:13:CC:5D:68:1E
mtu 1500
evpn: tenantC
controller evpn01
vrf-vxlan 5002
advertise-subnets 1
ipam pve
mac DA:05:CF:4D:CC:1F
mtu 1500
Last edited: