I believe that the below requires an expert to answer.
I am trying to build a cloud with tenants using PROXMOX & SDN, where I do have different customers with duplicated subnets while using BGP Peering for every customer to connect to his on Premises site using MPLS on Cisco Router. I have achieved the requirements through the below and the example is for two customers:
-I have total of 3 nodes, two (Node1 & Node2) of them will be the exit nodes, 1 (Node3) will host the workloads.
- Created EVPN Controller & added the 3 nodes.
- Created two EVPN Zones (Customer1 & Customer2) which would add on the FRR VRF_Customer1 & VRF_Customer2
- Created two VNets every Vnet represents a customer and has it's EVPN Zones associated with it (Segregation on the FRR through VRF).
(e.g. VNET-C1 associated with EVPN"Customer1" & VNET-C2 associated with EVPN"Customer2")
-Created duplicated subnets (e.g. VNET-C1 has 10.1.1.0/24 & 10.2.2.0/24.......... VNET-V2 has 10.1.1.0/24 & 10.2.2.0/24)
- Assigned the VMs for Customer1 with VNET-C1 & assigned the VMs for Customer2 with VNET-C2.
- On exit nodes (Node1 & Node2), created a Linux VLANs for Customer1 & Customer2 which I will use these interface for BGP Peering.
(e.g. For Customer1 created vmbr2.9 on Node1 & Node2 where Node1 has 10.9.9.1/29 & Node2 has 10.9.9.2/29 )
(e.g. For Customer2 created vmbr2.10 on Node1 & Node2 where Node1 has 10.10.10.1/29 & Node2 has 10.10.10.2/29 )
All the above is fine & clear which can be done through the PROXMOX & SDN GUI. The below is the manual work that I have done to complete the scenario:
-Added under vmbr2.9 on Node1 & Node2 vrf "VRF_Customer1" in the network interfaces file, to ensure that this network 10.9.9.0/29 is isolated for Customer1.
-Added under vmbr2.10 on Node1 & Node2 vrf "VRF_Customer2" in the network interfaces file, to ensure that this network 10.10.10.0/29 is isolated for Customer2.
(e.g.
iface vmbr2.9 inet static
address 10.9.9.1/29
vrf VRF_Customer1
)
- If I do the BGP peering with Cisco Router using BGP Controller then the BGP peering on Customer1 "neighbor 10.9.9.3 remote-as 65000" is added under the Default VRF which would not provide reachability to the customers VMs, so I have to go manual on every FRR and add the neighbor command under vrf
(e.g. on FRR Node1 & FRR Node2
router bgp 65014 vrf vrf_Customer1
neighbor 10.9.9.3 remote-as 65000
)
After applying the manual configuration above it works like a spark.
============================
1-Issue: Once I click apply config on the SDN, let's say I added new subnet under VNET-C1 and clicked apply the manual BGP neighbor command under the VRF is wiped.
so how to make it permanent or if there is way to do it from GUI.
2- Question: is there an way to achieve my scenario with a different way while applying segregation to different customer & use duplicated subnets.
============================
I am trying to build a cloud with tenants using PROXMOX & SDN, where I do have different customers with duplicated subnets while using BGP Peering for every customer to connect to his on Premises site using MPLS on Cisco Router. I have achieved the requirements through the below and the example is for two customers:
-I have total of 3 nodes, two (Node1 & Node2) of them will be the exit nodes, 1 (Node3) will host the workloads.
- Created EVPN Controller & added the 3 nodes.
- Created two EVPN Zones (Customer1 & Customer2) which would add on the FRR VRF_Customer1 & VRF_Customer2
- Created two VNets every Vnet represents a customer and has it's EVPN Zones associated with it (Segregation on the FRR through VRF).
(e.g. VNET-C1 associated with EVPN"Customer1" & VNET-C2 associated with EVPN"Customer2")
-Created duplicated subnets (e.g. VNET-C1 has 10.1.1.0/24 & 10.2.2.0/24.......... VNET-V2 has 10.1.1.0/24 & 10.2.2.0/24)
- Assigned the VMs for Customer1 with VNET-C1 & assigned the VMs for Customer2 with VNET-C2.
- On exit nodes (Node1 & Node2), created a Linux VLANs for Customer1 & Customer2 which I will use these interface for BGP Peering.
(e.g. For Customer1 created vmbr2.9 on Node1 & Node2 where Node1 has 10.9.9.1/29 & Node2 has 10.9.9.2/29 )
(e.g. For Customer2 created vmbr2.10 on Node1 & Node2 where Node1 has 10.10.10.1/29 & Node2 has 10.10.10.2/29 )
All the above is fine & clear which can be done through the PROXMOX & SDN GUI. The below is the manual work that I have done to complete the scenario:
-Added under vmbr2.9 on Node1 & Node2 vrf "VRF_Customer1" in the network interfaces file, to ensure that this network 10.9.9.0/29 is isolated for Customer1.
-Added under vmbr2.10 on Node1 & Node2 vrf "VRF_Customer2" in the network interfaces file, to ensure that this network 10.10.10.0/29 is isolated for Customer2.
(e.g.
iface vmbr2.9 inet static
address 10.9.9.1/29
vrf VRF_Customer1
)
- If I do the BGP peering with Cisco Router using BGP Controller then the BGP peering on Customer1 "neighbor 10.9.9.3 remote-as 65000" is added under the Default VRF which would not provide reachability to the customers VMs, so I have to go manual on every FRR and add the neighbor command under vrf
(e.g. on FRR Node1 & FRR Node2
router bgp 65014 vrf vrf_Customer1
neighbor 10.9.9.3 remote-as 65000
)
After applying the manual configuration above it works like a spark.
============================
1-Issue: Once I click apply config on the SDN, let's say I added new subnet under VNET-C1 and clicked apply the manual BGP neighbor command under the VRF is wiped.
so how to make it permanent or if there is way to do it from GUI.
2- Question: is there an way to achieve my scenario with a different way while applying segregation to different customer & use duplicated subnets.
============================