Hi @spirit, thank you for the amazing SDN support in Proxmox!
I'm trying to interoperate with an EBGP overlay/EBGP underlay configuration generated by Arista's "validated designs" (AVD) Ansible collection. I have some Proxmox hosts that'll act as L3 leaf nodes in this network, connected directly to an Arista L3 spine switch. The Proxmox hosts need to peer at the EVPN/VTEP layer with some Arista L3 leaf switches that are connected to the same L3 spines.
It's been a bit challenging to reconcile the Proxmox SDN with Arista's configuration schemas, but I think I've finally got it working, with one caveat. I need to be able to override the `vxlan-local-tunnelip` for my SDN-generated `vrfvx` and `vxlan` interfaces.
The SDN generator appears to choose as its `vxlan-local-tunnelip` the same IP that it chooses to peer with the L3 spine. In my case, that's an EBGP underlay IP chosen from a point-to-point (/31) uplink pool in the range `10.255.255.0/26`, which I've assigned to the Proxmox host's uplink Ethernet interface.
However, in the Arista validated designs, the VTEP loopback pool is chosen from a different range (in my case, it's `10.255.1.0/27`). I've created a `dummy` interface on the Proxmox host with an IP from that pool, and it has connectivity via the EVPN BGP overlay to the Arista L3 leaf nodes in my network, but the EBGP underlay point-to-point IP does not, so the Arista L3 leaf nodes mark the VTEP routes from the Proxmox nodes as invalid.
If I manually edit the generated SDN config to use the Proxmox host's VTEP loopback IP from the `10.255.1.0/27` pool as the `vxlan-local-tunnelip` and restart the host's networking, then everything works great!
So, in a nutshell, I'd like a way to specify that a particular loopback/dummy device is used as the `vxlan-local-tunnelip`, just like you can specify a particular loopback/dummy device in the SDN BGP controller config. Is there any way to do this in the existing implementation?
For reference, here's my SDN config from one of my Proxmox hosts, plus some additional `dummy` interfaces that I've defined to match the Arista validated design.
I'm trying to interoperate with an EBGP overlay/EBGP underlay configuration generated by Arista's "validated designs" (AVD) Ansible collection. I have some Proxmox hosts that'll act as L3 leaf nodes in this network, connected directly to an Arista L3 spine switch. The Proxmox hosts need to peer at the EVPN/VTEP layer with some Arista L3 leaf switches that are connected to the same L3 spines.
It's been a bit challenging to reconcile the Proxmox SDN with Arista's configuration schemas, but I think I've finally got it working, with one caveat. I need to be able to override the `vxlan-local-tunnelip` for my SDN-generated `vrfvx` and `vxlan` interfaces.
The SDN generator appears to choose as its `vxlan-local-tunnelip` the same IP that it chooses to peer with the L3 spine. In my case, that's an EBGP underlay IP chosen from a point-to-point (/31) uplink pool in the range `10.255.255.0/26`, which I've assigned to the Proxmox host's uplink Ethernet interface.
However, in the Arista validated designs, the VTEP loopback pool is chosen from a different range (in my case, it's `10.255.1.0/27`). I've created a `dummy` interface on the Proxmox host with an IP from that pool, and it has connectivity via the EVPN BGP overlay to the Arista L3 leaf nodes in my network, but the EBGP underlay point-to-point IP does not, so the Arista L3 leaf nodes mark the VTEP routes from the Proxmox nodes as invalid.
If I manually edit the generated SDN config to use the Proxmox host's VTEP loopback IP from the `10.255.1.0/27` pool as the `vxlan-local-tunnelip` and restart the host's networking, then everything works great!
So, in a nutshell, I'd like a way to specify that a particular loopback/dummy device is used as the `vxlan-local-tunnelip`, just like you can specify a particular loopback/dummy device in the SDN BGP controller config. Is there any way to do this in the existing implementation?
For reference, here's my SDN config from one of my Proxmox hosts, plus some additional `dummy` interfaces that I've defined to match the Arista validated design.
Code:
auto lo
iface lo inet loopback
iface eno1np0 inet manual
mtu 9214
iface eno2np1 inet manual
mtu 9214
iface enxbe3af2b6059f inet manual
iface enp65s0f0 inet manual
iface enp65s0f1 inet manual
auto enp129s0f4
iface enp129s0f4 inet static
address 10.255.255.33/31
mtu 9214
#P2P link to spine
iface enp129s0f4d1 inet manual
mtu 9214
auto vmbr0
iface vmbr0 inet static
address 10.0.6.23/24
gateway 10.0.6.1
bridge-ports eno1np0
bridge-stp off
bridge-fd 0
mtu 9214
source /etc/network/interfaces.d/*
Code:
auto apple
iface apple
address 10.8.1.1/24
hwaddress 00:00:00:00:00:01
bridge_ports vxlan_apple
bridge_stp off
bridge_fd 0
mtu 9214
alias APPLE
ip-forward on
arp-accept on
vrf vrf_T1
auto vrf_T1
iface vrf_T1
vrf-table auto
post-up ip route add vrf vrf_T1 unreachable default metric 4278198272
auto vrfbr_T1
iface vrfbr_T1
bridge-ports vrfvx_T1
bridge_stp off
bridge_fd 0
mtu 9214
vrf vrf_T1
auto vrfvx_T1
iface vrfvx_T1
vxlan-id 10
vxlan-local-tunnelip 10.255.255.33 # Here is the problem, needs to be 10.255.1.11
bridge-learning off
bridge-arp-nd-suppress on
mtu 9214
auto vxlan_apple
iface vxlan_apple
vxlan-id 10100
vxlan-local-tunnelip 10.255.255.33 # As above.
bridge-learning off
bridge-arp-nd-suppress on
mtu 9214
Code:
auto dummy0
iface dummy0 inet static
address 10.255.0.11/32
link-type dummy
# EVPN_Overlay_Peering
auto dummy1
iface dummy1 inet static
address 10.255.1.11/32
link-type dummy
# VTEP_VXLAN_Tunnel_Source, this is the interface/IP that should be used for the VTEP
YAML:
evpn: dc1
asn 65000
peers 10.255.0.1
bgp: bgpdc1-node05
asn 65016
node dc1-node05
peers 10.255.255.32
bgp-multipath-as-path-relax 0
ebgp 1
ebgp-multihop 3
loopback dummy0
YAML:
evpn: T1
controller dc1
vrf-vxlan 10
ipam pve
mac 00:00:00:00:00:01
mtu 9214
YAML:
vnet: apple
zone T1
tag 10100
YAML:
subnet: T1-10.8.1.0-24
vnet apple
gateway 10.8.1.1
Last edited: