Hi,
I have a setup of 3 Racks, each of them having two ToR L3 Switches and at least 1 proxmox Server in each rack.
The switches (Mellanox SN2700) cannot do EVPN also being an exit node, so I tried the SDN Feature from Proxmox for our proxmox setup.
The switches are fully routed. There is no Layer 2 network spanned across the racks. The connection between the switch and the proxmox node is also using link local IPs and BGP.
So, I went on and created:
- an EVPN controller with all nodes being an exit node
- an EVPN zone
- a VNet + IPv4 and IPv6 Subnet
- a BGP controller per proxmox node referencing all 4 external BGP Sessions to switch1/switch2 in the same rack (1x IPv4 and 1x IPv6 per link)
It seems to work like a charm with IPv4. Especially FRR announces /32 routes to the ToR Switches at the end (awesome!).
But when it comes to IPv6, it does not make sense to me.
I can only activate one BGP controller per Proxmox Node. So I inserted all IPv4/IPv6 neighbors in it.
The resulting FRR config is:
As you might be able to see: It does not announce any IPv6 Unicast AFI/SAFI. Even the IPv6 sessions only carry IPv4 Unicast AFI/SAFI. ... which is weird.
On top, there is an IPv6 route, which points to an IPv4 target, which FRR rejects.
Is IPv6 inside Proxmox SDN supported? What must be done for this?
I started patching around the Perl scripts (yeah, I finally can use my rusty Perl skills professionally) and I'm confident, I can fix this. Will there be a will from somebody else to help me upstream this?
Here you can find my hackings: https://github.com/bebehei/pve-network/tree/3q-changes
I have a setup of 3 Racks, each of them having two ToR L3 Switches and at least 1 proxmox Server in each rack.
The switches (Mellanox SN2700) cannot do EVPN also being an exit node, so I tried the SDN Feature from Proxmox for our proxmox setup.
The switches are fully routed. There is no Layer 2 network spanned across the racks. The connection between the switch and the proxmox node is also using link local IPs and BGP.
So, I went on and created:
- an EVPN controller with all nodes being an exit node
- an EVPN zone
- a VNet + IPv4 and IPv6 Subnet
- a BGP controller per proxmox node referencing all 4 external BGP Sessions to switch1/switch2 in the same rack (1x IPv4 and 1x IPv6 per link)
It seems to work like a charm with IPv4. Especially FRR announces /32 routes to the ToR Switches at the end (awesome!).
But when it comes to IPv6, it does not make sense to me.
I can only activate one BGP controller per Proxmox Node. So I inserted all IPv4/IPv6 neighbors in it.
The resulting FRR config is:
Code:
[...]
ip route 2a14:30c0:2::/64 10.255.255.2 xvrf_evpn
[...]
router bgp 62359
[...]
neighbor BGP peer-group
neighbor BGP remote-as 62359
neighbor BGP bfd
neighbor 100.128.121.101 peer-group BGP
neighbor 100.128.122.101 peer-group BGP
neighbor fc00:128::121:101 peer-group BGP
neighbor fc00:128::122:101 peer-group BGP
neighbor VTEP peer-group
neighbor VTEP remote-as 62359
neighbor VTEP bfd
neighbor VTEP update-source dummy_evpn
neighbor 37.58.31.231 peer-group VTEP
!
address-family ipv4 unicast
neighbor BGP activate
neighbor BGP soft-reconfiguration inbound
import vrf vrf_evpn
exit-address-family
!
address-family ipv6 unicast
import vrf vrf_evpn
exit-address-familyAs you might be able to see: It does not announce any IPv6 Unicast AFI/SAFI. Even the IPv6 sessions only carry IPv4 Unicast AFI/SAFI. ... which is weird.
On top, there is an IPv6 route, which points to an IPv4 target, which FRR rejects.
Is IPv6 inside Proxmox SDN supported? What must be done for this?
I started patching around the Perl scripts (yeah, I finally can use my rusty Perl skills professionally) and I'm confident, I can fix this. Will there be a will from somebody else to help me upstream this?
Here you can find my hackings: https://github.com/bebehei/pve-network/tree/3q-changes
