Node to Node communication not working with EVPN

B

bilby91

New Member
Jan 8, 2024
12
0
1
Hello,

I've been reading all the troubleshooting posts about EVPN trying to figure out a solution to the problem I'm having but no luck yet.

I'm experimenting in a 2-node Proxmox cluster:

vmbr0 interfaces:

Node 1: 10.0.1.131/24
Node 2: 10.0.1.132/24

Proxmox firewall is disabled for the moment, plan to use it in the future.

I'm following this tutorial https://pve.proxmox.com/pve-docs/chapter-pvesdn.html#pvesdn_setup_example_evpn

I'm trying to get the VM in Node 1 (vnet1) to ping the VM in Node 2 (vnet2) but I'm getting `Host Unreachable`.

VNET 1 has the 10.0.5.0/24 network, gateway 10.0.5.1. VNET 2 has 10.0.6.0/24 network, gateway 10.0.6.1.


My configurations look like this:

controllers.cfg
Code: 
evpn: myevpncl
        asn 65000
        peers 10.0.1.131,10.0.1.132

zones.cfg

Code: 
evpn: myevpnzn
        controller myevpncl
        vrf-vxlan 10000
        exitnodes-primary pve
        ipam pve
        mac BC:24:11:2A:62:B7
        mtu 1450
        nodes pve,pve-home-2

vnets.cfg
Code: 
vnet: myvnet1
        zone myevpnzn
        tag 11000

vnet: myvnet2
        zone myevpnzn
        tag 12000

subnets.cfg
Code: 
evpn: myevpnzn
        controller myevpncl
        vrf-vxlan 10000
        exitnodes-primary pve
        ipam pve
        mac BC:24:11:2A:62:B7
        mtu 1450
        nodes pve,pve-home-2

Node specific configurations:

Node 1:

/etc/network/interfaces.d/sdn
Code: 
#version:47

auto myvnet1
iface myvnet1
        address 10.0.5.1/24
        hwaddress BC:24:11:2A:62:B7
        bridge_ports vxlan_myvnet1
        bridge_stp off
        bridge_fd 0
        mtu 1450
        ip-forward on
        arp-accept on
        vrf vrf_myevpnzn

auto myvnet2
iface myvnet2
        address 10.0.6.1/24
        hwaddress BC:24:11:2A:62:B7
        bridge_ports vxlan_myvnet2
        bridge_stp off
        bridge_fd 0
        mtu 1450
        ip-forward on
        arp-accept on
        vrf vrf_myevpnzn

auto vrf_myevpnzn
iface vrf_myevpnzn
        vrf-table auto
        post-up ip route add vrf vrf_myevpnzn unreachable default metric 4278198272

auto vrfbr_myevpnzn
iface vrfbr_myevpnzn
        bridge-ports vrfvx_myevpnzn
        bridge_stp off
        bridge_fd 0
        mtu 1450
        vrf vrf_myevpnzn

auto vrfvx_myevpnzn
iface vrfvx_myevpnzn
        vxlan-id 10000
        vxlan-local-tunnelip 10.0.1.131
        bridge-learning off
        bridge-arp-nd-suppress on
        mtu 1450

auto vxlan_myvnet1
iface vxlan_myvnet1
        vxlan-id 11000
        vxlan-local-tunnelip 10.0.1.131
        bridge-learning off
        bridge-arp-nd-suppress on
        mtu 1450

auto vxlan_myvnet2
iface vxlan_myvnet2
        vxlan-id 12000
        vxlan-local-tunnelip 10.0.1.131
        bridge-learning off
        bridge-arp-nd-suppress on
        mtu 1450

/etc/frr/frr.conf

Code: 
frr version 8.5.1
frr defaults datacenter
hostname pve
log syslog informational
service integrated-vtysh-config
!
!
vrf vrf_myevpnzn
 vni 10000
exit-vrf
!
router bgp 65000
 bgp router-id 10.0.1.131
 no bgp hard-administrative-reset
 no bgp graceful-restart notification
 no bgp default ipv4-unicast
 coalesce-time 1000
 neighbor VTEP peer-group
 neighbor VTEP remote-as 65000
 neighbor VTEP bfd
 neighbor 10.0.1.132 peer-group VTEP
 !
 address-family l2vpn evpn
  neighbor VTEP route-map MAP_VTEP_IN in
  neighbor VTEP route-map MAP_VTEP_OUT out
  neighbor VTEP activate
  advertise-all-vni
 exit-address-family
exit
!
router bgp 65000 vrf vrf_myevpnzn
 bgp router-id 10.0.1.131
 no bgp hard-administrative-reset
 no bgp graceful-restart notification
exit
!
route-map MAP_VTEP_IN permit 1
exit
!
route-map MAP_VTEP_OUT permit 1
exit
!
line vty

Node 2:

/etc/network/interfaces.d/sdn
Code: 
#version:47

auto myvnet1
iface myvnet1
        address 10.0.5.1/24
        hwaddress BC:24:11:2A:62:B7
        bridge_ports vxlan_myvnet1
        bridge_stp off
        bridge_fd 0
        mtu 1450
        ip-forward on
        arp-accept on
        vrf vrf_myevpnzn

auto myvnet2
iface myvnet2
        address 10.0.6.1/24
        hwaddress BC:24:11:2A:62:B7
        bridge_ports vxlan_myvnet2
        bridge_stp off
        bridge_fd 0
        mtu 1450
        ip-forward on
        arp-accept on
        vrf vrf_myevpnzn

auto vrf_myevpnzn
iface vrf_myevpnzn
        vrf-table auto
        post-up ip route add vrf vrf_myevpnzn unreachable default metric 4278198272

auto vrfbr_myevpnzn
iface vrfbr_myevpnzn
        bridge-ports vrfvx_myevpnzn
        bridge_stp off
        bridge_fd 0
        mtu 1450
        vrf vrf_myevpnzn

auto vrfvx_myevpnzn
iface vrfvx_myevpnzn
        vxlan-id 10000
        vxlan-local-tunnelip 10.0.1.132
        bridge-learning off
        bridge-arp-nd-suppress on
        mtu 1450

auto vxlan_myvnet1
iface vxlan_myvnet1
        vxlan-id 11000
        vxlan-local-tunnelip 10.0.1.132
        bridge-learning off
        bridge-arp-nd-suppress on
        mtu 1450

auto vxlan_myvnet2
iface vxlan_myvnet2
        vxlan-id 12000
        vxlan-local-tunnelip 10.0.1.132
        bridge-learning off
        bridge-arp-nd-suppress on
        mtu 1450

/etc/frr/frr.conf

Code: 
frr version 8.5.1
frr defaults datacenter
hostname pve-home-2
log syslog informational
service integrated-vtysh-config
!
!
vrf vrf_myevpnzn
 vni 10000
exit-vrf
!
router bgp 65000
 bgp router-id 10.0.1.132
 no bgp hard-administrative-reset
 no bgp graceful-restart notification
 no bgp default ipv4-unicast
 coalesce-time 1000
 neighbor VTEP peer-group
 neighbor VTEP remote-as 65000
 neighbor VTEP bfd
 neighbor 10.0.1.131 peer-group VTEP
 !
 address-family l2vpn evpn
  neighbor VTEP route-map MAP_VTEP_IN in
  neighbor VTEP route-map MAP_VTEP_OUT out
  neighbor VTEP activate
  advertise-all-vni
 exit-address-family
exit
!
router bgp 65000 vrf vrf_myevpnzn
 bgp router-id 10.0.1.132
 no bgp hard-administrative-reset
 no bgp graceful-restart notification
exit
!
route-map MAP_VTEP_IN permit 1
exit
!
route-map MAP_VTEP_OUT permit 1
exit
!
line vty

Any guidance or suggestions would be more than welcome!
 
Last edited:
It seems that the BGP daemon is not running ?

Code: 
root@pve:~# vtysh -c 'show bgp summary'
bgpd is not running

I can confirm that as per systemctl, it's started and active.

Code: 
root@pve:~# systemctl status frr
● frr.service - FRRouting
     Loaded: loaded (/lib/systemd/system/frr.service; enabled; preset: enabled)
     Active: active (running) since Wed 2024-01-24 20:07:40 -03; 8min ago
       Docs: https://frrouting.readthedocs.io/en/latest/setup.html
    Process: 1306621 ExecStart=/usr/lib/frr/frrinit.sh start (code=exited, status=0/SUCCESS)
   Main PID: 1306631 (watchfrr)
     Status: "FRR Operational"
      Tasks: 7 (limit: 18693)
     Memory: 9.3M
        CPU: 196ms
     CGroup: /system.slice/frr.service
             ├─1306631 /usr/lib/frr/watchfrr -d -F traditional zebra staticd
             ├─1306642 /usr/lib/frr/zebra -d -F traditional -A 127.0.0.1 -s 90000000
             └─1306647 /usr/lib/frr/staticd -d -F traditional -A 127.0.0.1
 
I didn't have bgdp daemon enabled in `/etc/frr/daemons`

Now the output of `vtysh -c 'show bgp summary'` is looking healthy on both nodes.

Node 1
Code: 
root@pve:~# vtysh -c 'show bgp summary'

L2VPN EVPN Summary (VRF default):
BGP router identifier 10.0.1.131, local AS number 65000 vrf-id 0
BGP table version 0
RIB entries 7, using 1344 bytes of memory
Peers 1, using 724 KiB of memory
Peer groups 1, using 64 bytes of memory

Neighbor               V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt Desc
pve-home-2(10.0.1.132) 4      65000        80        79        0    0    0 00:03:22            6        5 N/A

Total number of neighbors 1

Node 2
Code: 
root@pve-home-2:~# vtysh -c 'show bgp summary'

L2VPN EVPN Summary (VRF default):
BGP router identifier 10.0.1.132, local AS number 65000 vrf-id 0
BGP table version 0
RIB entries 7, using 1344 bytes of memory
Peers 1, using 724 KiB of memory
Peer groups 1, using 64 bytes of memory

Neighbor        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt Desc
pve(10.0.1.131) 4      65000        88        89        0    0    0 00:03:50            5        6 N/A

Total number of neighbors 1

I still can't ping across VMs :(
 
Spirit,

Sorry about that. I originally started this thread and it disappeared. I suspect it might have been because I did several posts on my post and it was considered spammy. We can continue on the other one where information is more consolidated.

Thanks!
 
You must log in or register to reply here.
Top Bottom