Hello,
I've went through the docs: https://pve.proxmox.com/pve-docs/chapter-pvesdn.html#pvesdn_setup_example_evpn
But I can't get to ping VM2 (
I've setup all the evpn settings via Proxmox UI according to the docs, also clicked apply on any changes. I tried with and without
Another difference I spotted while node 1 sdn file has
node 2 sdn file has
Maybe a bug in Proxmox that it not updates both sdn files properly? This has to do if you select the node as exit node or not, either is gets this "route del ..." OR "route add ..." entry.
For my initial Promox setup, I was using
Appreciate any help.
UPDATE:
Deleted all settings and started again, went in the same order to create e.g. vnets and subnets (before I created all vnets then afterwards the subnets). Maybe this was causing some issues, don't know how I caused the generation be that wrong. As for now, both
Getting closer, but unfortunately I still the same "Destination Host Unreachable" error.
After configuring evpn, if executing
Both proxmox hosts are exactly on the same versions:
Node 1 interfaces file
Node 2 interfaces file
Node1 sdn file
Node 2 sdn file now looks identical except
I've went through the docs: https://pve.proxmox.com/pve-docs/chapter-pvesdn.html#pvesdn_setup_example_evpn
But I can't get to ping VM2 (
vnet3, node 2) from within VM1 (vnet3, node 1) on another node. Currently just testing with vnet3 and vmbr2.I've setup all the evpn settings via Proxmox UI according to the docs, also clicked apply on any changes. I tried with and without
SNAT checkbox for my subnet vnet3. The strange thing is, my last try was SNAT checked, but it just updated/ added iptables -t nat -A POSTROUTING ... to node 1 sdn file. Node 2 sdn file is missing that line. Also tried with and without gateway 192.168.0.254 in the nodes /etc/network/interfaces files.Another difference I spotted while node 1 sdn file has
post-up ip route del vrf vrf_evpnzon1 unreachable default metric 4278198272node 2 sdn file has
post-up ip route add vrf vrf_evpnzon1 unreachable default metric 4278198272.For my initial Promox setup, I was using
Masquerading (NAT) with iptables. So either the diff between node 1 sdn and node 2sdn file is the culprit or if this is correct that they have to differ like that, then maybe something with my nat based interface files.Appreciate any help.
UPDATE:
Deleted all settings and started again, went in the same order to create e.g. vnets and subnets (before I created all vnets then afterwards the subnets). Maybe this was causing some issues, don't know how I caused the generation be that wrong. As for now, both
/etc/network/interfaces.d/sdn (except the vxlan-local-tunnelip) are identical.Getting closer, but unfortunately I still the same "Destination Host Unreachable" error.
After configuring evpn, if executing
ifreload -a on a Proxmox node, I now get warning: vrf_evpnzon0: post-up cmd 'ip route del vrf vrf_evpnzon0 unreachable default metric 4278198272' failed: returned 2 (RTNETLINK answers: No such process)Both proxmox hosts are exactly on the same versions:
Code:
proxmox-ve: 7.2-1 (running kernel: 5.15.74-1-pve)
pve-manager: 7.2-11 (running version: 7.2-11/b76d3178)
pve-kernel-5.15: 7.2-14
pve-kernel-helper: 7.2-14
pve-kernel-5.15.74-1-pve: 5.15.74-1
pve-kernel-5.15.64-1-pve: 5.15.64-1
pve-kernel-5.15.30-2-pve: 5.15.30-3
ceph-fuse: 15.2.16-pve1
corosync: 3.1.5-pve2
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.24-pve1
libproxmox-acme-perl: 1.4.2
libproxmox-backup-qemu0: 1.3.1-1
libpve-access-control: 7.2-4
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.2-4
libpve-guest-common-perl: 4.1-4
libpve-http-server-perl: 4.1-5
libpve-network-perl: 0.7.1
libpve-storage-perl: 7.2-10
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 5.0.0-3
lxcfs: 4.0.12-pve1
novnc-pve: 1.3.0-3
proxmox-backup-client: 2.2.7-1
proxmox-backup-file-restore: 2.2.7-1
proxmox-mini-journalreader: 1.3-1
proxmox-widget-toolkit: 3.5.1
pve-cluster: 7.2-2
pve-container: 4.2-3
pve-docs: 7.2-2
pve-edk2-firmware: 3.20220526-1
pve-firewall: 4.2-6
pve-firmware: 3.5-6
pve-ha-manager: 3.4.0
pve-i18n: 2.7-2
pve-qemu-kvm: 7.0.0-4
pve-xtermjs: 4.16.0-1
qemu-server: 7.2-5
smartmontools: 7.2-pve3
spiceterm: 3.2-2
swtpm: 0.8.0~bpo11+2
vncterm: 1.7-1
zfsutils-linux: 2.1.6-pve1Node 1 interfaces file
Code:
auto lo
iface lo inet loopback
iface lo inet6 loopback
auto enp0s31f6
iface enp0s31f6 inet static
address IPADDRESS
netmask 255.255.255.192
gateway IPADDRESS
iface enp0s31f6 inet6 static
address IPV6ADDRESS
netmask 64
gateway IPV6ADDRESS
...
auto vmbr3
iface vmbr3 inet static
address 10.1.3.1/24
bridge-ports none
bridge-stp off
bridge-fd 0
mtu 1500
post-up echo 1 > /proc/sys/net/ipv4/ip_forward
post-up iptables -t nat -A POSTROUTING -s '10.1.3.0/24' -o enp0s31f6 -j MASQUERADE
post-down iptables -t nat -D POSTROUTING -s '10.1.3.0/24' -o enp0s31f6 -j MASQUERADE
source /etc/network/interfaces.d/*Node 2 interfaces file
Code:
auto lo
iface lo inet loopback
iface lo inet6 loopback
auto enp0s31f6
iface enp0s31f6 inet static
address IPADDRESS
netmask 255.255.255.192
gateway IPADDRESS
iface enp0s31f6 inet6 static
address IPV6ADDRESS
netmask 64
gateway IPV6ADDRESS
...
auto vmbr3
iface vmbr3 inet static
address 10.1.3.2/24
bridge-ports none
bridge-stp off
bridge-fd 0
mtu 1500
post-up echo 1 > /proc/sys/net/ipv4/ip_forward
post-up iptables -t nat -A POSTROUTING -s '10.1.3.0/24' -o enp0s31f6 -j MASQUERADE
post-down iptables -t nat -D POSTROUTING -s '10.1.3.0/24' -o enp0s31f6 -j MASQUERADE
source /etc/network/interfaces.d/*Node1 sdn file
Code:
#version:20
auto vnet0
iface vnet0
address 10.0.1.1/24
hwaddress A...
bridge_ports vxlan_vnet0
bridge_stp off
bridge_fd 0
mtu 1450
ip-forward on
arp-accept on
vrf vrf_evpnzon0
auto vnet1
iface vnet1
address 10.0.2.1/24
hwaddress A...
bridge_ports vxlan_vnet1
bridge_stp off
bridge_fd 0
mtu 1450
ip-forward on
arp-accept on
vrf vrf_evpnzon0
auto vrf_evpnzon0
iface vrf_evpnzon0
vrf-table auto
post-up ip route del vrf vrf_evpnzon0 unreachable default metric 4278198272
auto vrfbr_evpnzon0
iface vrfbr_evpnzon0
bridge-ports vrfvx_evpnzon0
bridge_stp off
bridge_fd 0
mtu 1450
vrf vrf_evpnzon0
auto vrfvx_evpnzon0
iface vrfvx_evpnzon0
vxlan-id 10000
vxlan-local-tunnelip 10.1.3.1
bridge-learning off
bridge-arp-nd-suppress on
mtu 1450
auto vxlan_vnet0
iface vxlan_vnet0
vxlan-id 11000
vxlan-local-tunnelip 10.1.3.1
bridge-learning off
bridge-arp-nd-suppress on
mtu 1450
auto vxlan_vnet1
iface vxlan_vnet1
vxlan-id 12000
vxlan-local-tunnelip 10.1.3.1
bridge-learning off
bridge-arp-nd-suppress on
mtu 1450Node 2 sdn file now looks identical except
vxlan-local-tunnelip 10.1.3.2.
Last edited: