VLAN SDN VNet Stops OSPF From Working

[PrimeYeti]

Currently I have a VLAN SDN zone setup with 3 nodes in it. I also have FRR setup on each node that is supposed to use OSPF to get its default gateways from some upstream routers, however, on the routers the neighbor state doesn't change from 'Init' indicating that there isn't bidirectional OSPF communication.

Everything else works it is just the OSPF on the nodes specifically that are having the problem. FRR configured on a VM on the nodes is also fine.

After some testing I thought it was just an MTU issue, however, even after ensuring that MTU on all interfaces is configured to 1500, this still doesn't work. I then created a separate zone that is exactly the same as the original zone except it doesn't have any VNets in. As soon as I put one of the nodes into this zone the OSPF neighborship comes up on this node. As soon as I add a VNet to the zone, the neighborship then drops again.

Any idea why adding a VNet would kill OSPF and how I could resolve it?

 

[PrimeYeti]

Is anyone able to assist here, please?

 

[shanreich]

Can you post your exact configuration as well as some debug output (in the state where OSPF is not working)?

pveversion -v

cat /etc/frr/frr.conf
vtysh -c 'show ip ospf summary'
vtysh -c 'show ip ospf interface'
vtysh -c 'show ip ospf neighbor detail'

cat /etc/network/interfaces
cat /etc/network/interfaces.d/sdn

 

[PrimeYeti]

pveversion -v

proxmox-ve: 8.3.0 (running kernel: 6.8.12-4-pve)
pve-manager: 8.3.0 (running version: 8.3.0/c1689ccb1065a83b)
proxmox-kernel-helper: 8.1.0
proxmox-kernel-6.8: 6.8.12-4
proxmox-kernel-6.8.12-4-pve-signed: 6.8.12-4
ceph-fuse: 17.2.7-pve3
corosync: 3.1.7-pve3
criu: 3.17.1-2
frr-pythontools: 10.3-0~deb12u1
glusterfs-client: 10.3-5
ifupdown2: 3.2.0-1+pmx11
ksm-control-daemon: 1.5-1
libjs-extjs: 7.0.0-5
libknet1: 1.28-pve1
libproxmox-acme-perl: 1.5.1
libproxmox-backup-qemu0: 1.4.1
libproxmox-rs-perl: 0.3.4
libpve-access-control: 8.2.0
libpve-apiclient-perl: 3.3.2
libpve-cluster-api-perl: 8.0.10
libpve-cluster-perl: 8.0.10
libpve-common-perl: 8.2.9
libpve-guest-common-perl: 5.1.6
libpve-http-server-perl: 5.1.2
libpve-network-perl: 0.10.0
libpve-rs-perl: 0.9.0
libpve-storage-perl: 8.2.9
libspice-server1: 0.15.1-1
lvm2: 2.03.16-2
lxc-pve: 6.0.0-1
lxcfs: 6.0.0-pve2
novnc-pve: 1.5.0-1
proxmox-backup-client: 3.2.9-1
proxmox-backup-file-restore: 3.2.9-1
proxmox-firewall: 0.6.0
proxmox-kernel-helper: 8.1.0
proxmox-mail-forward: 0.3.1
proxmox-mini-journalreader: 1.4.0
proxmox-offline-mirror-helper: 0.6.7
proxmox-widget-toolkit: 4.3.1
pve-cluster: 8.0.10
pve-container: 5.2.2
pve-docs: 8.3.1
pve-edk2-firmware: 4.2023.08-4
pve-esxi-import-tools: 0.7.2
pve-firewall: 5.1.0
pve-firmware: 3.14-1
pve-ha-manager: 4.0.6
pve-i18n: 3.3.1
pve-qemu-kvm: 9.0.2-4
pve-xtermjs: 5.3.0-3
qemu-server: 8.3.0
smartmontools: 7.3-pve1
spiceterm: 3.3.0
swtpm: 0.8.0+pve1
vncterm: 1.8.0
zfsutils-linux: 2.2.6-pve1

cat /etc/frr/frr.conf
I know the config is correct as it works whenever I take the VNet out of the VLAN Zone

vtysh -c 'show ip ospf summary'
This is just empty

vtysh -c 'show ip ospf interface'

vmbr0 is up

  ifindex 9, MTU 1500 bytes, BW 20000 Mbit <UP,LOWER_UP,BROADCAST,RUNNING,MULTICAST>

  Internet Address *.*.*.*, Broadcast *.*.*.*, Area *.*.*.*

  MTU mismatch detection: enabled

  Router ID *.*.*.*, Network Type BROADCAST, Cost: 5

  Transmit Delay is 1 sec, State DR, Priority 1

  Designated Router (ID) *.*.*.* Interface Address *.*.*.*

  No backup designated router on this network

  Saved Network-LSA sequence number *

  Multicast group memberships: OSPFAllRouters OSPFDesignatedRouters

  Timer intervals configured, Hello 10s, Dead 40s, Wait 40s, Retransmit 5

    Hello due in 9.752s

  Neighbor Count is 0, Adjacent neighbor count is 0

  Graceful Restart hello delay: 10s

  LSA retransmissions: 10523

vtysh -c 'show ip ospf neighbor detail'
This is just empty

cat /etc/network/interfaces

auto lo

iface lo inet loopback



auto eno1

iface eno1 inet manual

        mtu 1500

        up ip link set eno1 promisc on



auto eno2

iface eno2 inet manual

        mtu 1500

        up ip link set eno2 promisc on



iface eno3 inet manual



iface eno4 inet manual



auto bond0

iface bond0 inet manual

        bond-slaves eno1 eno2

        bond-miimon 100

        bond-mode 802.3ad

        bond-xmit-hash-policy layer3+4

        mtu 1500

        up ip link set bond0 promisc on



auto vmbr0

iface vmbr0 inet static

        address *.*.*.*

        gateway *.*.*.*

        bridge-ports bond0

        bridge-stp off

        bridge-fd 0

        bridge-vlan-aware yes

        bridge-vids 2-4094

        mtu 1500



iface vmbr0 inet6 static

        address *:*:*:*:*:*:*:*

        gateway *:*:*:*:*:*:*:*



source /etc/network/interfaces.d/*

cat /etc/network/interfaces.d/sdn

#version:83



auto Private

iface Private

        bridge_ports vmbr0.113

        bridge_stp off

        bridge_fd 0

        mtu 1500



auto Public

iface Public

        bridge_ports vmbr0.112

        bridge_stp off

        bridge_fd 0

        mtu 1500