openfab full mesh dual setup?

[tenfoldinaus]

hiya guys,

i have a full mesh ceph setup on 200gb nics

but i would like to create a second full mesh setup for my cluster to seperate cluster traffic
i have another 2x 100gb nics spare in each node interfaces enp4s0f0np0 and enp4s0f1np1
id like this to be the second seperate routed full mesh

but i cannot get it working for the life of me, and it keeps showing up as connected over the same first network ie.10.15.15.50
i know i must be doing something silly or wrong

frr.conf output

frr defaults traditional
hostname node3
log syslog warning
ip forwarding
no ipv6 forwarding
service integrated-vtysh-config
!
interface lo
ip address 10.15.15.52/32
ip address 192.168.50.4/32
ip router openfabric 1
openfabric passive
!
interface enp69s0f0np0
ip router openfabric 1
openfabric csnp-interval 2
openfabric hello-interval 1
openfabric hello-multiplier 2
!
interface enp101s0f0np0
ip router openfabric 1
openfabric csnp-interval 2
openfabric hello-interval 1
openfabric hello-multiplier 2
!
line vty
!
router openfabric 1
net 49.0001.3333.3333.3333.00
lsp-gen-interval 1
max-lsp-lifetime 600
lsp-refresh-interval 180
!
interface lo
ip address 192.168.50.4/32
ip router openfabric 2
openfabric passive
!
interface enp4s0f0np0
ip router openfabric 2
openfabric csnp-interval 2
openfabric hello-interval 1
openfabric hello-multiplier 2
!
interface enp4s0f1np1
ip router openfabric 2
openfabric csnp-interval 2
openfabric hello-interval 1
openfabric hello-multiplier 2
!
line vty
!
router openfabric 2
net 49.0001.6666.6666.6666.00
lsp-gen-interval 1
max-lsp-lifetime 600
lsp-refresh-interval 180


Vtysh output

root@ns4:~# vtysh

Hello, this is FRRouting (version 8.5.2).
Copyright 1996-2005 Kunihiro Ishiguro, et al.

ns4# show openfabric route
Area 1:
IS-IS L2 IPv4 routing table:

Prefix Metric Interface Nexthop Label(s)
---------------------------------------------------------------
10.15.15.50/32 20 enp101s0f0np0 10.15.15.50 -
10.15.15.51/32 20 enp69s0f0np0 10.15.15.51 -
10.15.15.52/32 0 - - -
192.168.50.2/32 20 enp101s0f0np0 10.15.15.50 -
192.168.50.4/32 0 - - -
192.168.50.5/32 20 enp69s0f0np0 10.15.15.51 -

Area 2:
IS-IS L2 IPv4 routing table:

ns4#

 

[twofourniner]

Hello!

Curious if you ever were able to get this to work and if so, how you did it. If you were not able to get it to work, did you happen try to update the config for Ceph to make sure the public and cluster networks were the two different IPs as you had set them up in frr?