Hi there,
currently i´m trying to setup a second openfabric area. I´m running a 3 node PVE cluster.
I setted up a ceph fully routed mesh network with the offical guide. So far so good, everything is running perfectly.
Every of my 3 nodes have each two additional 10G NICs, which i want to use for cluster network/migration network.
Currently i´m trying this configuration:
The config ist on all 3 nodes the same except the hostname, router nets and the loopback address.
I tried to configure the loopback in the /etc/network/interfaces like this, because when i configure it in the frr config it is not bonded to the new openfabric router. But both methods are leading to the same result. The lo:10 isnt bonding to the new openfabric router:
show openfabric interface:
Here it is really clear, that new interface with ip isnt attached to the second openfabric router.
Is there anyway to configure a second loopback for this which is working in the routed network?
Thank you for your help.
currently i´m trying to setup a second openfabric area. I´m running a 3 node PVE cluster.
I setted up a ceph fully routed mesh network with the offical guide. So far so good, everything is running perfectly.
Every of my 3 nodes have each two additional 10G NICs, which i want to use for cluster network/migration network.
Currently i´m trying this configuration:
Code:
frr defaults traditional
hostname vm003
log syslog warning
ip forwarding
no ipv6 forwarding
service integrated-vtysh-config
!
interface lo
ip address 10.254.120.52/32
ip router openfabric 1
openfabric passive
!
interface lo:10 <- new additional loopback
ip router openfabric 2
openfabric passive
!
interface eno1np0
ip router openfabric 1
openfabric csnp-interval 2
openfabric hello-interval 1
openfabric hello-multiplier 2
!
interface eno2np1
ip router openfabric 1
openfabric csnp-interval 2
openfabric hello-interval 1
openfabric hello-multiplier 2
!
interface ens8f0np0 <- NEW additional 10G interface
ip router openfabric 2
openfabric csnp-interval 2
openfabric hello-interval 1
openfabric hello-multiplier 2
!
interface ens8f1np1 <- NEW additional 10G interface
ip router openfabric 2
openfabric csnp-interval 2
openfabric hello-interval 1
openfabric hello-multiplier 2
!
line vty
!
router openfabric 1
net 49.0001.3333.3333.3333.00
lsp-gen-interval 1
max-lsp-lifetime 600
lsp-refresh-interval 180
!
router openfabric 2 <- NEW additional router
net 49.0002.3333.3333.3333.00
lsp-gen-interval 1
max-lsp-lifetime 600
lsp-refresh-interval 180
!
The config ist on all 3 nodes the same except the hostname, router nets and the loopback address.
I tried to configure the loopback in the /etc/network/interfaces like this, because when i configure it in the frr config it is not bonded to the new openfabric router. But both methods are leading to the same result. The lo:10 isnt bonding to the new openfabric router:
Code:
auto lo
iface lo inet loopback
auto lo:10
iface lo:10 inet static
address 10.254.121.52/32
auto ens4f0np0
iface ens4f0np0 inet manual
auto eno1np0 <- first openfabric
iface eno1np0 inet manual
mtu 9000
auto eno2np1 <- first openfabric
iface eno2np1 inet manual
mtu 9000
auto ens4f1np1
iface ens4f1np1 inet manual
auto ens8f0np0 <- second openfabric
iface ens8f0np0 inet manual
mtu 9000
auto ens8f1np1 <- second openfabric
iface ens8f1np1 inet manual
mtu 9000
auto bond0
iface bond0 inet manual
bond-slaves ens4f0np0 ens4f1np1
bond-miimon 100
bond-mode 802.3ad
bond-xmit-hash-policy layer2+3
auto vmbr0
iface vmbr0 inet manual
bridge-ports bond0
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4094
post-up devlink dev eswitch set pci/0000:af:00.0 mode switchdev
post-up devlink dev eswitch set pci/0000:af:00.1 mode switchdev
auto mgmt
iface mgmt inet static
address 10.XXX.99.77/24
gateway 10.XXX.99.1
vlan-id 99
vlan-raw-device vmbr0
post-up /usr/bin/systemctl restart frr.service
source /etc/network/interfaces.d/*
show openfabric interface:
Code:
vm003# show openfabric interface
Area 1:
Interface CircId State Type Level
lo 0x0 Up loopback L2
eno1np0 0x0 Up p2p L2
eno2np1 0x0 Up p2p L2
Area 2:
Interface CircId State Type Level
ens8f0np0 0x0 Up p2p L2
ens8f1np1 0x0 Up p2p L2
vm003#
Here it is really clear, that new interface with ip isnt attached to the second openfabric router.
Is there anyway to configure a second loopback for this which is working in the routed network?
Thank you for your help.
Last edited: