[SOLVED] SDN nodes out of sync.

V

volga629

Member
May 11, 2022
27
0
6
Hello Everyones,
Scenario: cluster 4 nodes
Issue: Some nodes get out of sync ( impossible push any configuration )


Log: pvestatd[1971268]: local sdn network configuration is too old, please reload


I checked SDN configuration files and they missing configuration.

How to restore good state ?
 
volga629 said:
Hello Everyones,
Scenario: cluster 4 nodes
Issue: Some nodes get out of sync ( impossible push any configuration )


Log: pvestatd[1971268]: local sdn network configuration is too old, please reload


I checked SDN configuration files and they missing configuration.

How to restore good state ?
Click to expand...
Hi,

do you have included
"source /etc/network/interfaces.d/*"

in /etc/network/interfaces ?

do you see the reload task for theses nodes ?

if yes, can you send the result of "ifquery -c -a" ?
 
Thank you for reply
spirit said:
do you have included
"source /etc/network/interfaces.d/*"
Click to expand...
Yes it included. I see in ifquery
spirit said:
do you see the reload task for theses nodes ?
Click to expand...
No I don't see the running task, because in web ui, mode is pending and it will execute task only on available nodes.

spirit said:
if yes, can you send the result of "ifquery -c -a" ?
Click to expand...

ifquery output


Code: 
root@aitdalpve02:/etc/network# ifquery -c -a
auto Vnet1
iface Vnet1                                                         [pass]
    bridge-ports vxlan_Vnet1                                    [pass]
    bridge-fd 0                                                 [pass]
    bridge-stp no                                               [pass]
    vrf vrf_vmbr2                                               [pass]
    mtu 9000                                                    [pass]
    hwaddress d6:ad:ac:b0:d9:35                                 [pass]
    ip-forward on                                               [pass]
    address                                                     [pass]

auto Vnet2
iface Vnet2                                                         [pass]
    bridge-ports vxlan_Vnet2                                    [pass]
    bridge-fd 0                                                 [pass]
    bridge-stp no                                               [pass]
    vrf vrf_vmbr2                                               [pass]
    mtu 9000                                                    [pass]
    hwaddress d6:ad:ac:b0:d9:35                                 [pass]
    alias MirtaPBX                                              [pass]
    ip-forward on                                               [pass]
    address                                                     [pass]

auto vrf_vmbr2
iface vrf_vmbr2                                                     [pass]
    post-up ip route add vrf vrf_vmbr2 unreachable default metric 4278198272         []
    vrf-table 1001                                              [pass]

auto vrfbr_vmbr2
iface vrfbr_vmbr2                                                   [pass]
    bridge-ports vrfvx_vmbr2                                    [pass]
    bridge-fd 0                                                 [pass]
    bridge-stp no                                               [pass]
    vrf vrf_vmbr2                                               [pass]
    mtu 9000                                                    [pass]

auto vrfvx_vmbr2
iface vrfvx_vmbr2                                                   [pass]
    vxlan-id 10000                                              [pass]
    vxlan-local-tunnelip                                      [pass]
    bridge-learning off                                         [pass]
    mtu 9000                                                    [pass]

auto vxlan_Vnet1
iface vxlan_Vnet1                                                   [pass]
    vxlan-id 1010                                               [pass]
    vxlan-local-tunnelip                                        [pass]
    bridge-learning off                                         [pass]
    mtu 9000                                                    [pass]

auto vxlan_Vnet2
iface vxlan_Vnet2                                                   [pass]
    vxlan-id 1011                                               [pass]
    vxlan-local-tunnelip                                        [pass]
    bridge-learning off                                         [pass]
    mtu 9000                                                    [pass]

auto lo
iface lo inet loopback

auto Mgmt-VRF
iface Mgmt-VRF                                                      [pass]
    post-up ip route add vrf Mgmt-VRF unreachable default metric 4278198272         []
    vrf-table 1199                                              [pass]

auto enp10s0
iface enp10s0 inet manual                                           [pass]
    post-up ethtool -G enp10s0 rx 4096 tx 4096                      []
    post-up ip link set dev enp10s0 txqueuelen 10000                []
    mtu 9000                                                    [pass]

auto enp11s0
iface enp11s0 inet manual                                           [pass]
    post-up ethtool -G enp11s0 rx 4096 tx 4096                      []
    post-up ip link set dev enp11s0 txqueuelen 10000                []
    mtu 9000                                                    [pass]

auto eno1
iface eno1 inet manual

auto eno2
iface eno2 inet manual

auto enp137s0
iface enp137s0 inet manual                                          [pass]
    post-up ethtool -G enp137s0 rx 4096 tx 4096                     []
    post-up ip link set dev enp137s0 txqueuelen 10000               []
    mtu 9000                                                    [pass]

auto enp138s0
iface enp138s0 inet manual                                          [pass]
    post-up ethtool -G enp138s0 rx 4096 tx 4096                     []
    post-up ip link set dev enp138s0 txqueuelen 10000               []
    mtu 9000                                                    [pass]

auto vlan99
iface vlan99 inet dhcp                                              [pass]
    vrf Mgmt-VRF                                                [pass]

auto vlan995
iface vlan995 inet static                                           [pass]
    address                                                         [pass]

auto vlan998
iface vlan998 inet static                                           [pass]
    address                                                        [pass]

auto bond0
iface bond0 inet manual                                             [pass]
    vrf Mgmt-VRF                                                [pass]

auto bond1
iface bond1 inet manual

auto bond2
iface bond2 inet manual

auto vmbr0
iface vmbr0 inet manual                                             [pass]
    vrf Mgmt-VRF                                                [pass]

auto vmbr1
iface vmbr1 inet manual

auto vmbr2
iface vmbr2 inet manual

auto vlan991
iface vlan991 inet manual                                           [pass]
    address                                                        [pass]
 
spirit said:
can you send me the log of the task "reloadnetworkall", when you click on the "apply" button on main section section ?

also, can you send me the content of /etc/pve/sdn/*.cfg files ?
Click to expand...
That interesting


Code: 
proxy handler failed: ssh: connect to host 172.22.100.6 port 22: Connection refused
command 'pvesh set /nodes/aitdalpve03/network' failed: exit code 255
proxy handler failed: ssh: connect to host 172.22.100.5 port 22: Connection refused
command 'pvesh set /nodes/aitdalpve02/network' failed: exit code 255
TASK OK

not sure yet why connection refused . SSH listen on all interfaces

Code: 
root@aitdalpve02:~# lsof -i :22
COMMAND     PID USER   FD   TYPE    DEVICE SIZE/OFF NODE NAME
sshd       3068 root    3u  IPv4     57770      0t0  TCP *:ssh (LISTEN)
sshd       3068 root    4u  IPv6     57772      0t0  TCP *:ssh (LISTEN)
 
Last edited:
Thank you for all help.
Issue has been resolved .
Ssh daemon was configured to use Mgmt VRF.
I configured secondary ssh instance and everything is reloads and applies as expected.

Code: 
ExecStart=/usr/sbin/ip vrf exec Mgmt-VRF
 
volga629 said:
Thank you for all help.
Issue has been resolved .
Ssh daemon was configured to use Mgmt VRF.
I configured secondary ssh instance and everything is reloads and applies as expected.

Code: 
ExecStart=/usr/sbin/ip vrf exec Mgmt-VRF
Click to expand...
ok thanks !

be carefull, because for vm live migration for example, proxmox also use ssh tunnel, and it's really run in the main vrf.
 
You must log in or register to reply here.
Top Bottom