Netzwerkprobleme zwischen den Knoten seit 7.1 bis 7.3

[atlasloewenherz]

moin Zusammen,

Ich habe 3 Nodes je 4 Nic eno1,2,3,4 diese sind jeweils eine Bridge zugeordnet vmbr0, vmbr1,vmbr2, vmbr3

Node1:
vmbr0: 10.10.10.10/24
vmbr1: 10.10.11.11/24
vmbr2: 10.10.12.12/24
vmbr3: 10.10.13.13/24

Node2:
vmbr0: 10.10.10.11/24
vmbr1: 10.10.11.12/24
vmbr2: 10.10.12.13/24
vmbr3: 10.10.13.14/24

Node3:
vmbr0: 10.10.10.12/24
vmbr1: 10.10.11.13/24
vmbr2: 10.10.12.14/24
vmbr3: 10.10.13.15/24


Node1 und Node2 verlieren mehrfach die Verbindung am Tag dmesg aus dem Node1:

[Fri Dec  9 11:39:16 2022] device tap1998i0 entered promiscuous mode
[Fri Dec  9 11:39:16 2022] vmbr1: port 2(tap1998i0) entered blocking state
[Fri Dec  9 11:39:16 2022] vmbr1: port 2(tap1998i0) entered disabled state
[Fri Dec  9 11:39:16 2022] vmbr1: port 2(tap1998i0) entered blocking state
[Fri Dec  9 11:39:16 2022] vmbr1: port 2(tap1998i0) entered forwarding state
[Fri Dec  9 11:39:26 2022] sd 7:0:0:0: alua: supports implicit TPGS

Node2

[Fri Dec  9 12:20:00 2022] vmbr0: port 2(veth125i0) entered blocking state
[Fri Dec  9 12:20:00 2022] vmbr0: port 2(veth125i0) entered disabled state
[Fri Dec  9 12:20:00 2022] device veth125i0 entered promiscuous mode
[Fri Dec  9 12:20:00 2022] eth0: renamed from vethGWVVLu
[Fri Dec  9 12:20:02 2022] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[Fri Dec  9 12:20:02 2022] vmbr0: port 2(veth125i0) entered blocking state
[Fri Dec  9 12:20:02 2022] vmbr0: port 2(veth125i0) entered forwarding state

hätte jemand eine Idee wie ich mindestens das Problem eingrenzen könnte?

pve-manager/7.3-3/c3928077 (running kernel: 5.15.74-1-pve)

root@pve01:~# uname -a
Linux pve01 5.15.74-1-pve #1 SMP PVE 5.15.74-1 (Mon, 14 Nov 2022 20:17:15 +0100) x86_64 GNU/Linux

Node1 lspci:

00:00.0 Host bridge: Intel Corporation 8th Gen Core 4-core Workstation Processor Host Bridge/DRAM Registers [Coffee Lake S] (rev 07)
00:01.0 PCI bridge: Intel Corporation 6th-10th Gen Core Processor PCIe Controller (x16) (rev 07)
00:12.0 Signal processing controller: Intel Corporation Cannon Lake PCH Thermal Controller (rev 10)
00:14.0 USB controller: Intel Corporation Cannon Lake PCH USB 3.1 xHCI Host Controller (rev 10)
00:14.2 RAM memory: Intel Corporation Cannon Lake PCH Shared SRAM (rev 10)
00:16.0 Communication controller: Intel Corporation Cannon Lake PCH HECI Controller (rev 10)
00:16.4 Communication controller: Intel Corporation Cannon Lake PCH HECI Controller #2 (rev 10)
00:17.0 SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)
00:1b.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #21 (rev f0)
00:1c.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #1 (rev f0)
00:1d.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #9 (rev f0)
00:1d.1 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #10 (rev f0)
00:1d.2 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #11 (rev f0)
00:1d.3 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #12 (rev f0)
00:1f.0 ISA bridge: Intel Corporation Device a30a (rev 10)
00:1f.5 Serial bus controller [0c80]: Intel Corporation Cannon Lake PCH SPI Controller (rev 10)
01:00.0 System peripheral: Hewlett-Packard Company Integrated Lights-Out Standard Slave Instrumentation & System Support (rev 07)
01:00.1 VGA compatible controller: Matrox Electronics Systems Ltd. MGA G200eH3 (rev 02)
01:00.2 System peripheral: Hewlett-Packard Company Integrated Lights-Out Standard Management Processor Support and Messaging (rev 07)
01:00.4 USB controller: Hewlett-Packard Company iLO5 Virtual USB Controller
02:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
02:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
02:00.2 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
02:00.3 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)

Node2 lspci:

00:00.0 Host bridge: Intel Corporation 8th Gen Core 4-core Workstation Processor Host Bridge/DRAM Registers [Coffee Lake S] (rev 07)
00:01.0 PCI bridge: Intel Corporation 6th-10th Gen Core Processor PCIe Controller (x16) (rev 07)
00:12.0 Signal processing controller: Intel Corporation Cannon Lake PCH Thermal Controller (rev 10)
00:14.0 USB controller: Intel Corporation Cannon Lake PCH USB 3.1 xHCI Host Controller (rev 10)
00:14.2 RAM memory: Intel Corporation Cannon Lake PCH Shared SRAM (rev 10)
00:16.0 Communication controller: Intel Corporation Cannon Lake PCH HECI Controller (rev 10)
00:16.4 Communication controller: Intel Corporation Cannon Lake PCH HECI Controller #2 (rev 10)
00:17.0 SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)
00:1b.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #21 (rev f0)
00:1c.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #1 (rev f0)
00:1d.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #9 (rev f0)
00:1d.1 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #10 (rev f0)
00:1d.2 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #11 (rev f0)
00:1d.3 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #12 (rev f0)
00:1f.0 ISA bridge: Intel Corporation Device a30a (rev 10)
00:1f.5 Serial bus controller [0c80]: Intel Corporation Cannon Lake PCH SPI Controller (rev 10)
01:00.0 System peripheral: Hewlett-Packard Company Integrated Lights-Out Standard Slave Instrumentation & System Support (rev 07)
01:00.1 VGA compatible controller: Matrox Electronics Systems Ltd. MGA G200eH3 (rev 02)
01:00.2 System peripheral: Hewlett-Packard Company Integrated Lights-Out Standard Management Processor Support and Messaging (rev 07)
01:00.4 USB controller: Hewlett-Packard Company iLO5 Virtual USB Controller
02:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
02:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
02:00.2 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
02:00.3 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)

 

[Richard]

moin Zusammen,

Ich habe 3 Nodes je 4 Nic eno1,2,3,4 diese sind jeweils eine Bridge zugeordnet vmbr0, vmbr1,vmbr2, vmbr3

Node1:
vmbr0: 10.10.10.10/24
vmbr1: 10.10.11.11/24
vmbr2: 10.10.12.12/24
vmbr3: 10.10.13.13/24

Node2:
vmbr0: 10.10.10.11/24
vmbr1: 10.10.11.12/24
vmbr2: 10.10.12.13/24
vmbr3: 10.10.13.14/24

Node3:
vmbr0: 10.10.10.12/24
vmbr1: 10.10.11.13/24
vmbr2: 10.10.12.14/24
vmbr3: 10.10.13.15/24


Node1 und Node2 verlieren mehrfach die Verbindung am Tag dmesg aus dem Node1:

[Fri Dec  9 11:39:16 2022] device tap1998i0 entered promiscuous mode
[Fri Dec  9 11:39:16 2022] vmbr1: port 2(tap1998i0) entered blocking state
[Fri Dec  9 11:39:16 2022] vmbr1: port 2(tap1998i0) entered disabled state
[Fri Dec  9 11:39:16 2022] vmbr1: port 2(tap1998i0) entered blocking state
[Fri Dec  9 11:39:16 2022] vmbr1: port 2(tap1998i0) entered forwarding state
[Fri Dec  9 11:39:26 2022] sd 7:0:0:0: alua: supports implicit TPGS

Da verlieren nicht die (physikalischen) nodes die Verbindung sondern die VM1998 bzw. der Container 125 verlieren diese oder werden abgeschaltet o.ä. Für Details müsste man in die log files (/var/log/syslog*) sehen.

 

[Falk R.]

Vermutlich hast du auch Spanning Tree im Netzwerk an, deshalb gehen die Ports immer kurz in blocking state und nach Loop Prüfung in forward state.