This is an odd behavior I'm seeing on two servers so far. Not sure why. I did some searches here but wasn't sure what to try for, nothing relevant seemed to come up.
I picked up some SFP+ Intel 10Gbit NICs (82599ES, X520-DA2), connected to each other for high speed VM communication.
So far they work great, VMs move fast between the cluster. The issue comes when I look at the system with ip a on the CLI after a while.
I'll see a dozen of these
11: fwpr104p0@fwln104i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000
link/ether 32:90:44:95:da:be brd ff:ff:ff:ff:ff:ff
12: fwln104i0@fwpr104p0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr104i0 state UP group default qlen 1000
link/ether b6:3b:48:82:e0:0e brd ff:ff:ff:ff:ff:ff
14: tap107i0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master fwbr107i0 state UNKNOWN group default qlen 1000
..
24: fwpr108p1@fwln108i1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr2EXT state UP group default qlen 1000
link/ether 56:09:e2:de:6f:16 brd ff:ff:ff:ff:ff:ff
25: fwln108i1@fwpr108p1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr108i1 state UP group default qlen 1000
link/ether 4e:33:25:42:9f:74 brd ff:ff:ff:ff:ff:ff
67: tap105i0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr0 state UNKNOWN group default qlen 1000
link/ether 0a:ec:83:8d:83:95 brd ff:ff:ff:ff:ff:ff
I counted, the Intel based one goes 1-25, and then 67, the amd based server goes from 1-49 interfaces.
And this is on both servers. I'm pretty sure the cards are good, they work, they're fast, but the behavior is strange. On boot I only get the basic proper interfaces there, but as time goes on it adds more of these.
Thing is they look fine in the web interface. Networking seems to work even with this odd behavior.
Box 1
02:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
02:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
05:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
05:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
08:00.0 Ethernet controller: Intel Corporation Ethernet Controller I225-V (rev 01)
Box 2
01:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
01:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
07:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
09:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller (rev 05)
I picked up some SFP+ Intel 10Gbit NICs (82599ES, X520-DA2), connected to each other for high speed VM communication.
So far they work great, VMs move fast between the cluster. The issue comes when I look at the system with ip a on the CLI after a while.
I'll see a dozen of these
11: fwpr104p0@fwln104i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000
link/ether 32:90:44:95:da:be brd ff:ff:ff:ff:ff:ff
12: fwln104i0@fwpr104p0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr104i0 state UP group default qlen 1000
link/ether b6:3b:48:82:e0:0e brd ff:ff:ff:ff:ff:ff
14: tap107i0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master fwbr107i0 state UNKNOWN group default qlen 1000
..
24: fwpr108p1@fwln108i1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr2EXT state UP group default qlen 1000
link/ether 56:09:e2:de:6f:16 brd ff:ff:ff:ff:ff:ff
25: fwln108i1@fwpr108p1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr108i1 state UP group default qlen 1000
link/ether 4e:33:25:42:9f:74 brd ff:ff:ff:ff:ff:ff
67: tap105i0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr0 state UNKNOWN group default qlen 1000
link/ether 0a:ec:83:8d:83:95 brd ff:ff:ff:ff:ff:ff
I counted, the Intel based one goes 1-25, and then 67, the amd based server goes from 1-49 interfaces.
And this is on both servers. I'm pretty sure the cards are good, they work, they're fast, but the behavior is strange. On boot I only get the basic proper interfaces there, but as time goes on it adds more of these.
Thing is they look fine in the web interface. Networking seems to work even with this odd behavior.
Box 1
02:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
02:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
05:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
05:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
08:00.0 Ethernet controller: Intel Corporation Ethernet Controller I225-V (rev 01)
Box 2
01:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
01:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
07:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
09:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller (rev 05)
