vmbr0: received packet on eth0 with own address as source address

[Belokan]

Hello all and sorry for the title of the post but I failed searching for a better one ... So I simply used the error message itself.

Since some days (let's say 2 weeks) one of my PVE node started to log in kernel.log, messages and syslog the following messages over and over:

root@pve1:/var/log# tail /var/log/messages
Sep 30 14:17:08 pve1 kernel: [21336.626777] net_ratelimit: 26 callbacks suppressed
Sep 30 14:17:08 pve1 kernel: [21336.626792] vmbr0: received packet on eth0 with own address as source address
Sep 30 14:17:08 pve1 kernel: [21336.626807] vmbr0: received packet on eth0 with own address as source address
Sep 30 14:17:08 pve1 kernel: [21336.626864] vmbr0: received packet on eth0 with own address as source address
Sep 30 14:17:08 pve1 kernel: [21336.840947] vmbr0: received packet on eth0 with own address as source address
Sep 30 14:17:08 pve1 kernel: [21337.069275] vmbr0: received packet on eth0 with own address as source address
Sep 30 14:17:09 pve1 kernel: [21337.368542] vmbr0: received packet on eth0 with own address as source address
Sep 30 14:17:09 pve1 kernel: [21337.639985] vmbr0: received packet on eth0 with own address as source address
Sep 30 14:17:09 pve1 kernel: [21337.842956] vmbr0: received packet on eth0 with own address as source address
Sep 30 14:17:10 pve1 kernel: [21338.172809] vmbr0: received packet on eth0 with own address as source address
root@pve1:/var/log#

If it was just a problem with messages flood, I could live with that but network is also nearly unresponsive on the node itself. Lost packets with ping, timeouts to services like DNS/SMTP, impossible to achieve an apt-get update/apt-get upgrade because of timeouts.

I've dist-upgraded the node to latest 4.3.1/kernel-4.4.19 using the content of /var/cache/apt/archives of another node and it does not solve the problem. The 2 others nodes are OK (one with the same hardware configuration).

On the other hand, VMs hosted on the node do not have the same problem and their network works as a charm ... No packet lost, download/upload, updates. So it is not an hardware problem.

eth1 is connect to a dedicated switch together with the second node and both NAS eth1 in order to isolate DRBD traffic over iSCSI ...

Here are some (maybe) relevant files:

root@pve1:~# cat /etc/network/interfaces
# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage part of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

iface eth0 inet manual

auto eth1
iface eth1 inet static
address 10.0.0.151
netmask 255.255.255.0

auto vmbr0
iface vmbr0 inet static
address 192.168.1.150
netmask 255.255.255.0
gateway 192.168.1.1
bridge_ports eth0
bridge_stp on
bridge_fd 0

I've set bridge_stp on for test but it does not help ...

root@pve1:~# cat /etc/hosts
127.0.0.1 localhost.localdomain localhost
192.168.1.150 pve1.home pve1 pvelocalhost

# The following lines are desirable for IPv6 capable hosts

::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts


192.168.1.100 ds412 ds412.home
192.168.1.101 ds415 ds415.home

192.168.1.151 pve2 pve2.home
192.168.1.152 pve3 pve3.home

10.0.0.100 ds412pm
10.0.0.101 ds415pm
10.0.0.151 pve1pm
10.0.0.152 pve2pm

------------------------------------------------

root@pve1:~# ifconfig -a
eth0 Link encap:Ethernet HWaddr 00:19:99:e4:2c:7b
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:3015438 errors:0 dropped:0 overruns:0 frame:0
TX packets:2622371 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1068023781 (1018.5 MiB) TX bytes:502621647 (479.3 MiB)
Interrupt:20 Memory:f7c00000-f7c20000

eth1 Link encap:Ethernet HWaddr a0:ce:c8:08:c9:65
inet addr:10.0.0.151 Bcast:10.0.0.255 Mask:255.255.255.0
inet6 addr: fe80::a2ce:c8ff:fe08:c965/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:451191 errors:0 dropped:0 overruns:0 frame:0
TX packets:309699 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:315754410 (301.1 MiB) TX bytes:639168821 (609.5 MiB)

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:253 errors:0 dropped:0 overruns:0 frame:0
TX packets:253 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:64142 (62.6 KiB) TX bytes:64142 (62.6 KiB)

tap110i0 Link encap:Ethernet HWaddr 0e:35:38:d0:1d:e6
inet6 addr: fe80::c35:38ff:fed0:1de6/64 Scope:Link
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
RX packets:92643 errors:0 dropped:0 overruns:0 frame:0
TX packets:138957 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:50170299 (47.8 MiB) TX bytes:71671657 (68.3 MiB)

tap112i0 Link encap:Ethernet HWaddr f2:52:21:de:ed:7c
inet6 addr: fe80::f052:21ff:fede:ed7c/64 Scope:Link
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
RX packets:16912 errors:0 dropped:0 overruns:0 frame:0
TX packets:56753 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1679698 (1.6 MiB) TX bytes:5346390 (5.0 MiB)

vmbr0 Link encap:Ethernet HWaddr 00:19:99:e4:2c:7b
inet addr:192.168.1.150 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::219:99ff:fee4:2c7b/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:2621971 errors:0 dropped:4820 overruns:0 frame:0
TX packets:2501502 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:927962815 (884.9 MiB) TX bytes:439551839 (419.1 MiB)

------------------------------------------------

root@pve1:~# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.1.1 0.0.0.0 UG 0 0 0 vmbr0
10.0.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 vmbr0

Thanks in advance for insights ... I'm a bit lost here.

Olivier

 

[morph027]

Can you post the output oft "brctl show" ?

Looks like a loop due to VMs (we see this sometimes on our test cluster, when VMs with multiple NICs on the same bridge are running wild )

 

[Belokan]

Hello !

Here you are:

root@pve1:~# brctl show
bridge name bridge id STP enabled interfaces
vmbr0 8000.001999e42c7b yes eth0
tap110i0
tap112i0

root@pve1:~# brctl showmacs vmbr0
port no mac addr is local? ageing timer
1 00:04:4b:48:6e:4e no 31.84
1 00:11:32:25:f2:df no 4.91
1 00:11:32:43:08:af no 4.42
1 00:15:17:2e:9f:37 no 9.35
1 00:19:99:e4:2c:7b yes 0.00
1 00:19:99:e4:2c:7b yes 0.00
1 00:1b:a9:66:63:2e no 8.76
1 00:62:6e:58:5b:05 no 52.25
1 00:62:6e:65:d0:42 no 10.26
1 08:00:27:d7:b4:7e no 0.02
3 0e:35:38:d0:1d:e6 yes 0.00
3 0e:35:38:d0:1d:e6 yes 0.00
1 28:10:7b:4e:08:c8 no 7.88
1 3a:35:61:65:33:30 no 3.44
1 3c:46:d8:be:e9:d9 no 1.47
1 62:36:66:39:62:64 no 0.00
3 62:62:61:39:31:61 no 1.47
2 82:04:f9:0b:63:bd no 7.46
1 90:1b:0e:34:f0:f2 no 0.36
1 b0:b2:8f:73:fb:b8 no 12.71
1 b8:27:eb:6a:b0:9d no 65.18
1 c4:a8:1d:8f:c0:20 no 7.46
1 c4:d6:55:3f:8a:17 no 230.81
1 ce:91:41:e2:e9:00 no 5.64
2 f2:52:21:de:ed:7c yes 0.00
2 f2:52:21:de:ed:7c yes 0.00

 

[Belokan]

BTW, I've restarted the node without eth1 and vmbr0 with eth0 configured as static. I don't have the messages any more (hopefully) but network connection is not better (timeouts, etc, ...)

 

[Belokan]

Up ?