Packet Drops on Bonded Interface

[darulehsan]

Hello, I am experiencing slow but consistent packet drops (twice per minute) on my bonded interfaces.

13: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue maste
r vmbr1152 state UP mode DEFAULT group default qlen 1000
    link/ether 00:1b:21:bd:33:bc brd ff:ff:ff:ff:ff:ff
    RX: bytes  packets  errors  dropped overrun mcast
    17093      134      0       22      0       134
    TX: bytes  packets  errors  dropped carrier collsns
    3688       30       0       0       0       0

My /etc/network/interfaces are as followed: -

iface ens6f0 inet manual

auto eno1
iface eno1 inet manual

auto eno2
iface eno2 inet manual

iface eno3 inet manual

iface eno4 inet manual

auto ens6f1
iface ens6f1 inet manual

auto bond0
iface bond0 inet manual
        bond-slaves ens6f0 ens6f1
        bond-miimon 100
        bond-mode 802.3ad

auto vmbr0
iface vmbr0 inet static
        address 10.11.51.10/24
        gateway 10.11.51.1
        bridge-ports eno1
        bridge-stp off
        bridge-fd 0
        
auto vmbr1152
iface vmbr1152 inet static
        address 10.11.152.10/24
        bridge-ports bond0
        bridge-stp off
        bridge-fd 0

The interfaces are currently using 10G DAC cables that are connected to a 10G Cisco Nexus 3000 Series switch. I have tried the same bonding configuration with 1G Ethernet connection and did not encounter any packet drops.

Currently there are I have bonded 2 ports (ens6f0, ens6f1) under the same NIC with LACP configured on both server and switch. The bonding (bond0) is then assigned under vmbr1152. I have tried assigning an IP on bond0 rather than vmbr1152 and the packet loss have stopped. However, I would like to know why are there packet drops when it's using Linux Bridge?

Thank you

 

[spirit]

I have tried assigning an IP on bond0 rather than vmbr1152 "

you can't assign an ip address on a interface if the interface is in a bridge. (kernel will refuse it).

 

[darulehsan]

you can't assign an ip address on a interface if the interface is in a bridge. (kernel will refuse it).

Hi, my apologies, I forgot to mention that when I assigned an IP to the bridge, I have removed the IP on the bond beforehand.