[PVE9.1|OVS|LACP] Issues with interface pinning and openvswitch LACP on fresh install

F

fufufighter

Member
Aug 25, 2022
4
0
6
Hi everyone
I was wondering if openvswitch was broken on the latest version of PVE, especially regarding LACP bonding in balance-tcp mode. Here's my story:

I've had a node for a while which was built about 3 years ago with OVS bridges and has seen numerous upgrades and experiments which led to its demise when I upgraded from 8.4 to 9.1. After some troubleshooting I finally managed to recover it and boot, set up an LACP OVS bond to my L2 switch and everything was running fine.

At that point, knowing that the next upgrade might again break things I'd have to fix, I figured this was a good time to start from a fresh install and rid myself of past mistakes and whatever timebombs I might have left unexploded. I backed /etc with my working network config and all VMs and went ahead.

And that's where trouble began. I set everything back up again, manually, using /etc/network/interfaces as a reference, but the bond never came up. I could set trunking with all the required vlans on each port and interface individually using OVS bridges on individual nics, but they never worked in an OVS bond, despite trying numerous options. A friend of mine experienced the same phenomenon on a fresh install, which I of course blamed on his skill, and had to use Linux bridges in order to get a working LACP trunk.

I had a feeling in my gut that something was different from the last time I installed: network interface pinning. It turns out that reinstalling without pinning and leaving the default interface names works, I was able to set up a LACP bond, so I guess openvswitch does not yet work well with the generated names.

For the exact fix: I reinstalled without enabling the interface name pinning at install and it worked. And I guess it's not in the documentation yet. I might be wrong, I just went on this hunch and it worked out, so feel free to share your experiences.

I don't know if the hardware is relevant here but just in case, the server is a minisforum HM90.
 
+1 I'm experiencing the same issue. When provisioning new servers using LACP and OVS with network interface pinning (systemd link files), our bonds fail to come up properly.
 
Would you mind posting the exact configuration contained in /etc/network/interfaces ? That would help pinpointing / reproducing the issue and providing a fix.
 
Hi shanreich,

I don't have the exact non-working config anymore but I have an intermediate one while I was troubleshooting, let me know if that works for you. Please note that at this point the two NICs weren't both yet in the bond for troubleshooting purposes, but before that they were.

Code: 
auto lo
iface lo inet loopback

auto nic1
iface nic1 inet manual
#2,5 Gbps

iface wlp3s0 inet manual

auto nic0
iface nic0 inet manual

auto vlan999
iface vlan999 inet manual
    ovs_type OVSIntPort
    ovs_bridge ovsbr1
    ovs_options tag=999
#pfsync

auto vlan162
iface vlan162 inet manual
    ovs_type OVSIntPort
    ovs_bridge ovsbr5
    ovs_options tag=162
#lxc

auto vlan161
iface vlan161 inet static
    address [IPv4 address]
    ovs_type OVSIntPort
    ovs_bridge ovsbr2
    ovs_options tag=161
#lab

auto vlan10
iface vlan10 inet manual
    ovs_type OVSIntPort
    ovs_bridge ovsbr3
    ovs_options tag=10
#ceph

auto vlan11
iface vlan11 inet manual
    ovs_type OVSIntPort
    ovs_bridge ovsbr4
    ovs_options tag=11
#ceph

auto vlan100
iface vlan100 inet manual
    ovs_type OVSIntPort
    ovs_bridge ovsbr0
    ovs_options tag=100
#lan

auto vlan160
iface vlan160 inet static
    address [IPv4 address]
    gateway [GW address]
    ovs_type OVSIntPort
    ovs_bridge ovsbr0
    ovs_options tag=160
#infra

auto vlan9
iface vlan9 inet manual
    ovs_type OVSIntPort
    ovs_bridge ovsbr0
    ovs_options tag=9

auto vlan101
iface vlan101 inet manual
    ovs_type OVSIntPort
    ovs_bridge ovsbr0
    ovs_options tag=101
#iot

auto bond0
iface bond0 inet manual
    ovs_bonds nic0
    ovs_type OVSBond
    ovs_bridge ovsbr0
    ovs_options bond_mode=balance-tcp lacp=active other_config:lacp-time=fast

auto vmbr1
iface vmbr1 inet static
    address [IPv4 address]
    bridge-ports nic1
    bridge-stp off
    bridge-fd 0

auto ovsbr0
iface ovsbr0 inet manual
    ovs_type OVSBridge
    ovs_ports bond0 vlan100 vlan160 vlan9 vlan101
#wan

auto ovsbr1
iface ovsbr1 inet manual
    ovs_type OVSBridge
    ovs_ports vlan999
#pfsync

auto ovsbr2
iface ovsbr2 inet manual
    ovs_type OVSBridge
    ovs_ports vlan161
#lab

auto ovsbr3
iface ovsbr3 inet static
    address [IPv4 address]
    ovs_type OVSBridge
    ovs_ports vlan10
#ceph

auto ovsbr4
iface ovsbr4 inet static
    address [IPv4 address]
    ovs_type OVSBridge
    ovs_ports vlan11
#ceph

auto ovsbr5
iface ovsbr5 inet static
    address [IPv4 address]
    ovs_type OVSBridge
    ovs_ports vlan162
#lxc

source /etc/network/interfaces.d/*

My current config is this one:

Code: 
auto lo
iface lo inet loopback

iface wlp3s0 inet manual

auto enp4s0
iface enp4s0 inet manual
#2,5 Gbps

auto eno1
iface eno1 inet manual
#1 Gbps

auto vlan10
iface vlan10 inet static
    address [IPv4 address]
    ovs_type OVSIntPort
    ovs_bridge ovsbr0
    ovs_options tag=10
#ceph1

auto vlan11
iface vlan11 inet static
    address [IPv4 address]
    ovs_type OVSIntPort
    ovs_bridge ovsbr0
    ovs_options tag=11
#ceph2

auto vlan101
iface vlan101 inet manual
    ovs_type OVSIntPort
    ovs_bridge ovsbr0
    ovs_options tag=101
#iot

auto vlan160
iface vlan160 inet static
    address [IPv4 address]
    gateway [IPv4 gw]
    ovs_type OVSIntPort
    ovs_bridge ovsbr0
    ovs_options tag=160
#infra

auto vlan162
iface vlan162 inet manual
    ovs_type OVSIntPort
    ovs_bridge ovsbr1
    ovs_options tag=162
#lxc

auto vlan161
iface vlan161 inet static
    address [IPv4 address]
    ovs_type OVSIntPort
    ovs_bridge ovsbr1
    ovs_options tag=161
#lab

auto vlan9
iface vlan9 inet static
    address [IPv4 address]
    ovs_type OVSIntPort
    ovs_bridge ovsbr0
    ovs_options tag=9
#wan

auto vlan999
iface vlan999 inet manual
    ovs_type OVSIntPort
    ovs_bridge ovsbr0
    ovs_options tag=999
#pfsync

auto vlan100
iface vlan100 inet manual
    ovs_type OVSIntPort
    ovs_bridge ovsbr0
    ovs_options tag=100
#lan

auto bond0
iface bond0 inet manual
    ovs_bonds enp4s0 eno1
    ovs_type OVSBond
    ovs_bridge ovsbr0
    ovs_options other_config:lacp-time=fast lacp=active bond_mode=balance-tcp

auto ovsbr0
iface ovsbr0 inet manual
    ovs_type OVSBridge
    ovs_ports bond0 vlan10 vlan11 vlan101 vlan160 vlan9 vlan999 vlan100

auto ovsbr1
iface ovsbr1 inet manual
    ovs_type OVSBridge
    ovs_ports vlan162 vlan161

source /etc/network/interfaces.d/*

Thanks for looking into it!
 
Last edited:
You must log in or register to reply here.
Top Bottom