I had no problem using OVS to bond 2 Mellanox interfaces on PVE 5. Now I'm on 6 and I'm having a lot of issues, and I'm not the only one. Other recent threads are here:
https://forum.proxmox.com/threads/proxmox-6-network-wont-start.56362/
https://forum.proxmox.com/threads/pve-6-and-mellanox-4-x-drivers.56553/
I followed the tutorial but it's just not working for me. I have the latest Proxmox version as of today, with everything updated. I have 2 Mellanox interfaces I want to bond, on the same NIC: the Mellanox ConnectX-3 (MCX312A-XCBT, 2x SFP+ ports). Using
These are the two interfaces I want to bond together: enp132s0 + enp132s0d1. This is currently my /etc/network/interfaces (I followed the proxmox OVS manual on the website):
Note that ifconfig is used while it's depreciated. Surely this can't be correct? What should I replace it with?
The above simply doesn't work. When I type
And from the syslog:
How do I get this to work? And why is
https://forum.proxmox.com/threads/proxmox-6-network-wont-start.56362/
https://forum.proxmox.com/threads/pve-6-and-mellanox-4-x-drivers.56553/
I followed the tutorial but it's just not working for me. I have the latest Proxmox version as of today, with everything updated. I have 2 Mellanox interfaces I want to bond, on the same NIC: the Mellanox ConnectX-3 (MCX312A-XCBT, 2x SFP+ ports). Using
ip a I get this:
Code:
4: enp132s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 00:02:c9:3b:61:10 brd ff:ff:ff:ff:ff:ff
5: enp132s0d1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 00:02:c9:3b:61:11 brd ff:ff:ff:ff:ff:ffThese are the two interfaces I want to bond together: enp132s0 + enp132s0d1. This is currently my /etc/network/interfaces (I followed the proxmox OVS manual on the website):
Code:
allow-vmbr2 bond0
iface bond0 inet manual
ovs_bonds enp132s0 enp132s0d1
ovs_type OVSBond
ovs_bridge vmbr2
mtu 9000
ovs_options bond_mode=balance-tcp other_config:lacp-time=fast lacp=active
pre-up ( ifconfig enp132s0 mtu 9000 && ifconfig enp132s0d1 mtu 9000 )
# Force the MTU of the physical interfaces to be jumbo-frame capable.
# This doesn't mean that any OVSIntPorts must be jumbo-capable.
# We cannot, however set up definitions for eth0 and eth1 directly due
# to what appear to be bugs in the initialization process.
auto lo
iface lo inet loopback
iface eno1 inet manual
iface eno2 inet manual
iface enp132s0 inet manual
iface enp132s0d1 inet manual
auto vmbr0
iface vmbr0 inet static
address 192.168.1.23
netmask 255.255.255.0
gateway 192.168.1.1
bridge-ports eno1
bridge-stp off
bridge-fd 0
allow-ovs vmbr2
auto vmbr2
iface vmbr2 inet manual
ovs_type OVSBridge
ovs_ports bond0
mtu 9000
# NOTE: we MUST mention bond0, vlan50, and vlan55 even though each
# of them lists ovs_bridge vmbr0! Not sure why it needs this
# kind of cross-referencing but it won't work without it!Note that ifconfig is used while it's depreciated. Surely this can't be correct? What should I replace it with?
The above simply doesn't work. When I type
dmesg | grep -i enp132s0 I get this output:
Code:
root@proxmox:~# dmesg | grep -i enp132s0
[ 12.432376] mlx4_core 0000:84:00.0 enp132s0: renamed from eth0
[ 12.473761] mlx4_core 0000:84:00.0 enp132s0d1: renamed from eth0
[ 15.459747] mlx4_en: enp132s0: Link Up
[ 15.564489] mlx4_en: enp132s0d1: Link Up
[ 1073.864057] mlx4_en: enp132s0: Link Down
[ 1082.575190] mlx4_en: enp132s0d1: Link Down
[ 1095.603476] mlx4_en: enp132s0: Link Up
[ 1099.024206] mlx4_en: enp132s0d1: Link Up
[ 1113.176462] mlx4_en: enp132s0: Link Down
[ 1116.346555] mlx4_en: enp132s0d1: Link Down
[ 1121.184069] mlx4_en: enp132s0: Link Up
[ 1122.601733] mlx4_en: enp132s0d1: Link UpAnd from the syslog:
Code:
Sep 14 14:19:48 proxmox systemd[1]: Starting Open vSwitch...
Sep 14 14:19:48 proxmox zed[2215]: eid=2 class=config_sync pool_guid=0x713F0C2B1A688975
Sep 14 14:19:48 proxmox openvswitch-switch[2209]: ovsdb-server is already running.
Sep 14 14:19:48 proxmox openvswitch-switch[2209]: ovs-vswitchd is already running.
Sep 14 14:19:48 proxmox ovs-vsctl[2261]: ovs|00001|vsctl|INFO|Called as ovs-vsctl --no-wait set Open_vSwitch . external-ids:hostname=proxmox.vice
Sep 14 14:19:48 proxmox openvswitch-switch[2209]: Enabling remote OVSDB managers.
Sep 14 14:19:48 proxmox zed[2272]: eid=3 class=pool_import pool_guid=0x713F0C2B1A688975
Sep 14 14:19:48 proxmox ovs-vsctl[2277]: ovs|00001|vsctl|INFO|Called as ovs-vsctl --timeout=5 -- --may-exist add-br vmbr2 --
Sep 14 14:19:48 proxmox systemd-udevd[920]: Using default interface naming scheme 'v240'.
Sep 14 14:19:48 proxmox systemd-udevd[920]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Sep 14 14:19:48 proxmox systemd-udevd[920]: Could not generate persistent MAC address for ovs-system: No such file or directory
Sep 14 14:19:48 proxmox kernel: device ovs-system entered promiscuous mode
Sep 14 14:19:48 proxmox kernel: netlink: 'ovs-vswitchd': attribute type 5 has an invalid length.
Sep 14 14:19:48 proxmox zed[2320]: eid=4 class=history_event pool_guid=0x713F0C2B1A688975
Sep 14 14:19:48 proxmox zed[2343]: eid=5 class=config_sync pool_guid=0x713F0C2B1A688975
Sep 14 14:19:48 proxmox systemd-udevd[920]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Sep 14 14:19:48 proxmox systemd-udevd[920]: Could not generate persistent MAC address for vmbr2: No such file or directory
Sep 14 14:19:48 proxmox kernel: netlink: 'ovs-vswitchd': attribute type 5 has an invalid length.
Sep 14 14:19:48 proxmox kernel: device vmbr2 entered promiscuous mode
Sep 14 14:19:48 proxmox openvswitch-switch[2209]: /bin/sh: 1: ifconfig: not found
Sep 14 14:19:48 proxmox openvswitch-switch[2209]: ifup: failed to bring up bond0
Sep 14 14:19:48 proxmox systemd[1]: Started Open vSwitch.
Sep 14 14:19:48 proxmox systemd[1]: Starting Raise network interfaces...
Sep 14 14:19:48 proxmox systemd-udevd[920]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Sep 14 14:19:48 proxmox systemd-udevd[920]: Could not generate persistent MAC address for vmbr0: No such file or directoryHow do I get this to work? And why is
ifconfig still used when it's depreciated, shouldn't the tutorial be updated?
