Proxmox newbie - networking issue

[Harald Niehus]

Hi all,

I'm pretty new to ProxMoxVE and have difficulties with getting the networking part. The ProxMox host is connected to our physical network twice:
- one 1GbE network adapter connected to our management switch
- two 10GbE network adapter connected to our main switch, LACP bonded
Both switches are connected to the same physical network, sharing the same IP address range. This all worked well with VMWare, but I can't seem to get it up and running with ProxMox and OVS.
I have assigned the ProxMox a fixed IP Address on the management port and am able to reach it, with the WebConsole as well as through SSH.
I then created an OVS Bridge and an OVS Bond and attached a Windows Server VM. I assigned the Windows VM a static IP Address from the same physical network. However, I can't get a proper network connection. Neither can I ping the gateway from the VM, nor anything else in the physical network.

I've attached my network config file and the output from ip addr down below.

Can anyone tell me what's going wrong?

Thanks,
Harald

My \etc\network\interfaces file:
------------------------------
# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage part of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

allow-vmbr0 bond0
iface bond0 inet manual
ovs_bonds eno49 eno50
ovs_type OVSBond
ovs_bridge vmbr0
ovs_options bond_mode=balance-tcp lacp=active

auto lo
iface lo inet loopback

auto eno1
iface eno1 inet static
address 192.168.100.43
netmask 255.255.255.0
gateway 192.168.100.1
network 192.168.100.0

allow-ovs vmbr0

iface eno2 inet manual

iface eno3 inet manual

iface eno4 inet manual

iface eno49 inet manual

iface eno50 inet manual

auto vmbr0
iface vmbr0 inet manual
ovs_type OVSBridge
ovs_ports bond0

------------------------------




My output from ip addr:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 14:02:ec:43:64:98 brd ff:ff:ff:ff:ff:ff
inet 192.168.100.43/24 brd 192.168.100.255 scope global eno1
valid_lft forever preferred_lft forever
inet6 fe80::1602:ecff:fe43:6498/64 scope link
valid_lft forever preferred_lft forever
3: eno2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 14:02:ec:43:64:99 brd ff:ff:ff:ff:ff:ff
4: eno3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 14:02:ec:43:64:9a brd ff:ff:ff:ff:ff:ff
5: eno4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 14:02:ec:43:64:9b brd ff:ff:ff:ff:ff:ff
6: eno49: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master ovs-system state UP group default qlen 1000
link/ether 5c:b9:01:9a:bd:d8 brd ff:ff:ff:ff:ff:ff
inet6 fe80::5eb9:1ff:fe9a:bdd8/64 scope link
valid_lft forever preferred_lft forever
7: eno50: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master ovs-system state UP group default qlen 1000
link/ether 5c:b9:01:9a:bd:d9 brd ff:ff:ff:ff:ff:ff
inet6 fe80::5eb9:1ff:fe9a:bdd9/64 scope link
valid_lft forever preferred_lft forever
8: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether ea:a8:80:90:39:79 brd ff:ff:ff:ff:ff:ff
9: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
link/ether 5c:b9:01:9a:bd:d8 brd ff:ff:ff:ff:ff:ff
inet6 fe80::5eb9:1ff:fe9a:bdd8/64 scope link
valid_lft forever preferred_lft forever
10: bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
link/ether 3a:82:ba:33:8d:39 brd ff:ff:ff:ff:ff:ff
inet6 fe80::3882:baff:fe33:8d39/64 scope link
valid_lft forever preferred_lft forever
11: tap100i0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master ovs-system state UNKNOWN group default qlen 1000
link/ether fe:9e:dd:04:2c:c4 brd ff:ff:ff:ff:ff:ff

 

[Richard]

Possibly the bond is not active. Check it by

ovs-appctl bond/show bond0

Follow also the packetflow by

tcpdump -e -n -i eno49
tcpdump -e -n -i eno50
tcpdump -e -n -i tap100i0

 

[Harald Niehus]

Hi Richard,

thanks for your reply it seems bond0 is inactive, but not sure why this is the case:

:~# ovs-appctl bond/show bond0
no such bond
ovs-appctl: ovs-vswitchd: server returned an error

Any idea how to get it activated?

Also, here is output from syslog:
root@ssvdmzpve-01:~# grep bond0 /var/log/syslog
Nov 26 10:11:58 ssvdmzpve-01 ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl --timeout=5 -- --fake-iface add-bond vmbr0 bond0 eno49 eno50 lacp=active bond_mode=balance-tcp lacp-time=fast --
Nov 26 10:11:58 ssvdmzpve-01 ifup[1697]: bond0: ERROR while getting interface flags: No such device
Nov 26 10:11:59 ssvdmzpve-01 ifup[1697]: ifup: interface bond0 already configured
Nov 26 10:45:50 ssvdmzpve-01 ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl --timeout=5 -- --fake-iface add-bond vmbr0 bond0 eno49 eno50 lacp=active bond_mode=balance-tcp lacp-time=fast --
Nov 26 10:45:50 ssvdmzpve-01 ifup[1707]: bond0: ERROR while getting interface flags: No such device
Nov 26 10:45:51 ssvdmzpve-01 ifup[1707]: ifup: interface bond0 already configured

 

[Richard]

If you not are yet become familiar with ovs control commands by running

ovs-vsctl --help

or

man ovs-vsctl

Check if the bond is known in ovs by running

ovs-vsctl show

If not (probable) try to add it by using explicitly ovs-vsctl commands, e.g.:

ovs-vsctl add-bond vmbr0 bond0 eno49 eno50

 

[Harald Niehus]

This is strange:
Regardless of whatever bond_mode I set in the interfaces file (balance-tcp, balance-slb, active-backup), I can't get the bond running, and the slaves remain disabled. I assume that the lacp link is configured correctly on the physical switch, because it worked previously under ESXi.

The two NICs are definitely active and listening on the network (at least I receive some output for either of them if I do tcpdump)

Here is my current output of ovs-appctl bond/show bond0

---- bond0 ----
bond_mode: active-backup
bond may use recirculation: no, Recirc-ID : -1
bond-hash-basis: 0
updelay: 0 ms
downdelay: 0 ms
lacp_status: configured
active slave mac: 00:00:00:00:00:00(none)

slave eno49: disabled
may_enable: false

slave eno50: disabled
may_enable: false

 

[Richard]

Here is my current output of ovs-appctl bond/show bond0

---- bond0 ----
bond_mode: active-backup
bond may use recirculation: no, Recirc-ID : -1
bond-hash-basis: 0
updelay: 0 ms
downdelay: 0 ms
lacp_status: configured
active slave mac: 00:00:00:00:00:00(none)

slave eno49: disabled
may_enable: false

slave eno50: disabled
may_enable: false

Looks like the counterpart (switchport) does not answer.

If you run e.g.

tcpdump -e -n -i eno49

you should get messages like

08:53:27.939832 9e:87:48:cb:50:53 > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110

For lacp protocol investigation see e.g. https://wiki.wireshark.org/LinkAggregationControlProtocol

 

[Harald Niehus]

Hi Richard,

thanks for your inputs. After all my hard trials to get OVS working to no avail, I finally switched to native Linux bond/bridge and guess what? Everything just works fine!!

Not sure if there's kinda bug in OVS, or any kind of undocumented option that I didn't set, or a driver or kernel module missing ... I don't know.

Regards,
Harald