Proxmox 6.1-7 and OpenvSwitch not starting at boot

[dlopez]

Hi everyone, I've a newly installed Proxmox 6.1-7 and I'm trying to get OpenvSwitch to work without much success.

Everythin works as expected with the following configuration but after every reboot I have to open a console in the server and run "ifup vmbr0" because the interfaces wont start on boot.

# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

allow-vmbr0 bond0
iface bond0 inet manual
ovs_bonds eno1 eno2 enp2s0f0 enp2s0f1
ovs_type OVSBond
ovs_bridge vmbr0
ovs_options lacp=active bond_mode=balance-tcp

auto lo
iface lo inet loopback

allow-bond0 eno1

iface eno1 inet manual

allow-bond0 enp2s0f0

iface enp2s0f0 inet manual

allow-bond0 enp2s0f1

iface enp2s0f1 inet manual

allow-bond0 eno2

iface eno2 inet manual

allow-vmbr0 vlan1
iface vlan1 inet static
address 192.168.0.2
netmask 24
gateway 192.168.0.1
ovs_type OVSIntPort
ovs_bridge vmbr0

allow-vmbr0 vlan2
iface vlan2 inet static
address 192.168.2.2
netmask 24
ovs_type OVSIntPort
ovs_bridge vmbr0
ovs_options tag=2

allow-ovs vmbr0
iface vmbr0 inet manual
ovs_type OVSBridge
ovs_ports bond0 vlan1 2

Also the packages versions in case is needed:

root@pve:~# pveversion --v
proxmox-ve: 6.1-2 (running kernel: 5.3.18-1-pve)
pve-manager: 6.1-7 (running version: 6.1-7/13e58d5e)
pve-kernel-5.3: 6.1-4
pve-kernel-helper: 6.1-4
pve-kernel-5.3.18-1-pve: 5.3.18-1
pve-kernel-5.3.10-1-pve: 5.3.10-1
ceph: 14.2.6-pve1
ceph-fuse: 14.2.6-pve1
corosync: 3.0.3-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: residual config
ifupdown2: 2.0.1-1+pve4
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.14-pve1
libpve-access-control: 6.0-6
libpve-apiclient-perl: 3.0-3
libpve-common-perl: 6.0-12
libpve-guest-common-perl: 3.0-3
libpve-http-server-perl: 3.0-4
libpve-storage-perl: 6.1-4
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 3.2.1-1
lxcfs: 3.0.3-pve60
novnc-pve: 1.1.0-1
openvswitch-switch: 2.10.0+2018.08.28+git.8ca7c82b7d+ds1-12+deb10u1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.1-3
pve-cluster: 6.1-4
pve-container: 3.0-19
pve-docs: 6.1-4
pve-edk2-firmware: 2.20191127-1
pve-firewall: 4.0-10
pve-firmware: 3.0-5
pve-ha-manager: 3.0-8
pve-i18n: 2.0-4
pve-qemu-kvm: 4.1.1-2
pve-xtermjs: 4.3.0-1
qemu-server: 6.1-5
smartmontools: 7.1-pve2
spiceterm: 3.1-1
vncterm: 1.6-1
zfsutils-linux: 0.8.3-pve1

I've setup the proxmox no subscription repo and also installed net-tools and ifupdown2 package.

Is there anything I'm missing?

Thanks in advance.

 

[spirit]

Could be ifupdown2. I have see some race, and send a fix recently. I m currently working on a native ovs module for ifupdown2, it should fix all problem and add reload. Could you already send your etc network interface ?

 

[spirit]

(if you use only openvswitch, for now, you don't need ifupdown2, it's only able to reload classic linux bridge/interfaces currently.

 

[dlopez]

Hi spirit, the interface file is in the first post. Also I've tried removing ifupdown2, just in case it was causing any issues, with no success.

 

[spirit]

Hi spirit, the interface file is in the first post. Also I've tried removing ifupdown2, just in case it was causing any issues, with no success.

allow-ovs vmbr0
iface vmbr0 inet manual
ovs_type OVSBridge
ovs_ports bond0 vlan1 2

ovs_ports is wrong

allow-ovs vmbr0
iface vmbr0 inet manual
ovs_type OVSBridge
ovs_ports bond0 vlan1 vlan2

 

[spirit]

also remove the "allow-bond0 X". (it shouldnt be necessary)

 

[spirit]

after fixing the configuration,
can you send the result of "ovs-vsctl show" && "ip addr" ?

 

[dlopez]

ovs_ports is wrong

You're that was a typo from my part. The interfaces file was as you said, meaning with vlan2 instead of just 2.

also remove the "allow-bond0 X". (it shouldnt be necessary)

That was automatically added from the WebUI, but I'll try removing it and let you know the result ASAP.

 

[dlopez]

Hi, I'm still having the same problem. Here's the output for the "ovs-vsctl show" && "ip addr" :

8e65eaaf-e7aa-4e52-ac08-a46db0a32408
Bridge "vmbr0"
Port "vlan1"
Interface "vlan1"
type: internal
Port "vlan2"
tag: 2
Interface "vlan2"
type: internal
Port "vmbr0"
Interface "vmbr0"
type: internal
Port "bond0"
Interface "enp2s0f0"
Interface "eno2"
Interface "enp2s0f1"
Interface "eno1"
ovs_version: "2.10.1"

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp2s0f0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master ovs-system state DOWN group default qlen 1000
link/ether b0:26:28:f1:a6:84 brd ff:ff:ff:ff:ff:ff
3: enp2s0f1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master ovs-system state DOWN group default qlen 1000
link/ether b0:26:28:f1:a6:85 brd ff:ff:ff:ff:ff:ff
4: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master ovs-system state UP group default qlen 1000
link/ether 6c:2b:59:ad:3b:36 brd ff:ff:ff:ff:ff:ff
inet6 fe80::6e2b:59ff:fead:3b36/64 scope link
valid_lft forever preferred_lft forever
5: eno2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master ovs-system state DOWN group default qlen 1000
link/ether 6c:2b:59:ad:3b:37 brd ff:ff:ff:ff:ff:ff
6: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether da:50:4c:4d:e5:6e brd ff:ff:ff:ff:ff:ff
7: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
link/ether 6c:2b:59:ad:3b:36 brd ff:ff:ff:ff:ff:ff
inet6 fe80::6e2b:59ff:fead:3b36/64 scope link
valid_lft forever preferred_lft forever
8: vlan2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
link/ether f6:6c:bf:15:5b:76 brd ff:ff:ff:ff:ff:ff
inet6 fe80::f46c:bfff:fe15:5b76/64 scope link
valid_lft forever preferred_lft forever
9: vlan1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
link/ether 5e:98:39:48:da:0a brd ff:ff:ff:ff:ff:ff
inet6 fe80::5c98:39ff:fe48:da0a/64 scope link
valid_lft forever preferred_lft forever
10: bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
link/ether 7e:3a:01:ae:62:7e brd ff:ff:ff:ff:ff:ff
inet6 fe80::7c3a:1ff:feae:627e/64 scope link
valid_lft forever preferred_lft forever

 

[spirit]

are you still on ifupdown2 ?

because I have found a bug with setting ip address on ovs, and it's look like your problem)
for ifupdown2, I'm currently working on a native ovs plugin. but I recommand to switch back to ifupdown1 for now.

 

[dlopez]

are you still on ifupdown2 ?

because I have found a bug with setting ip address on ovs, and it's look like your problem)
for ifupdown2, I'm currently working on a native ovs plugin. but I recommand to switch back to ifupdown1 for now.

Hi spirit, it seems you where right. Switch back to ifupdown, instead of ifupdown2, has fixed the issue.

I'm going to test this in two more servers that I'm installing and I'll let you know.

Thanks.

 

[spirit]

Hi spirit, it seems you where right. Switch back to ifupdown, instead of ifupdown2, has fixed the issue.

I'm going to test this in two more servers that I'm installing and I'll let you know.

Thanks.

Ok. great. (the bug is that ifupdown2 is running too fast for setup ips after ovs interface creations. I have fixed this with a native ovs plugin for ifupdown2 (with reloading working for ovs!). It should be available soon.

 

[dlopez]

Ok. great. (the bug is that ifupdown2 is running too fast for setup ips after ovs interface creations. I have fixed this with a native ovs plugin for ifupdown2 (with reloading working for ovs!). It should be available soon.

Hi spirit, I tested the solution in two more servers and it works as expected.

Please let me know when the native plugin is available, so I can test it out. Also, I don't know if this is something it could be done in that plugin, but it'd be nice if the MTU of the OVS Bond interfaces could be set from the WEBUI.

 

[spirit]

Please let me know when the native plugin is available, so I can test it out.

Hi, I have make a build with the plugin for testing
http://odisoweb1.odiso.net/ifupdown2_2.0.1-1+pve4_all.deb

(you need to edit your /etc/network/interfaces, and changes allow-ovs, allow-vmbr... , with "auto ..." like other interfaces. This will be changed automatically with the official release, because it need another patch in another package)

Also, I don't know if this is something it could be done in that plugin, but it'd be nice if the MTU of the OVS Bond interfaces could be set from the WEBUI.

it's already available in gui in last version of pve-manager ! (it's setting ovs_mtu)
https://git.proxmox.com/?p=proxmox-...it;h=aedaf2b6f1a480fae5bb30a97b3ba5988bc8f735

not sure if it's already in no subscription repo