Installing updates stops ovs

[David Herselman]

We periodically login to system to retrieve and install updates using the following commands:

apt-get update; apt-get -y dist-upgrade; apt-get -y autoremove;

Lost connection to systems running OVS, herewith the last output:

Unpacking pve-manager (4.3-12) over (4.3-9) ...
Preparing to unpack .../proxmox-ve_4.3-72_all.deb ...
Unpacking proxmox-ve (4.3-72) over (4.3-71) ...
Preparing to unpack .../openvswitch-switch_2.6.0-2_amd64.deb ...

Had to connect to the system using the Intel RMM (Remote Management Module) and then perform the following actions to restore connectivity:

/etc/init.d/openvswitch-switch start

We subsequently had to reference VM network configurations (grep net /etc/pve/local/qemu-server/*.conf) and reattach the tun interfaces to OVS:

ovs-vsctl add-port vmbr0 tap100i0 tag=847
ovs-vsctl add-port vmbr0 tap101i0 tag=3352

Surely Proxmox should be vetting these updates before releasing them? OVS should either not be shutdown during the update process (perhaps requiring a manual system reboot) or recover again sufficiently after installing updates as not to disrupt services...

 

[David Herselman]

We prefer to statically name network interfaces so we have a systemd network link file for each physical and VLAN interface, ie:

/etc/systemd/network/10-eth0.link

[Match]
MACAddress=00:1e:67:1c:7c:50
Path=pci*

[Link]
Name=eth0

/etc/systemd/network/9-vlan.link

[Match]
Type=vlan

[Link]
NamePolicy=kernel

Herewith a simplified version of our network interface configuration file:
/etc/network/interfaces

auto lo
iface lo inet loopback

allow-vmbr0 bond0
iface bond0 inet manual
        ovs_bridge vmbr0
        ovs_type OVSBond
        ovs_bonds eth0 eth1
        pre-up ( ifconfig eth0 mtu 9216 && ifconfig eth1 mtu 9216 )
#       ovs_options bond_mode=active-backup tag=1 vlan_mode=native-untagged
        ovs_options bond_mode=balance-tcp lacp=active other_config:lacp-time=fast tag=1 vlan_mode=native-untagged
        mtu 9216

auto vmbr0
allow-ovs vmbr0
iface vmbr0 inet manual
        ovs_type OVSBridge
        ovs_ports bond0 vlan1
        mtu 9216

allow-vmbr0 vlan1
iface vlan1 inet static
        ovs_type OVSIntPort
        ovs_bridge vmbr0
        ovs_options tag=1
        ovs_extra set interface ${IFACE} external-ids:iface-id=$(hostname -s)-${IFACE}-vif
        address 198.19.17.30
        netmask 255.255.255.240
        gateway 198.19.17.17
        mtu 1500

 

[dietmar]

Proxmox should be vetting these updates before releasing them? OVS should either not be shutdown during the update process (perhaps requiring a manual system reboot) or recover again sufficiently after installing updates as not to disrupt services...

We always test carefully. I additionally reviewed the codebase, and I cannot really find an error. On my test setups, I can restart the OVS service without any problem - running VMs and containers keeps working.

 

[David Herselman]

This occurred on two separate systems, both of which are the only ones we're currently trialling OVS...
Apart from VLAN tags the setup is almost identical.
One system is a Dell R710 though and the other an Intel 'white box' (ie unbranded).
I'll try restarting OVS to see if it reconnect virtual machine tap interfaces normally but both systems had ovsswitch-switch in a stopped state after upgrading.

Perhaps you could try with my network config, perhaps systemd interface reservations somehow make a difference?

 

[dragon2611]

Weird on my machines they all came back by themselves after a few seconds.

 

[wolfgang]

Hi,

@David
Can you send me your config to test, because i can't reproduce it here.

 

[jacques]

Hi,

We had the same thing happen on our two node cluster. Both servers are HP ProLiant DL380 G6 with 4x Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet (rev 20). Restarting OVS restored connectivity to the hosts, but the VMs had to be rebooted too, probably due to the ports no longer being attached...

auto lo
iface lo inet loopback

allow-vmbr0 bond0
iface bond0 inet manual
        ovs_bridge vmbr0
        ovs_type OVSBond
        ovs_bonds eth0 eth1
        ovs_options bond_mode=balance-tcp lacp=active other_config:lacp-time=fast

auto vmbr0
allow-ovs vmbr0
iface vmbr0 inet manual
        ovs_type OVSBridge
        ovs_ports bond0 vlan10

allow-vmbr0 vlan10
iface vlan10 inet static
        ovs_type OVSIntPort
        ovs_bridge vmbr0
        ovs_options tag=10
        ovs_extra set interface ${IFACE} external-ids:iface-id=$(hostname -s)-${IFACE}-vif
        address 192.168.0.50
        netmask 255.255.255.0
        gateway 192.168.0.1