Openvswitch init problems at boot

L

lowk3y

New Member
Jun 29, 2014
5
0
1
I have some strange issues regarding openvswitch in PVE 3.2. I did clean install of Debian stable, add PVE 3.2 repositories and install PVE 3.2 using instructions on wiki - http://pve.proxmox.com/wiki/Install_Proxmox_VE_on_Debian_Wheezy

at the end I've installed openvswitch (apt-get install openvswitch-switch openvswitch-common) and when system is restarted, openvswitch seems to be running if I print process list ..

Code: 
vm02:/# ps axf|grep openv    
1766 ?        S<     0:00  \_ ovsdb-server /etc/openvswitch/conf.db -vconsole:emer -vsyslog:err -vfile:info --remote=punix:/var/run/openvswitch/db.sock --private-key=db:Open_vSwitch,SSL,private_key --certificate=db:Open_vSwitch,SSL,certificate --bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert --no-chdir --log-file=/var/log/openvswitch/ovsdb-server.log --pidfile=/var/run/openvswitch/ovsdb-server.pid --detach --monitor
1778 ? S<L   0:00  \_ ovs-vswitchd unix:/var/run/openvswitch/db.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --no-chdir --log-file=/var/log/openvswitch/ovs-vswitchd.log --pidfile=/var/run/openvswitch/ovs-vswitchd.pid --detach --monitor

but I cannot bring up any OVS interfaces or control openvswitch process - it seems as unix:/var/run/openvswitch/db.sock is not there or /run filesystem remounted as tmpfs after openvswitch process is started, because there isn't any /var/run/openvswitch*.

Code: 
vm02:/# ovs-vsctl show 
2014-06-29T21:58:37Z|00001|reconnect|WARN|unix:/var/run/openvswitch/db.sock: connection attempt failed (No such file or directory) 
ovs-vsctl: unix:/var/run/openvswitch/db.sock: database connection failed (No such file or directory)

there isn't anything useful in logs.

Are there any known problems, because running openvswitch (packages from debian repositories or recompiled OVS 2.1) on other (non-PVE) debian boxes works without any problems.
 
dietmar said:
Please use OVS packages from our repositories.
Click to expand...

I am ..

Code: 
vm02:~# dpkg -l|grep openvswitch
ii  openvswitch-common               2.0.90-3                      amd64        Open vSwitch common components
ii  openvswitch-switch               2.0.90-3                      amd64        Open vSwitch switch implementations

#apt-cache show openvswitch-switch|less
Package: openvswitch-switch
Priority: extra
Section: net
Installed-Size: 4769
Maintainer: Open vSwitch developers <dev@openvswitch.org>
Architecture: amd64
Source: openvswitch
Version: 2.0.90-3
Depends: libc6 (>= 2.12), libssl1.0.0 (>= 1.0.0), python, openvswitch-common (= 2.0.90-3), module-init-tools, procps, uuid-runtime, netbase, python-argparse
Suggests: openvswitch-datapath-module
Filename: dists/wheezy/pve-no-subscription/binary-amd64/openvswitch-switch_2.0.90-3_amd64.deb
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.....
 
dietmar said:
That should work,at least it work when using the proxmox default installer.
Click to expand...

Strange; if you install PVE on Debian stable and openvswitch from PVE repositories this doesn't work. If you do clean install with Proxmox PVE install ISO it works.


But in second case, now that OVS is correctly initialized on boot I have some other issues regarding /etc/network/interfaces - the "up" command are not executed on boot. For instance

Code: 
...
auto vmbr0
iface vmbr0 inet manual
   ovs_type  OVSBridge
   ovs_ports eth0 wan0
   ovs_extra set Bridge vmbr0 stp_enable=true
   mtu 9000
   up sysctl -w net.ipv6.conf.$IFACE.disable_ipv6=1

...

allow-vmbr0 wan0
iface wan0 inet static
    address 19x.xx.xx.xx
    netmask 255.255.255.224
    gateway 19x.xx.xx.xy
    ovs_bridge vmbr0
    ovs_type OVSIntPort
    ovs_options vlan_mode=access
    ovs_options tag=10
    up sysctl -w net.ipv6.conf.$IFACE.disable_ipv6=1

once system is up and running and I do
Code: 
ifdown vmbr0 && ifup vmbr0
and those 'up' commands are executed.
 
Try this:
post-up sysctl -w net.ipv6.conf.$IFACE.disable_ipv6=1

Maybe the interface is not available when the up is executed.
 
There is a bug in the openvswitch-switch package causing the underlying error state that the instate file (/run/network/ifstate) does not exist before the startup script is run. This file is created when /etc/init.d/networking is run, however, openvswitch-switch starts before networking (required), so therefore it also needs to check and create the directory and state file the same as /etc/init.d/networking.

The following patch is required to have OVS start correctly under PVE:

Code: 
# /etc/default/openvswitch-switch -- add the following code snipped to correctly bring up OVS interfaces at boot

# --
# HACK < START
# --
# Hack to allow OVS bridges to be brought up correctly due to ifup dependencies
#--


RUN_DIR="/run/network"
IFSTATE="$RUN_DIR/ifstate"


check_ifstate() {
    if [ ! -d "$RUN_DIR" ] ; then
        if ! mkdir -p "$RUN_DIR" ; then
            log_failure_msg "can't create $RUN_DIR"
            exit 1
        fi
    fi
    if [ ! -r "$IFSTATE" ] ; then
        if ! :> "$IFSTATE" ; then
            log_failure_msg "can't initialise $IFSTATE"
            exit 1
        fi
    fi
}


check_ifstate


# --
# HACK < END
# --

Here is an example configuration that works (snippet from /etc/network/interfaces):

Code: 
allow-ovs vmbr0
iface vmbr0 inet static
        address   192.168.0.2
        netmask   255.255.255.0
        gateway   192.168.0.1
        ovs_type  OVSBridge
        ovs_ports bond0


allow-vmbr0 bond0
iface bond0 inet manual
        ovs_bridge  vmbr0
        ovs_type    OVSBond
        ovs_bonds   eth0 eth1
        ovs_options bond_mode=balance-slb lacp=off
 
Thanks for pushing that upstream Dietmar - I had to patch this a few months ago and was not aware that a fix was finally in-train.

This is not patched in my running production systems and until it is, the above modification to /etc/default/openvswitch-switch will ensure OVS will correctly start at boot time and will have no detrimental effect when the patch is available for general consumption.
 
(PVE from ISO 3.3 / Kernel 3.10.0-4-pve / OVS 2.3.0-1 )

Hi,
I reactivated this thread as I have a similar issue when rebooting my host...
During the boot process I can see (unix:/var/run/openvswitch/db.sock: database connection failed (No such file or directory) everytime OVS try to do an action, it is like networking process is starting after it. To double check, after booting and doing a /etc/init.d/networking restart everyting is allright and my OVS is running well with all I set on /etc/network/interfaces.

I checked and the OVS 3.3 is patched so I don't know what can still avoid OVS to start properly?

Many thanks,
 
(Kernel 2.6.32-34-pve / OVS 2.3.0-1 )

Same problem. It seems openvswitch configuration was hapenning before ovsdb-server and ovsdb-server were started. Below my boot log:

I tried the /etc/default/openvswitch hack without success.

root@proxmox0:~# cat /var/log/boot
Sat Dec 13 00:15:34 2014: Setting parameters of disc: (none).
Sat Dec 13 00:15:34 2014: Activating swap...done.
Sat Dec 13 00:15:34 2014: Checking root file system...fsck from util-linux 2.20.1
Sat Dec 13 00:15:34 2014: /dev/mapper/pve-root: clean, 46982/4882432 files, 681934/19529728 blocks
Sat Dec 13 00:15:34 2014: done.
Sat Dec 13 00:15:34 2014: Loading kernel module fuse.
Sat Dec 13 00:15:34 2014: Cleaning up temporary files... /tmp.
Sat Dec 13 00:15:34 2014: Setting up LVM Volume Groups...done.
Sat Dec 13 00:15:35 2014: Activating lvm and md swap...done.
Sat Dec 13 00:15:35 2014: Checking file systems...fsck from util-linux 2.20.1
Sat Dec 13 00:15:35 2014: /dev/mapper/pve-data: clean, 32322/13312000 files, 18859984/53239808 blocks
Sat Dec 13 00:15:35 2014: /dev/sda2: clean, 246/130560 files, 75083/522240 blocks
Sat Dec 13 00:15:35 2014: done.
Sat Dec 13 00:15:35 2014: Mounting local filesystems...done.
Sat Dec 13 00:15:36 2014: Activating swapfile swap...done.
Sat Dec 13 00:15:36 2014: Cleaning up temporary files....
Sat Dec 13 00:15:36 2014: Setting kernel variables ...done.
Sat Dec 13 00:15:36 2014: Configuring network interfaces...
Sat Dec 13 00:15:36 2014: Waiting for vmbr0 to get ready (MAXWAIT is 2 seconds).
Sat Dec 13 00:15:37 2014: ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl --timeout=5 -- --may-exist add-br vmbr1 --
Sat Dec 13 00:15:37 2014: ovs-vsctl: ovs|00002|vsctl|ERR|unix:/var/run/openvswitch/db.sock: database connection failed (No such file or directory)
Sat Dec 13 00:15:37 2014: ovs-vsctl: unix:/var/run/openvswitch/db.sock: database connection failed (No such file or directory)
Sat Dec 13 00:15:37 2014: ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl --timeout=5 -- --may-exist add-port vmbr1 eth1 --
Sat Dec 13 00:15:37 2014: ovs-vsctl: ovs|00002|vsctl|ERR|unix:/var/run/openvswitch/db.sock: database connection failed (No such file or directory)
Sat Dec 13 00:15:37 2014: ovs-vsctl: unix:/var/run/openvswitch/db.sock: database connection failed (No such file or directory)
Sat Dec 13 00:15:37 2014: ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl --timeout=5 -- --may-exist add-port vmbr1 wan0 -- set Interface wan0 type=internal -- set interface wan0 external-ids:iface-id=proxmox0-wan0-vif
Sat Dec 13 00:15:37 2014: ovs-vsctl: ovs|00002|vsctl|ERR|unix:/var/run/openvswitch/db.sock: database connection failed (No such file or directory)
Sat Dec 13 00:15:37 2014: ovs-vsctl: unix:/var/run/openvswitch/db.sock: database connection failed (No such file or directory)
Sat Dec 13 00:15:37 2014: wan0: ERROR while getting interface flags: No such device
Sat Dec 13 00:15:37 2014: Cannot find device "wan0"
Sat Dec 13 00:15:37 2014: Failed to bring up wan0.
Sat Dec 13 00:15:37 2014: done.
Sat Dec 13 00:15:37 2014: Starting rpcbind daemon....
Sat Dec 13 00:15:37 2014: Starting NFS common utilities: statd idmapd.
Sat Dec 13 00:15:37 2014: Cleaning up temporary files....
Sat Dec 13 00:15:38 2014: Setting console screen modes and fonts.
Sat Dec 13 00:15:38 2014: ^[]R^[[9;30]^[[14;30]Setting up X socket directories... /tmp/.X11-unix /tmp/.ICE-unix.
Sat Dec 13 00:15:38 2014: Starting iSCSI initiator service: iscsid.
Sat Dec 13 00:15:38 2014: Setting up iSCSI targets:
Sat Dec 13 00:15:38 2014: iscsiadm: No records found
Sat Dec 13 00:15:38 2014: .
Sat Dec 13 00:15:38 2014: Mounting network filesystems:.
Sat Dec 13 00:15:38 2014: INIT: Entering runlevel: 2
Sat Dec 13 00:15:38 2014: Using makefile-style concurrent boot in runlevel 2.
Sat Dec 13 00:15:38 2014: Starting rpcbind daemon...Already running..
Sat Dec 13 00:15:38 2014: Starting NFS common utilities: statd idmapd.
Sat Dec 13 00:15:38 2014: Starting KSM control daemon: ksmtuned.
Sat Dec 13 00:15:38 2014: Checking vzevent kernel module .....done
Sat Dec 13 00:15:38 2014: Starting vzeventd:
Sat Dec 13 00:15:39 2014: Starting enhanced syslogd: rsyslogd.
Sat Dec 13 00:15:39 2014: Starting RRDtool data caching daemon: rrdcached.
Sat Dec 13 00:15:39 2014: Starting deferred execution scheduler: atd.
Sat Dec 13 00:15:39 2014: Inserting openvswitch module.
Sat Dec 13 00:15:39 2014: /etc/openvswitch/conf.db does not exist ... (warning).
Sat Dec 13 00:15:39 2014: Creating empty database /etc/openvswitch/conf.db.
Sat Dec 13 00:15:39 2014: Starting ovsdb-server.
Sat Dec 13 00:15:39 2014: Configuring Open vSwitch system IDs.
Sat Dec 13 00:15:39 2014: Starting ovs-vswitchd.
Sat Dec 13 00:15:39 2014: Enabling remote OVSDB managers.
Sat Dec 13 00:15:39 2014: ifup: interface vmbr1 already configured
Sat Dec 13 00:15:39 2014: Starting NTP server: ntpd.
Sat Dec 13 00:15:40 2014: Starting OpenBSD Secure Shell server: sshd.
Sat Dec 13 00:15:41 2014: Starting Postfix Mail Transport Agent: postfix.
Sat Dec 13 00:15:41 2014: Starting pve cluster filesystem : pve-cluster.
Sat Dec 13 00:15:42 2014: clvmd: cluster not configured.
Sat Dec 13 00:15:42 2014: Starting periodic command scheduler: cron.
Sat Dec 13 00:15:42 2014: Starting PVE firewall logger: pvefw-logger.
Sat Dec 13 00:15:44 2014: Starting OpenVZ: ..done
Sat Dec 13 00:15:44 2014: Bringing up interface venet0: ..done
Sat Dec 13 00:15:44 2014: Starting Proxmox VE firewall: pve-firewall.
Sat Dec 13 00:15:44 2014: Starting PVE Daemon: pvedaemon.
Sat Dec 13 00:15:46 2014: Starting PVE Status Daemon: pvestatd.
Sat Dec 13 00:15:46 2014: Starting PVE API Proxy Server: pveproxy.
Sat Dec 13 00:15:47 2014: Starting PVE SPICE Proxy Server: spiceproxy.
Sat Dec 13 00:15:47 2014: Starting VMs and Containers
Click to expand...
 
You must log in or register to reply here.
Top Bottom