Multiple NIC's but only one working

Kenneth_H

Well-Known Member
Feb 28, 2017
30
1
48
31
Hi
I have recently installed deployed a new Proxmox VE 4.4 on an older SuperMicro server.
The server has 2 onboard Intel Server NIC's and two PCIe cards containing dual Intel Server NIC's.
All NIC's are shown as Intel Corporation 82576 Gigabit Network Connection (lspci | grep -i eth)
Already upon installation I had issues, as I could not get it use the onboard NIC that is connected to our management network. Instead it choose a port on the first addin NIC, but after installation, all 6 NIC's show up in the web interface.
Now the next step is that I want to create OVS-bridge to run my VM's on.
I have created the vmbr1 bridge, assigned it an IP-addres and restarted the server, but the vmbr1 bridge is still marked as active=no.
I do not know if the second cable is connected to the device that proxmox detects as eth1, but I do also not know how to find out which interfaces are seen as having a physical link.

Could someone tell me how to find out which links have a physical link running and also how I can get hte vmbr1 running?
 
Hi
I managed to use ethtool --identify ethX to find the mapping of physical NIC to software ethX and have changed the mapping of vmbr0 to use the onboard NIC.
I have also created a OVS bridge between two ports on the same addin NIC and checked with ethtool ethX that the two interfaces are having "Link Detected: yes" and they do. I also verified with some NIC's that are not yet connected, but have a OVS bridge created and they show "Link Detected: no".
The strange thing is that if I reboot the server, they still show in the web GUI as not active, but if I then run "service networking restart", the OVS bridges come online, including the ones that have no physical link detected. All bridges are configured to auto start and are also represented so in the web GUI. After getting the bridges to show as active I can now boot a VM using a given bridge that has an actual physical link, but the DHCP-server on the physical network cannot be reached and therefore the VM cannot get an IP-address. The same goes for assigning a static IP.
It seems the bridge is stated as active in the Web GUI, but there is no link to the actual network.
In this lab setup we are not running any kind of VLAN, so that we can keep it simple.

One thing that I found in /var/log/daemon.log is this:
Code:
Mar  1 13:40:12 vkd-vmhost1 ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl --timeout=5 -- --may-exist add-br vmbr1 --
Mar  1 13:40:12 vkd-vmhost1 ovs-vsctl: ovs|00002|vsctl|ERR|unix:/var/run/openvswitch/db.sock: database connection failed (No such file or directory)
Mar  1 13:40:12 vkd-vmhost1 networking[1897]: ovs-vsctl: unix:/var/run/openvswitch/db.sock: database connection failed (No such file or directory)
Mar  1 13:40:12 vkd-vmhost1 ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl --timeout=5 -- --may-exist add-port vmbr1 eth0 --
Mar  1 13:40:12 vkd-vmhost1 ovs-vsctl: ovs|00002|vsctl|ERR|unix:/var/run/openvswitch/db.sock: database connection failed (No such file or directory)
Mar  1 13:40:12 vkd-vmhost1 networking[1897]: ovs-vsctl: unix:/var/run/openvswitch/db.sock: database connection failed (No such file or directory)
Mar  1 13:40:12 vkd-vmhost1 ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl --timeout=5 -- --may-exist add-port vmbr1 eth1 --
Mar  1 13:40:12 vkd-vmhost1 ovs-vsctl: ovs|00002|vsctl|ERR|unix:/var/run/openvswitch/db.sock: database connection failed (No such file or directory)
Mar  1 13:40:12 vkd-vmhost1 networking[1897]: ovs-vsctl: unix:/var/run/openvswitch/db.sock: database connection failed (No such file or directory)
Mar  1 13:40:12 vkd-vmhost1 networking[1897]: Cannot find device "vmbr1"
Mar  1 13:40:12 vkd-vmhost1 networking[1897]: Failed to bring up vmbr1.
It seems the OVS service is missing some kind of database file that prevents it from running correctly.

If I create the bridges as Linux bridges and reboot, then it does all seem to work.
 
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!