[SOLVED] No network on server unless i ifdown / ifup vmbr0

[CLE]

Hello all,

happy new year everyone

I configured Proxmox VE 6.1-5 on a Debian Buster install (freshly set up for that purpose) according to the entry in the wiki.

The server is running in my homelab/-network, IP range is 192.168.222.0/24, gateway is at .254, DNS & DHCP is at .253

The server has the IP 192.168.222.4, configured on the vmbr0 OVS Bridge device which is bound to eno1 (the server has 4 NICs in total).



/etc/network/interfaces

# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

source /etc/network/interfaces.d/*

auto lo
iface lo inet loopback

allow-hotplug eno1

iface eno2 inet manual

iface eno3 inet manual

iface enp129s0f3 inet manual

allow-vmbr0 eno1
iface eno1 inet manual
        ovs_type OVSPort
        ovs_bridge vmbr0

allow-ovs vmbr0
iface vmbr0 inet static
        address  192.168.222.4
        netmask  24
        gateway  192.168.222.254
        ovs_type OVSBridge
        ovs_ports eno1

As far as i can tell, nothing out of the ordinary - but my problem is that the network interface on vmbr0 doesn't seem to properly come up after the server has booted.

I have no network unless i do a ifdown vmbr0, followed by ifup vmbr0 - i actually have no idea why it will not come up by itself or what the reason may be it doesn't behave as expected.... After this little "raindance", everything works normally - i have network in a Debian VM as well as a LXC container i have tested so far...

I'd be very grateful if anyone of you can provide clues what may be the issue - if i omitted something important, please don't hesitate to ask!

Thanks in advance!

 

[jrshaw]

I was going to say, try do an IP scan if you have a GUI VM on all your IPs, I had something similar but with my VMs, in the end I found out MAC addresses were being spoofed and doing ifdown/ifup brought it back to life. Not sure if this will be the same, as its your hostbox and not sure how you'll do a scan on it. You won't be able to see MAC spoofing through Proxmox or the VMs etc. thats why I used advanced IP scanner which is were I noticed it.

 

[jrshaw]

Also, when setting them to active they should stay up permenently, is there not an option on the web interface to do this? Should be at the top Were you see create, edit etc.

 

[CLE]

I‘m not sure i got you right, but the container and the VM both start on server boot automatically, so, in theory, they should even be online if the PVE webinterface is not - but they aren‘t

 

[jrshaw]

Lets try explain it simply. In the Proxmox entry on the GUI were you build the virtual bridges, they should be set to active and start up automatically without intevention from a human.

I have added a spare virtual bridge on mine to explain a bit better, when you click on the inactive bridges, the greyed out "Apply" configuration button should then light up which means once you click it, it will automatically set the bridges to ACTIVE and start them from boot.

If an error pops up saying ifupdown 2 not installed, then run.

apt install ifupdown2

Hopefully this will resolve your issue.

 

[CLE]

The bridge vmbr0 is, as can be seen in my screenshot, set to autostart - also if i got that right "ifupdown2" isn't supposed to be used with Open VSwitch (which my bridge is)....

 

[spirit]

can you send the result after boot , of:

#systemctl status networking
#systemctl status openvswitch

?

 

[CLE]

"systemctl status openvswitch" comes back with "Unit openvswitch.service could not be found" - Are you thinking of an other service (tab completion gives openvswitch-nonetwork.service as well as openvswitch-switch.service)?

 

[CLE]

Here's a screenshot of the output of "systemctl status openvswitch-switch.service" as well as "systemctl status openvswitch-nonetwork.service":

Edit: in the meanwhile, i've set vmbr0 to dhcp - however, the main problem persists, somehow it seems the bridge is up before eno1 is really ready (at least that is what i suspect, as i can get a DHCP offer if i again do the "raindance"...)

 

[CLE]

Ok, i've set vmbr0 again to the static configuration as per my original post (sorry, not meaning to shift goalposts, just getting impatient )

Output of "systemctl status openvswitch-switch":

● openvswitch-switch.service - Open vSwitch
Loaded: loaded (/lib/systemd/system/openvswitch-switch.service; enabled; vendor preset: enabled)
Active: active (exited) since Sun 2020-01-05 14:14:10 CET; 3min 35s ago
Process: 1042 ExecStart=/etc/init.d/openvswitch-switch start (code=exited, status=0/SUCCESS)
Main PID: 1042 (code=exited, status=0/SUCCESS)

Jan 05 14:14:10 server systemd[1]: Starting Open vSwitch...
Jan 05 14:14:10 server openvswitch-switch[1042]: ovsdb-server is already running.
Jan 05 14:14:10 server openvswitch-switch[1042]: ovs-vswitchd is already running.
Jan 05 14:14:10 server ovs-vsctl[1104]: ovs|00001|vsctl|INFO|Called as ovs-vsctl --no-wait set Open
Jan 05 14:14:10 server openvswitch-switch[1042]: Enabling remote OVSDB managers.
Jan 05 14:14:10 server ovs-vsctl[1127]: ovs|00001|vsctl|INFO|Called as ovs-vsctl --timeout=5 -- --m
Jan 05 14:14:10 server openvswitch-switch[1042]: ifup: waiting for lock on /run/network/ifstate.eno
Jan 05 14:14:10 server openvswitch-switch[1042]: ifup: interface eno1 already configured
Jan 05 14:14:10 server systemd[1]: Started Open vSwitch.

The output of the above doesn't change after i do the "ifdown/ifup vmbr0" commands (just fyi), yet networking starts to work (cannot even ping the gateway before).

 

[jrshaw]

But eno1 isn't set to auto start, so the moment you reboot your machine, the network will go down due to eno1 not starting on boot. Try adding auto eno1 so it looks like (if this is the config to eno1):

auto eno1
allow-vmbr0 eno1
iface eno1 inet manual
ovs_type OVSPort
ovs_bridge vmbr0

Reboot, try again and let us know if it works Maybe worth adding auto vmbr0 at the top of that ones config.

 

[CLE]

Tried it, behaviour is still the same though...

Fun fact: even without "auto eno1" and "auto vmbr0" both interfaces are UP according to ifconfig.

 

[jrshaw]

Tbh bud, I'm still quite new to Proxmox myself I'm just trying to help best I can, someone will probably have a better answer or solution.

What does systemctl status networking say?

Have you also got a VM with a GUI i.e linux or windows desktop? Maybe worth trying an IP scan and see if MAC addresses are spoofing, as with my original post, Proxmox won't show this, when you do ifconfig etc you will see different MAC addresses, so might be worth getting network sorted, download Advanced ip scanner to a desktop with GUI reboot your server then jump on to the desktop through console and scan when the network is down see if MACs are being spoofed.

 

[spirit]

But eno1 isn't set to auto start, so the moment you reboot your machine, the network will go down due to eno1 not starting on boot. Try adding auto eno1 so it looks like (if this is the config to eno1):

auto eno1
allow-vmbr0 eno1
iface eno1 inet manual
ovs_type OVSPort
ovs_bridge vmbr0

Reboot, try again and let us know if it works Maybe worth adding auto vmbr0 at the top of that ones config.

you shouldn't use auto... with ovs or bridged ovs interfaces.
or you can have race between ovs network script "allow..." and ifupdown "auto ..."

 

[CLE]

I know - i've read as much somewhere on a OVS-related website... ...and also have, after the try, reversed that change....

The original question still stands tho - why doesn't my networking...work... unless i ifdown/ifup vmbr0?

 

[coNemniatEND]

I have the same issue, proxmox 6.1 ifupdown2 installed and when the server starts all the interfaces are down unless I "ifup" them manually.

Is the fix to revert back to ifupdown-v1?

 

[spirit]

I have the same issue, proxmox 6.1 ifupdown2 installed and when the server starts all the interfaces are down unless I "ifup" them manually.

Is the fix to revert back to ifupdown-v1?

Systemctl status networking ? (I have see some case where service was not enabled After ifupdown2 install )

 

[CLE]

I have the same issue ....

No, you don't - necessarily...

I don't have ifupdown2 installed as i use OpenVSwitch, as those are mutually exclusive... If you have ifupdown2 installed and are using OVS, you're doing it wrong (and, please, open your own thread - thank you!)

Systemctl status networking ? (I have see some case where service was not enabled After ifupdown2 install )

For me it just says :

● networking.service - Raise network interfaces
Loaded: loaded (/lib/systemd/system/networking.service; enabled; vendor preset: enabled)
Active: active (exited) since Mon 2020-01-06 09:09:58 CET; 3min 26s ago
Docs: man:interfaces(5)
Process: 1221 ExecStart=/sbin/ifup -a --read-environment (code=exited, status=0/SUCCESS)
Main PID: 1221 (code=exited, status=0/SUCCESS)

Jan 06 09:09:58 server systemd[1]: Starting Raise network interfaces...
Jan 06 09:09:58 server systemd[1]: Started Raise network interfaces.

dpkg -l ifupdown2
Gewünscht=Unbekannt/Installieren/R=Entfernen/P=Vollständig Löschen/Halten
| Status=Nicht/Installiert/Config/U=Entpackt/halb konFiguriert/
Halb installiert/Trigger erWartet/Trigger anhängig
|/ Fehler?=(kein)/R=Neuinstallation notwendig (Status, Fehler: GROSS=schlecht)
||/ Name Version Architektur Beschreibung
+++-==============-============-============-=================================
un ifupdown2 <keine> <keine> (keine Beschreibung vorhanden)

No ifupdown2 here

 

[spirit]

@CLE : ok networking is enabled. I'm going to test your config today.

also, could you replace:

allow-hotplug eno1

by

iface eno1 inet manual

?

 

[syfy323]

I know - i've read as much somewhere on a OVS-related website... ...and also have, after the try, reversed that change....

The original question still stands tho - why doesn't my networking...work... unless i ifdown/ifup vmbr0?

You might hit this Bug with OVS:
https://bugzilla.proxmox.com/show_bug.cgi?id=2475
https://forum.proxmox.com/threads/proxmox-5-4-to-6-0-strange-network-issues.56086/