Adding Bridge without rebooting

[Ben McGuire]

We hope there is a solution to being able to modify the network without rebooting the entire server.

We have read that you can directly modify the /etc/network/interfaces file and you dont have to reboot but this has not worked for us.

We need to two things. One to be able to add a bridge without restarting and two to be able to add nat forwarding rules to a bridge without rebooting.

I have changed the file above but Proxmox is still telling me to reboot when viewing the network tab in proxmox.

--- /etc/network/interfaces 2016-09-22 21:31:26.425698412 +0200
+++ /etc/network/interfaces.new 2016-09-25 14:27:42.772806681 +0200
@@ -30,7 +30,6 @@
post-up iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 22222 -j DNAT --to 192.168.0.10:22
post-down iptables -t nat -D PREROUTING -i vmbr0 -p tcp --dport 22222 -j DNAT --to 192.168.0.10:22


Can any one provide their solution to activate network changes without rebooting the server.

 

[duelle]

1. Add bridge: I am not sure whether adding a bridge is possible without rebooting. One thing you could try is to restart the debian networking service, but I am not sure whether this works or if this has any side-effects on proxmox. To do so you could run service networking restart
   Perhaps you can find more details on what this command can change and what not in the documentation of the networking service (possibly).
2. Add PREROUTING rule: If I got you correctly, you just want to add the rule iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 22222 -j DNAT --to 192.168.0.10:22 to the current iptables rules? If so, you could just execute this command as root. Your changes to the file itself would then be applied on the next reboot and you would not have to execute this again manually.

Especially for number 1 I am not sure whether this is doable and what effects the restart of the networking service of debian might have. So you do this on your own risk of course.

 

[wolfgang]

Hi,

Yes it is possible. The reboot is only a protection, so you have to think twice before you look out .

You can direct edit the /etc/network/interfaces.
The /etc/network/interfaces.new will copied over /etc/network/interfaces when you reboot.
This you can do this also manually.

After changing this file you have to apply this settings.
You can use for this ifdown <net> and ifup <net>.
But if you work on ssh be aware you disconnect the network.

 

[Ben McGuire]

Hi,

Yes it is possible. The reboot is only a protection, so you have to think twice before you look out .

You can direct edit the /etc/network/interfaces.
The /etc/network/interfaces.new will copied over /etc/network/interfaces when you reboot.
This you can do this also manually.

After changing this file you have to apply this settings.
You can use for this ifdown <net> and ifup <net>.
But if you work on ssh be aware you disconnect the network.

Hello

We did do exactly what you state above a few weeks ago except we did not use the ifdown and ifup so hopefully this will be the problem... I shall give it go go today and report back.

Thank you

 

[Ben McGuire]

We ran ifdown && ifup and on both eth0 and vmbr0 and eth0 said the interface is not configured and same with vmbr0. After these commands all VMs lost connection and had to be rebooted.
Now even after running the commands we still see the network configurations in the network tab saying we need to reboot so the command did not work.. Do we need to run the commands on eth0 or the vmbr0? The nat rues have been added to vmbr1 which is set a private IP as vmbr0 has our public IP assigned. I guess I will have to give customers notice and reboot the whole server as the command did not work.

 

[wolfgang]

The best way is to force the ifdown on all net devices (vmbr1->eth0)
and the next thing is no VM should use the bridge.

 

[weerok]

Hello,

If vmbr have attached taps, after debian network restart, taps lose link to their vmbr's

# brctl show
bridge name bridge id STP enabled interfaces
vmbr3320 8000.f40343440ad8 no bond1.3320
tap12765i1

#/etc/init.d/networking stop && /etc/init.d/networking start

# brctl show
bridge name bridge id STP enabled interfaces
vmbr3320 8000.f40343440ad8 no bond1.3320

So I made a workaround script:
- Network restart take about 36 seconds (on my setup), but changes are applied without reboot,
- Don't judge my scripting skills
- Script export vmbr's taps, restart network, and reattach them

#!/bin/bash

echo ==== Proxmox Network-tap-ssh RESTART
brctl show | grep 'vmbr' | awk '{print $1}' | while read line
do
>$line
brctl show $line | egrep -v 'bridge name|vmbr' | awk {'print $1'} >> $line
done

/etc/init.d/networking stop && /etc/init.d/networking start

ls |grep vmbr | while read line
do
input="$line"
while IFS=',' read -r f1 f2 f3 f4 f5 f6 f7 f8 f9 f10
do
ip link set dev $f1 $f2 $f3 $f4 $f5 $f6 $f7 $f8 $f9 master "$input"
done < "$input"
done

 

[smartynov]

I ran into the same problem and found a solution here, thank you. However, I think it is better to read taps configuration from VM .conf files rather than trying (which btw does not work for me because of a slightly different output format).

Therefore I ended up with the following one-liner, which I apply after "service networking restart":

grep -FH bridge= /etc/pve/nodes/*/qemu-server/*.conf \
 | perl -nle 'print "tap$1i$2 master $3" if /\/(\d+).conf:net(\d+):.*?bridge=(vmbr\d+)/' \
 | xargs -l1 echo ip link set

Please use at your own risk only and make a DRY RUN FIRST. If you see reasonable commands, remove the word "echo" from last part of the command and run again to invoke actual changes.

 

[Stoiko Ivanov]

You could also try installing ifupdown2, which has an ifreload command, which should keep all tap-devices attached to the bridge

 

[Marathon]

What is actually really working here ? I installed ifupdown2 but it doesn't seem to work with:

pvesh set /nodes/hv-01/network

Also the earlier mentioned scripts don't seem to (re)set the tap interfaces.

 

[Stoiko Ivanov]

AFAIK the integration of ifupdown2 is not in the API yet (and probably will not be put there unless we change all setups to use ifupdown2)

what should work is running ifreload on the command line

hope this helps!

 

[Marathon]

Good to know! I will check it out if I fixed my bridge issue like here: https://forum.proxmox.com/threads/b...use-of-bridge-ports-issues.53227/#post-245884

Thanks!

 

[guiambros]

Therefore I ended up with the following one-liner, which I apply after "service networking restart":

grep -FH bridge= /etc/pve/nodes/*/qemu-server/*.conf \
 | perl -nle 'print "tap$1i$2 master $3" if /\/(\d+).conf:net(\d+):.*?bridge=(vmbr\d+)/' \
 | xargs -l1 echo ip link set

Thanks @smartynov, this worked for me!

Just note that the tap interfaces changed to fwpr###p# in recent versions, so you may need to adjust accordingly. Here's what I'm using as reload-network.sh:

#!/bin/bash
/etc/init.d/networking restart

grep -FH bridge= /etc/pve/nodes/*/qemu-server/*.conf \
 | perl -nle 'print "fwpr$1p$2 master $3" if /\/(\d+).conf:net(\d+):.*?bridge=(vmbr\d+)/' \
 | xargs -l1 ip link set

 

[ozgurerdogan]

Thanks @smartynov, this worked for me!

Just note that the tap interfaces changed to fwpr###p# in recent versions, so you may need to adjust accordingly. Here's what I'm using as reload-network.sh:

#!/bin/bash
/etc/init.d/networking restart

grep -FH bridge= /etc/pve/nodes/*/qemu-server/*.conf \
 | perl -nle 'print "fwpr$1p$2 master $3" if /\/(\d+).conf:net(\d+):.*?bridge=(vmbr\d+)/' \
 | xargs -l1 ip link set

Is this still working?

 

[ozgurerdogan]

This one seems to be working:

pvesh set /nodes/s9/network > s9 && grep -FH bridge= /etc/pve/nodes/*/qemu-server/*.conf \
 | perl -nle 'print "tap$1i$2 master $3" if /\/(\d+).conf:net(\d+):.*?bridge=(vmbr\d+)/' \
 | xargs -l1 ip link set

 s9 = is hostname
 and not tap$1p$2 but tap$1i$2

 

[Kyle]

Thanks to those who have contributed solutions on the thread

For what its worth I've captured the solutions I know about on a blog post.

Proxmox: solutions to restore guest networking after vm bridge interface maintenance on the host - without a reboot

Critique / edit suggestions welcome.

 

[forresthopkinsa]

@Kyle this is extremely useful, thanks!