Bridge won't start because of bridge-ports issues

Marathon

Member
Apr 7, 2019
40
0
6
49
Hi,

Sorry for the delay but I needed spare server for this to test at the moment and other upcoming things as well.

I just tried without changing anything in the interfaces file:"

Code:
ifreload -a
Code:
# cat /proc/sys/net/ipv4/ip_forward
1
Code:
# grep "" /proc/sys/net/ipv4/conf/*/forwarding
/proc/sys/net/ipv4/conf/all/forwarding:1
/proc/sys/net/ipv4/conf/bond0/forwarding:1
/proc/sys/net/ipv4/conf/default/forwarding:1
/proc/sys/net/ipv4/conf/eth0/forwarding:0
/proc/sys/net/ipv4/conf/eth1/forwarding:0
/proc/sys/net/ipv4/conf/eth2/forwarding:0
/proc/sys/net/ipv4/conf/lo/forwarding:0
/proc/sys/net/ipv4/conf/tap201i0/forwarding:1
/proc/sys/net/ipv4/conf/tap201i1/forwarding:1
/proc/sys/net/ipv4/conf/tap202i0/forwarding:1
/proc/sys/net/ipv4/conf/tap202i1/forwarding:1
/proc/sys/net/ipv4/conf/tap211i0/forwarding:1
/proc/sys/net/ipv4/conf/tap211i1/forwarding:1
/proc/sys/net/ipv4/conf/tap212i0/forwarding:1
/proc/sys/net/ipv4/conf/tap212i1/forwarding:1
/proc/sys/net/ipv4/conf/tap231i0/forwarding:1
/proc/sys/net/ipv4/conf/tap232i0/forwarding:1
/proc/sys/net/ipv4/conf/tun0/forwarding:1
/proc/sys/net/ipv4/conf/vmbr1001/forwarding:0
/proc/sys/net/ipv4/conf/vmbr101/forwarding:0
/proc/sys/net/ipv4/conf/vmbr11/forwarding:1

So it seems that non-taps are failing.
 

spirit

Well-Known Member
Apr 2, 2010
3,507
155
63
www.odiso.com
Hi,

Sorry for the delay but I needed spare server for this to test at the moment and other upcoming things as well.

I just tried without changing anything in the interfaces file:"

Code:
ifreload -a
Code:
# cat /proc/sys/net/ipv4/ip_forward
1
Code:
# grep "" /proc/sys/net/ipv4/conf/*/forwarding
/proc/sys/net/ipv4/conf/all/forwarding:1
/proc/sys/net/ipv4/conf/bond0/forwarding:1
/proc/sys/net/ipv4/conf/default/forwarding:1
/proc/sys/net/ipv4/conf/eth0/forwarding:0
/proc/sys/net/ipv4/conf/eth1/forwarding:0
/proc/sys/net/ipv4/conf/eth2/forwarding:0
/proc/sys/net/ipv4/conf/lo/forwarding:0
/proc/sys/net/ipv4/conf/tap201i0/forwarding:1
/proc/sys/net/ipv4/conf/tap201i1/forwarding:1
/proc/sys/net/ipv4/conf/tap202i0/forwarding:1
/proc/sys/net/ipv4/conf/tap202i1/forwarding:1
/proc/sys/net/ipv4/conf/tap211i0/forwarding:1
/proc/sys/net/ipv4/conf/tap211i1/forwarding:1
/proc/sys/net/ipv4/conf/tap212i0/forwarding:1
/proc/sys/net/ipv4/conf/tap212i1/forwarding:1
/proc/sys/net/ipv4/conf/tap231i0/forwarding:1
/proc/sys/net/ipv4/conf/tap232i0/forwarding:1
/proc/sys/net/ipv4/conf/tun0/forwarding:1
/proc/sys/net/ipv4/conf/vmbr1001/forwarding:0
/proc/sys/net/ipv4/conf/vmbr101/forwarding:0
/proc/sys/net/ipv4/conf/vmbr11/forwarding:1

So it seems that non-taps are failing.
it was forwarding:1 everywhere before reload ?
can you post your full /etc/network/interfaces ?

(from my test, ifreload disable it with manual interface (without ip address))
 

Marathon

Member
Apr 7, 2019
40
0
6
49
It's the same config as in my startpost.

I see that ethX are forwarding:0 after the reload which are forwarding:1 before. The bridges, except vmbr11, are all :0 before and after.

Does that help ?
 
Last edited:

Marathon

Member
Apr 7, 2019
40
0
6
49
New update, this makes it work again:

Code:
echo '1' => /proc/sys/net/ipv4/conf/eth<X>/forwarding
 

Marathon

Member
Apr 7, 2019
40
0
6
49
I'm able to reproduce too on ethX interface.

as workaround, can you try to add "ip-forward on" in your eth in /etc/network/interface ?
That works indeed!

I must say I only need to do it on the ethX interface that has an address. I have 2 interfaces in a bond and they can live with :0 as the bridge takes over I think ?

I have read in the link you gave me earlier it's a bug that Cumulus wants/needs to fix ?

Thanks a lot for all the effort!
 

spirit

Well-Known Member
Apr 2, 2010
3,507
155
63
www.odiso.com
That works indeed!

I must say I only need to do it on the ethX interface that has an address. I have 2 interfaces in a bond and they can live with :0 as the bridge takes over I think ?

As forwarding occur between 2 interfaces with ip address, it make sense.

- for a "standalone" interface, without bond, not in a bridge with static ip -> enable forward on the interface
- for a bond interface, not in a bridge with static ip -> enable forward on the bond
- for a bond, and a standalone interface in a bridge, with a static ip on the bridge -> enable forward on the bridge


>>I have read in the link you gave me earlier it's a bug that Cumulus wants/needs to fix ?
yes. I think the correct behaviour is to not change the value, if ip-forward is not defined. (for now, they assume that it's 0, if not defined)
Thanks a lot for all the effort!
Thanks for the report !
 

Marathon

Member
Apr 7, 2019
40
0
6
49
As forwarding occur between 2 interfaces with ip address, it make sense.

- for a "standalone" interface, without bond, not in a bridge with static ip -> enable forward on the interface
- for a bond interface, not in a bridge with static ip -> enable forward on the bond
- for a bond, and a standalone interface in a bridge, with a static ip on the bridge -> enable forward on the bridge



yes. I think the correct behaviour is to not change the value, if ip-forward is not defined. (for now, they assume that it's 0, if not defined)

Thanks for the report !
You were very welcome, great!

To be precise; We need the "workaround" or the package will be updated in some time and will float into Proxmox ? I'm not sure who maintains what at the moment but that it's managed well is for sure! :) (need to read some more about Cumulus)
 

Marathon

Member
Apr 7, 2019
40
0
6
49
Another update on this.

IPv6 doesn't seem to forward at all, only one bridge is in :1 mode, vmbr11 and none of the interfaces is with ip-forward on on the IPv6 interfaces/aliases.

Maybe another bug, but a terrible one :(
 

spirit

Well-Known Member
Apr 2, 2010
3,507
155
63
www.odiso.com
You were very welcome, great!

To be precise; We need the "workaround" or the package will be updated in some time and will float into Proxmox ? I'm not sure who maintains what at the moment but that it's managed well is for sure! :) (need to read some more about Cumulus)
Proxmox have his own ifupdown2 package, with custom patchs. So generally, I'm looking with ifupdown2 devs if I can have the patch/fix upstream, and if it take too long, I'm already make patch in the proxmox package.


IPv6 doesn't seem to forward at all, only one bridge is in :1 mode, vmbr11 and none of the interfaces is with ip-forward on on the IPv6 interfaces/aliases.
you can also add "ip6-forward on".

Also, with ifupdown2, you don't need to create a :1 interface for ipv6, you can simply create an interface, without inet static or inet6 static.
with multiple ip address (ipv4 or ipv6, mixed)
Code:
auto ethx
iface ethx    
    address 10.0.0.1/24
    address  2001:db8:1f89::/48
    address 192.168.0.1/16
    ip-forward on
    ip6-forward on
 

Marathon

Member
Apr 7, 2019
40
0
6
49
Proxmox have his own ifupdown2 package, with custom patchs. So generally, I'm looking with ifupdown2 devs if I can have the patch/fix upstream, and if it take too long, I'm already make patch in the proxmox package.
I already thought, good work! It's nice to work with a reference and see if changes just should be made and Cumulus is also just seeing if there are bugs and like input from the Proxmox side.

you can also add "ip6-forward on".

Also, with ifupdown2, you don't need to create a :1 interface for ipv6, you can simply create an interface, without inet static or inet6 static.
with multiple ip address (ipv4 or ipv6, mixed)
Code:
auto ethx
iface ethx   
    address 10.0.0.1/24
    address  2001:db8:1f89::/48
    address 192.168.0.1/16
    ip-forward on
    ip6-forward on
OK, but how would this work for the gateway ? you get a gateway and gateway6 ? I try to keep it also a little but Proxmox GUI like, I think that really needs an upgrade then.

This doesn't change the forward state so far so I need to investigate further.

Thanks again!
 

Marathon

Member
Apr 7, 2019
40
0
6
49
OK I'm testing this out and it's strange.

I can stil ping my Proxmox bridge but nothing to the outside world. On my Proxmox box itself I can ping/traceroute the outside world. So I'm unsure what goes wrong here.
 

spirit

Well-Known Member
Apr 2, 2010
3,507
155
63
www.odiso.com
OK I'm testing this out and it's strange.

I can stil ping my Proxmox bridge but nothing to the outside world. On my Proxmox box itself I can ping/traceroute the outside world. So I'm unsure what goes wrong here.
what is the result of

grep "" /proc/sys/net/ipv4/conf/*/forwarding

before and after reload ?
 

Marathon

Member
Apr 7, 2019
40
0
6
49
what is the result of

grep "" /proc/sys/net/ipv4/conf/*/forwarding

before and after reload ?
IPv4 works, IPv6 doesn't. Sorry for the missing detail!

IPv6 has eth0 on forward, the vmbr11 and the rest is 0.
 

spirit

Well-Known Member
Apr 2, 2010
3,507
155
63
www.odiso.com
IPv4 works, IPv6 doesn't. Sorry for the missing detail!

IPv6 has eth0 on forward, the vmbr11 and the rest is 0.
does is ipv6 routing works before reload ?
and is
net.ipv6.conf.all.forwarding=1 enabled in sysctl.conf ?


what is the value of "cat /proc/sys/net/ipv6/conf/all/forwarding" ?


(also, do you have the same problem with option "ip6-forward yes" ?)


I'll try to reproduce with ipv6 today or tomorrow.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!