Bridge won't start because of bridge-ports issues

Discussion in 'Proxmox VE: Networking and Firewall' started by Marathon, Apr 9, 2019.

  1. Marathon

    Marathon Member

    Joined:
    Apr 7, 2019
    Messages:
    40
    Likes Received:
    0
    Will do ASAP as I need to use some textbox for it :)

    Thanks for the link, we start to become a team ;)
     
  2. Marathon

    Marathon Member

    Joined:
    Apr 7, 2019
    Messages:
    40
    Likes Received:
    0
    Hi,

    Sorry for the delay but I needed spare server for this to test at the moment and other upcoming things as well.

    I just tried without changing anything in the interfaces file:"

    Code:
    ifreload -a
    Code:
    # cat /proc/sys/net/ipv4/ip_forward
    1
    Code:
    # grep "" /proc/sys/net/ipv4/conf/*/forwarding
    /proc/sys/net/ipv4/conf/all/forwarding:1
    /proc/sys/net/ipv4/conf/bond0/forwarding:1
    /proc/sys/net/ipv4/conf/default/forwarding:1
    /proc/sys/net/ipv4/conf/eth0/forwarding:0
    /proc/sys/net/ipv4/conf/eth1/forwarding:0
    /proc/sys/net/ipv4/conf/eth2/forwarding:0
    /proc/sys/net/ipv4/conf/lo/forwarding:0
    /proc/sys/net/ipv4/conf/tap201i0/forwarding:1
    /proc/sys/net/ipv4/conf/tap201i1/forwarding:1
    /proc/sys/net/ipv4/conf/tap202i0/forwarding:1
    /proc/sys/net/ipv4/conf/tap202i1/forwarding:1
    /proc/sys/net/ipv4/conf/tap211i0/forwarding:1
    /proc/sys/net/ipv4/conf/tap211i1/forwarding:1
    /proc/sys/net/ipv4/conf/tap212i0/forwarding:1
    /proc/sys/net/ipv4/conf/tap212i1/forwarding:1
    /proc/sys/net/ipv4/conf/tap231i0/forwarding:1
    /proc/sys/net/ipv4/conf/tap232i0/forwarding:1
    /proc/sys/net/ipv4/conf/tun0/forwarding:1
    /proc/sys/net/ipv4/conf/vmbr1001/forwarding:0
    /proc/sys/net/ipv4/conf/vmbr101/forwarding:0
    /proc/sys/net/ipv4/conf/vmbr11/forwarding:1
    

    So it seems that non-taps are failing.
     
  3. spirit

    spirit Well-Known Member

    Joined:
    Apr 2, 2010
    Messages:
    3,323
    Likes Received:
    135
    it was forwarding:1 everywhere before reload ?
    can you post your full /etc/network/interfaces ?

    (from my test, ifreload disable it with manual interface (without ip address))
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  4. Marathon

    Marathon Member

    Joined:
    Apr 7, 2019
    Messages:
    40
    Likes Received:
    0
    It's the same config as in my startpost.

    I see that ethX are forwarding:0 after the reload which are forwarding:1 before. The bridges, except vmbr11, are all :0 before and after.

    Does that help ?
     
    #24 Marathon, May 3, 2019
    Last edited: May 3, 2019
  5. Marathon

    Marathon Member

    Joined:
    Apr 7, 2019
    Messages:
    40
    Likes Received:
    0
    New update, this makes it work again:

    Code:
    echo '1' => /proc/sys/net/ipv4/conf/eth<X>/forwarding
     
  6. spirit

    spirit Well-Known Member

    Joined:
    Apr 2, 2010
    Messages:
    3,323
    Likes Received:
    135
    I'm able to reproduce too on ethX interface.

    as workaround, can you try to add "ip-forward on" in your eth in /etc/network/interface ?
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  7. Marathon

    Marathon Member

    Joined:
    Apr 7, 2019
    Messages:
    40
    Likes Received:
    0
    That works indeed!

    I must say I only need to do it on the ethX interface that has an address. I have 2 interfaces in a bond and they can live with :0 as the bridge takes over I think ?

    I have read in the link you gave me earlier it's a bug that Cumulus wants/needs to fix ?

    Thanks a lot for all the effort!
     
  8. spirit

    spirit Well-Known Member

    Joined:
    Apr 2, 2010
    Messages:
    3,323
    Likes Received:
    135

    As forwarding occur between 2 interfaces with ip address, it make sense.

    - for a "standalone" interface, without bond, not in a bridge with static ip -> enable forward on the interface
    - for a bond interface, not in a bridge with static ip -> enable forward on the bond
    - for a bond, and a standalone interface in a bridge, with a static ip on the bridge -> enable forward on the bridge


    yes. I think the correct behaviour is to not change the value, if ip-forward is not defined. (for now, they assume that it's 0, if not defined)
    Thanks for the report !
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  9. Marathon

    Marathon Member

    Joined:
    Apr 7, 2019
    Messages:
    40
    Likes Received:
    0
    You were very welcome, great!

    To be precise; We need the "workaround" or the package will be updated in some time and will float into Proxmox ? I'm not sure who maintains what at the moment but that it's managed well is for sure! :) (need to read some more about Cumulus)
     
  10. Marathon

    Marathon Member

    Joined:
    Apr 7, 2019
    Messages:
    40
    Likes Received:
    0
    Another update on this.

    IPv6 doesn't seem to forward at all, only one bridge is in :1 mode, vmbr11 and none of the interfaces is with ip-forward on on the IPv6 interfaces/aliases.

    Maybe another bug, but a terrible one :(
     
  11. spirit

    spirit Well-Known Member

    Joined:
    Apr 2, 2010
    Messages:
    3,323
    Likes Received:
    135
    Proxmox have his own ifupdown2 package, with custom patchs. So generally, I'm looking with ifupdown2 devs if I can have the patch/fix upstream, and if it take too long, I'm already make patch in the proxmox package.


    you can also add "ip6-forward on".

    Also, with ifupdown2, you don't need to create a :1 interface for ipv6, you can simply create an interface, without inet static or inet6 static.
    with multiple ip address (ipv4 or ipv6, mixed)
    Code:
    auto ethx
    iface ethx    
        address 10.0.0.1/24
        address  2001:db8:1f89::/48
        address 192.168.0.1/16
        ip-forward on
        ip6-forward on
    
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  12. Marathon

    Marathon Member

    Joined:
    Apr 7, 2019
    Messages:
    40
    Likes Received:
    0
    I already thought, good work! It's nice to work with a reference and see if changes just should be made and Cumulus is also just seeing if there are bugs and like input from the Proxmox side.

    OK, but how would this work for the gateway ? you get a gateway and gateway6 ? I try to keep it also a little but Proxmox GUI like, I think that really needs an upgrade then.

    This doesn't change the forward state so far so I need to investigate further.

    Thanks again!
     
  13. spirit

    spirit Well-Known Member

    Joined:
    Apr 2, 2010
    Messages:
    3,323
    Likes Received:
    135
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  14. Marathon

    Marathon Member

    Joined:
    Apr 7, 2019
    Messages:
    40
    Likes Received:
    0

    Great, going to try it in a few hours! Is the bond change applied as well (yet) ?
     
  15. spirit

    spirit Well-Known Member

    Joined:
    Apr 2, 2010
    Messages:
    3,323
    Likes Received:
    135
    for bond-primary option support, no. (this will take more time)

    I have only fixed the ip_forward issue.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  16. Marathon

    Marathon Member

    Joined:
    Apr 7, 2019
    Messages:
    40
    Likes Received:
    0
    Sorry for the delay, I will report back asap :) Thanks again!
     
  17. Marathon

    Marathon Member

    Joined:
    Apr 7, 2019
    Messages:
    40
    Likes Received:
    0
    OK I'm testing this out and it's strange.

    I can stil ping my Proxmox bridge but nothing to the outside world. On my Proxmox box itself I can ping/traceroute the outside world. So I'm unsure what goes wrong here.
     
  18. spirit

    spirit Well-Known Member

    Joined:
    Apr 2, 2010
    Messages:
    3,323
    Likes Received:
    135
    what is the result of

    grep "" /proc/sys/net/ipv4/conf/*/forwarding

    before and after reload ?
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  19. Marathon

    Marathon Member

    Joined:
    Apr 7, 2019
    Messages:
    40
    Likes Received:
    0
    IPv4 works, IPv6 doesn't. Sorry for the missing detail!

    IPv6 has eth0 on forward, the vmbr11 and the rest is 0.
     
  20. spirit

    spirit Well-Known Member

    Joined:
    Apr 2, 2010
    Messages:
    3,323
    Likes Received:
    135
    does is ipv6 routing works before reload ?
    and is
    net.ipv6.conf.all.forwarding=1 enabled in sysctl.conf ?


    what is the value of "cat /proc/sys/net/ipv6/conf/all/forwarding" ?


    (also, do you have the same problem with option "ip6-forward yes" ?)


    I'll try to reproduce with ipv6 today or tomorrow.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice