VLAN bond setup not working

[mikey4323]

Hi everyone,

I think I've read all the documentation I'm comfortable with both linux and networking but I can't seem to get a lacp/trunk setup working.

So my setup is this.

I got a server with 6 ethernet interfaces running proxmox 2.1.

And I wanted to have a configuration something like this.

eth0 > management , single ip
eth1 > unused, spare
eth2 bonded together working an lacp port-channel and 802.1q trunk
eth3 2nd member of this port-channel
eth4 > internet connection (don't want this in my port-channel)
eth5 > 2nd internet connection (also don't want this in the lacp channel)

The portchannel + trunk is going towards a single cisco switch with the config:
!
interface Port-channel48
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1,304
switchport mode trunk
spanning-tree portfast trunk
!
interface GigabitEthernet1/0/17
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1,304
switchport mode trunk
channel-protocol lacp
channel-group 48 mode active
spanning-tree portfast trunk
!
interface GigabitEthernet1/0/23
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1,304
switchport mode trunk
channel-protocol lacp
channel-group 48 mode active
spanning-tree portfast trunk
!

Nothing fancy, I can see that the channel is up and working from the switch side
48 Po48(SU) LACP Gi1/0/17(P) Gi1/0/23(P)

And the configuration of the server interface is (following the vlan guide on the site):
auto lo
iface lo inet loopback
iface eth0 inet manual
iface eth1 inet manual
iface eth2 inet manual
iface eth3 inet manual
iface eth4 inet manual
iface eth5 inet manual

auto vmbr2000
iface vmbr2000 inet static
address x.x.x.x
netmask 255.255.255.252
bridge_ports eth0
bridge_stp off
bridge_fd 0
up /sbin/route add -netx.x.x.x/24 gw x.x.x.x
down /sbin/route del -net x.x.x.x/24 gw x.x.x.x


auto vmbr2004
iface vmbr2004 inet static
address y.y.y.y
netmask 255.255.255.y
gateway y.y.y.y
bridge_ports eth4
bridge_stp off
bridge_fd 0

auto vmbr2005
iface vmbr2005 inet static
address z.z.z.z
netmask 255.255.255.z
bridge_ports eth5
bridge_stp off
bridge_fd 0

auto bond0
iface bond0 inet manual
slaves eth2 eth3
bond_miimon 100
bond_mode 802.3ad

# Trunk native VLAN1
auto vmbr1
iface vmbr1 inet static
address q.q.q.q
netmask 255.255.255.q
bridge_ports bond0
bridge_stp off
bridge_fd 0
# Trunk VLAN304
auto vlan304
iface vlan304 inet manual
vlan_raw_device bond0

auto vmbr304
iface vmbr304 inet static
address r.r.r.r
netmask 255.255.255.r
bridge_ports vlan304
bridge_stp off
bridge_fd 0
post-up ip route add table vlan304 default via r.r.r.r dev vmbr304
post-up ip rule add from r.r.r.r/26 table vlan304
post-down ip route del table vlan304 default via r.r.r.r dev vmbr304
post-down ip rule del from r.r.r.r/26 table vlan304

and in /etc/iproute2/rt_tables I added:
# Table for vlan304
304 vlan304

Now all the networking is working except the tagged vlans in this case vlan 304.

Also in the syslog i see error messages like: Deleting interface #10 vmbr304 interface stats: received=0, sent=0, dropped=0, active_time=1501 secs

I'm a bit lost in what I've done wrong or how i can fix things, any ideas hopefully?

 

[udo]

Hi,
I'm not familiar with bonding but you can try to use a config like this:

auto bond0
iface bond0 inet static
    slaves eth2 eth3
    address  0.0.0.0
    netmask  0.0.0.0
    bond_miimon 100
    bond_mode 802.3ad

auto bond0.304
iface eth0.304 inet static
        address  0.0.0.0
        netmask  0.0.0.0


auto vmbr304
iface vmbr304 inet static
    address r.r.r.r
    netmask 255.255.255.r
    bridge_ports bond0.304
    bridge_stp off
    bridge_fd 0

BTW. You only need an IP on an bridge, if the pve-gui must be accessible fron this network and the normal routing don't work for this.

Point 2 is, that pve normaly need an vmbr0 (or eth0) with ip for cluster communication - so perhaps it's better to use vmbr0 instead of vmbr2000?

Udo

 

[mikey4323]

Udo,

Thanks I will try that but I do wonder what the IP addressing should be in the example above as both bond0.304 as the vmbr304 have than the exact same IP information or do you mean that I have to chose on of the two?

 

[mikey4323]

Hi Udo,

I tried your syntax way of things but still a nogo. Starting to wonder if I first have to try it without the bond.

 

[spirit]

If you use only kvm guest,you can also do


auto bond0
iface bond0 inet static
slaves eth2 eth3
address 0.0.0.0
netmask 0.0.0.0
bond_miimon 100
bond_mode 802.3ad




auto vmbr1
iface vmbr1 inet static
bridge_ports bond0
bridge_stp off
bridge_fd 0




and manage vlans directly on your kvm guest networks card configuration


(This will create news bridges with tagged bond automaticly)