Hiho!
I'm now trying for several hours to make bonding work between proxmox and the cisco.
The persisting problem is: only one link is active, so there is always only the speed of one link :-(
I will paste some console output here:
cisco-configuration:
====================
interface Port-channel3
switchport
switchport access vlan 22
switchport trunk encapsulation dot1q
switchport trunk native vlan 22
switchport trunk allowed vlan 1,22,33,34
switchport mode trunk
!
interface GigabitEthernet3/9
description Barny I - 1
switchport
switchport access vlan 22
switchport trunk allowed vlan 1,22,33,34
switchport mode trunk
channel-group 3 mode active
ip dhcp snooping trust
!
!
interface GigabitEthernet3/13
description Barny I - 2
switchport
switchport access vlan 22
switchport trunk native vlan 22
switchport trunk allowed vlan 1,22,33,34
switchport mode trunk
channel-group 3 mode active
ip dhcp snooping trust
cisco console output:
====================
mom#sh lacp 3 internal
Flags: S - Device is requesting Slow LACPDUs
F - Device is requesting Fast LACPDUs
A - Device is in Active mode P - Device is in Passive mode
Channel group 3
LACP port Admin Oper Port Port
Port Flags State Priority Key Key Number State
Gi3/9 SA bndl 32768 0x3 0x1003 0x30A 0x3D
Gi3/13 SA bndl 32768 0x3 0x3 0x30E 0x3D
mom#
-------
mom#show etherchannel 3 port-channel
Port-channels in the group:
----------------------
Port-channel: Po3 (Primary Aggregator)
------------
Age of the Port-channel = 66d:22h:25m:09s
Logical slot/port = 14/3 Number of ports = 1
HotStandBy port = null
Port state = Port-channel Ag-Inuse
Protocol = LACP
Fast-switchover = disabled
Direct Load Swap = disabled
Ports in the Port-channel:
Index Load Port EC state No of bits
------+------+--------+------------------+-----------
0 FF Gi3/13 Active 8
Time since last port bundled: 0d:00h:05m:25s Gi3/13
Time since last port Un-bundled: 0d:00h:07m:54s Gi3/9
Port-channel: Po3A
------------
Age of the Port-channel = 0d:00h:05m:27s
Logical slot/port = 14/7 Number of ports = 1
HotStandBy port = null
Port state = Port-channel Ag-Inuse
Protocol = LACP
Fast-switchover = disabled
Direct Load Swap = disabled
Ports in the Port-channel:
Index Load Port EC state No of bits
------+------+--------+------------------+-----------
0 FF Gi3/9 Active 8
Time since last port bundled: 0d:00h:05m:26s Gi3/9
Last applied Hash Distribution Algorithm: Fixed
Channel-group Iedge Counts:
--------------------------:
Access ref count : 0
Iedge session count : 0
mom#
-----
mom#sh interfaces Port-channel3
Port-channel3 is up, line protocol is up (connected)
Hardware is EtherChannel, address is 0009.1258.aefc (bia 0009.1258.aefc)
MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 1000Mb/s
input flow-control is off, output flow-control is off
Members in this channel: Gi3/13
ARP type: ARPA, ARP Timeout 04:00:00
Last input never, output never, output hang never
Last clearing of "show interface" counters never
Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 44000 bits/sec, 43 packets/sec
1724936763 packets input, 1205960876246 bytes, 0 no buffer
Received 58351 broadcasts (44184 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 0 multicast, 0 pause input
0 input packets with dribble condition detected
2340288793 packets output, 2972632224427 bytes, 0 underruns
0 output errors, 0 collisions, 2 interface resets
0 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 pause output
0 output buffer failures, 0 output buffers swapped out
mom#
server console output:
======================
barny-i:~# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.5.0 (November 4, 2008)
Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer2 (0)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
802.3ad info
LACP rate: slow
Aggregator selection policy (ad_select): stable
Active Aggregator Info:
Aggregator ID: 2
Number of ports: 1
Actor Key: 17
Partner Key: 4099
Partner Mac Address: 00:18:74:1e:41:80
Slave Interface: eth0
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:22:19:7a:90:95
Aggregator ID: 1
Slave Interface: eth1
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:22:19:7a:90:97
Aggregator ID: 2
barny-i:~#
------
barny-i:/etc/network# cat interfaces
# network interface settings
auto lo
iface lo inet loopback
iface eth0 inet manual
iface eth1 inet manual
auto bond0
iface bond0 inet manual
slaves eth0 eth1
bond_miimon 100
bond_mode 802.3ad
up ifconfig bond0 0.0.0.0 up
auto bond0.1
iface bond0.1 inet manual
vlan-raw-device bond0
auto bond0.22
iface bond0.22 inet manual
vlan-raw-device bond0
auto bond0.33
iface bond0.33 inet manual
vlan-raw-device bond0
auto bond0.34
iface bond0.34 inet manual
vlan-raw-device bond0
auto vmbr0
iface vmbr0 inet static
address XXXXXX
netmask 255.255.252.0
gateway XXXXXXXX
broadcast XXXXXXXX
bridge_ports bond0.22
bridge_stp off
bridge_fd 0
auto vmbr1
iface vmbr1 inet static
address 192.168.10.102
netmask 255.255.255.0
bridge_ports bond0
bridge_stp off
bridge_fd 0
auto vmbr33
iface vmbr33 inet static
address 192.168.20.102
netmask 255.255.255.0
bridge_ports bond0.33
bridge_stp off
bridge_fd 0
auto vmbr34
iface vmbr34 inet static
address 10.0.34.102
netmask 255.255.255.0
bridge_ports bond0.34
bridge_stp off
bridge_fd 0
barny-i:/etc/network#
barny-i:/etc/network# ifconfig eth1
eth1 Link encap:Ethernet Hardware Adresse 00:22:19:7a:90:95
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metrik:1
RX packets:2375715 errors:0 dropped:0 overruns:0 frame:0
TX packets:3077643 errors:0 dropped:0 overruns:0 carrier:0
Kollisionen:0 Sendewarteschlangenlänge:1000
RX bytes:1448360035 (1.3 GiB) TX bytes:3982042220 (3.7 GiB)
Interrupt:16 Speicher:f4000000-f4012800
barny-i:/etc/network# ifconfig eth0
eth0 Link encap:Ethernet Hardware Adresse 00:22:19:7a:90:95
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metrik:1
RX packets:122513 errors:0 dropped:0 overruns:0 frame:0
TX packets:109 errors:0 dropped:0 overruns:0 carrier:0
Kollisionen:0 Sendewarteschlangenlänge:1000
RX bytes:20153741 (19.2 MiB) TX bytes:13116 (12.8 KiB) (some LACP-handshake???)
Interrupt:16 Speicher:f8000000-f8012800
barny-i:/etc/network#
I marked two lines red, they say, there is only one port active. I tested the speed by simultaneous downloading from severeal computers - it's really exactly one port.
The same Problem I have on the second machine, wich is connected with four cables, but because it's running all live-systems right now I'm testing with that machine.
Is there something fundamentally wrong with my configuration?
Any help appreciated...
Regards from Aachen, Germany,
Thomas
I'm now trying for several hours to make bonding work between proxmox and the cisco.
The persisting problem is: only one link is active, so there is always only the speed of one link :-(
I will paste some console output here:
cisco-configuration:
====================
interface Port-channel3
switchport
switchport access vlan 22
switchport trunk encapsulation dot1q
switchport trunk native vlan 22
switchport trunk allowed vlan 1,22,33,34
switchport mode trunk
!
interface GigabitEthernet3/9
description Barny I - 1
switchport
switchport access vlan 22
switchport trunk allowed vlan 1,22,33,34
switchport mode trunk
channel-group 3 mode active
ip dhcp snooping trust
!
!
interface GigabitEthernet3/13
description Barny I - 2
switchport
switchport access vlan 22
switchport trunk native vlan 22
switchport trunk allowed vlan 1,22,33,34
switchport mode trunk
channel-group 3 mode active
ip dhcp snooping trust
cisco console output:
====================
mom#sh lacp 3 internal
Flags: S - Device is requesting Slow LACPDUs
F - Device is requesting Fast LACPDUs
A - Device is in Active mode P - Device is in Passive mode
Channel group 3
LACP port Admin Oper Port Port
Port Flags State Priority Key Key Number State
Gi3/9 SA bndl 32768 0x3 0x1003 0x30A 0x3D
Gi3/13 SA bndl 32768 0x3 0x3 0x30E 0x3D
mom#
-------
mom#show etherchannel 3 port-channel
Port-channels in the group:
----------------------
Port-channel: Po3 (Primary Aggregator)
------------
Age of the Port-channel = 66d:22h:25m:09s
Logical slot/port = 14/3 Number of ports = 1
HotStandBy port = null
Port state = Port-channel Ag-Inuse
Protocol = LACP
Fast-switchover = disabled
Direct Load Swap = disabled
Ports in the Port-channel:
Index Load Port EC state No of bits
------+------+--------+------------------+-----------
0 FF Gi3/13 Active 8
Time since last port bundled: 0d:00h:05m:25s Gi3/13
Time since last port Un-bundled: 0d:00h:07m:54s Gi3/9
Port-channel: Po3A
------------
Age of the Port-channel = 0d:00h:05m:27s
Logical slot/port = 14/7 Number of ports = 1
HotStandBy port = null
Port state = Port-channel Ag-Inuse
Protocol = LACP
Fast-switchover = disabled
Direct Load Swap = disabled
Ports in the Port-channel:
Index Load Port EC state No of bits
------+------+--------+------------------+-----------
0 FF Gi3/9 Active 8
Time since last port bundled: 0d:00h:05m:26s Gi3/9
Last applied Hash Distribution Algorithm: Fixed
Channel-group Iedge Counts:
--------------------------:
Access ref count : 0
Iedge session count : 0
mom#
-----
mom#sh interfaces Port-channel3
Port-channel3 is up, line protocol is up (connected)
Hardware is EtherChannel, address is 0009.1258.aefc (bia 0009.1258.aefc)
MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 1000Mb/s
input flow-control is off, output flow-control is off
Members in this channel: Gi3/13
ARP type: ARPA, ARP Timeout 04:00:00
Last input never, output never, output hang never
Last clearing of "show interface" counters never
Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 44000 bits/sec, 43 packets/sec
1724936763 packets input, 1205960876246 bytes, 0 no buffer
Received 58351 broadcasts (44184 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 0 multicast, 0 pause input
0 input packets with dribble condition detected
2340288793 packets output, 2972632224427 bytes, 0 underruns
0 output errors, 0 collisions, 2 interface resets
0 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 pause output
0 output buffer failures, 0 output buffers swapped out
mom#
server console output:
======================
barny-i:~# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.5.0 (November 4, 2008)
Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer2 (0)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
802.3ad info
LACP rate: slow
Aggregator selection policy (ad_select): stable
Active Aggregator Info:
Aggregator ID: 2
Number of ports: 1
Actor Key: 17
Partner Key: 4099
Partner Mac Address: 00:18:74:1e:41:80
Slave Interface: eth0
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:22:19:7a:90:95
Aggregator ID: 1
Slave Interface: eth1
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:22:19:7a:90:97
Aggregator ID: 2
barny-i:~#
------
barny-i:/etc/network# cat interfaces
# network interface settings
auto lo
iface lo inet loopback
iface eth0 inet manual
iface eth1 inet manual
auto bond0
iface bond0 inet manual
slaves eth0 eth1
bond_miimon 100
bond_mode 802.3ad
up ifconfig bond0 0.0.0.0 up
auto bond0.1
iface bond0.1 inet manual
vlan-raw-device bond0
auto bond0.22
iface bond0.22 inet manual
vlan-raw-device bond0
auto bond0.33
iface bond0.33 inet manual
vlan-raw-device bond0
auto bond0.34
iface bond0.34 inet manual
vlan-raw-device bond0
auto vmbr0
iface vmbr0 inet static
address XXXXXX
netmask 255.255.252.0
gateway XXXXXXXX
broadcast XXXXXXXX
bridge_ports bond0.22
bridge_stp off
bridge_fd 0
auto vmbr1
iface vmbr1 inet static
address 192.168.10.102
netmask 255.255.255.0
bridge_ports bond0
bridge_stp off
bridge_fd 0
auto vmbr33
iface vmbr33 inet static
address 192.168.20.102
netmask 255.255.255.0
bridge_ports bond0.33
bridge_stp off
bridge_fd 0
auto vmbr34
iface vmbr34 inet static
address 10.0.34.102
netmask 255.255.255.0
bridge_ports bond0.34
bridge_stp off
bridge_fd 0
barny-i:/etc/network#
barny-i:/etc/network# ifconfig eth1
eth1 Link encap:Ethernet Hardware Adresse 00:22:19:7a:90:95
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metrik:1
RX packets:2375715 errors:0 dropped:0 overruns:0 frame:0
TX packets:3077643 errors:0 dropped:0 overruns:0 carrier:0
Kollisionen:0 Sendewarteschlangenlänge:1000
RX bytes:1448360035 (1.3 GiB) TX bytes:3982042220 (3.7 GiB)
Interrupt:16 Speicher:f4000000-f4012800
barny-i:/etc/network# ifconfig eth0
eth0 Link encap:Ethernet Hardware Adresse 00:22:19:7a:90:95
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metrik:1
RX packets:122513 errors:0 dropped:0 overruns:0 frame:0
TX packets:109 errors:0 dropped:0 overruns:0 carrier:0
Kollisionen:0 Sendewarteschlangenlänge:1000
RX bytes:20153741 (19.2 MiB) TX bytes:13116 (12.8 KiB) (some LACP-handshake???)
Interrupt:16 Speicher:f8000000-f8012800
barny-i:/etc/network#
The same Problem I have on the second machine, wich is connected with four cables, but because it's running all live-systems right now I'm testing with that machine.
Is there something fundamentally wrong with my configuration?
Any help appreciated...
Regards from Aachen, Germany,
Thomas
Last edited: