Total bandwith capped

darkrow

New Member
May 29, 2012
1
0
1
Hello,
i am running proxmox 2.1-1, with a windows server 2003 guest and a debian openvz container.

The machine is not stressed much, but is in production.

This is the motherboard the machine is running on: http://www.asus.com/Motherboards/Intel_Socket_1155/P8Z68VGEN3/#specifications

The machine also has two extra NICS: http://www.zyxel.it/web/product_fam...yGroupNo=B889E035-2E0A-4E5A-8D9F-7DC24E52C2FD

I am using one of the two Zyxel nics as eth0. They have a chipset from Realtek

Code:
root@proxmox:~# lspci
00:00.0 Host bridge: Intel Corporation 2nd Generation Core Processor Family DRAM Controller (rev 09)
00:01.0 PCI bridge: Intel Corporation Xeon E3-1200/2nd Generation Core Processor Family PCI Express Root Port (rev 09)
00:02.0 VGA compatible controller: Intel Corporation 2nd Generation Core Processor Family Integrated Graphics Controller (rev 09)
00:16.0 Communication controller: Intel Corporation 6 Series/C200 Series Chipset Family MEI Controller #1 (rev 04)
00:19.0 Ethernet controller: Intel Corporation 82579V Gigabit Network Connection (rev 05)
00:1a.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #2 (rev 05)
00:1b.0 Audio device: Intel Corporation 6 Series/C200 Series Chipset Family High Definition Audio Controller (rev 05)
00:1c.0 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 1 (rev b5)
00:1c.1 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 2 (rev b5)
00:1c.2 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 3 (rev b5)
00:1c.3 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 4 (rev b5)
00:1c.4 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 5 (rev b5)
00:1c.6 PCI bridge: Intel Corporation 82801 PCI Bridge (rev b5)
00:1c.7 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 8 (rev b5)
00:1d.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #1 (rev 05)
00:1f.0 ISA bridge: Intel Corporation Z68 Express Chipset Family LPC Controller (rev 05)
00:1f.2 SATA controller: Intel Corporation 6 Series/C200 Series Chipset Family SATA AHCI Controller (rev 05)
00:1f.3 SMBus: Intel Corporation 6 Series/C200 Series Chipset Family SMBus Controller (rev 05)
03:00.0 USB controller: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller
05:00.0 IDE interface: JMicron Technology Corp. JMB362 SATA Controller (rev 10)
06:00.0 USB controller: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller
07:00.0 PCI bridge: ASMedia Technology Inc. ASM1083/1085 PCIe to PCI Bridge (rev 01)
08:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8169 Gigabit Ethernet (rev 10)
08:01.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8169 Gigabit Ethernet (rev 10)

I have the pve-firmware package installed and that takes care of the e1000 drivers for the Intel integrated NIC. I have downloaded and compiled the Realtek driver from realtek.com and it is functioning properly:

Code:
root@proxmox:~# lsmod | grep r8169
r8169                  40016  0
mii                     5287  1 r8169
root@proxmox:~# modinfo r8169
filename:       /lib/modules/2.6.32-12-pve/kernel/drivers/net/r8169.ko
version:        6.017.00-NAPI
license:        GPL
description:    RealTek RTL-8169 Gigabit Ethernet driver
author:         Realtek and the Linux r8169 crew <netdev@vger.kernel.org>
srcversion:     231149B837AF2B63F7C0271
alias:          pci:v00001186d00004300sv00001186sd00004C00bc*sc*i*
alias:          pci:v000010ECd00008169sv*sd*bc*sc*i*
alias:          pci:v000010ECd00008167sv*sd*bc*sc*i*
depends:
vermagic:       2.6.32-12-pve SMP mod_unload modversions
parm:           speed:force phy operation. Deprecated by ethtool (8). (array of int)
parm:           duplex:force phy operation. Deprecated by ethtool (8). (array of int)
parm:           autoneg:force phy operation. Deprecated by ethtool (8). (array of int)
parm:           rx_copybreak:Copy breakpoint for copy-only-tiny-frames (int)
parm:           use_dac:Enable PCI DAC. Unsafe on 32 bit PCI slot. (int)
parm:           debug:Debug verbosity level (0=none, ..., 16=all) (int)

The issue i am having is that the network speed gets capped at 0.16MBps
This is happening on both HOST and GUEST OSes

I managed to fix the issue by:
  • # ethtool -s eth0 speed 100 duplex full autoneg off
  • shutting down the machine
  • unplugging the cable from eth0
  • booting the machine fully
  • plugging back the patch cable

It reappeared less than 24 hours later.

The windows guest is using vmbr0 with virtio drivers (installed the binaries from redhat and patched the registry)
The OpenVZ container is also using vmbr0 (which uses eth0)

I do not suppose this is a cable issue, the machine is on a 3 meters cat5e cable and that is connected to a Cisco smart-managed GBit switch. I set the port to 100M Full Duplex and turned auto-negotiation off from the switch interface but this did not help.

I conducted tests with iperf and it clearly shows there is something capping the network speed:

Code:
bin/iperf.exe -c 192.168.1.26 -P 1 -i 1 -p 5001 -f M -t 10 -T 1
------------------------------------------------------------
Client connecting to 192.168.1.26, TCP port 5001
TCP window size: 0.01 MByte (default)
------------------------------------------------------------
[156] local 192.168.1.37 port 56270 connected with 192.168.1.26 port 5001
[ ID] Interval       Transfer     Bandwidth
[156]  0.0- 1.0 sec  0.24 MBytes  0.24 MBytes/sec
[156]  1.0- 2.0 sec  0.16 MBytes  0.16 MBytes/sec
[156]  2.0- 3.0 sec  0.16 MBytes  0.16 MBytes/sec
[156]  3.0- 4.0 sec  0.16 MBytes  0.16 MBytes/sec
[156]  4.0- 5.0 sec  0.16 MBytes  0.16 MBytes/sec
[156]  5.0- 6.0 sec  0.16 MBytes  0.16 MBytes/sec
[156]  6.0- 7.0 sec  0.16 MBytes  0.16 MBytes/sec
[156]  7.0- 8.0 sec  0.16 MBytes  0.16 MBytes/sec
[156]  8.0- 9.0 sec  0.16 MBytes  0.16 MBytes/sec
[156]  9.0-10.0 sec  0.16 MBytes  0.16 MBytes/sec
[156]  0.0-10.2 sec  1.66 MBytes  0.16 MBytes/sec
Done.

The above is on eth2 (the intel integrated network card), but the results are the same for eth0.

I'll paste the network configuration aswell:
Code:
root@proxmox:~# ifconfig -a
eth0      Link encap:Ethernet  HWaddr cc:5d:4e:38:90:16
          inet6 addr: fe80::ce5d:4eff:fe38:9016/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:75621398 errors:0 dropped:30661 overruns:0 frame:0
          TX packets:28785647 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:106106404013 (98.8 GiB)  TX bytes:4775175251 (4.4 GiB)
          Interrupt:18 Base address:0x2000

eth1      Link encap:Ethernet  HWaddr cc:5d:4e:38:8d:3c
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
          Interrupt:19 Base address:0x8000

eth2      Link encap:Ethernet  HWaddr c8:60:00:73:2e:a8
          inet addr:192.168.1.26  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::ca60:ff:fe73:2ea8/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:4334 errors:0 dropped:0 overruns:0 frame:0
          TX packets:56 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:420138 (410.2 KiB)  TX bytes:5352 (5.2 KiB)
          Interrupt:18 Memory:fbc00000-fbc20000

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:1790635 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1790635 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:306125264 (291.9 MiB)  TX bytes:306125264 (291.9 MiB)

tap100i0  Link encap:Ethernet  HWaddr 2e:90:e2:af:a4:5f
          inet6 addr: fe80::2c90:e2ff:feaf:a45f/64 Scope:Link
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
          RX packets:25975902 errors:0 dropped:0 overruns:0 frame:0
          TX packets:74042174 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500
          RX bytes:4061506816 (3.7 GiB)  TX bytes:105405234144 (98.1 GiB)

venet0    Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
          inet6 addr: fe80::1/128 Scope:Link
          UP BROADCAST POINTOPOINT RUNNING NOARP  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:3 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

veth101.0 Link encap:Ethernet  HWaddr 9a:09:8c:31:3e:8e
          inet6 addr: fe80::9809:8cff:fe31:3e8e/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:215457 errors:0 dropped:0 overruns:0 frame:0
          TX packets:206122 errors:0 dropped:357 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:298050566 (284.2 MiB)  TX bytes:21027303 (20.0 MiB)

vmbr0     Link encap:Ethernet  HWaddr cc:5d:4e:38:90:16
          inet addr:192.168.1.249  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::ce5d:4eff:fe38:9016/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1633427 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1145538 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:672732380 (641.5 MiB)  TX bytes:334730205 (319.2 MiB)

Please help me out on this, cause i am clueless on how to fix this permanently.