Hello guys i am doing a lot of testing with 3 supermicro servers as a replacement to vmware. Proxmox seems to fit the bill except for the simple fact the trough put from the host and VM is deplorable. now im not saying that proxmox behaves this way on all hardware but its all i am seeing. i have dual port 10gig intel x520-da2.
From the VM's the most i get on iperf is around 800mbs the host is at 1.5gps. I will add my config below i bonded the 2 10gig nics in lacp. i have also tried some tuning on the vm's with no result better than this. Does proxmox not like Intel network cards? Or is it the face the nics are pretty old? Any help would be welcome Thanks.
pve host iperf test from my pc
Connecting to host 10.0.100.4, port 5201
[ 4] local 10.0.9.6 port 50068 connected to 10.0.100.4 port 5201
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 184 MBytes 1.54 Gbits/sec
[ 4] 1.00-2.00 sec 184 MBytes 1.54 Gbits/sec
[ 4] 2.00-3.00 sec 185 MBytes 1.55 Gbits/sec
[ 4] 3.00-4.00 sec 186 MBytes 1.56 Gbits/sec
[ 4] 4.00-5.00 sec 185 MBytes 1.55 Gbits/sec
[ 4] 5.00-6.00 sec 185 MBytes 1.55 Gbits/sec
[ 4] 6.00-7.00 sec 180 MBytes 1.51 Gbits/sec
[ 4] 7.00-8.00 sec 170 MBytes 1.42 Gbits/sec
[ 4] 8.00-9.00 sec 175 MBytes 1.46 Gbits/sec
[ 4] 9.00-10.00 sec 186 MBytes 1.56 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 1.78 GBytes 1.53 Gbits/sec sender
[ 4] 0.00-10.00 sec 1.78 GBytes 1.53 Gbits/sec receiver
VM iperf test
Connecting to host 10.0.100.4, port 5201
[ 4] local 10.0.9.6 port 50068 connected to 10.0.100.4 port 5201
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 184 MBytes 1.54 Gbits/sec
[ 4] 1.00-2.00 sec 184 MBytes 1.54 Gbits/sec
[ 4] 2.00-3.00 sec 185 MBytes 1.55 Gbits/sec
[ 4] 3.00-4.00 sec 186 MBytes 1.56 Gbits/sec
[ 4] 4.00-5.00 sec 185 MBytes 1.55 Gbits/sec
[ 4] 5.00-6.00 sec 185 MBytes 1.55 Gbits/sec
[ 4] 6.00-7.00 sec 180 MBytes 1.51 Gbits/sec
[ 4] 7.00-8.00 sec 170 MBytes 1.42 Gbits/sec
[ 4] 8.00-9.00 sec 175 MBytes 1.46 Gbits/sec
[ 4] 9.00-10.00 sec 186 MBytes 1.56 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 1.78 GBytes 1.53 Gbits/sec sender
[ 4] 0.00-10.00 sec 1.78 GBytes 1.53 Gbits/sec receiver
auto lo
iface lo inet loopback
auto enp5s0f0
iface enp5s0f0 inet manual
iface eno1 inet manual
iface eno2 inet manual
auto enp5s0f1
iface enp5s0f1 inet manual
auto bond0
iface bond0 inet manual
bond-slaves enp5s0f0 enp5s0f1
bond-miimon 100
bond-mode 802.3ad
bond-xmit-hash-policy layer2+3
auto vmbr0
iface vmbr0 inet static
address 10.0.100.4/24
gateway 10.0.100.1
bridge-ports bond0
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4094
auto vlan50
iface vlan50 inet static
05:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
05:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
root@p1:~# ethtool -S enp5s0f0
NIC statistics:
rx_packets: 4323546
tx_packets: 829218
rx_bytes: 6244181053
tx_bytes: 92740826
rx_pkts_nic: 4323546
tx_pkts_nic: 829218
rx_bytes_nic: 6261917940
tx_bytes_nic: 100675757
lsc_int: 2
tx_busy: 0
non_eop_descs: 0
rx_errors: 0
tx_errors: 0
rx_dropped: 0
tx_dropped: 0
multicast: 4327
broadcast: 5491
rx_no_buffer_count: 0
collisions: 0
rx_over_errors: 0
rx_crc_errors: 0
rx_frame_errors: 0
hw_rsc_aggregated: 0
hw_rsc_flushed: 0
fdir_match: 4126093
fdir_miss: 189007
fdir_overflow: 0
rx_fifo_errors: 0
rx_missed_errors: 0
tx_aborted_errors: 0
tx_carrier_errors: 0
tx_fifo_errors: 0
tx_heartbeat_errors: 0
tx_timeout_count: 0
tx_restart_queue: 0
rx_length_errors: 0
rx_long_length_errors: 0
rx_short_length_errors: 0
tx_flow_control_xon: 0
rx_flow_control_xon: 0
tx_flow_control_xoff: 0
rx_flow_control_xoff: 0
rx_csum_offload_errors: 0
alloc_rx_page: 16538
alloc_rx_page_failed: 0
alloc_rx_buff_failed: 0
rx_no_dma_resources: 0
os2bmc_rx_by_bmc: 0
os2bmc_tx_by_bmc: 0
os2bmc_tx_by_host: 0
os2bmc_rx_by_host: 0
tx_hwtstamp_timeouts: 0
tx_hwtstamp_skipped: 0
rx_hwtstamp_cleared: 0
tx_ipsec: 0
rx_ipsec: 0
fcoe_bad_fccrc: 0
rx_fcoe_dropped: 0
rx_fcoe_packets: 0
rx_fcoe_dwords: 0
fcoe_noddp: 0
fcoe_noddp_ext_buff: 0
tx_fcoe_packets: 0
tx_fcoe_dwords: 0
tx_queue_0_packets: 77700
tx_queue_0_bytes: 6734318
tx_queue_1_packets: 41042
tx_queue_1_bytes: 7760651
tx_queue_2_packets: 129169
tx_queue_2_bytes: 9421084
tx_queue_3_packets: 69142
tx_queue_3_bytes: 7963403
tx_queue_4_packets: 24139
tx_queue_4_bytes: 3069657
tx_queue_5_packets: 28111
tx_queue_5_bytes: 4801145
tx_queue_6_packets: 35360
tx_queue_6_bytes: 4130059
tx_queue_7_packets: 77590
tx_queue_7_bytes: 5886461
tx_queue_8_packets: 51862
tx_queue_8_bytes: 5237961
tx_queue_9_packets: 27412
tx_queue_9_bytes: 2768962
tx_queue_10_packets: 19575
tx_queue_10_bytes: 4516051
tx_queue_11_packets: 13810
tx_queue_11_bytes: 2643773
tx_queue_12_packets: 77136
tx_queue_12_bytes: 6283101
tx_queue_13_packets: 35097
tx_queue_13_bytes: 3342546
tx_queue_14_packets: 96406
tx_queue_14_bytes: 11973286
tx_queue_15_packets: 17333
tx_queue_15_bytes: 4022562
tx_queue_16_packets: 248
tx_queue_16_bytes: 133372
tx_queue_17_packets: 166
tx_queue_17_bytes: 102686
tx_queue_18_packets: 614
tx_queue_18_bytes: 284073
tx_queue_19_packets: 107
tx_queue_19_bytes: 58222
tx_queue_20_packets: 444
tx_queue_20_bytes: 198484
tx_queue_21_packets: 11
tx_queue_21_bytes: 2436
tx_queue_22_packets: 249
tx_queue_22_bytes: 171508
tx_queue_23_packets: 58
tx_queue_23_bytes: 36510
tx_queue_24_packets: 159
tx_queue_24_bytes: 121666
tx_queue_25_packets: 65
tx_queue_25_bytes: 30519
tx_queue_26_packets: 3069
tx_queue_26_bytes: 243002
tx_queue_27_packets: 287
tx_queue_27_bytes: 317255
tx_queue_28_packets: 2379
tx_queue_28_bytes: 279459
tx_queue_29_packets: 137
tx_queue_29_bytes: 44255
tx_queue_30_packets: 104
tx_queue_30_bytes: 39059
tx_queue_31_packets: 237
tx_queue_31_bytes: 123300
tx_queue_32_packets: 0
tx_queue_32_bytes: 0
tx_queue_33_packets: 0
tx_queue_33_bytes: 0
tx_queue_34_packets: 0
tx_queue_34_bytes: 0
tx_queue_35_packets: 0
tx_queue_35_bytes: 0
tx_queue_36_packets: 0
tx_queue_36_bytes: 0
tx_queue_37_packets: 0
tx_queue_37_bytes: 0
tx_queue_38_packets: 0
tx_queue_38_bytes: 0
tx_queue_39_packets: 0
tx_queue_39_bytes: 0
tx_queue_40_packets: 0
tx_queue_40_bytes: 0
tx_queue_41_packets: 0
tx_queue_41_bytes: 0
tx_queue_42_packets: 0
tx_queue_42_bytes: 0
tx_queue_43_packets: 0
tx_queue_43_bytes: 0
tx_queue_44_packets: 0
tx_queue_44_bytes: 0
tx_queue_45_packets: 0
tx_queue_45_bytes: 0
tx_queue_46_packets: 0
tx_queue_46_bytes: 0
tx_queue_47_packets: 0
tx_queue_47_bytes: 0
tx_queue_48_packets: 0
tx_queue_48_bytes: 0
tx_queue_49_packets: 0
tx_queue_49_bytes: 0
tx_queue_50_packets: 0
tx_queue_50_bytes: 0
tx_queue_51_packets: 0
tx_queue_51_bytes: 0
tx_queue_52_packets: 0
tx_queue_52_bytes: 0
tx_queue_53_packets: 0
tx_queue_53_bytes: 0
tx_queue_54_packets: 0
tx_queue_54_bytes: 0
tx_queue_55_packets: 0
tx_queue_55_bytes: 0
tx_queue_56_packets: 0
tx_queue_56_bytes: 0
tx_queue_57_packets: 0
tx_queue_57_bytes: 0
tx_queue_58_packets: 0
tx_queue_58_bytes: 0
tx_queue_59_packets: 0
tx_queue_59_bytes: 0
tx_queue_60_packets: 0
tx_queue_60_bytes: 0
tx_queue_61_packets: 0
tx_queue_61_bytes: 0
tx_queue_62_packets: 0
tx_queue_62_bytes: 0
tx_queue_63_packets: 0
tx_queue_63_bytes: 0
rx_queue_0_packets: 89699
rx_queue_0_bytes: 118978225
rx_queue_1_packets: 187074
rx_queue_1_bytes: 275203826
rx_queue_2_packets: 795180
rx_queue_2_bytes: 1178942038
rx_queue_3_packets: 464245
rx_queue_3_bytes: 684758117
rx_queue_4_packets: 91940
rx_queue_4_bytes: 134380627
rx_queue_5_packets: 360893
rx_queue_5_bytes: 413090028
rx_queue_6_packets: 215327
rx_queue_6_bytes: 315754354
rx_queue_7_packets: 229326
rx_queue_7_bytes: 338777695
rx_queue_8_packets: 183658
rx_queue_8_bytes: 269688044
rx_queue_9_packets: 125053
rx_queue_9_bytes: 182689370
rx_queue_10_packets: 145318
rx_queue_10_bytes: 213278828
rx_queue_11_packets: 192756
rx_queue_11_bytes: 283601456
rx_queue_12_packets: 541484
rx_queue_12_bytes: 801871054
rx_queue_13_packets: 3685
rx_queue_13_bytes: 2299939
rx_queue_14_packets: 612523
rx_queue_14_bytes: 907331420
rx_queue_15_packets: 84420
rx_queue_15_bytes: 123184068
rx_queue_16_packets: 34
rx_queue_16_bytes: 4022
rx_queue_17_packets: 5
rx_queue_17_bytes: 410
rx_queue_18_packets: 246
rx_queue_18_bytes: 214392
rx_queue_19_packets: 39
rx_queue_19_bytes: 3384
rx_queue_20_packets: 274
rx_queue_20_bytes: 64886
rx_queue_21_packets: 0
rx_queue_21_bytes: 0
rx_queue_22_packets: 51
rx_queue_22_bytes: 8699
rx_queue_23_packets: 3
rx_queue_23_bytes: 272
rx_queue_24_packets: 80
rx_queue_24_bytes: 21438
rx_queue_25_packets: 22
rx_queue_25_bytes: 2024
rx_queue_26_packets: 68
rx_queue_26_bytes: 6180
rx_queue_27_packets: 15
rx_queue_27_bytes: 1070
rx_queue_28_packets: 19
rx_queue_28_bytes: 15491
rx_queue_29_packets: 0
rx_queue_29_bytes: 0
rx_queue_30_packets: 95
rx_queue_30_bytes: 8516
rx_queue_31_packets: 14
rx_queue_31_bytes: 1180
rx_queue_32_packets: 0
rx_queue_32_bytes: 0
rx_queue_33_packets: 0
rx_queue_33_bytes: 0
rx_queue_34_packets: 0
rx_queue_34_bytes: 0
rx_queue_35_packets: 0
rx_queue_35_bytes: 0
rx_queue_36_packets: 0
rx_queue_36_bytes: 0
rx_queue_37_packets: 0
rx_queue_37_bytes: 0
rx_queue_38_packets: 0
rx_queue_38_bytes: 0
rx_queue_39_packets: 0
rx_queue_39_bytes: 0
rx_queue_40_packets: 0
rx_queue_40_bytes: 0
rx_queue_41_packets: 0
rx_queue_41_bytes: 0
rx_queue_42_packets: 0
rx_queue_42_bytes: 0
rx_queue_43_packets: 0
rx_queue_43_bytes: 0
rx_queue_44_packets: 0
rx_queue_44_bytes: 0
rx_queue_45_packets: 0
rx_queue_45_bytes: 0
rx_queue_46_packets: 0
rx_queue_46_bytes: 0
rx_queue_47_packets: 0
rx_queue_47_bytes: 0
rx_queue_48_packets: 0
rx_queue_48_bytes: 0
rx_queue_49_packets: 0
rx_queue_49_bytes: 0
rx_queue_50_packets: 0
rx_queue_50_bytes: 0
rx_queue_51_packets: 0
rx_queue_51_bytes: 0
rx_queue_52_packets: 0
rx_queue_52_bytes: 0
rx_queue_53_packets: 0
rx_queue_53_bytes: 0
rx_queue_54_packets: 0
rx_queue_54_bytes: 0
rx_queue_55_packets: 0
rx_queue_55_bytes: 0
rx_queue_56_packets: 0
rx_queue_56_bytes: 0
rx_queue_57_packets: 0
rx_queue_57_bytes: 0
rx_queue_58_packets: 0
rx_queue_58_bytes: 0
rx_queue_59_packets: 0
rx_queue_59_bytes: 0
rx_queue_60_packets: 0
rx_queue_60_bytes: 0
rx_queue_61_packets: 0
rx_queue_61_bytes: 0
rx_queue_62_packets: 0
rx_queue_62_bytes: 0
rx_queue_63_packets: 0
rx_queue_63_bytes: 0
tx_pb_0_pxon: 0
tx_pb_0_pxoff: 0
tx_pb_1_pxon: 0
tx_pb_1_pxoff: 0
tx_pb_2_pxon: 0
tx_pb_2_pxoff: 0
tx_pb_3_pxon: 0
tx_pb_3_pxoff: 0
tx_pb_4_pxon: 0
tx_pb_4_pxoff: 0
tx_pb_5_pxon: 0
tx_pb_5_pxoff: 0
tx_pb_6_pxon: 0
tx_pb_6_pxoff: 0
tx_pb_7_pxon: 0
tx_pb_7_pxoff: 0
rx_pb_0_pxon: 0
rx_pb_0_pxoff: 0
rx_pb_1_pxon: 0
rx_pb_1_pxoff: 0
rx_pb_2_pxon: 0
rx_pb_2_pxoff: 0
rx_pb_3_pxon: 0
rx_pb_3_pxoff: 0
rx_pb_4_pxon: 0
rx_pb_4_pxoff: 0
rx_pb_5_pxon: 0
rx_pb_5_pxoff: 0
rx_pb_6_pxon: 0
rx_pb_6_pxoff: 0
rx_pb_7_pxon: 0
rx_pb_7_pxoff: 0
From the VM's the most i get on iperf is around 800mbs the host is at 1.5gps. I will add my config below i bonded the 2 10gig nics in lacp. i have also tried some tuning on the vm's with no result better than this. Does proxmox not like Intel network cards? Or is it the face the nics are pretty old? Any help would be welcome Thanks.
pve host iperf test from my pc
Connecting to host 10.0.100.4, port 5201
[ 4] local 10.0.9.6 port 50068 connected to 10.0.100.4 port 5201
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 184 MBytes 1.54 Gbits/sec
[ 4] 1.00-2.00 sec 184 MBytes 1.54 Gbits/sec
[ 4] 2.00-3.00 sec 185 MBytes 1.55 Gbits/sec
[ 4] 3.00-4.00 sec 186 MBytes 1.56 Gbits/sec
[ 4] 4.00-5.00 sec 185 MBytes 1.55 Gbits/sec
[ 4] 5.00-6.00 sec 185 MBytes 1.55 Gbits/sec
[ 4] 6.00-7.00 sec 180 MBytes 1.51 Gbits/sec
[ 4] 7.00-8.00 sec 170 MBytes 1.42 Gbits/sec
[ 4] 8.00-9.00 sec 175 MBytes 1.46 Gbits/sec
[ 4] 9.00-10.00 sec 186 MBytes 1.56 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 1.78 GBytes 1.53 Gbits/sec sender
[ 4] 0.00-10.00 sec 1.78 GBytes 1.53 Gbits/sec receiver
VM iperf test
Connecting to host 10.0.100.4, port 5201
[ 4] local 10.0.9.6 port 50068 connected to 10.0.100.4 port 5201
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 184 MBytes 1.54 Gbits/sec
[ 4] 1.00-2.00 sec 184 MBytes 1.54 Gbits/sec
[ 4] 2.00-3.00 sec 185 MBytes 1.55 Gbits/sec
[ 4] 3.00-4.00 sec 186 MBytes 1.56 Gbits/sec
[ 4] 4.00-5.00 sec 185 MBytes 1.55 Gbits/sec
[ 4] 5.00-6.00 sec 185 MBytes 1.55 Gbits/sec
[ 4] 6.00-7.00 sec 180 MBytes 1.51 Gbits/sec
[ 4] 7.00-8.00 sec 170 MBytes 1.42 Gbits/sec
[ 4] 8.00-9.00 sec 175 MBytes 1.46 Gbits/sec
[ 4] 9.00-10.00 sec 186 MBytes 1.56 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 1.78 GBytes 1.53 Gbits/sec sender
[ 4] 0.00-10.00 sec 1.78 GBytes 1.53 Gbits/sec receiver
auto lo
iface lo inet loopback
auto enp5s0f0
iface enp5s0f0 inet manual
iface eno1 inet manual
iface eno2 inet manual
auto enp5s0f1
iface enp5s0f1 inet manual
auto bond0
iface bond0 inet manual
bond-slaves enp5s0f0 enp5s0f1
bond-miimon 100
bond-mode 802.3ad
bond-xmit-hash-policy layer2+3
auto vmbr0
iface vmbr0 inet static
address 10.0.100.4/24
gateway 10.0.100.1
bridge-ports bond0
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4094
auto vlan50
iface vlan50 inet static
05:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
05:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
root@p1:~# ethtool -S enp5s0f0
NIC statistics:
rx_packets: 4323546
tx_packets: 829218
rx_bytes: 6244181053
tx_bytes: 92740826
rx_pkts_nic: 4323546
tx_pkts_nic: 829218
rx_bytes_nic: 6261917940
tx_bytes_nic: 100675757
lsc_int: 2
tx_busy: 0
non_eop_descs: 0
rx_errors: 0
tx_errors: 0
rx_dropped: 0
tx_dropped: 0
multicast: 4327
broadcast: 5491
rx_no_buffer_count: 0
collisions: 0
rx_over_errors: 0
rx_crc_errors: 0
rx_frame_errors: 0
hw_rsc_aggregated: 0
hw_rsc_flushed: 0
fdir_match: 4126093
fdir_miss: 189007
fdir_overflow: 0
rx_fifo_errors: 0
rx_missed_errors: 0
tx_aborted_errors: 0
tx_carrier_errors: 0
tx_fifo_errors: 0
tx_heartbeat_errors: 0
tx_timeout_count: 0
tx_restart_queue: 0
rx_length_errors: 0
rx_long_length_errors: 0
rx_short_length_errors: 0
tx_flow_control_xon: 0
rx_flow_control_xon: 0
tx_flow_control_xoff: 0
rx_flow_control_xoff: 0
rx_csum_offload_errors: 0
alloc_rx_page: 16538
alloc_rx_page_failed: 0
alloc_rx_buff_failed: 0
rx_no_dma_resources: 0
os2bmc_rx_by_bmc: 0
os2bmc_tx_by_bmc: 0
os2bmc_tx_by_host: 0
os2bmc_rx_by_host: 0
tx_hwtstamp_timeouts: 0
tx_hwtstamp_skipped: 0
rx_hwtstamp_cleared: 0
tx_ipsec: 0
rx_ipsec: 0
fcoe_bad_fccrc: 0
rx_fcoe_dropped: 0
rx_fcoe_packets: 0
rx_fcoe_dwords: 0
fcoe_noddp: 0
fcoe_noddp_ext_buff: 0
tx_fcoe_packets: 0
tx_fcoe_dwords: 0
tx_queue_0_packets: 77700
tx_queue_0_bytes: 6734318
tx_queue_1_packets: 41042
tx_queue_1_bytes: 7760651
tx_queue_2_packets: 129169
tx_queue_2_bytes: 9421084
tx_queue_3_packets: 69142
tx_queue_3_bytes: 7963403
tx_queue_4_packets: 24139
tx_queue_4_bytes: 3069657
tx_queue_5_packets: 28111
tx_queue_5_bytes: 4801145
tx_queue_6_packets: 35360
tx_queue_6_bytes: 4130059
tx_queue_7_packets: 77590
tx_queue_7_bytes: 5886461
tx_queue_8_packets: 51862
tx_queue_8_bytes: 5237961
tx_queue_9_packets: 27412
tx_queue_9_bytes: 2768962
tx_queue_10_packets: 19575
tx_queue_10_bytes: 4516051
tx_queue_11_packets: 13810
tx_queue_11_bytes: 2643773
tx_queue_12_packets: 77136
tx_queue_12_bytes: 6283101
tx_queue_13_packets: 35097
tx_queue_13_bytes: 3342546
tx_queue_14_packets: 96406
tx_queue_14_bytes: 11973286
tx_queue_15_packets: 17333
tx_queue_15_bytes: 4022562
tx_queue_16_packets: 248
tx_queue_16_bytes: 133372
tx_queue_17_packets: 166
tx_queue_17_bytes: 102686
tx_queue_18_packets: 614
tx_queue_18_bytes: 284073
tx_queue_19_packets: 107
tx_queue_19_bytes: 58222
tx_queue_20_packets: 444
tx_queue_20_bytes: 198484
tx_queue_21_packets: 11
tx_queue_21_bytes: 2436
tx_queue_22_packets: 249
tx_queue_22_bytes: 171508
tx_queue_23_packets: 58
tx_queue_23_bytes: 36510
tx_queue_24_packets: 159
tx_queue_24_bytes: 121666
tx_queue_25_packets: 65
tx_queue_25_bytes: 30519
tx_queue_26_packets: 3069
tx_queue_26_bytes: 243002
tx_queue_27_packets: 287
tx_queue_27_bytes: 317255
tx_queue_28_packets: 2379
tx_queue_28_bytes: 279459
tx_queue_29_packets: 137
tx_queue_29_bytes: 44255
tx_queue_30_packets: 104
tx_queue_30_bytes: 39059
tx_queue_31_packets: 237
tx_queue_31_bytes: 123300
tx_queue_32_packets: 0
tx_queue_32_bytes: 0
tx_queue_33_packets: 0
tx_queue_33_bytes: 0
tx_queue_34_packets: 0
tx_queue_34_bytes: 0
tx_queue_35_packets: 0
tx_queue_35_bytes: 0
tx_queue_36_packets: 0
tx_queue_36_bytes: 0
tx_queue_37_packets: 0
tx_queue_37_bytes: 0
tx_queue_38_packets: 0
tx_queue_38_bytes: 0
tx_queue_39_packets: 0
tx_queue_39_bytes: 0
tx_queue_40_packets: 0
tx_queue_40_bytes: 0
tx_queue_41_packets: 0
tx_queue_41_bytes: 0
tx_queue_42_packets: 0
tx_queue_42_bytes: 0
tx_queue_43_packets: 0
tx_queue_43_bytes: 0
tx_queue_44_packets: 0
tx_queue_44_bytes: 0
tx_queue_45_packets: 0
tx_queue_45_bytes: 0
tx_queue_46_packets: 0
tx_queue_46_bytes: 0
tx_queue_47_packets: 0
tx_queue_47_bytes: 0
tx_queue_48_packets: 0
tx_queue_48_bytes: 0
tx_queue_49_packets: 0
tx_queue_49_bytes: 0
tx_queue_50_packets: 0
tx_queue_50_bytes: 0
tx_queue_51_packets: 0
tx_queue_51_bytes: 0
tx_queue_52_packets: 0
tx_queue_52_bytes: 0
tx_queue_53_packets: 0
tx_queue_53_bytes: 0
tx_queue_54_packets: 0
tx_queue_54_bytes: 0
tx_queue_55_packets: 0
tx_queue_55_bytes: 0
tx_queue_56_packets: 0
tx_queue_56_bytes: 0
tx_queue_57_packets: 0
tx_queue_57_bytes: 0
tx_queue_58_packets: 0
tx_queue_58_bytes: 0
tx_queue_59_packets: 0
tx_queue_59_bytes: 0
tx_queue_60_packets: 0
tx_queue_60_bytes: 0
tx_queue_61_packets: 0
tx_queue_61_bytes: 0
tx_queue_62_packets: 0
tx_queue_62_bytes: 0
tx_queue_63_packets: 0
tx_queue_63_bytes: 0
rx_queue_0_packets: 89699
rx_queue_0_bytes: 118978225
rx_queue_1_packets: 187074
rx_queue_1_bytes: 275203826
rx_queue_2_packets: 795180
rx_queue_2_bytes: 1178942038
rx_queue_3_packets: 464245
rx_queue_3_bytes: 684758117
rx_queue_4_packets: 91940
rx_queue_4_bytes: 134380627
rx_queue_5_packets: 360893
rx_queue_5_bytes: 413090028
rx_queue_6_packets: 215327
rx_queue_6_bytes: 315754354
rx_queue_7_packets: 229326
rx_queue_7_bytes: 338777695
rx_queue_8_packets: 183658
rx_queue_8_bytes: 269688044
rx_queue_9_packets: 125053
rx_queue_9_bytes: 182689370
rx_queue_10_packets: 145318
rx_queue_10_bytes: 213278828
rx_queue_11_packets: 192756
rx_queue_11_bytes: 283601456
rx_queue_12_packets: 541484
rx_queue_12_bytes: 801871054
rx_queue_13_packets: 3685
rx_queue_13_bytes: 2299939
rx_queue_14_packets: 612523
rx_queue_14_bytes: 907331420
rx_queue_15_packets: 84420
rx_queue_15_bytes: 123184068
rx_queue_16_packets: 34
rx_queue_16_bytes: 4022
rx_queue_17_packets: 5
rx_queue_17_bytes: 410
rx_queue_18_packets: 246
rx_queue_18_bytes: 214392
rx_queue_19_packets: 39
rx_queue_19_bytes: 3384
rx_queue_20_packets: 274
rx_queue_20_bytes: 64886
rx_queue_21_packets: 0
rx_queue_21_bytes: 0
rx_queue_22_packets: 51
rx_queue_22_bytes: 8699
rx_queue_23_packets: 3
rx_queue_23_bytes: 272
rx_queue_24_packets: 80
rx_queue_24_bytes: 21438
rx_queue_25_packets: 22
rx_queue_25_bytes: 2024
rx_queue_26_packets: 68
rx_queue_26_bytes: 6180
rx_queue_27_packets: 15
rx_queue_27_bytes: 1070
rx_queue_28_packets: 19
rx_queue_28_bytes: 15491
rx_queue_29_packets: 0
rx_queue_29_bytes: 0
rx_queue_30_packets: 95
rx_queue_30_bytes: 8516
rx_queue_31_packets: 14
rx_queue_31_bytes: 1180
rx_queue_32_packets: 0
rx_queue_32_bytes: 0
rx_queue_33_packets: 0
rx_queue_33_bytes: 0
rx_queue_34_packets: 0
rx_queue_34_bytes: 0
rx_queue_35_packets: 0
rx_queue_35_bytes: 0
rx_queue_36_packets: 0
rx_queue_36_bytes: 0
rx_queue_37_packets: 0
rx_queue_37_bytes: 0
rx_queue_38_packets: 0
rx_queue_38_bytes: 0
rx_queue_39_packets: 0
rx_queue_39_bytes: 0
rx_queue_40_packets: 0
rx_queue_40_bytes: 0
rx_queue_41_packets: 0
rx_queue_41_bytes: 0
rx_queue_42_packets: 0
rx_queue_42_bytes: 0
rx_queue_43_packets: 0
rx_queue_43_bytes: 0
rx_queue_44_packets: 0
rx_queue_44_bytes: 0
rx_queue_45_packets: 0
rx_queue_45_bytes: 0
rx_queue_46_packets: 0
rx_queue_46_bytes: 0
rx_queue_47_packets: 0
rx_queue_47_bytes: 0
rx_queue_48_packets: 0
rx_queue_48_bytes: 0
rx_queue_49_packets: 0
rx_queue_49_bytes: 0
rx_queue_50_packets: 0
rx_queue_50_bytes: 0
rx_queue_51_packets: 0
rx_queue_51_bytes: 0
rx_queue_52_packets: 0
rx_queue_52_bytes: 0
rx_queue_53_packets: 0
rx_queue_53_bytes: 0
rx_queue_54_packets: 0
rx_queue_54_bytes: 0
rx_queue_55_packets: 0
rx_queue_55_bytes: 0
rx_queue_56_packets: 0
rx_queue_56_bytes: 0
rx_queue_57_packets: 0
rx_queue_57_bytes: 0
rx_queue_58_packets: 0
rx_queue_58_bytes: 0
rx_queue_59_packets: 0
rx_queue_59_bytes: 0
rx_queue_60_packets: 0
rx_queue_60_bytes: 0
rx_queue_61_packets: 0
rx_queue_61_bytes: 0
rx_queue_62_packets: 0
rx_queue_62_bytes: 0
rx_queue_63_packets: 0
rx_queue_63_bytes: 0
tx_pb_0_pxon: 0
tx_pb_0_pxoff: 0
tx_pb_1_pxon: 0
tx_pb_1_pxoff: 0
tx_pb_2_pxon: 0
tx_pb_2_pxoff: 0
tx_pb_3_pxon: 0
tx_pb_3_pxoff: 0
tx_pb_4_pxon: 0
tx_pb_4_pxoff: 0
tx_pb_5_pxon: 0
tx_pb_5_pxoff: 0
tx_pb_6_pxon: 0
tx_pb_6_pxoff: 0
tx_pb_7_pxon: 0
tx_pb_7_pxoff: 0
rx_pb_0_pxon: 0
rx_pb_0_pxoff: 0
rx_pb_1_pxon: 0
rx_pb_1_pxoff: 0
rx_pb_2_pxon: 0
rx_pb_2_pxoff: 0
rx_pb_3_pxon: 0
rx_pb_3_pxoff: 0
rx_pb_4_pxon: 0
rx_pb_4_pxoff: 0
rx_pb_5_pxon: 0
rx_pb_5_pxoff: 0
rx_pb_6_pxon: 0
rx_pb_6_pxoff: 0
rx_pb_7_pxon: 0
rx_pb_7_pxoff: 0
