ProxMox VE 8.2.2 and Broadcom 25GB MTU settings?

K

kevin.p

New Member
Apr 4, 2024
2
0
1
I am running some Dell R650 servers with Broadcom broadcom adv. quad 25gb NICs BCM957504-N425G

I know I have the jumbo MTU set correctly on storage, switches, etc. In ProxMox I have the MTU set to 9216 on the interfaces, OVS Bridge, and OVS IntPorts.

Cluster and storage are set to go over these networks.

I feel like I am not getting proper VM migration or Disk Migration performance and I want to check the MTU on this 25GB NIC. I have not found a setting on the NIC BIOS that says anything about the MTU, nor have I found documentation about what the default MTU is.

On VM and disk migrations, I am getting about 120MiB/s.

Has anyone had any luck with these NICs?
 
If the performance is that low, then MTU is probably not your issue. Can you post the output of the following commands:

Code: 
ip a
cat /etc/network/interfaces
 
Sure thing , thanks for looking

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host noprefixroute
valid_lft forever preferred_lft forever
2: eno8303: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr0 state UP group default qlen 1000
link/ether a8:3c:a5:3b:2e:4a brd ff:ff:ff:ff:ff:ff
altname enp4s0f0
3: eno12399np0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9216 qdisc mq master ovs-system state UP group default qlen 1000
link/ether d4:04:e6:2c:39:d0 brd ff:ff:ff:ff:ff:ff
altname enp81s0f0np0
inet6 fe80::d604:e6ff:fe2c:39d0/64 scope link
valid_lft forever preferred_lft forever
4: eno12409np1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9216 qdisc mq master ovs-system state UP group default qlen 1000
link/ether d4:04:e6:2c:39:d1 brd ff:ff:ff:ff:ff:ff
altname enp81s0f1np1
inet6 fe80::d604:e6ff:fe2c:39d1/64 scope link
valid_lft forever preferred_lft forever
5: eno8403: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether a8:3c:a5:3b:2e:4b brd ff:ff:ff:ff:ff:ff
altname enp4s0f1
6: eno12419np2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
link/ether d4:04:e6:2c:39:d2 brd ff:ff:ff:ff:ff:ff
altname enp81s0f2np2
7: ens1f0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 20:3a:43:04:0c:48 brd ff:ff:ff:ff:ff:ff
altname enp138s0f0
8: eno12429np3: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
link/ether d4:04:e6:2c:39:d3 brd ff:ff:ff:ff:ff:ff
altname enp81s0f3np3
9: ens1f1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 20:3a:43:04:0c:49 brd ff:ff:ff:ff:ff:ff
altname enp138s0f1
10: ens1f2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 20:3a:43:04:0c:4a brd ff:ff:ff:ff:ff:ff
altname enp138s0f2
11: ens1f3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 20:3a:43:04:0c:4b brd ff:ff:ff:ff:ff:ff
altname enp138s0f3
12: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether c2:89:f0:f5:3c:ef brd ff:ff:ff:ff:ff:ff
13: vmbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9216 qdisc noqueue state UNKNOWN group default qlen 1000
link/ether d4:04:e6:2c:39:d0 brd ff:ff:ff:ff:ff:ff
inet6 fe80::c048:7ff:fe60:7149/64 scope link
valid_lft forever preferred_lft forever
14: vlan132_cluster: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9216 qdisc noqueue state UNKNOWN group default qlen 1000
link/ether ca:0f:e0:2b:8c:94 brd ff:ff:ff:ff:ff:ff
inet 10.107.132.12/24 scope global vlan132_cluster
valid_lft forever preferred_lft forever
inet6 fe80::c80f:e0ff:fe2b:8c94/64 scope link
valid_lft forever preferred_lft forever
15: vlan2110_nfs: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9216 qdisc noqueue state UNKNOWN group default qlen 1000
link/ether 86:f4:b4:92:9d:d9 brd ff:ff:ff:ff:ff:ff
inet 10.107.110.12/24 scope global vlan2110_nfs
valid_lft forever preferred_lft forever
inet6 fe80::84f4:b4ff:fe92:9dd9/64 scope link
valid_lft forever preferred_lft forever
16: bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9216 qdisc noqueue state UNKNOWN group default qlen 1000
link/ether ee:d1:f2:e9:6f:9e brd ff:ff:ff:ff:ff:ff
inet6 fe80::ecd1:f2ff:fee9:6f9e/64 scope link
valid_lft forever preferred_lft forever
17: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether a8:3c:a5:3b:2e:4a brd ff:ff:ff:ff:ff:ff
inet 10.107.121.12/24 scope global vmbr0
valid_lft forever preferred_lft forever
inet6 fe80::aa3c:a5ff:fe3b:2e4a/64 scope link
valid_lft forever preferred_lft forever

# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

iface eno8303 inet manual

iface ens1f2 inet manual

iface ens1f3 inet manual

iface eno8403 inet manual

auto eno12399np0
iface eno12399np0 inet manual
mtu 9216

auto eno12409np1
iface eno12409np1 inet manual
mtu 9216

auto eno12419np2
iface eno12419np2 inet manual

auto eno12429np3
iface eno12429np3 inet manual

iface ens1f0 inet manual

iface ens1f1 inet manual

auto vlan132_cluster
iface vlan132_cluster inet static
address 10.107.132.12/24
ovs_type OVSIntPort
ovs_bridge vmbr1
ovs_mtu 9216
ovs_options tag=132

auto vlan2110_nfs
iface vlan2110_nfs inet static
address 10.107.110.12/24
ovs_type OVSIntPort
ovs_bridge vmbr1
ovs_mtu 9216
ovs_options tag=2110

auto bond0
iface bond0 inet manual
ovs_bonds eno12399np0 eno12409np1
ovs_type OVSBond
ovs_bridge vmbr1
ovs_mtu 9216
ovs_options bond_mode=balance-tcp lacp=active

auto vmbr0
iface vmbr0 inet static
address 10.107.121.12/24
gateway 10.107.121.1
bridge-ports eno8303
bridge-stp off
bridge-fd 0

auto vmbr1
iface vmbr1 inet manual
ovs_type OVSBridge
ovs_ports bond0 vlan132_cluster vlan2110_nfs
ovs_mtu 9216

source /etc/network/interfaces.d/*
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom