Hello,
I have 2 servers with each one Mellanox ConnectX-4 100 GbE QSFP installed. >I connected the 2 with a Mellanox 100gbe DAC cable. and both idracs show me a link with of 100gbit/s. However when I run ipref3 on both ends I only get around 34gbit/s. I have read that iperf3 might be limited to 1 cpu core only so I ran 3 parallel streams which results in the 3 streams getting 18-19gbit/s each so ~ 55gbit/s. which is better than 34gbit/s but far from 100gbits. what can I still try? do I have to enable any specific features in the drivers?
one of the servers is a
dell r730xd with dual Intel(R) Xeon(R) CPU E5-2690 v3 @ 2.60GHz and 128gb ecc ram
and the other a
dell r740xd with dual Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz and 512gb ecc ram
the dell r730xd is running truenas Scale latest version
the dell r740xd is running proxmox VE latest version
are there any driver settings or something I need to change? like HW offlodaing or similar?
on the r740xd with dual Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz I tried doing the grub config with
like stated here: https://forum.proxmox.com/threads/m...imited-to-bitrate-34gbits-s.96378/post-421663
but I can't seem to get it working. Even with the line added top the grub config, after reboot it still shows "9" instead of "0"
when running
cat /sys/module/intel_idle/parameters/max_cstate
any help is appreciated
thanks
I have 2 servers with each one Mellanox ConnectX-4 100 GbE QSFP installed. >I connected the 2 with a Mellanox 100gbe DAC cable. and both idracs show me a link with of 100gbit/s. However when I run ipref3 on both ends I only get around 34gbit/s. I have read that iperf3 might be limited to 1 cpu core only so I ran 3 parallel streams which results in the 3 streams getting 18-19gbit/s each so ~ 55gbit/s. which is better than 34gbit/s but far from 100gbits. what can I still try? do I have to enable any specific features in the drivers?
one of the servers is a
dell r730xd with dual Intel(R) Xeon(R) CPU E5-2690 v3 @ 2.60GHz and 128gb ecc ram
and the other a
dell r740xd with dual Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz and 512gb ecc ram
the dell r730xd is running truenas Scale latest version
the dell r740xd is running proxmox VE latest version
are there any driver settings or something I need to change? like HW offlodaing or similar?
on the r740xd with dual Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz I tried doing the grub config with
Code:
intel_idle.max_cstate=0 processor.max_cstate=0 idle=poll intel_pstate=disablelike stated here: https://forum.proxmox.com/threads/m...imited-to-bitrate-34gbits-s.96378/post-421663
but I can't seem to get it working. Even with the line added top the grub config, after reboot it still shows "9" instead of "0"
when running
cat /sys/module/intel_idle/parameters/max_cstate
any help is appreciated
thanks