Hi all,
I've been trying to troubleshoot an odd networking issue but have ended up a bit stumped.
There seems to be some similar, older threads, but they all either ended up unresolved or the problem seemed of a different nature.
When I test the network speed from a Linux VM to a different Linux machine on the local network, I get Network Speeds of a bit under 9 Gbit/s and many thousands of retries. This only applies to traffic originating FROM the VM - incoming traffic is at 9.4 Gbit/s with 0 retries.
The same test from VM to PVE Host and PVE Host to Linux Workstation is absolutely fine though, VM to VM is fine as well - in both directions.
It really only occurs VM -->> Linux Workstation and in no other configuration I've found so far.
What I've tried so far:
- Switch from my OpenvSwitch Configuration back to a simple non-vlan-aware Linux Bridge.
- Try different Clients on the Network to rule out an odd issue with the Workstation.
- Change the ethtool ring buffer size.
- Try a different VM with a different Linux Guest OS
The Machine is configured with:
- Epyc 8224
- 4x32GB DDR5 4800
- Mellanox ConnectX-4Lx connected via DAC Cables to Switch
- Micron 7450 Max NVME ZFS Mirror
The VMs and Linux Workstations are all running various versions of Debian or Fedora.
The VMs are using virtio NICs and the firewall feature is disabled.
I've been trying to troubleshoot an odd networking issue but have ended up a bit stumped.
There seems to be some similar, older threads, but they all either ended up unresolved or the problem seemed of a different nature.
When I test the network speed from a Linux VM to a different Linux machine on the local network, I get Network Speeds of a bit under 9 Gbit/s and many thousands of retries. This only applies to traffic originating FROM the VM - incoming traffic is at 9.4 Gbit/s with 0 retries.
The same test from VM to PVE Host and PVE Host to Linux Workstation is absolutely fine though, VM to VM is fine as well - in both directions.
It really only occurs VM -->> Linux Workstation and in no other configuration I've found so far.
What I've tried so far:
- Switch from my OpenvSwitch Configuration back to a simple non-vlan-aware Linux Bridge.
- Try different Clients on the Network to rule out an odd issue with the Workstation.
- Change the ethtool ring buffer size.
- Try a different VM with a different Linux Guest OS
The Machine is configured with:
- Epyc 8224
- 4x32GB DDR5 4800
- Mellanox ConnectX-4Lx connected via DAC Cables to Switch
- Micron 7450 Max NVME ZFS Mirror
The VMs and Linux Workstations are all running various versions of Debian or Fedora.
The VMs are using virtio NICs and the firewall feature is disabled.
iperf3 from VM to Linux Workstation:
iperf3 from VM to PVE Host:
iperf3 from PVE Host to Linux Workstation:
Code:
Connecting to host 10.0.20.120, port 5201
[ 5] local 10.0.20.45 port 57932 connected to 10.0.20.120 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 831 MBytes 6.96 Gbits/sec 4078 1010 KBytes
[ 5] 1.00-2.00 sec 1.03 GBytes 8.87 Gbits/sec 1397 1.38 MBytes
[ 5] 2.00-3.00 sec 1.06 GBytes 9.07 Gbits/sec 3310 1.14 MBytes
[ 5] 3.00-4.00 sec 1.06 GBytes 9.14 Gbits/sec 2849 899 KBytes
[ 5] 4.00-5.00 sec 1.04 GBytes 8.97 Gbits/sec 2965 1.07 MBytes
[ 5] 5.00-6.00 sec 1.04 GBytes 8.94 Gbits/sec 3633 1022 KBytes
[ 5] 6.00-7.00 sec 1.04 GBytes 8.92 Gbits/sec 918 1.40 MBytes
[ 5] 7.00-8.00 sec 1.07 GBytes 9.18 Gbits/sec 4654 1.05 MBytes
[ 5] 8.00-9.00 sec 1.04 GBytes 8.91 Gbits/sec 792 1.56 MBytes
[ 5] 9.00-10.00 sec 1.04 GBytes 8.94 Gbits/sec 1312 1.40 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 10.2 GBytes 8.79 Gbits/sec 25908 sender
[ 5] 0.00-10.00 sec 10.2 GBytes 8.79 Gbits/sec receiver
iperf Done.iperf3 from VM to PVE Host:
Code:
Connecting to host 10.0.20.25, port 5201
[ 5] local 10.0.20.45 port 50124 connected to 10.0.20.25 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 1.43 GBytes 12.3 Gbits/sec 0 1.83 MBytes
[ 5] 1.00-2.00 sec 1.54 GBytes 13.2 Gbits/sec 0 1.83 MBytes
[ 5] 2.00-3.00 sec 1.74 GBytes 15.0 Gbits/sec 0 1.83 MBytes
[ 5] 3.00-4.00 sec 1.53 GBytes 13.2 Gbits/sec 0 1.83 MBytes
[ 5] 4.00-5.00 sec 1.49 GBytes 12.8 Gbits/sec 0 1.83 MBytes
[ 5] 5.00-6.00 sec 1.66 GBytes 14.3 Gbits/sec 0 1.94 MBytes
[ 5] 6.00-7.00 sec 1.38 GBytes 11.8 Gbits/sec 0 1.94 MBytes
[ 5] 7.00-8.00 sec 1.35 GBytes 11.6 Gbits/sec 0 1.94 MBytes
[ 5] 8.00-9.00 sec 680 MBytes 5.71 Gbits/sec 0 1.94 MBytes
[ 5] 9.00-10.00 sec 1.45 GBytes 12.5 Gbits/sec 0 1.94 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 15.6 GBytes 13.4 Gbits/sec 0 sender
[ 5] 0.00-10.00 sec 15.6 GBytes 13.4 Gbits/sec receiver
iperf Done.iperf3 from PVE Host to Linux Workstation:
Code:
Connecting to host 10.0.20.120, port 5201
[ 5] local 10.0.20.25 port 47834 connected to 10.0.20.120 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 1.09 GBytes 9.38 Gbits/sec 27 1.20 MBytes
[ 5] 1.00-2.00 sec 1.09 GBytes 9.40 Gbits/sec 0 1.20 MBytes
[ 5] 2.00-3.00 sec 1.09 GBytes 9.38 Gbits/sec 0 1.34 MBytes
[ 5] 3.00-4.00 sec 1.09 GBytes 9.40 Gbits/sec 0 1.49 MBytes
[ 5] 4.00-5.00 sec 1.09 GBytes 9.38 Gbits/sec 0 1.51 MBytes
[ 5] 5.00-6.00 sec 1.09 GBytes 9.40 Gbits/sec 0 1.51 MBytes
[ 5] 6.00-7.00 sec 1.09 GBytes 9.40 Gbits/sec 0 1.51 MBytes
[ 5] 7.00-8.00 sec 1.09 GBytes 9.38 Gbits/sec 0 1.51 MBytes
[ 5] 8.00-9.00 sec 1.09 GBytes 9.40 Gbits/sec 0 1.51 MBytes
[ 5] 9.00-10.00 sec 1.09 GBytes 9.38 Gbits/sec 0 1.51 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 10.9 GBytes 9.39 Gbits/sec 27 sender
[ 5] 0.00-10.00 sec 10.9 GBytes 9.39 Gbits/sec receiver
iperf Done.
Last edited: