I'm trying to hunt down a network throughput issue I'm having in my virtualized environment. It was suggested that perhaps my OVS config could be to blame and so I wanted to get a sanity check on what I'm working with.
X-post: https://www.reddit.com/r/Proxmox/comments/r9jbci/low_throughput_across_vlans_virtualized_pfsense/
X-post: https://www.reddit.com/r/Proxmox/co..._vlans_virtualized_pfsense/hndeeqq/?context=3
I am getting <200Mbps throughput across VLANs and am looking for help. There are no network devices between the computers being tested, they're on the same physical machine along with the pfsense VM and all share the same NIC.
Here you see Untagged Hypervisor <-> Tagged VM. Low Bandwidth and high Rtry
Additional Details:
Yes I followed the config per here: https://docs.netgate.com/pfsense/en/latest/recipes/virtualize-proxmox-ve.html and have disabled hardware checksums
Network Config Details:
Hypervisor
VM
Additional Troubleshooting steps:
- I ruled out a faulty NIC. I get 1G throughput and no retries when testing from Hypervisor <-> Untagged VM
- I performed a packet capture on pfSense in case that's helpful. I didn't see anything blaringly wrong on it but I have very little experience reading packet captures.
- I did have snort IDS enabled on the untagged interface, I disabled it and there was no difference
- I monitor my VMs via a Grafana dashboard, this isn't a case of the CPU being maxed out and limiting bandwidth.
- There are no limiters or traffic shaping going on.
Hardware/Software Details:
Broadcom Limited NetXtreme BCM5720 Gigabit Network Interface Card
pfSense v2.4.5-RELEASE-p1 - VM has 6 CPUs + 2GB ECC RAM dedicated on a Xeon E7 v2
Proxmox Virtual Environment v6.4-13
Open vSwitch v2.12.3
VMs were both running Ubuntu 20.04
X-post: https://www.reddit.com/r/Proxmox/comments/r9jbci/low_throughput_across_vlans_virtualized_pfsense/
X-post: https://www.reddit.com/r/Proxmox/co..._vlans_virtualized_pfsense/hndeeqq/?context=3
I am getting <200Mbps throughput across VLANs and am looking for help. There are no network devices between the computers being tested, they're on the same physical machine along with the pfsense VM and all share the same NIC.
Here you see Untagged Hypervisor <-> Tagged VM. Low Bandwidth and high Rtry
Code:
jon@Serv:~$ iperf -e -c zoneminder -n 250M
------------------------------------------------------------
Client connecting to zoneminder, TCP port 5001 with pid 23648
Write buffer size: 128 KByte
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[ 3] local 192.168.1.129 port 60166 connected with 192.168.3.10 port 5001
[ ID] Interval Transfer Bandwidth Write/Err Rtry Cwnd/RTT
[ 3] 0.00-10.93 sec 250 MBytes 192 Mbits/sec 2000/0 146 1030K/39198 usAdditional Details:
Yes I followed the config per here: https://docs.netgate.com/pfsense/en/latest/recipes/virtualize-proxmox-ve.html and have disabled hardware checksums
Network Config Details:
Hypervisor
VM
Additional Troubleshooting steps:
- I ruled out a faulty NIC. I get 1G throughput and no retries when testing from Hypervisor <-> Untagged VM
Code:
jon@TracheServ:~$ iperf -e -c nfs -n 250M
------------------------------------------------------------
Client connecting to nfs, TCP port 5001 with pid 16667
Write buffer size: 128 KByte
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[ 3] local 192.168.1.129 port 59384 connected with 192.168.1.139 port 5001
[ ID] Interval Transfer Bandwidth Write/Err Rtry Cwnd/RTT
[ 3] 0.00-0.19 sec 250 MBytes 11.1 Gbits/sec 2000/0 0 1240K/156 us- I performed a packet capture on pfSense in case that's helpful. I didn't see anything blaringly wrong on it but I have very little experience reading packet captures.
- I did have snort IDS enabled on the untagged interface, I disabled it and there was no difference
- I monitor my VMs via a Grafana dashboard, this isn't a case of the CPU being maxed out and limiting bandwidth.
- There are no limiters or traffic shaping going on.
Hardware/Software Details:
Broadcom Limited NetXtreme BCM5720 Gigabit Network Interface Card
pfSense v2.4.5-RELEASE-p1 - VM has 6 CPUs + 2GB ECC RAM dedicated on a Xeon E7 v2
Proxmox Virtual Environment v6.4-13
Open vSwitch v2.12.3
VMs were both running Ubuntu 20.04
Last edited:
