Degraded Upload Speed in OPNsense VM

d3dl3g

New Member
Sep 10, 2023
14
0
1
hi all,

im going to post a copy/paste from my post on the OPNsense forum just to save time really, in the hope that you might help with things to check on the proxmox side. i state below that my suspicion is with OPN, buuut after some further tests, im not too certain any more. ive spooled a second OPN and tested on a total of 6 interfaces, but im seeing the same results, so im think it *may* be the way proxmox is handling traffic.

i have tested on 2 sperate proxmox servers (v8.2.4), both "pinned" to kernel "6.5.13-5-pve"im pretty sure at this stage ive ruled out hardware, but im unsure as to why suddenly upload speeds dropped off a cliff

Good evening everyone.

Im a little bit stumped by this issue.
Since Monday 17th at 13.00 (GMT) i have had degraded upload speeds. (see below for rudimentary table, taken from log export in speedtest widget, timestamps are in "epoch" format https://www.epochconverter.com/ )

I have a 900mbit FTTP service, and use mimugmail's speed test widget (https://github.com/mimugmail/opn-repo) to monitor speeds. every hour, on the hour, from within OPN. this is not an asyncronous connection its "900 down/900 up" (FTR i can regularly speed test at >900 down/ >800 up before this issue arose)

My network HW is ONT > HP Procurve switch > Dell Power Connect switch > Proxmox Server > OPNsense VM

Proxmox and OPNsense fully up-to-date with latest "stable" releases. neither are of the "enterprise" variety.
The issue has spanned OPNsense 24.1.8 into OPNsense 24.1.9 (release date for .9, i believe, was 18/06)

I have confirmed the following... Shut down all other VMs/CTs Within Proxmox, leaving ONLY OPN VM running. this resulted in no change to the poor upload speeds... iperf3 at any step in above chain results in maximum 1g throughput. when running iperf from ONT side to Proxmox Server i saw a max of 1300 retries but the throughput was there.
i further tested speeds with my ISP router at every stage that there was an ethernet connection... this resulted in >900down, >800up
Given the combination of these 2 results i think i can safely say my line and HW are good.
i talked with ISP, they informed me that they do not apply throttling to the line, regardless of usage or attached router. i have no reason to not believe them at this stage as the testing appears to confirm that the line speed is obtainable, at least with their kit.

My suspicion is now firmly with OPN. No settings were changed in relation to WAN, i did set a port in NAT Port Forward towards my reverse proxy, but thats it, since testing i have disabled this PF and confirmed the attached floating rule has also been disabled.

im not really sure of what to check/adjust. any help greatly appreciated and warmly recieved.

Timestamp DlSpeed UlSpeed Latency
1718600429 908.81 406 12.66
1718604028 843.66 361.76 15.37
1718607627 842.27 255.55 12.28
1718611225 899.97 266.33 9.04
1718614824 878.21 298.25 10.83
1718618436 737.92 277.4 12.54
1718622031 843.74 356.48 11.87
1718625635 837.8 404.11 14.78
1718629224 825 140.19 13.62
1718762425 909.74 22.65 12.45
1718766017 873.95 14.52 9.37
1718769621 841.62 15.98 10.1
1718773228 851.58 15.68 10.72
1718776830 904.69 25.16 14.81
1718780419 877.29 15.86 9.58
1718784027 798.76 7.31 16.35
1718787630 790.9 25.49 13.92
1718791227 487.07 21.18 12.8
1718794827 890.61 20.5 9.54

Speed test widget reports
3341 probes (and counting)
Avg Down:- 809.15 Mbps (min: 14.78 Mbps, max: 939.75 Mbps)
Avg Up:- 308.96 Mbps (min: 0.53 Mbps, max: 827.16 Mbps)
* "Avg Up" is skewed slightly due to multiple <30mbit results over the past couple of days, but only by around 50Mbps
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!