Proxmox & OPNsense - 10% of the performance vs. Bare Metal, what did I do wrong?

[optical]

Hi all, having some problems which I hope I can resolve because I *REALLY* want to run Proxmox on this machine and not be stuck with just OPNsense running on bare metal as it's infinitely less useful like this.

I have a super simple setup:

10gb port out on my ISP router and PPPoE credentials
Dual Port 2.5GbE NIC in my Proxmox machine, with OPNsense installed in a VM
When I run OPNsense on either live USB, or installed to bare metal, performance is fantastic and works exactly as intended:

As seen here, 2500Base-T is the link speed, and my speed tests are fantastic across any devices attached to the OPNsense - absolutely no problems observed


The settings on OPNsense ended up being very straight forward so I don't think I messed up any major settings between the two of them. They simply needed WAN port designation, then LAN. Then I run the setup wizard, and designate WAN to PPPoE IPv4 using my login & password and external IP is assigned with no issues in both situations


As far as I can tell, Proxmox is also able at the OS level to see everything as 2.5GbE with no problems. `ethtool` reports 2500Base-T just like it does on bare metal OPNsense


However now we see in our OPNsense installation the link speed is only 1000Base-T instead of the 2500Base-T it should be



And as we can see, my speeds have never been worse, this is even worse than the ISP router - it's exactly 10% of my full speed, should be 2500 and I get 250mbps

I'm willing to assume I simply did something wrong inside Proxmox itself or misconfigured the VM somehow, much appreciated in advance for any ideas!

Have a great day Proxmox crew!

 

[Neobin]

A good start would be, to show your actual VM-config (qm config VMID).

My first assumption, based on your screenshot, would be, that you are not using: host as CPU-type and another guess would be, that you are using: e1000 for your virtual NIC(s), instead of: virtio.

Additionally: What physical NICs are those exactly?

 

[Dunuin]

For bare metal performance, it would also be useful to PCI passthrough the two NICs into the OPNsense VM with a third (and maybe forth) NIC for PVE.

 

[optical]

A good start would be, to show your actual VM-config (qm config VMID).

My first assumption, based on your screenshot, would be, that you are not using: host as CPU-type and another guess would be, that you are using: e1000 for your virtual NIC(s), instead of: virtio.

Additionally: What physical NICs are those exactly?

Hi Neobin! Thanks for getting back to me. Here is the output of the config:


I am using host indeed, it's one of the first things I tried changing when I found sparse discussion on it via Google before making a forum post! And I also increased cpuunits based on some other advice in the same thread with no difference made.. You are right, it appears set to E1000, which I guess didn't even register with me at the time that that might mean "ethernet 1000mbps", oops, this was just a default value that I apparently have never thought to change in the past.. I am a novice user.. I will try switching this right away, I just am starting to try and figure this out again after a long day of work

The NIC is dual port i225-V, I had that written in my OP and somehow it got erased! My apologies!!

 

[optical]

For bare metal performance, it would also be useful to PCI passthrough the two NICs into the OPNsense VM with a third (and maybe forth) NIC for PVE.

Hi Dunuin, thanks for the reply.. I have no expansion slots available to add another NIC as it's a small form factor unit, unless something is possible with an empty USB port? Also have an available USB-C port too actually.. I'm not sure what to do

 

[tomtom13]

any progress on that ? I was considering doing a failover dual router on proxmox and I'm trying to figure out whenever those machines will need extra nics (not needing those would be beneficial !)

 

[Dunuin]

No additional NICs required in case there is a tagged vlan capable switch. Without that you would of cause need at least 2 NICs (WAN + LAN).

 

[tomtom13]

wait, he was running those of the bridges, does it mean that he's got a line throughput on vmbr ?

 

[spirit]

e1000 nic are software only, and should reach around 300mbit/s.

you need to use virtio nic to have acceleration.

 

[tomtom13]

That's great, BUT, I would like to either:
1. see the original poster comeback and state whenever they achived the original speed
2. see some rudimentary test results of BSD in VM using VIRTIO for both interfaces, under normal routing load.