Help with High CPU Usage in OPNSense/Pfsense VM inside Proxmox

[saluteferux]

Hi everyone,
I'm experiencing an issue with high CPU usage in my OPNSense VM running inside Proxmox and could really use some help.
Setup Details:

• Host Configuration:
  • Proxmox Host
  • 64 GB RAM
  • Intel i7-8700 (mitigations=off)
  • IOMMU enabled (iommu=on and iommu=pt in /etc/kernel/cmdline)
  • Proxmox installed on SSD NVME in ZFS RAID0
• VM Configuration:
  • OPNSense VM
  • 8GB RAM
  • 1 socket, 4 CPU threads (using host cpu type, testing with others doesn't change anything)
  • Passing through 4 ports of Intel I226V 2.5GbE. PCI-E is on in the settings of pci devices.
  • Q35, OVMF (UEFI)

Issue:
When I run a speed test, downloading at speeds of 2.1 Gbps to 2.5 Gbps, Proxmox reports 80-100% CPU usage. However, inside the OPNSense dashboard, the CPU usage is reported as 5-20%. (Same tested with pfsense)
I’m trying to understand why the CPU usage is so high on Proxmox.
Things I've Tried:

• Ensured that IOMMU is enabled and configured properly.
• Verified that CPU mitigations are turned off for performance.
• Tested with Pfsense
• Tested OpenWRT and the CPU Usage doesn't go above 10-15% at full 2.5Gb speed so this is a problem with the senses...

Questions:

1. Why is there such a discrepancy between CPU usage reported by Proxmox and OPNSense?
2. Is there anything I can do to optimize or reduce the CPU usage on Proxmox when running high-speed network tasks?

Any insights or suggestions would be greatly appreciated!
Thanks in advance!

 

[KevinS]

login through the CLI and confirm the actual CPU usage, maybe the dashboard is not reporting the right stats.

 

[floh8]

there exist tuning guides:

https://binaryimpulse.com/2022/11/opnsense-performance-tuning-for-multi-gigabit-internet/
https://medium.com/@truvis.thornton...it-internet-and-better-speeds-in-cfc80c49c544
https://www.neelc.org/posts/multicore-pppoe/

 

[PHXX]

I noticed the same problem,
but only recently with the latest updates of opnsense (since the last major version probably).

I think I solved the problem (it is still under observation):
I was previously in CPU "host" and I went back to CPU "qemu64" (+ flag aes on).

Of course, also check in VM options that QEMU guest agent = ON
and that in opnsense the QEMU guest agent service is installed and enabled.