pfSense - Fast on bare metal, slow on PVE

[scsirob]

For my homelab I have used a generic firewall with pfSense for years. My current Internet speed is 1Gbit. Upgrading to faster hardware in preparation to ftth, I got a barebones miniserver with 3x 2.5Gbit and 2x 10Gbit SFP+, 4-core N150 CPU with 16G memory. I'm not using the 10Gbit NICs for now.

PfSense bare metal on this hardware has no trouble handling the full 1Gbit Internet on the 2.5Gbit NICs, the CPU doesn't get over 10% load

I configured Proxmox VE 9.0.11 on this server, and added a pfSense VM with PCI passthrough to 2x 2.5Gbit NICs and one bridge to connect Proxmox management and the LAN interface for pfSense on the 3rd NIC. Assigned 4 vCPUs and 4GB memory. The CPU seems to do just fine, but I can't get beyond 300Mbit/s on this setup. I disabled the hardware accelleration as documented by Netgate. No errors or warnings anywhere. What would be the first steps to diagnose this?

 

[bmj]

Is your VM set up with CPU-Type host, and Chipset Q35? If you use i440fx instead of q35, it can only do PCI passthrough instead of PCIe passthrough, so it will probably be slower as the host has to do some extra work.

But Paravirtualized interfaces should be faster anyway, but the Problem is that pfSense cant do multiqueuing with its vtnet driver. Either you can use OPNsense, or you need to use a modified kernel for pfsense.

 

[scsirob]

Thanks for the reply. The VM is configured as CPU type host, chipset Q35. I have tried disabling PCIe passthrough and use paravirtualized interfaces instead, no improvement. I have now tried OPNsense as well, used the system tunables and settings recommended by others. CPU now at 50 - 60% but none seems to matter for the network throughput, the setup tops out at 300Mbit/s.

As this is the main firewall to the house I have unfortunately moved back to pfSense on bare metal. It saturates my 1Gbit line at less than 10% CPU without a sweat.

 

[proxuser77]

have unfortunately moved back to pfSense on bare metal

It makes more sense anyway. ;-) I mean, I understand that it's 2025 and we all want to protect the environment, and that energy is becoming more expensive. However, keeping the router separate from the rest of the infrastructure still seems like a good idea to me.