[SOLVED] PfSense and/or bridge limiting bandwidth ?

[alxml]

Hi All,

I'm going to go mad as i don't understand what's happening here,
got an homeserver and fiber connection but got an issue doing LAN 2 WAN debit tests.

Server is Ryzen 9 5950X and 32G ram

My Pfsense is on both LAN/WAN ( 4 cores : 1Gb Ram )
Go some VM's on LAN also, connected using Linux bridge ( using bond0 of 2 x 1Gbit NIC"s)
My wan is using my motherboard 2.5G
My Fiber connection is a 5G down / 700 mb UP

When i put a vm on WAN bridge ( direct to router : vmbr0), test on fast.com gives me the max value available 2.5Gbps)

When i do the same test using the same VM but link on vmbr1 aka LAN , mas speed is ~700mbps !
It seems that the bandwidth is limited like if i was connected using a switch , however i thought that given that my VM are on the same bridge (Pfsense + my VM on LAN Linux Bridge)

I tested iperfs3 also using my Pfsense as server and the debit is a way better than the one using internet

if i do the same on WAN interface of my PfSense as target


fast on vmbr1 (WAN):

fast on vmbr0 (LAN Through PfSense )




I don't understand as LAN is using virtIO( so 10Gbps announced) , enough CPU , memory ?
My PFsense is also i guess well configured :



in advance, thanks for your tips !

 

[boopzz]

I think the problem here is the kernel in the virtualisation.
Proxmox is running Linux and pfSense is running FreeBSD which is different. So there's essentially an emulation layer happening. What you could do to prove this is to run a firewall with a Linux kernel to prove it. Something like IPFire I believe is running the Linux kernel.

When I tested opnSense and IPFire years ago I had the same, IPFire was basically line speed, something like 970 (for me 1G) and on opnsense I was hitting between 700-800Mb but for me this is fine.
Also run opnSense rather than pfSense, it was forked years ago and is much better

 

[alxml]

Hi , Thanks , i moved to opnsense and noticed one thing.
If i disable ipv6 when i have the VM linked directly to WAN , the bandwidth drops to the same value as when i'm using the LAN through OpnSense,
I'm now trying to enable ipv6 on opnsense to check if that could solve my issue, as ipv6 not having NAT, it could explain the speed difference?
Thanks also for opnsense which seems much better and have a friendlier UI !

 

[alxml]

Hi, quick update as i configured my opnsense using ipv6 and now all issues are gone, bandwidth is now 2.5Gps on my whole Proxmox and all Vm (limited by NIC and my ISP Router for now)
Thanks for the advice to move to opnsense @boopzz.

 

[pkarc]

I did solve this by changing the VM Ethernet adapter Model, was using Intel E1000 just changed to VirtIO and now PfSense/VMs/CTs using full bandwidth