[SOLVED] Very poor pfsense performance with PCI passthrough

[uncleiroh]

Okay, so I have been able to pass-through my intel nic card to every single guest linux operating system just fine, and performance has been great. I can install/boot pfsense perfectly fine without having my nic passed through to it, but right when I pass through the ethernet card, the performance of the vm is horrible, the boot times are abysmal, and the cpu is always at 25-50% at all times.

I find this very odd, because I was able to run a pfsense vm on another host os on the same machine with the same nic card passed through to it just fine. I've tried almost every solution available, save for disabling checksum, but the problem that I have is that I cannot even navigate to the page where I need to disable the checksum, and I'm not sure what the commandline argument is to disable checksum. Any help on this issue would be much appreciated.

 

[longshot902]

I also have pfsense virtualized on proxmox.

I had abysmal performance using virtio network drivers. Changed the network driver to e1000 and it fixed my performance network wise. I never checked if this affected CPU performance.

EDIT:
Wanted to include that I do not use any passthrough. Identical performance to physical pfsense hardware (before virtualized). I just setup two virtual nics using e1000 driver and it runs great.

 

[whitewater]

Hello uncleiroh,
why do you need to use PCI passthrough for pfsense ?
witch proxmox version do you use ?

i have 5 pfsense in production with pve 3.4.
i use virtual network card virtio, with pfsense 2.2.6. No problem.
Like longshot say, e1000 is better.

my experience, with pfsense version <= 2.1.5 : use e1000. better, stable. virtio isn't stable.
with last pfsense version, no problems since maybie 6 months (and bridged network card. no PCI passthrough).

 

[uncleiroh]

Hello uncleiroh,
why do you need to use PCI passthrough for pfsense ?
witch proxmox version do you use ?

I am on proxmox 4.1. I want PCI passthrough to isolate my network related tasks to the VM, I've also read that throughput isn't as good as pci passthrough. I am running ipcop right now with pci passthrough, and it's working well save for the lack of features.

I was using unraid before with pfsense running with no problems with pci passthrough, which I find pretty odd because unraid also uses kvm for virtualization.

 

[whitewater]

i must correct my post first.
with pfsense, since 2.2.x, i use virtio. no problem.
pve 3.4. i will test pve 4 later.

always on my experience, i have, on the same node, xivo and pfsense.
no voip lack performance.

pci passthrough should be better perfomance.
I do not know enough to give an opinion.

 

[uncleiroh]

@whitewater so it would be worthwhile to try different proxmox versions?

 

[whitewater]

at the moment, i haven't time enough to go with pve 4, but i will do it to have lastest version.
it's works with pve 3.4. So, i think it will work with pve 4.

to compare, if you have enought time, try with virtual network bridge, to see if you have same cpu use,
and / or pve 3.4 if you haven't got any success.

 

[uncleiroh]

to compare, if you have enought time, try with virtual network bridge, to see if you have same cpu use,
and / or pve 3.4 if you haven't got any success.

For reference, here is what I've added to my 106.conf file to use passthrough

hostpci0: 01:00

Just added 2 network devices to the machine and pfsense is booting fine with e1000 and virtio network bridging. It's using 1-3% of the CPU right now with network bridging, so I'm pretty convinced that it's the pci passthrough that is the issue.

 

[whitewater]

had you got see this wiki ?
https://pve.proxmox.com/wiki/Pci_passthrough

i had just see it and it mention

Note:

PCI passthrough is an experimental feature in Proxmox VE

So, don't use it in production environment.
it's ok with bridged, so, go ahead and give it a try

(all my virtual networks cards in pfsense are virtio model).

 

[uncleiroh]

**update**

I fixed it by adding the pvetest repo to my host and installing the latest kernel. Anyone looking in the future, here is how I did it:

echo 'deb http://download.proxmox.com/debian jessie pvetest' >> /etc/apt/sources.list
apt-get update
apt-get upgrade
apt-get install pve-kernel-4.2.8-1

and then I removed the bridges and restarted the machine.

 

[whitewater]

Thanks for your feedback

 

[martink]

for testing purposes we want to use pfsense 2.2.6 with a direct nic in proxmox 4.2

has anyone a valid and working configuration ? a working conf file ?

Regards

Martin

 

[quangnhut123]

I wonder to have a complete setup tutorial with pfsense 2.3 and proxmox 4.4 with virtio driver!

 

[whitewater]

hello quangnhut,
with last version pfsense and proxmox, you can do can install pfsense with virtio network card.
install it like a real hardware.
configure pppoe if needed, all network cards.

in my case, i must do this :
menu system, advanced, networking.
Hardware Checksum Offloading : enable this "Disable hardware checksum offload"
(disable by default)

try that, working for me like this, since maybie more a year.

 

[quangnhut123]

hello quangnhut,
with last version pfsense and proxmox, you can do can install pfsense with virtio network card.
install it like a real hardware.
configure pppoe if needed, all network cards.

in my case, i must do this :
menu system, advanced, networking.
Hardware Checksum Offloading : enable this "Disable hardware checksum offload"
(disable by default)

try that, working for me like this, since maybie more a year.

Hello you mean no need to do passthrough ?
Just use virtio network card ?

I try to do this on my vmware. I install proxmox on vmware with 2 Nic. I bridge 2 nic with vmbr0 and vmbr1 for eth0 and eth1 in proxmox. The eth0 is only one card have internet connection via Nat mode in vmware.
Then i install pfsense and assign 2 nic vmbr0 and vmbr1 as Wan and Lan. After setup complete i can go to pfsense and i create new vm in proxmox to test by point default gateway to.my pfsense but have no internet connection. Dhcp of pfsense can lease ip as usual but no internet at all.

When i try to use with virtio nic card for pfsense i cannot start up vm.

Could you help me to install pfsense to handle all connection before any of vm in proxmox ?

 

[whitewater]

I do not know what you're trying to do, but you get complicated to put proxmox on vmware.
Installs proxmox in bare metal and nothing else, even for tests.

 

[quangnhut123]

I do not know what you're trying to do, but you get complicated to put proxmox on vmware.
Installs proxmox in bare metal and nothing else, even for tests.

Ah i plan to do on lab first before buy new metal server

 

[whitewater]

For my part, I could not help you there, I do not know vmware and putting proxmox on top is complicating life.
You will lose less time using an old computer than testing with vmware.

 

[quangnhut123]

For my part, I could not help you there, I do not know vmware and putting proxmox on top is complicating life.
You will lose less time using an old computer than testing with vmware.

Ah just want to make sure how to setup right way. If it cannot work because hardware on top Vmware so we can move to another solution.
Thanks for your reply but can you tell me how to setup pfsense as vm in proxmox and handle all connection before in and out in proxmox VM ?