Getting Pfsense up and running in Proxmox

I

iPanini

Well-Known Member
Mar 30, 2019
56
2
48
65

about 2 hours ago

Hi,

I'm seeking some help with getting started with Pfsense running virtualized in Proxmox.
As this is a bit in between Pfsense and Proxmox, I have also posted in Netgate forum.

What I have:

  • Dell Optiplex 980, with i5 cpu
  • onboard nic
  • 2x Intel 1 GB nic
  • Proxmox 8.2.7
Network detail:
For the time being, I'd like to integrate the Pfsense vm in my local network. The range is 192.168.1.0/24.
192.168.1.1 is my router, therefore I have tried to configure my WAN nic as 192.168.1.2.

Here's a view of the hardware config in Proxmox:
2025-03-20 at 22.57.58_CleanShot.png

I have tried as much as possible to follow the Netgate Docs. At first I assumed there was a mixup between the WAN and LAN nic, but even switching and retesting did not help.
Whenever I get to the stage where Pfsense will check network connectivity, the install / connection fails.

I would like to get to a point where I can manually ping networks, LAN or WAN to see and find out what works or why it does not work.

Question:
Once exited from the installer, I land in the terminal.
What commands can I enter on the cli to:

  • check availability of the NIC's
  • manually check functioning of the NIC's by pinging e.g. my network modem
  • manually check the MAC address that is assigned to the 2 INTEL NIC's
Any further info that might help is hugely appreciated!
If you need me to post more info, please ask.

Thanks for helping out!

image of WAN nic
2025-03-20 at 23.03.02_CleanShot.png

image of LAN nic
2025-03-20 at 23.04.06_CleanShot.png
 
Hello,

With Pfsense/Opnsense you can see menu to do action in cli.
For full shell, select "8"

do you have this option ?
 
Just looking at the options, I see that "hardware" checksumming is turned on. That does not work because of an incompatibility with the vtnet drivers and FreeBSD. You can find the fix and more of such pitfalls here.
 
Last edited:
meyergru said:
Just looking at the options, I see that "hardware" checksumming is turned on. That does not work because of an incompatibility with the vtnet drivers and FreeBSD. You can find the fix and more of such pitfalls here.
Click to expand...
Thanks for helping out!

I've no clue as to where to disable cheksumming. Did search and the only thing a bit helpfull to me is this post at it-notes.dragas.net.
Where it says:
Code: 
To remedy the situation, I made the following modifications:

Added the following to /boot/loader.conf:
hw.vtnet.X.csum_disable=1
hw.vtnet.lro_disable=1
Integrated these lines into /etc/sysctl.conf:
net.link.bridge.pfil_member=0
net.link.bridge.pfil_bridge=0
net.link.bridge.pfil_onlyip=0
And appended to /etc/rc.local (which I already use for initialization):
ifconfig vtnet0 -rxcsum

1. Is this what your mean?
2. Where would I need to put that conf? In the /boot/loader.conf from the pfsense VM?
 
Can you log into the web UI at all?

In your WAN interface set up screen, you will need to uncheck the box at the bottom that says "Block private networks and loop back addresses" if you leave this checked, you will not be able to get a WAN IP address from your router.

Screenshot 2025-03-21 081247.png



What commands can I enter on the cli to:
  • check availability of the NIC's
  • manually check functioning of the NIC's by pinging e.g. my network modem
  • manually check the MAC address that is assigned to the 2 INTEL NIC's
Click to expand...

"ip -a" ought to give you the ip address and mac address of each interface. "ping 192.168.1.1" will let you ping your router. "ctrl-c" will rend the ping process
 
Also turn off the firewalls in Proxmox, at the VM and node and data center levels until you finish trouble shooting the issues. If, once you trouble shoot everything and you still want to use the Proxmox firewalls, you will need to add appropriate rules to pass and block traffic as desired. I generally let pfSense handle all my firewall rules, and leave the Proxmox firewalls disabled.

Also I think your life will be a lot easier if you don't create vmbr's for your NICs. If you have enough spare NICs in your machine it is far easier to simply pass those through to pfSense, and they will be used solely by pfSense, just the same as if you had a physical box running pfSense. The only downside is you have to connect more cables to your router and switches than you would otherwise need.

2025-03-20 at 22.57.58_CleanShot.png
 
Last edited:
Some more trying to better understand:
I have been spending quite some time following guides on the internet and such.

Here are my thoughts / questions:
I'm running on older hardware, so I wonder if it supports IOMMU. Did some searching and changed some settings in the BIOS
louie1961 said:
Also turn off the firewalls in Proxmox, at the VM and node and data center levels until you finish trouble shooting the issues. If, once you trouble shoot everything and you still want to use the Proxmox firewalls, you will need to add appropriate rules to pass and block traffic as desired. I generally let pfSense handle all my firewall rules, and leave the Proxmox firewalls disabled.

Also I think your life will be a lot easier if you don't create vmbr's for your NICs. If you have enough spare NICs in your machine it is far easier to simply pass those through to pfSense, and they will be used solely by pfSense, just the same as if you had a physical box running pfSense. The only downside is you have to connect more cables to your router and switches than you would otherwise need.

View attachment 83931
Click to expand...
louie1961 said:
Also I think your life will be a lot easier if you don't create vmbr's for your NICs. If you have enough spare NICs in your machine it is far easier to simply pass those through to pfSense, and they will be used solely by pfSense, just the same as if you had a physical box running pfSense. The only downside is you have to connect more cables to your router and switches than you would otherwise need.
Click to expand...
Thanks for chiming in!
Apparently I had not fully read your post, sorry for that.

In fact I was wondering about the same. It is just that quasi all the guides & examples that I found use vmbr's, that's why I went that route.

But.. After fully reading your reply, I followed suite, but then the vm will not boot:
Code: 
Error: start failed: QEMU exited with code 1

Do you have any idea what this means? How to solve / circumvent?
 
iPanini said:
Some more trying to better understand:
I have been spending quite some time following guides on the internet and such.

Here are my thoughts / questions:
I'm running on older hardware, so I wonder if it supports IOMMU. Did some searching and changed some settings in the BIOS


Thanks for chiming in!
Apparently I had not fully read your post, sorry for that.

In fact I was wondering about the same. It is just that quasi all the guides & examples that I found use vmbr's, that's why I went that route.

But.. After fully reading your reply, I followed suite, but then the vm will not boot:
Code: 
Error: start failed: QEMU exited with code 1

Do you have any idea what this means? How to solve / circumvent?
Click to expand...
Might it be because this system does not support IOMMU?
This is what my system shows in BIOS:
As far as I'm aware, everything related to Virtualization is enabled.
The single setting that was not enabled was this VT for Direct i/o, and as I am not familiar with IOMMU, I have guessed that this was approximately the same.
IMG_8010.jpg
 
For vtnet, you have to set only hw.vtnet.csum_disable=1 in /etc/loader.conf, not for each NIC individually, see: https://man.freebsd.org/cgi/man.cgi?vtnet(4). Do not forget to reboot afterwards and check if the checksum flags are off. This is only a temporary measure to get the interfaces running in order to be able to configure the firewall at all.

For OpnSense, this setting is the default anyway, just because it does not work and therefore, this is the better way to do it. I do not do pfSense, so I do not know where you can set tuneables in the GUI, though.

But there are more things that will not work out-of-the-box, just follow the link to see which these are.

BTW: If you do not have many NICs, pass-through will not help and is usually only done for WAN anyway. You probably do not want to tunnel traffic from one pass-through interface via a switch to the same Proxmox machine again when all of that can be done inside the machine without bottlenecks.
The recommendation to pass interfaces into VMs comes from darker times where vtnet was not as optimized as it is today. This is also covered in the linked guide.
 
Last edited:
Some more info:
- As a matter of fact, I started out following Techno Tim's youtube video. A bit later I realized that it dated May 16, 2020.. so that explained, things have obviously changed, and I could not get it to work.
- I have also been looking at youtube video's from Lawrence Systems, and he talks about differences between pfsense and opnsense.
- Then I set out to search what's available in the famous Proxmox helper scripts, there I found a script to install opnsense, so I tried that.

From my experience, neither is working, pfsense gets stuck in the set-up of WAN and LAN (using vmbr, not pasthrough) and fails during "Trying to reach the Netgate Servers".
OPNsense on the other hand, at least (in my opinion) finishes the setup completely. At least I can use the browser to login to the webinterface of OPNsense.
 
Update:

Ok, in the mean time I've done more experimenting and testing, during which the count of vm's is now
- 4 pfsense vm's
- 3 opensense vm's

What are the results?
- I've been experimenting a bit with Proxmox firewall, and for the time being I have following result:
- 1 opensense vm works (partly)
- this opensense vm is reachable on the LAN, by using it's web-interface
- in the web-interface, I can see things that work, e.g. web-interface > interfaces > diagnostics > dns lookup gives me correct replies when I enter a couple of known websites

What does not work (yet)?
- I have connected a hub to the LAN interface
- then a macbook to this hub
=> but the macbook does not get an IP address

Where can I check settings for DHCP serving?
I notice a section Kea DHCP, and there the IP range that I entered during setup shows correctly.
But my macbook does not get an IP address?
- In the "Leases" section, I do see a single IP address, notably the first of the assigned range (192.168.1.100) (range = 192.168.1.100 ~ 192.168.1.150), it shows as a android-blablabla (I do not use android, but may have some devices in the house that I'm not immediately familiar with)
- pinging this 192.168.1.100 device from my laptop works..

So, mission for now: how to debug this DHCP situation?
 
Follow this video. This is how I virtualized pfSense way back when, and just to make sure it still works, I downloaded the latest pfSense ISO and spun this up on my sandbox server this morning. It worked perfectly for me. I had two extra NICs lying around that I popped into my server and everything worked great.

https://www.youtube.com/watch?v=hdoBQNI_Ab8
 
louie1961 said:
Follow this video. This is how I virtualized pfSense way back when, and just to make sure it still works, I downloaded the latest pfSense ISO and spun this up on my sandbox server this morning. It worked perfectly for me. I had two extra NICs lying around that I popped into my server and everything worked great.

https://www.youtube.com/watch?v=hdoBQNI_Ab8
Click to expand...
Thanks!, I'll have a look! At the moment I'm partly up-and-running with Proxmox/opnsene...
Only DHCP does not seem to work, but I still need to do more testing.
 
iPanini said:
Thanks!, I'll have a look! At the moment I'm partly up-and-running with Proxmox/opnsene...
Only DHCP does not seem to work, but I still need to do more testing.
Click to expand...
Ha ha!, that's exactly the one I set out to follow. Does not work out of the box (anymore)
At least for me, pci pasthrough does not seem to function as expected => QEMU exited with code 1
I'm not aware what this error means, and can't seem to find proper explanation on the www.
 
For completnes sake:
I have attached a monitor to my Proxmox box, can't do a screengrab but here's at least a picture showing lspci.
IMG_8016.jpg

Further:
In the mean time I've done minimal reading on DHCP config of opnsense, this has learned me that there are 2 DHCP servers on-board, ISC DHCP and Kea.
I also learn that ISC DHCP is probably to get deprecated in the semi-near future.

So far it seems that on my system, either the LAN adapter or the DHCP on that LAN adapter is not working.
(how to eliminate / check the fact that LAN adapter / vtnet1 is properly working?)
Next step would then be to check functioning of DHCP (Kea)

Thanks for any tip or help!
 
You must log in or register to reply here.
Top Bottom