Straggling on setting Pfsense vm on Proxmox

[gdmax]

Hi Guys,
After a long research between several Hypervisor, i decided to go with Proxmox for my home network.

I started with Pfsense on a Phisical PC and it worked fine.
Now i am trying to install the Pfsense as VM on my new Dell T320 Server along side other Vm's such freenas, and gaming server.

But can't understand what i am missing out, tried all kind of tutorials arround with no luck.

My idea is that all trafic from the router, WAN will enter the Dell HV on the WAN Nic vtnet0,
goes through the Pfsense FW and out to vtnet1 to the Dell 5324 switch on trunk port number 24

On the Dell switch the Default Gateway is 192.168.1.1

I have attached my network separation pdf file of my planning

Any help or some directions would be appreciated

some details:
Using Proxmox 6.2-6
Having Dell 5324 switch with 3 vlans
Port: 1-10 Vlan-1 192.168.1.1 - 192.168.1.254
Port: 11-16 Vlan-2 192.168.50.1 - 192.168.50.254
Port: 17-23 Vlan-3 192.168.200.1 - 169.168.200.254
Port: 24 Trunk from FW vtnet1 NIC

 

[maverickws]

Hi there,

Both your interfaces are on the same /24 network. Your WAN and LAN cannot be the same /24 network.

1st define your gateway.
2nd define a network for all the machines behind the pfSense. Let's say it's network 192.168.254.0/24

Assign an interface (eg vmbr0) as WAN
Assign an interface (eg vmbr1) as LAN.

Now, neither one of these interfaces needs an IP set on the proxmox bridge. You'll give them addresses in pfSense. (note about where to put proxmox's ip below).
For WAN you'll assign an IP that all traffic will go through to reach the internet
For LAN you'll assign an unique network for the machines behind pfsense, and let the router give ip addresses via DHCP, or manually assign leases to mac addresses.

What it seems to me is you have an upstream router with the 192.168.1.0/24 network configured and you're handing out ip's from this range to all interfaces on the pfsense.
Configure the proxmox ip address on the eno1 interfaces, not the bridge. You can have your Proxmox eno1 interface with 192.168.1.100 and your pfSense WAN interface, set on the pfsense, 192.168.1.1.

192.168.1.1 is your WAN, and 192.168.1.100 is the upstream gateway. Just make sure 192.168.1.1 isn't your home router as well... that won't work.

 

[vsp2979]

Hi maverickws,

I am stuck in the similar situation, wondering if you can guide and I hope OP is fine with the question.

It makes sense to configure IP address on en01 interface, but where do you set WAN interface for PfSense? In PfSense interfaces or Proxmox bridges?

 

[maverickws]

It makes sense to configure IP address on en01 interface, but where do you set WAN interface for PfSense? In PfSense interfaces or Proxmox bridges?

Hi @vsp2979,

It depends on the configuration you choose. If you choose a bridged setup, when you create a vmbr interface you'll have it bridged to the physical address and you don't need to enable net.ipv4.ip_forward=0 (bc they're bridged);
If you choose a routed setup you must enable net.ipv4.ip_forward=1 and you also must add a route.

So in either way, my config is individual IP plus /29 subnet.
The individual ip is assigned to the physical interface, and to the vmbr interface I left the config empty.

Now here what I did to make my config a breeze was the following:

Since the gateway for the /29 is outside of the subnet, pfSense rejects it by default. You have to go on the webgui and allow a gateway outside of the subnet on the advanced settings. (literature: pfSense - routing/gateway settings)
So for this what I did was I created a second vmbr interface which will be the LAN interface. also without config all is done on the pfSense.
When you install the pfSense by default it has the LAN DHCP active, so using the console you create a VM (i used red hat linux), the network interface you assign to it is the same vmbr on the pfSense LAN side. When you boot this machine it'll get a LAN ip address, and you can go on the console, open the browser, navigate to the router address and finish the configuration.

in this given example I configure the WAN interface with an IP of the given /29 subnet etc the normal WAN config by the book.

In the meanwhile I would like to say I have ditched Proxmox since tom banned me from this forum for a week. I actually have to thank him, as I was making too many questions about failing features and bad implemented things, shit that just doesn't work well here. I don't know how many people here had to resort to the solution of formatting and reinstalling Proxmox hosts because things just break.
I installed XCP-ng, using XCP-ng Center with crossover/wine and xen-orchestra (trying both) but everything works marvellously. Got my pfSense there working like a charm, the /29 ip's are added as virtual ip's then I use 1:1 NAT to forward traffic.

But if you have more questions just drop a message if I can help!! have a nice one

 

[gdmax]

Thank for the initial help
Finally get it running connected to a simple WiFi router with some tweaks

But when I am trying to connect the Lan to A dell 5324 switch with some vlan's to the port 24 (trunk) with no success getting clients receive Dhcp ip's

Can't figure it out.
I'll give it more time, if can't solve this issues, i'll try my luck with different HV

Thanks

 

[vsp2979]

Appreciate the in-deapth response, maverickws. Will give it a go and respond back. Also, very glad to hear about your experience, will try it as well.

 

[bobmc]

Assuming your internet router is normally on your lan at 192.168.1.1 and is also your dhcp server. In this scenario a virtual instance of PFsense will become the new firewall and router for your LAN clients, physical, and virtual

Decide what the new network range will be for your LAN as trying to use 192.168.1.x on both will just cause you headaches. Say 192.168.10.x instead. Decide what IP you want to assign as your new network gateway (this will be PFSense) and what IP do you want your proxmox host to have - say we will use 192.168.10.254 as your PFSense address and 192.168.10.1 as your proxmox host. Assign 192.168.10.1 as the IP address of vmbr1 with a gateway of 192.168.10.254.

vmbr0 (WAN) does not need an IP address assigned within Proxmox.

Create a VM with two network interfaces - one on vmbr0 (this will be the WAN interface) and one on vmbr1 (which will be LAN). Connect your ISP router to the second network port on your proxmox host. Install PFSense and set the WAN interface to DHCP and set the LAN address to 192.168.10.254

Enable the DHCP server on PFSense with the address pool in the same network as the LAN - say 192.168.10.10 to 192.168.10.100 with a gateway of 192.168.10.254

You should now be able to create a VM or a container, with a network adapter on vmbr1 and they should get a DHCP lease from PFSense, the same should also apply to any physical client. vmbr0 will not be used for anything other than PFSense


If your internet router is also providing your wifi service then the Wifi clients will still be on the 192.168.1.x network and won't be able to talk to the LAN side clients

To implement VLAN's, you will need to
a) make sure that vmbr1 is 'VLAN aware' in the proxmox gui
b) create a vlan port in PFsense and assign it to the LAN port
c) create a dhcp pool for the vlan in PFsense
d) tag the switch port for the vlan for physical clients or tag the network interface when creating a VM or container

 

[gdmax]

Hi,
In order to know where my path is going south regarding Pfsense as Vm, I decided to try installing it on bare metal PC and try to make it work with my dell 5324 switch.

Later I'll post my setup hoping you guys see where I lost it.

I am thinking maybe my luck of knowledge is the issue here.

Sorry for that guys, but I am a quick learner

 

[gdmax]

Hi,
On the bare metal PC Pfsense
Finally i have got the missing part ( as mentioned, luck of knowledge on switches configuration)
When setting a Vlan's on the switch (where i have setup port 24 as Trunk )
I found out that on each Vlan it is necessary to mark port 24 as T (Trunk )

After doing so, it works like a charm
Now i need to check it on the Pfsense VM on Proxmox ( i'll post later if all went well )

Added a pick of how it should be.

 

[gdmax]

Some update.
The Pfsense FW is running on the Proxmox HV
All Vlan's works fine
just a minor issue with my Tenda WiFi router unable to make it into Access Point ( cheap china product )
Other then that, all runs fine.

 

[gdmax]

Other issue that i have noticed that is the memory usage on the Proxmox HV for Pfsense is different then on the Pfsense dashboard
Why is that? and should the Pfsense VM consume so much memory ?




On the Pfsense Dashboard

 

[bobmc]

There's no guest agent running in the VM so proxmox has no knowledge of application memory use. 16GB is way more than PFsense needs, around 2GB should be fine - same with the boot disk try 4GB