Proxmox & pfsense configuration

R

richjim

New Member
Oct 2, 2020
6
1
3
52
I have an existing network that I am routing out my traffic to a public IP 1 using a physical router. I am trying to set up a Proxmox VM using Pfsense so I can move traffic through it using public IP 2. I would like to be able to configure my network devices and VMs to point to either the physical router or the pfsense by changing the gateway on the device depending on what public IP I would like them to use.

When I change the network settings in Prox for a VM to use vmbr2 and change the gateway on the VM to use pfsense (192.168.0.2) I can't get out to public IP 2.
Any assistance would be appreciated to get this accomplished.


I have a network with the following configuration
vmbr0 (eno1) Used to access Proxmox IP 192.168.0.5 GATE 192.168.0.1 which points to my PUBLIC 1 through a physical router (192.168.0.1)
vmbr1 (eno2) which I am using for pfsense WAN PUBLIC IP 2
vmbr2 (eno3) which I am using for pfsense LAN 192.168.0.2

prox.JPG
 

Attachments

  • prox.JPG
    prox.JPG
    140.7 KB · Views: 4
Last edited:
richjim said:
When I change the network settings in Prox for a VM to use vmbr2 and change the gateway on the VM to use pfsense (192.168.0.2) I can't get out to public IP 2.
Click to expand...
Can you clarify "can't get out to public IP 2"? Does L2 traffic propagate to the pfSense VM and is it an L3 issue (you might be able to verify this by checking the ARP tables of either machine), or can traffic from a VM on vmbr2 not reach the pfSense VM at all?
 
datdenkikniet said:
Can you clarify "can't get out to public IP 2"? Does L2 traffic propagate to the pfSense VM and is it an L3 issue (you might be able to verify this by checking the ARP tables of either machine), or can traffic from a VM on vmbr2 not reach the pfSense VM at all?
Click to expand...
Thanks for your quick response. I actually can't reach the pfsense from a VM on vmbr2
prox2.JPG
 
I did verify the pfsense LAN is configured for 192.168.02/24. Also, I just checked a physical machine and changed the gateway to 192.168.0.2 which worked! All its traffic went through pfsense > Public IP2.

The issue now is only with the VM's. When the VM's are using vmbr0 (192.168.0.1 gate) they can access the pfsense but traffic goes through public IP1. When I change the gateway to vmbr2 (192.68.0.2 gateway) I can't ping or access the pfsense.
 
richjim said:
I did verify the pfsense LAN is configured for 192.168.02/24. Also, I just checked a physical machine and changed the gateway to 192.168.0.2 which worked! All its traffic went through pfsense > Public IP2.

The issue now is only with the VM's. When the VM's are using vmbr0 (192.168.0.1 gate) they can access the pfsense but traffic goes through public IP1. When I change the gateway to vmbr2 (192.68.0.2 gateway) I can't ping or access the pfsense.
Click to expand...
I think a more detailed description (in terms of networks and segments) of what you are trying to accomplish would help a lot here.

It seems like both your gateways are in the same network (192.168.0.1 and 192.168.0.2 are both in 192.168.0.0/24), but for some reason they are on separate L2 segments (vmbr0 for your normal LAN and vmbr2 for your pfSense LAN). Is there any specific reason that you don't want to connect your pfSense LAN to vmbr0 as well, or why you do not want the pfSense LAN to be a different network (i.e. 192.168.1.0/24)? The first option might solve the issue, and the second one probably won't, but it *might* help debug the issue a little further.
 
There was no reason I created the additional VMBR2 other than I just followed the Proxmox/Pfsense install guide which had me create the 2 bridges. I went ahead and did a re-installed pfsense using VMBR0 (LAN) and VMBR1 (WAN) for pfsense.

Current config:
1601676896441.png

I am looking to use 2 gateways.
Gateway 192.168.0.1 > physical router >Public IP1
Gateway 192.168.0.2 > pfsense> Public IP2

After the reinstall here is what is happening:
Physical devices using gateway 192.168.0.1 can ping/access pfsense (192.168.0.2) and traffic goes through Public IP1
Physical devices using gateway 192.168.0.2 can ping/access the pfsense and traffic goes through Public IP2
VM's using gateway 192.168.0.1 can ping/access pfsense and traffic goes through Public IP1
This is exactly what I want.

Here is where the issue is:
Any VM's using gateway 192.168.0.2 with VMBR0 can ping/access pfsense but no traffic is passed to Public IP2

I hope I gave enough information.
 
Aha! That does sound like it's more of a pfSense issue than a Proxmox issue, though, especially if the VMs can reach the pfSense machine. How do you connect the physical machines to the virtual bridge? By plugging them/the switch they are connected to into eno1?
 
Though a little late to the party:

The problem is likely to also be solved by disabling "Hardware checksum offloading" (in "Avanced > Networking") in pfSense, since that seems to produce exactly this issue when VirtIO network devices are used. When using E1000 emulation the hardware checksum offloading does actually work, but the overhead is likely greater, as paravirtualized devices are easier to virtualize then hardware devices.
 
Last edited:
Found the same problem, took me 2 days for narrowing down the problem and finding this post here. Great work. For disabling the hardware checksum offload you need to tick the checkbox in pfsense.

Tnx
Whalewatcher
 
You must log in or register to reply here.
Top Bottom