SOLVED: Virtualizing pfSense with OpenVSwitch and Multiple VLANs

Hazmat480

New Member
Dec 2, 2019
5
0
1
43
I am having an issue with pfSense Proxmox and VLAN's. My setup has pfSense virtualized with other VM's on the same host and 4 VLAN's. I have a Cisco 3850 switch that is my core. Originally I had all of my intervlan routing occuring inside the Cisco switch and anything that need to get to the WAN would be routed to the pfSense VM. So I had 3 physical interfaces on my Proxmox host; 1 - VM LAN traffic (trunk port), 1 - pfSense LAN traffic (routed port on cisco switch), 1 - pfSense WAN traffic. I also had a 4th interface so I used a OVSBond to bond the VM LAN interface and used link aggregation on the Cisco switch. All of this worked great. I had multiple VM's associated with different VLAN's and everything was communicating. I have since decided to make things more complicated. I want to be able to enforce policy between each VLAN, so for now I removed the physical interfaces for the VM LAN and setup a trunk port on the cisco switch, and removed the IP Addresses from the Cisco switch. I created VLAN's inside of pfSense and assigned them to the LAN physical interface. pfSense is able to ping devices connected to the management vlan of the cisco switch including the switch itself. But no device is able to ping pfSense. I have created a rule that allows all ICMP traffic for now until I can get this figured out. When I do packet capture I don't see any traffic coming into pfSense. So I am thinking that there is something with OVS that I am not configuring properly. Other two thoughts were do I need to change the native vlan on the Cisco switch, or do I need to use a different network device for the pfSense VM, currently VirtIO paravirtualized. Attached are diagrams of what I had that worked, and what I now have that doesn't work.
 

Attachments

  • Network Examples.pdf
    300 KB · Views: 198
I guess that you need to allow ping your pfsense in firewall > rules. You need rule for ICMP trafic for each VLAN.
 
I allowed ping on all interfaces, and configured to log both accept and reject requests. I do not see any come through, which is why I am thinking it is in the configuration of Proxmox and OpenVSwitch. Also, when I use pfSense packet capture it does not show anything being captured either. Thanks.
 
Did you check your VM network settings for right bridge and VLAN Tag?
 
I do not have any tags on the LAN bridge because it is a trunk. I am assuming that the bridge by default is a trunk, but may be wrong.PVE Network Interfaces.PNG
 
>>I do not have any tags on the LAN bridge because it is a trunk. I am assuming that the bridge by default is a trunk, but may be wrong.
yes, it's a trunk by default

>>I created VLAN's inside of pfSense and assigned them to the LAN physical interface. pfSense is able to ping devices connected to the management >>vlan of the cisco switch including the switch itself.

So if it's working from pfsense to device, the network is ok

>> But no device is able to ping pfSense
sound like firewall blocking incoming packets
 
In my case, my VM have untagged traffic on network device. Proxmox put Tag 7 and put it to vmbr1.

vmbr1 is connected to network card, for example enp3s0f1. This card is connected to my switch, which provide tagged traffic on this port.
 
So everything is working, as in most cases it was me not verifying everything. The VMs were still on vmbr3 and not moved to vmbr2. I did not have to do anything with VLAN tags. In case someone runs into this in the future, OVS bridges without a VLAN tag will act as a trunk port. Each VM is tagged with the proper vlan and does communicate across that vlan, the pfSense VM does not have a VLAN tag on the interface so it acts as a trunk port. My mistake, thank you everyone for your input. If someone would like in the future I can share screenshots of my configuration.
 
So everything is working, as in most cases it was me not verifying everything. The VMs were still on vmbr3 and not moved to vmbr2. I did not have to do anything with VLAN tags. In case someone runs into this in the future, OVS bridges without a VLAN tag will act as a trunk port. Each VM is tagged with the proper vlan and does communicate across that vlan, the pfSense VM does not have a VLAN tag on the interface so it acts as a trunk port. My mistake, thank you everyone for your input. If someone would like in the future I can share screenshots of my configuration.
Hello.
Coud you be so nice to share your screenshots , Thanks
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!