Proxmox/OPNsense

spetrillo

Member
Feb 15, 2024
269
14
18
Hello all,

I was wondering if anyone has been able to virtualize OPNsense on a Proxmox platform. I can do it via PCIe passthrough of the ethernet adapter but now I would like to virtualize said ports of the adapter. I cannot seem to get the networking down properly, so I am hoping someone can assist.

Thanks,
Steve
 
I am sorry you felt you needed to shoot me with your response. Yes I did check his post and I followed his guide but it did not work in my use case. I thought some others might have had either good or bad experiences.

Thanks!
 
Well you didn't wrote this in your first post so I couldn't know this. Why don't you describe what you are trying to achieve and what didn't work?
 
  • Like
Reactions: UdoB
My Proxmox PC has an onboard NIC and a 2 port 10 gig PCIe network adapter.

I have setup vmbr0(10Gport1) to be VLANs 2-4 and 20, while vmbr1(10Gport2) was setup with VLANs 10-12. OPNsense is configured as follows: vlan 2(LAN), vlan 3(OPT1), vlan 4(OPT2), vlan 20(OPT3), vlan 10(OPT4), vlan 12(OPT5). My WAN is configured for vmbr2(Onboard) and is set for DHCP. When I try to connection to the LAN(vlan 2) interface of the OPNsense VM I go nowhere. My PC is connected to a switch port, which is setup as untagged on VLAN 2. The actual NIC is also configured to be on VLAN 2. The Proxmox server NIC1 is connected to a switch port, which is setup as tagged for VLANs 2-4 and 20. I get no response from the LAN side of the OPNsense VM and I thought I would.
 
Hi,

Network side, all your VLAN are tagged ?
Tagged on the VM level or host level ?
Can you post your PVE network config and your VM config ?

Best regards,
 
  • Like
Reactions: Johannes S
I did this with pfSense, not OPNsense, but the differences should be immaterial. My hardware set up is an ASrock Industrial IMB-V2000M motherboard, with two built in Realtek NICs and an Intel X520 dual port SFP+ 10gbe NIC. The first step was to get the Realtek NICs operational, which may not apply to you. I used the the actual official Realtek Linux drivers and installed them using DKMS. Once I had all four NICs working, I assigned vmbr0 to one of the X520 ports. Note, I could have used any of the 4 NICs but I wanted a 10GBE connection to my 10gbe managed switch, and I decided to dedicate the two Realtek NICs to my redundant WAN ports (I have two internet providers, Xfinity and T-Mobile). Anyway, vmbr0 was assinged to a nic and I created two virtual NICs of of that: vmbro.100 and vmbr0.3. I assigned static IPs to both of these. VLAN 100 is my management VLAN and the proxmox web UI is on this IP address. VLAN 3 is my non-routed storage VLAN that connects all my proxmox nodes to my NAS, without having to leave the switch/be routed by pfSense. I also created vmbr1, vmbr2, and vmbr3. Each were assigned to one of the other NICs. Inside my pfSense VM, I added three virtual NICs, one assigned to vmbr0 which becomes the LAN port in pfSense, one assigned to vmbr1 which becomes the WAN port in pfSense, and one assigned to vmbr2, which is initially assigned to OPT1 in pfSense and later renamed/reassigned to WAN2.

the remaining bridge, vmbr3, is assigned a static IP that doesn't correspond to the IP ranges of any of my VLANs or anything else for that matter. It is the port I use to log into pfSense to manage everything in Proxmox should pfSense go down or need to be restored from backup. I have it set to 192.168.99.99 To use that port, I manually set the IP on my laptop to 192.168.99.100/255.255.0.0 and just plug then in direct, no switch or anything else in between. You will use this port initially to set up the pfsense VM and get pfsense (or OPNsense) installed. I used vmbr1 for the initial pfSense install, then I shut down the VM and added all the other virtual NICs. From the command line (Proxmox VM Spice console) I had to reassign the interfaces. After that I can log in from my normal network and go through the rest of the cofiguration.

Note that my setup is complicated because I am running my Proxmox gui on a VLAN that needed to be defined within pfSense. If you go the same route, you need to have a means of logging into Proxmox before pfSense is up and running. If you don't have enough NICs, you could maybe use something like a USB NIC.

For the pfSense VM I used OVMF (UEFI) as the bios, q35 as the machine type, Virtio SCSI single for the drive, and Virtio (paravirtualized) for my virtual network interfaces. I made sure firewall was unchecked and no vlan was noted in the VLAN field (leave it blank). I gave my VM 6 cores (just because this motherboard has 8 cores/16 threads total, and I have plenty to spare) and 4gb of memory, non ballooning.
 
Last edited:
  • Like
Reactions: Johannes S
I have setup vmbr0(10Gport1) to be VLANs 2-4 and 20, while vmbr1(10Gport2) was setup with VLANs 10-12. OPNsense is configured as follows: vlan 2(LAN), vlan 3(OPT1), vlan 4(OPT2), vlan 20(OPT3), vlan 10(OPT4), vlan 12(OPT5). My WAN is configured for vmbr2(Onboard) and is set for DHCP. When I try to connection to the LAN(vlan 2) interface of the OPNsense VM I go nowhere. My PC is connected to a switch port, which is setup as untagged on VLAN 2. The actual NIC is also configured to be on VLAN 2. The Proxmox server NIC1 is connected to a switch port, which is setup as tagged for VLANs 2-4 and 20. I get no response from the LAN side of the OPNsense VM and I thought I would.
That's not how this should work. All your VLANs need to be on one VTNET NIC. So in my setup vmbr0 (on the Proxmox side) becomes VTNET0, assigned to "LAN" in pfSense. Then inside pfSense I create the VLANs using VTNET0 as the parent interface. I have VTNET0 as LAN, VTNET1 as WAN and VTNET2 as my second WAN. All of my VLANs show up as "VLAN1 on VTNET0", "VLAN2 on VTNET0", etc., etc.
 
  • Like
Reactions: Johannes S
Hi,
I am running this exact setup right now since 1+ year. It works great.
However, I did not pass through the NICs. I have Intel I350-T4v2 and I have connected the individual ports of the NIC in Proxmox to individual bridges, also with VLANs, and then passed through the bridges to the opnsense VM with virtio and multiqueue.

Like shown in the attachments. The WAN port goes to a glass fiber media converter and is then directly connected to my ISP glass fiber. The other ports on the NIC I have used to connect the Wifi AP, some PCs and such. The performance is good for Gigabit, as I don't have a faster uplink, I could not test if that would also work.

I would say the advantage of using bridges is that you can also change your network card (i.e. copper, glass fiber, whatever) without affecting the opnsense. Also Linux has slightly better drivers.

Just make sure you assign no IP addresses for the WAN bridge, then you are safe because your Proxmox Environment can only be reached over that one bridge that has an address assigned.
 

Attachments

  • Screenshot_2025-11-10_19-35-28.png
    Screenshot_2025-11-10_19-35-28.png
    53.3 KB · Views: 7
  • Screenshot_2025-11-10_19-35-52.png
    Screenshot_2025-11-10_19-35-52.png
    67.3 KB · Views: 7
Last edited:
  • Like
Reactions: Johannes S
I did this with pfSense, not OPNsense, but the differences should be immaterial. My hardware set up is an ASrock Industrial IMB-V2000M motherboard, with two built in Realtek NICs and an Intel X520 dual port SFP+ 10gbe NIC. The first step was to get the Realtek NICs operational, which may not apply to you. I used the the actual official Realtek Linux drivers and installed them using DKMS. Once I had all four NICs working, I assigned vmbr0 to one of the X520 ports. Note, I could have used any of the 4 NICs but I wanted a 10GBE connection to my 10gbe managed switch, and I decided to dedicate the two Realtek NICs to my redundant WAN ports (I have two internet providers, Xfinity and T-Mobile). Anyway, vmbr0 was assinged to a nic and I created two virtual NICs of of that: vmbro.100 and vmbr0.3. I assigned static IPs to both of these. VLAN 100 is my management VLAN and the proxmox web UI is on this IP address. VLAN 3 is my non-routed storage VLAN that connects all my proxmox nodes to my NAS, without having to leave the switch/be routed by pfSense. I also created vmbr1, vmbr2, and vmbr3. Each were assigned to one of the other NICs. Inside my pfSense VM, I added three virtual NICs, one assigned to vmbr0 which becomes the LAN port in pfSense, one assigned to vmbr1 which becomes the WAN port in pfSense, and one assigned to vmbr2, which is initially assigned to OPT1 in pfSense and later renamed/reassigned to WAN2.

the remaining bridge, vmbr3, is assigned a static IP that doesn't correspond to the IP ranges of any of my VLANs or anything else for that matter. It is the port I use to log into pfSense to manage everything in Proxmox should pfSense go down or need to be restored from backup. I have it set to 192.168.99.99 To use that port, I manually set the IP on my laptop to 192.168.99.100/255.255.0.0 and just plug then in direct, no switch or anything else in between. You will use this port initially to set up the pfsense VM and get pfsense (or OPNsense) installed. I used vmbr1 for the initial pfSense install, then I shut down the VM and added all the other virtual NICs. From the command line (Proxmox VM Spice console) I had to reassign the interfaces. After that I can log in from my normal network and go through the rest of the cofiguration.

Note that my setup is complicated because I am running my Proxmox gui on a VLAN that needed to be defined within pfSense. If you go the same route, you need to have a means of logging into Proxmox before pfSense is up and running. If you don't have enough NICs, you could maybe use something like a USB NIC.

For the pfSense VM I used OVMF (UEFI) as the bios, q35 as the machine type, Virtio SCSI single for the drive, and Virtio (paravirtualized) for my virtual network interfaces. I made sure firewall was unchecked and no vlan was noted in the VLAN field (leave it blank). I gave my VM 6 cores (just because this motherboard has 8 cores/16 threads total, and I have plenty to spare) and 4gb of memory, non ballooning.
So you have a separate NIC that is your Proxmox GUI?
 
That's not how this should work. All your VLANs need to be on one VTNET NIC. So in my setup vmbr0 (on the Proxmox side) becomes VTNET0, assigned to "LAN" in pfSense. Then inside pfSense I create the VLANs using VTNET0 as the parent interface. I have VTNET0 as LAN, VTNET1 as WAN and VTNET2 as my second WAN. All of my VLANs show up as "VLAN1 on VTNET0", "VLAN2 on VTNET0", etc., etc.
So did you bond the two 10 gig ports in Proxmox, to effectively create one virtual NIC to OPnsense/pfSense? I could do that!
 
No, I did one bridge (vmbr) per NIC. Then I added three different virtual ethernet devices to my pfsense VM, one on vmbr0, one on vmbr1, and one on vmbr2. The virtual NIC on vmbr0 became my LAN port in pfsense. The virtual NIC on vmbr1 became my WAN port, and the virtual NIC on vmbr2 became my second WAN port (OPT1) on pfsense.
 
  • Like
Reactions: Johannes S
No, I did one bridge (vmbr) per NIC. Then I added three different virtual ethernet devices to my pfsense VM, one on vmbr0, one on vmbr1, and one on vmbr2. The virtual NIC on vmbr0 became my LAN port in pfsense. The virtual NIC on vmbr1 became my WAN port, and the virtual NIC on vmbr2 became my second WAN port (OPT1) on pfsense.
But if you have multiple vlans per NIC what do you present to pfSense/OPNsense? Are you not defining vlans on the pfSense/OPNsense initial install config screen?
 
No. I am not defining the VLANs on the initial install. I do that after I get it up and running on WAN and LAN first. I am not saying you couldn't do it the other way, I just never have. VLANs go on virtual NICs that OPNsense creates. You use a physical NIC, like VTNET1 (as the parent), to create virtual NICs, one for each VLAN. So, for example, VLAN 99 on VTNET1, would show up a VTNET1.99 as a virtual NIC. Without jumping into esoteric use cases, you basically want to have two physical NICs, one assigned to WAN and one assigned to LAN. And then you let the OPNsense software create virtual NICXs for each VLAN.

I guess I should check, but do you have a managed switch? Or are you trying to do this without a switch? My set up assumes that all traffic will flow on a trunked port to a managed switch. If you are not using a switch, then, I am really not sure how you would proceed. That is beyond my experience.
 
Last edited:
  • Like
Reactions: Johannes S