Port Forwarding Issue with pfSense VM on Proxmox (Dual WAN, DHCP ISP)

[whytf]

Hello,

I'm having trouble with port forwarding on a pfSense VM running on Proxmox. I have a dual WAN setup with one DHCP connection and one PPPoE connection.

I have confirmed with DHCP ISP that they use 1:1 NAT on their end.

My setup:

• Proxmox version: 8.1.4
• pfSense version: 2.7.2
• NIC: i340-t4, i219 (motherboard)
• Network configuration:
  • vmbr0 is assigned to LAN in pfsense and all other VMs in proxmox, it also has slaved physical port (i340-t4) that connects to rest of the lan
  • vmbr1 is assigned to WAN in pfsense and it has slaved physical port (i340-t4) to ISP1(DHCP)
  • vmbr2 is assigned to WAN2 in pfsense and it has slaved physical port (i340-t4) to ISP2(PPPoE)
  • vmbr4 is assigned for proxmox management/cluster only and it has slaved physical port (i219) that connects to same physical switch as vmbr0/rest of the lan

The issue:

• Port forwarding works when using NIC passthrough, but not when using virtIO
• Specifically, port forwarding doesn't work for the DHCP ISP connection when using virtIO, but does work with PPPoE ISP2

I have tried:

• disable hardware offloading in pfsense
• ethtool -K XXXX rx off tx off for physical ports as well as vmbr(0-4) on proxmox
• manually changing MAC Addresses on vmbr(0-4) in case there would be a conflict, especially vmbr1 having same MAC as the physical interface
• This is my /etc/network/interfaces with manual MAC Addresses, to test without that I just comment out the vmbr hwaddress lines:

  auto lo
  iface lo inet loopback
  
  auto enp1s0f2
  iface enp1s0f2 inet manual
  
  iface enp1s0f3 inet manual
  
  iface enp1s0f0 inet manual
          hwaddress XXXXXXXXXXX
  
  iface enp1s0f1 inet manual
  
  iface eno1 inet manual
  
  auto vmbr0
  iface vmbr0 inet manual
          bridge-ports enp1s0f3
          bridge-stp off
          bridge-fd 0
          hwaddress 90:e2:ba:37:0d:a0
  #LAN
  
  auto vmbr1
  iface vmbr1 inet manual
          bridge-ports enp1s0f0
          bridge-stp off
          bridge-fd 0
          hwaddress 90:e2:ba:37:0d:a1
  #Antik
  
  auto vmbr2
  iface vmbr2 inet manual
          bridge-ports enp1s0f1
          bridge-stp off
          bridge-fd 0
          hwaddress 90:e2:ba:37:0d:a2
  #Telekom
  
  auto vmbr4
  iface vmbr4 inet static
          address 192.168.0.70/16
          gateway 192.168.0.1
          bridge-ports eno1
          bridge-stp off
          bridge-fd 0
          hwaddress 50:65:f3:48:34:a4
  #PVE

But still I cannot get it to work.

Thanks for help.