1 Physical Nic - 1 IP - OPNsense in Proxmox - Routing issue

[vertigo262]

Hi,

Fairly new to Proxmox, coming from about 20 years of Vmware. This seems to be a common secenerio with Proxmox. But it've been racking my brain for about a week now. Scowering the Forums to try to resolve it.

I am limited to 1 physical NIC, in which I was planning on doing an Ethernet Passthrough. How it should be. I have access to other virtual IP's from the provider, but I think it's all going to lead back to a Double NAT scenerio

I have a basic IP masquarade on the Bridges as the ISP doesn't like multiple Mac Addresses Broadcasting. Neither do I actually.

Proxmox is running fine, with Internet WAN. Management Port. OPNSense is configured and routing basic traffic to VM's inside of it to the LAN virtual Bridge. Example, I can surf the web our ping from VM's from inside the OPNsense LAN ip gateway behind the OPNSense Firewall

But where I am stuck is, I can't Port Forwards anything from the outside on Public IP's and get it to OPNsense for port Forwards.

Example, Web Server on port 80,443, or Mailsever 587, 25, 143, you name it
Packets seem to stop before OPNsense. So there is no OPNsense Port Forwards

This appears to be a common scenerio Since tons of people are using Proxmox, and running firewalls inside of it.

Tried Multiple Varients of Network configs but Manual Bridge Routing isn't my thing.
Also, trying the Proxmox Firewall Forwarding which doesn't seem to work for me

Any Help would be appreciated

 

[shanreich]

If I understand correctly, you want to do DNAT - the network configuration you posted does MASQUERADE which does source NAT. The forward feature of the firewall is for firewalling traffic that gets forwarded - not for forwarding traffic itself.

An example for DNATing traffic coming in on the interface eth0 on TCP port 8080 to 192.0.2.1 port 80:

iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 8080 -j DNAT --to 192.0.2.1:80

 

[vertigo262]

You sort of confused me the way your worded things, I want to all traffic to forward from the outside, minus the mangement port, to the firewall. Obviously in the firewall, do whatever. Port Forward, and wall as firewall rules. Just like a ethernet passthrough. However I need the mangement port

"The forward feature of the firewall is for firewalling traffic that gets forwarded - not for forwarding traffic itself."

Sounds like your saying the traffic isn't being forwarded

And your example is by Pre-routing

So this is an Iptables port forward rule that is in Interfaces?
"iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 8080 -j DNAT --to 192.0.2.1:80"

What if I want all traffic to 192.0.2.1? So that OPNsense can do whatever on that subnet? not just port 8080

Or is this just an Iptables firewall rule to be entered in Iptables

 

[shanreich]

"The forward feature of the firewall is for firewalling traffic that gets forwarded - not for forwarding traffic itself."

Sounds like your saying the traffic isn't being forwarded

Exactly, if you configure a firewall rule with direction 'forward', then it does not mean that this traffic will get forwarded by the PVE host. It means if the PVE host is configured to act as a router (usually by enabling ip_forward sysctl), then you can create firewall rules for the traffic that gets forwarded by the PVE host (e.g. only allow access to a VM from certain IPs).

What if I want all traffic to 192.0.2.1? So that OPNsense can do whatever on that subnet? not just port 8080

Do I understand correctly? You have a public IP and want to expose certain services on that IP? In that case it would make more sense to configure the WAN IP address on your OPNsense, instead of using NAT on the PVE host, which just complicates things. Then use OPNsense to route / forward traffic from / to the VMs as you see fit. You need one bridge for the WAN and one for the LAN and use OPNsense to route between the networks.

I'd strongly suggest not exposing the PVE host via the public IP. Rather use a separate VLAN for management and another VLAN for the public VM traffic to separate your management network from your VM traffic. If you really need to access your PVE from outside, then you should use a VPN. Generally, if this is a home setup and you just want to access certain services from the outside, Wireguard might be a good fit for your use case instead of exposing everything publically.

 

[vertigo262]

h direction 'forward', then it does

 

[vertigo262]

My Solution was going from

iface eth0 inet static

to

iface eth0 inet manual

And adding a virtual IP address in the VM

I think the static routes were complicating the Bridging and Forwarding