Proxmox host traffic routing issue

[proless8]

I have an issue with my current network setup that for some reason is preventing the Proxmox host traffic to be routed back correctly to a Linux bridge.


I have OPNsense virtualized and behind it a private internal LAN network. I am using HAProxy to proxy my services externally using a private domain (lab.home) that is being forwarded to the Proxmox host, and it is working fine when connecting from my Home LAN (Laptop).

The issue is that when I try to access a service from the pve host itself, e.g. "curl https://app.lab.home/ --verbose" this returns the error below, and it is not an SSL error or Certificate issue, my custom root ca is already installed and trusted by the pve host. I am assuming that the pve host first is querying the DNS on my Home Router for the domain "lab.home" and it being redirected to itself but not forwarded to vmbr1 (OPNsense WAN) anymore ?!

What is wrong with my setup, and what am I missing here ? Any help is appreciated.

Curl error

* Trying 192.168.100.1:443...
* connect to 192.168.100.1 port 443 failed: Connection refused
* Failed to connect to app.lab.home port 443 after 1 ms: Couldn't connect to server
* Closing connection 0
curl: (7) Failed to connect to app.lab.home port 443 after 1 ms: Couldn't connect to server

/etc/network/interfaces

auto lo
iface lo inet loopback

iface enp1s0 inet manual

auto vmbr0
iface vmbr0 inet dhcp
        bridge-ports enp1s0
        bridge-stp off
        bridge-fd 0
        post-up echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up iptables -t nat -A PREROUTING -i vmbr0 -p tcp -m multiport ! --dport 22,8006 -j DNAT --to 172.16.1.1
        post-up iptables -t nat -A PREROUTING -i vmbr0 -p udp -m multiport ! --dport 22,8006 -j DNAT --to 172.16.1.1
        post-down iptables -t nat -D PREROUTING -i vmbr0 -p tcp -m multiport ! --dport 22,8006 -j DNAT --to 172.16.1.1
        post-down iptables -t nat -D PREROUTING -i vmbr0 -p udp -m multiport ! --dport 22,8006 -j DNAT --to 172.16.1.1

auto vmbr1
iface vmbr1 inet static
        address 172.16.1.0/31
        bridge-ports none
        bridge-stp off
        bridge-fd 0
        post-up iptables -t nat -A POSTROUTING -s '172.16.1.1/31' -o vmbr0 -j MASQUERADE
        post-down iptables -t nat -D POSTROUTING -s '172.16.1.1/31' -o vmbr0 -j MASQUERADE
# OPNsense WAN

auto vmbr2
iface vmbr2 inet static
        bridge-ports none
        bridge-stp off
        bridge-fd 0
# OPNsense LAN

source /etc/network/interfaces.d/*

 

[shanreich]

You need to configure a gateway that is able to route to the 192 network (in your case probably on the host, since you're doing masquerading). Judging from your network configuration you currently only have a route for 172.16.1.0/31

 

[proless8]

You need to configure a gateway that is able to route to the 192 network (in your case probably on the host, since you're doing masquerading). Judging from your network configuration you currently only have a route for 172.16.1.0/31

i was not sure about routes since i am kinda new the networking but running ip route on the host gives me this output:

default via 192.168.100.254 dev vmbr0
172.16.1.0/31 dev vmbr1 proto kernel scope link src 172.16.1.0
192.168.100.0/24 dev vmbr0 proto kernel scope link src 192.168.100.1

 

[shanreich]

Ah sorry, I overlooked the dhcp setting in your network configuration. Can you ping the host?
Is the DNS server returning the correct IP for your app? It seems like it is returning the IP of the router, is this where the app is located?

 

[proless8]

Yes, I am able to ping the host. Doing nslookup on "app.lab.home" is also returning the correct IP expected (host ip) → vmbr0 (192.168.100.1)

Server:         192.168.100.254
Address:        192.168.100.254#53

Name:   app.lab.home
Address: 192.168.100.1

The app (RestAPI) is located on the internal LAN, which is managed by OPNsense, I also have HAProxy setup to expose the services, by doing port forwarding for 80,443 to an internal IP. I can reach this app from my laptop, no issues there.