Access VM from PVE Host through Virtual Pfsense

[mthax]

Hi,
i have a Problem connecting Proxmox Backup Server to PVE from the PVE Host itself. So far everything is working, i have internet on the pve host, i have internet on the virtual lan behind the pfsense VM. Even the Proxmox Backup Server VM has Internet. I can reach the Proxmox Backup Server UI with the Browser through a haproxy. All needed ports are open
The PVE Host has its own IP, PFsense gets another Public IP i bought from my host, i can reach VMs from the Internet and i can reach the Internet from the VMs. But for example, when i want to ping the internal IP of the Backup (192.168.1.5) from the PVE Host i cant reach it. Vice Versa i cant reach the PVE Host Itself, but everything after it . I have the feeling i am missing something like a static route, but i am unsure what that should look like.
Did someone have the same problem and can someone point me in the right direction ?
Best greets

 

[ph0x]

You should come up with a little more detailed information about your network layout (which nodes/vm's connect through which gateway), maybe with a little sketch.
And the output of cat /etc/network/interfaces and ip r of pve and pbs.

 

[mthax]

### Hetzner Online GmbH installimage
source /etc/network/interfaces.d/*

auto lo
iface lo inet loopback
iface lo inet6 loopback

iface enp34s0 inet manual
iface enp34s0 inet6 manual

auto vmbr0
iface vmbr0 inet static
        address 95.217.113.xxx
        netmask 255.255.255.192
        gateway 95.217.113.193
        pointopoint 95.217.113.193
        dns-nameserver 1.1.1.2 213.133.98.98

        up route add -net 95.217.113.192 netmask 255.255.255.192 gw 95.217.113.193 dev enp34s0

        up ip route add 192.168.0.0/16 via 95.217.250.xxx dev vmbr0
        up ip route add 172.16.0.0/12 via 95.217.250.xxx dev vmbr0
        up ip route add 10.0.0.0/8 via 95.217.250.xxx dev vmbr0

        bridge_ports enp34s0
        bridge_stp off
        bridge_fd 0
        bridge_wait 0

iface vmbr0 inet6 static

        address  2a01:4f9:4a:xxx::2
        netmask  64
        gateway  fe80::1
        dns-nameservers 2606:4700:4700::1112 2a01:4f8:0:1::add:9898

auto vmbr1
iface vmbr1 inet manual

        bridge_ports none
        bridge_stp off
        bridge_fd 0
        bridge_wait 0

 

[ph0x]

The Interfaces file looks quite good for me.
I would guess that pfsense blocks the intended traffic. Can you ping pve and pbs from pfsense?

 

[mthax]

ip r on PVE Host

default via 95.217.113.193 dev vmbr0 onlink
95.217.113.193 dev vmbr0 proto kernel scope link src 95.217.113.242

ipr on Proxmox Backup Server

default via 192.168.1.1 dev ens18 proto kernel onlink
192.168.1.0/24 dev ens18 proto kernel scope link src 192.168.1.5

/etc/network/interfaces on Backup VM

auto lo
iface lo inet loopback

auto ens18
iface ens18 inet static
        address 192.168.1.5/24
        gateway 192.168.1.1

iface ens18 inet6 static
        address 2a01:4f9:4a:xxx::3
        netmask 64
        gateway 2a01:4f9:4a:xxx::1

 

[mthax]

The Interfaces file looks quite good for me.
I would guess that pfsense blocks the intended traffic. Can you ping pve and pbs from pfsense?

yes, its works great, but i cant ping from pve host to the vm

 

[ph0x]

The routes to the internal networks are missing on pve, therefore it tries to connect via your real gateway, not via pfsense. Can't see why they do not get applied. But you can add them yourself, just type the stuff after up into the pve console.
And second, pfsense has to be configured to allow pings from outside (not recommended).

 

[mthax]

I have echo request and reply allowed ( at least as long as i need to test things) so this should be no problem.

I allready thought about something like that, but with static routes, my head explodes. Its hard for me to think about how a correct route would look like in this case. Can i route to vmbr1 directly without going through the pfsense ? The pfsense VM get its WAN ip through dhcp (the pfsense vm has a spoofed mac on vmbr0 to get that additional IP i bought )

 

[ph0x]

Vmbr1 does not play a role as far as I can tell.
Type this into the console, it should do the trick:
ip route add 192.168.0.0/16 via 95.217.250.xxx dev vmbr0
Replace xxx with your actual address, of course.