Problems configuring VM builtin firewall with NAT + Port Forwarding + Hairpinning

[prngao]

I have a server with Proxmox 4.0-57 with the built-in firewall activated for the datacenter.

I have a single public IP address (e.g. 1.2.3.4) and my CTs/VMs have IPs in the subnet 10.10.10.0/24 and connected to internet using NAT:

## /etc/network/interfaces

iface lo inet loopback

auto eth0
iface eth0 inet static
        address 1.2.3.4
        netmask 255.255.0.0
        gateway 1.2.3.1
        post-up echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp

auto vmbr0
iface vmbr0 inet static
        address 10.10.10.1
        netmask 255.255.255.0
        bridge_ports none
        bridge_stp off
        bridge_fd 0

        post-up echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up   iptables -t nat -A POSTROUTING -s '10.10.10.0/24' -o eth0 -j MASQUERADE
        post-down iptables -t nat -D POSTROUTING -s '10.10.10.0/24' -o eth0 -j MASQUERADE

The VMs/CTs (100 and 110 for the following example) expose several services (let's say SSH and HTTPS).
Services on some VMs (such as VM110) should only be visible from a given public subnet (e.g. 1.2.0.0/16).
Since the ports to be published may collide, port forwarding is done in the following way:

VM100:22  <--> 1.2.3.4:10022
VM100:443 <--> 1.2.3.4:10443
VM110:22  <--> 1.2.3.4:11022

That is:

## More from /etc/network/interfaces

# 100
        post-up   iptables -t nat -A PREROUTING -p tcp -m tcp --dport 10022 -j DNAT --to-destination 10.10.10.100:22
        post-down iptables -t nat -D PREROUTING -p tcp -m tcp --dport 10022 -j DNAT --to-destination 10.10.10.100:22
        post-up   iptables -t nat -A PREROUTING -p tcp -m tcp --dport 10443 -j DNAT --to-destination 10.10.10.100:443
        post-down iptables -t nat -D PREROUTING -p tcp -m tcp --dport 10443 -j DNAT --to-destination 10.10.10.100:443
# 110
        post-up   iptables -t nat -A PREROUTING -s 1.2.0.0/16 -p tcp -m tcp --dport 11022 -j DNAT --to-destination 10.10.10.110:22
        post-down iptables -t nat -D PREROUTING -s 1.2.0.0/16 -p tcp -m tcp --dport 11022 -j DNAT --to-destination 10.10.10.110:22

Up to this point, everything works flawlessly.

Now, the tricky part:

I'd like to move some part of this config to the GUI (such as the allowed ip ranges for the SSH on VM110). This is possible by removing the "-s 1.2.0.0/16" argument in the VM110 rules, and configuring accordingly the VM firewall.

I have been able to make this proposed configuration work, activating the ProxMox built-in firewall at the CT/VM level together with the previous config for port forwarding (removing the "-s" arg.) by using the following rules:

## More from /etc/network/interfaces 

# Allow NAT working with the built-in firewall
        post-up   iptables -t raw -I PREROUTING  -i fwbr+ -j CT --zone 1
        post-down iptables -t raw -D PREROUTING  -i fwbr+ -j CT --zone 1

Additionally, with the CT/VM built-in firewall disabled, I have been able to configure hairpinning (connect in the internal network between CTs/VMs using their public IPORT) by using the following rules:

## More from /etc/network/interfaces

# 100 Harpinning
        post-up   iptables -t nat -A POSTROUTING -d 10.10.10.100 -p tcp -m multiport --dports 22,443 -j MASQUERADE
        post-down iptables -t nat -D POSTROUTING -d 10.10.10.100 -p tcp -m multiport --dports 22,443 -j MASQUERADE
# 110 Harpinning
        post-up   iptables -t nat -A POSTROUTING -d 10.10.10.110 -p tcp -m multiport --dports 22 -j MASQUERADE
        post-down iptables -t nat -D POSTROUTING -d 10.10.10.110 -p tcp -m multiport --dports 22 -j MASQUERADE

However, after several days trying, I have been completely unable to make both things work together (CT/VM firewall + hairpinning), and I have not been able to guess which is the network architecture with all the additional virtual NICs introduced by the firewall (fwbr101i0, fwln101i0, fwpr101i0, etc.).

Obviusly, disabling the firewall, enabling the previous hairpinning rules and controlling the traffic to the CTs/VMs using rules in the /etc/network/interfaces file is an option, but, is there a way to make hairpinning work together with the built-in firewall?

Any ideas?

Thanks in advance.

 

[jannispl]

I am running into the same problem. Have you found a solution to this?

 

[Dura]

Never found a way, so I don't use "hairpinning"