Proxmox Firewall config optimization

[pille99]

hello all,

right now i have the configuration like in the attachment (the top Picture) and i want to optimize to the bottom part of the picture

New should be:
1. Hetzner does the Port Filtering (only the desired ports are opened)
2. Proxmox Host does the Country filter, and IDS
3. the opnFirewall does forwarding, HAProxy, SSLoffloading, Monitoring, etc.

for the sake of simplicity. some questions plz
1. i will do the host firewall on Proxmox host, but the issue is that ipset cant handle big lists of IPs. i had previously a server with ipset and big lists, its possible and works like a carm. in proxmox the file crashes. does anybody has the same issue ?
2. how would look like a iptable firewall rule to forward traffic from vmbr0 to the SDN, EXT1 ?

 

[LnxBil]

i will do the host firewall on Proxmox host, but the issue is that ipset cant handle big lists of IPs. i had previously a server with ipset and big lists, its possible and works like a carm. in proxmox the file crashes. does anybody has the same issue ?

A list is always a bad idea, a tree would be much better and faster.

how would look like a iptable firewall rule to forward traffic from vmbr0 to the SDN, EXT1 ?

Simple iptables dnat route.
Another approach would be to NOT expose the PVE host itself to the internet and give the public IP to your pfsense. This setup needs to be done off-hetzner and then just copied to hetzner and it is the most secure but also one of the complicated ones. With this setup, your PVE will not be accessible and therefore not hackible from the internet. I would only use this setup myself if I'd use "real" servers with a "real" management console so that I can access the host without having direct access to the running os.

 

[pille99]

A list is always a bad idea, a tree would be much better and faster.


Simple iptables dnat route.
Another approach would be to NOT expose the PVE host itself to the internet and give the public IP to your pfsense. This setup needs to be done off-hetzner and then just copied to hetzner and it is the most secure but also one of the complicated ones. With this setup, your PVE will not be accessible and therefore not hackible from the internet. I would only use this setup myself if I'd use "real" servers with a "real" management console so that I can access the host without having direct access to the running os.

plz tell me what you mean with "tree" ? the IP List of Germany and Swiss contains around 100.00 ips. right now its on OPNSense and it works like a charm but i would love to get the forbitten traffic not to the VM Network.

the PVE Host itself (management) is completly blocked, except my IP (from the hetzner firewall), only the Public IP Traffic comes in. right now i have OPNsense (which is the same as pfsense) which is configured to the public IP. (4 Servers, on each node one) and they filter country for example. the allowed traffic flows than to the 5th firewall, which does the finally segmantation of the traffic. do you thin the config is ok ? i just find it more maintainance and management work as with only one firewall.

its just that i would love to get the middle step out, for performance and resource reason.

 

[LnxBil]

plz tell me what you mean with "tree" ? the IP List of Germany and Swiss contains around 100.00 ips. right now its on OPNSense and it works like a charm but i would love to get the forbitten traffic not to the VM Network.

I don't know if OPNSense has such a feature implemented, but iptables and one table/chain means that you have to traverse the list from top to bottom and if a rule matches, the package is dropped from further processing. A tree would be some logic that splits the traffic based on the ip octets so that you have a tetranary tree with a depth of 4. I've done this in the past and it'll improve the package traversal time through iptables tremendously with you have a lot of rules. The tree depends on the block list and maybe whole subnets can be blocked so that it is more efficient than a simple list traversal from top to bottom.

 

[pengu1n]

I guess is horses for courses. OPN is a firewall distribution and the parsing of large lists of ips is incredibly fast, and done for different services. Examples are the geoip blocking, alias from txt files, and more. Unfortunately you can't just bring the logic accross. OPN doesn't use iptables.

 

[pille99]

you guys think that the original configuration with 5 opnsense is a good config ?
i just find it a bit more work and it takes resources of the cluster

 

[pengu1n]

sorry what is 5 ? If it is in the attached file, I can't open from work machine.

 

[pille99]

sorry what is 5 ? If it is in the attached file, I can't open from work machine.

5 opnsense firewalls.
its pdf file.
in other words
right now i have nodes, each has its own opnsense (which would be 4 opnsense firewalls), these 4 opnsense are connected to a 5th opnsense

 

[pengu1n]

sorry for the delay.
One scenario is to have one OPN vm in front for all VMS in a multi-wan setup, but normally multi-node in high availability. These two OPN firewalls would do all the firewalling for all networks. However this is an advanced setup and needs good networking knowledge.
I think that's what you have on your optimised setup, just one node at the moment.
Having said that, with the little consumption of an OPNSense VM, I'd be inclined to leave it as it is, with one OPN for each network but do the by-country firewalling on each one. Super easy all around.

 

[pille99]

for now i leave it like it is. 4 opn, on each node 1, sdn with connection from opn 1-4 (10.69.1-4.1 to 10.69.1-4.254 - which is than the IP of the 5th opn)

 

[pille99]

this solution is very unpractically.
somebody know how i can forward traffic with iptables to SDN and there to a definde IP Address

 

[LnxBil]

somebody know how i can forward traffic with iptables to SDN and there to a definde IP Address

You need an ingress router and then simply do DNAT.

 

[pille99]

You need an ingress router and then simply do DNAT.

can you give me an example what exactly that is

 

[LnxBil]

can you give me an example what exactly that is

Exactly what you asked for: a machine that gets all traffic and "does stuff with it". Normally you would create a cluster web-ingress like in kubernetes that gets all web traffic and reverse proxies what you need. For that you would DNAT all incoming traffic via iptables to the machine that runs your reverse proxy (or cluster ingress ip for kubernetes) so that it manages everything. This could be your PFsense and you could do deep package inspection.

For a hetzner setup, you can create a failover HA-IP and use it for that. In this setup you don't need traffic forwarding, you just use the HA IP for any ingress and give it to your pfsense. This machine is HA in your cluster such that it'll automatically migrate to another machine if the host it runs on fails. You then also need to create a hook script that switches the HA-IP from one node to the other in the hetzner control panel. You then need only one HA-IP for that for a simple setup. You could use multiple HA-IPs (really one HA-IP, not a subnet) and use multiple ingress routes with pfsense (e.g. number of cluster nodes) and configure a round robin DNS name for them in order to distribute the connections between them.

Running such a setup with hetzner is possible but is not as cheap as just running a couple of hetzner servers. To have a proper PVE cluster, you would also need a custom cluster network and those HA-IPs than can be switches via the management console if you need that. We tried something similar and dismissed it due to the horendous costs in comparison to housing our own servers in a datacenter nearby.

For a simple web system, just use the following DNAT rules to run on your PVE host:

iptables -t nat -A PREROUTING -d <public-ip> -p tcp --dport 80    -j DNAT --to <private-pfsense-ip>:80
iptables -t nat -A PREROUTING -d <public-ip> -p tcp --dport 443   -j DNAT --to <private-pfsense-ip>:443