Request for documentation

[basisbit]

Hello,

I am asking this in behave of a group of regular proxmox users here in Germany:
Proxmox 4 introduced web-configurable PVE firewall. It allows for configuration to be done on datacenter level, on host level and on vm level. We wanted to set up default deny based firewalling and also have iptables forward single ports to specific clients. The documentation about the pve firewall in the wiki sadly does not specify at all what the order of those rules being applied is. For example, for inbound traffic on eth0, does it first apply allow and deny rules of datacenter in configured order and then apply the host+vm specific allow+deny rules as configured? Or does it apply them the other way round or does it first apply all allow rules and then the deny rules or does it only apply one of the configured rules sets? Also, when there are other iptables rules configured on that system (for port-forwarding, ...), when do these get applied? Before/after the PVE web configured rules? All in all, we ask for some diagram of the different firewall chains and when these are applied to be added to the wiki. Another question: how are already established connections treated by the web-configured firewall?

Thanks in advance,
basisbit

 

[dietmar]

or example, for inbound traffic on eth0, does it first apply allow and deny rules of datacenter in configured order and then apply the host+vm specific allow+deny rules as configured?

Host rules can overwrite datacenter rules. VM rules are independent.

Or does it apply them the other way round or does it first apply all allow rules and then the deny rules or does it only apply one of the configured rules sets? Also, when there are other iptables rules configured on that system (for port-forwarding, ...), when do these get applied? Before/after the PVE web configured rules?

That depends if you insert them before of after the pve-firewall rules.

All in all, we ask for some diagram of the different firewall chains and when these are applied to be added to the wiki.

You can simply analyze the output of iptables-save?

Another question: how are already established connections treated by the web-configured firewall?

The firewall accepts already established connections.

 

[basisbit]

Thanks a lot @dietmar for your quick reply!
So, the web-interface for firewall rules generates IP-tables rules and applies them after changes. Is there some section where we can insert custom rules so that the web-interface doesn't overwrite/delete those? (like nat-ing rules, port forwarding,...)

Also, I have to agree to what is being said here at the end: "The documentation for the Proxmox firewall, imho, really just sucks. I burned a lot of hours and brain cells only to come to the conclusion that Cascading doesn’t really happen.". The answer from @dietmar shows exactly the same. It doesn't actually answer my questions except for that informations about established connections always being allowed (which is another crucial information that is also not documented visibly in the wiki).

 

[dietmar]

So, the web-interface for firewall rules generates IP-tables rules and applies them after changes. Is there some section where we can insert custom rules so that the web-interface doesn't overwrite/delete those?

The pve-firewall does not delete other iptable chains.

Also, I have to agree to what is being said here at the end: "The documentation for the Proxmox firewall, imho, really just sucks. I burned a lot of hours and brain cells only to come to the conclusion that Cascading doesn’t really happen.".

I think the behavior is clearly documented in https://pve.proxmox.com/wiki/Firewall

The answer from @dietmar shows exactly the same. It doesn't actually answer my questions except for that informations about established connections always being allowed (which is another crucial information that is also not documented visibly in the wiki).

Feel free to improve the documentation and send patches ...

 

[basisbit]

Host rules can overwrite datacenter rules. VM rules are independent.
You can simply analyze the output of iptables-save?
The firewall accepts already established connections.

So I did some more experimenting with the proxmox pve firewall. From what I saw, establishes connections are not accepted atomatically by the firewall.

Maybe this will help others a bit, so here is an example iptables-save output for mostly just default, but default drop for outgoing traffic, too:

# Generated by iptables-save v1.4.21 on Tue Jan 31 21:01:58 2017
*filter
:INPUT ACCEPT [2:156]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [3:360]
: PVEFW-Drop - [0:0]
: PVEFW-DropBroadcast - [0:0]
: PVEFW-FORWARD - [0:0]
: PVEFW-FWBR-IN - [0:0]
: PVEFW-FWBR-OUT - [0:0]
: PVEFW-HOST-IN - [0:0]
: PVEFW-HOST-OUT - [0:0]
: PVEFW-INPUT - [0:0]
: PVEFW-OUTPUT - [0:0]
: PVEFW-Reject - [0:0]
: PVEFW-SET-ACCEPT-MARK - [0:0]
: PVEFW-logflags - [0:0]
: PVEFW-reject - [0:0]
: PVEFW-smurflog - [0:0]
: PVEFW-smurfs - [0:0]
: PVEFW-tcpflags - [0:0]
:fail2ban-ssh - [0:0]
-A INPUT -p tcp -m multiport --dports 22 -j fail2ban-ssh
-A INPUT -j PVEFW-INPUT
-A FORWARD -j PVEFW-FORWARD
-A OUTPUT -j PVEFW-OUTPUT
-A PVEFW-Drop -p tcp -m tcp --dport 43 -j PVEFW-reject
-A PVEFW-Drop -j PVEFW-DropBroadcast
-A PVEFW-Drop -p icmp -m icmp --icmp-type 3/4 -j ACCEPT
-A PVEFW-Drop -p icmp -m icmp --icmp-type 11 -j ACCEPT
-A PVEFW-Drop -m conntrack --ctstate INVALID -j DROP
-A PVEFW-Drop -p udp -m multiport --dports 135,445 -j DROP
-A PVEFW-Drop -p udp -m udp --dport 137:139 -j DROP
-A PVEFW-Drop -p udp -m udp --sport 137 --dport 1024:65535 -j DROP
-A PVEFW-Drop -p tcp -m multiport --dports 135,139,445 -j DROP
-A PVEFW-Drop -p udp -m udp --dport 1900 -j DROP
-A PVEFW-Drop -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -j DROP
-A PVEFW-Drop -p udp -m udp --sport 53 -j DROP
-A PVEFW-Drop -m comment --comment "PVESIG:zfGV4KTPaxGVOCwRUVqqqbR0IhM"
-A PVEFW-DropBroadcast -m addrtype --dst-type BROADCAST -j DROP
-A PVEFW-DropBroadcast -m addrtype --dst-type MULTICAST -j DROP
-A PVEFW-DropBroadcast -m addrtype --dst-type ANYCAST -j DROP
-A PVEFW-DropBroadcast -d 224.0.0.0/4 -j DROP
-A PVEFW-DropBroadcast -m comment --comment "PVESIG:NyjHNAtFbkH7WGLamPpdVnxHy4w"
-A PVEFW-FORWARD -m conntrack --ctstate INVALID -j DROP
-A PVEFW-FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A PVEFW-FORWARD -m physdev --physdev-in fwln+ --physdev-is-bridged -j PVEFW-FWBR-IN
-A PVEFW-FORWARD -m physdev --physdev-out fwln+ --physdev-is-bridged -j PVEFW-FWBR-OUT
-A PVEFW-FORWARD -m comment --comment "PVESIG:qnNexOcGa+y+jebd4dAUqFSp5nw"
-A PVEFW-FWBR-IN -m conntrack --ctstate INVALID,NEW -j PVEFW-smurfs
-A PVEFW-FWBR-IN -m comment --comment "PVESIG:Ijl7/xz0DD7LF91MlLCz0ybZBE0"
-A PVEFW-FWBR-OUT -m comment --comment "PVESIG:2jmj7l5rSw0yVb/vlWAYkK/YBwk"
-A PVEFW-HOST-IN -i lo -j ACCEPT
-A PVEFW-HOST-IN -m conntrack --ctstate INVALID -j DROP
-A PVEFW-HOST-IN -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A PVEFW-HOST-IN -m conntrack --ctstate INVALID,NEW -j PVEFW-smurfs
-A PVEFW-HOST-IN -p igmp -j RETURN
-A PVEFW-HOST-IN -d XXX.XXX.XXX.203/32 -p tcp -m set --match-set PVEFW-0-notinternal-v4 src -m tcp --dport 8006 -j RETURN
-A PVEFW-HOST-IN -d XXX.XXX.XXX.203/32 -p tcp -m tcp --dport 22 -j RETURN
-A PVEFW-HOST-IN -p tcp -m set --match-set PVEFW-0-management-v4 src -m tcp --dport 8006 -j RETURN
-A PVEFW-HOST-IN -p tcp -m set --match-set PVEFW-0-management-v4 src -m tcp --dport 5900:5999 -j RETURN
-A PVEFW-HOST-IN -p tcp -m set --match-set PVEFW-0-management-v4 src -m tcp --dport 3128 -j RETURN
-A PVEFW-HOST-IN -p tcp -m set --match-set PVEFW-0-management-v4 src -m tcp --dport 22 -j RETURN
-A PVEFW-HOST-IN -s XXX.XXX.XXX.0/24 -d XXX.XXX.XXX.0/24 -p udp -m udp --dport 5404:5405 -j RETURN
-A PVEFW-HOST-IN -s XXX.XXX.XXX.0/24 -p udp -m addrtype --dst-type MULTICAST -m udp --dport 5404:5405 -j RETURN
-A PVEFW-HOST-IN -j PVEFW-Drop
-A PVEFW-HOST-IN -j DROP
-A PVEFW-HOST-IN -m comment --comment "PVESIG:UDd8BahxEo5iwDeZegl13ED76iU"
-A PVEFW-HOST-OUT -o lo -j ACCEPT
-A PVEFW-HOST-OUT -m conntrack --ctstate INVALID -j DROP
-A PVEFW-HOST-OUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A PVEFW-HOST-OUT -p igmp -j RETURN
-A PVEFW-HOST-OUT -s XXX.XXX.XXX.203/32 -p tcp -m tcp --sport 22 -j RETURN
-A PVEFW-HOST-OUT -p udp -m set --match-set PVEFW-0-internal-v4 src -m set --match-set PVEFW-0-notdinky-v4 dst -m udp --dport 53 -j RETURN
-A PVEFW-HOST-OUT -p tcp -m set --match-set PVEFW-0-internal-v4 src -m set --match-set PVEFW-0-notdinky-v4 dst -m tcp --dport 53 -j RETURN
-A PVEFW-HOST-OUT -p tcp -m set --match-set PVEFW-0-internal-v4 src -m tcp --dport 80 -j RETURN
-A PVEFW-HOST-OUT -p tcp -m set --match-set PVEFW-0-internal-v4 src -m tcp --dport 443 -j RETURN
-A PVEFW-HOST-OUT -d XXX.XXX.XXX.0/24 -p tcp -m tcp --dport 8006 -j RETURN
-A PVEFW-HOST-OUT -d XXX.XXX.XXX.0/24 -p tcp -m tcp --dport 22 -j RETURN
-A PVEFW-HOST-OUT -d XXX.XXX.XXX.0/24 -p tcp -m tcp --dport 5900:5999 -j RETURN
-A PVEFW-HOST-OUT -d XXX.XXX.XXX.0/24 -p tcp -m tcp --dport 3128 -j RETURN
-A PVEFW-HOST-OUT -d XXX.XXX.XXX.0/24 -p udp -m udp --dport 5404:5405 -j RETURN
-A PVEFW-HOST-OUT -p udp -m addrtype --dst-type MULTICAST -m udp --dport 5404:5405 -j RETURN
-A PVEFW-HOST-OUT -j PVEFW-Drop
-A PVEFW-HOST-OUT -j DROP
-A PVEFW-HOST-OUT -m comment --comment "PVESIG:6tz4xMLO+VCozvY/lPrsG/CYCFg"
-A PVEFW-INPUT -j PVEFW-HOST-IN
-A PVEFW-INPUT -m comment --comment "PVESIG:+5iMmLaxKXynOB/+5xibfx7WhFk"
-A PVEFW-OUTPUT -j PVEFW-HOST-OUT
-A PVEFW-OUTPUT -m comment --comment "PVESIG:LjHoZeSSiWAG3+2ZAyL/xuEehd0"
-A PVEFW-Reject -p tcp -m tcp --dport 43 -j PVEFW-reject
-A PVEFW-Reject -j PVEFW-DropBroadcast
-A PVEFW-Reject -p icmp -m icmp --icmp-type 3/4 -j ACCEPT
-A PVEFW-Reject -p icmp -m icmp --icmp-type 11 -j ACCEPT
-A PVEFW-Reject -m conntrack --ctstate INVALID -j DROP
-A PVEFW-Reject -p udp -m multiport --dports 135,445 -j PVEFW-reject
-A PVEFW-Reject -p udp -m udp --dport 137:139 -j PVEFW-reject
-A PVEFW-Reject -p udp -m udp --sport 137 --dport 1024:65535 -j PVEFW-reject
-A PVEFW-Reject -p tcp -m multiport --dports 135,139,445 -j PVEFW-reject
-A PVEFW-Reject -p udp -m udp --dport 1900 -j DROP
-A PVEFW-Reject -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -j DROP
-A PVEFW-Reject -p udp -m udp --sport 53 -j DROP
-A PVEFW-Reject -m comment --comment "PVESIG:3gYHaSHlZx5luiKyM0oCsTVaXi4"
-A PVEFW-SET-ACCEPT-MARK -j MARK --set-xmark 0x80000000/0x80000000
-A PVEFW-SET-ACCEPT-MARK -m comment --comment "PVESIG:K9jRaFw5I2si1xj1eGi18ZF/Ng0"
-A PVEFW-logflags -j DROP
-A PVEFW-logflags -m comment --comment "PVESIG:ewllejV/lK5Rjmt/E3xIODQgfYg"
-A PVEFW-reject -m addrtype --dst-type BROADCAST -j DROP
-A PVEFW-reject -s 224.0.0.0/4 -j DROP
-A PVEFW-reject -p icmp -j DROP
-A PVEFW-reject -p tcp -j REJECT --reject-with tcp-reset
-A PVEFW-reject -p udp -j REJECT --reject-with icmp-port-unreachable
-A PVEFW-reject -p icmp -j REJECT --reject-with icmp-host-unreachable
-A PVEFW-reject -j REJECT --reject-with icmp-host-prohibited
-A PVEFW-reject -m comment --comment "PVESIG:KM/fOv4KvGn8XvMqxoiRCdvlji8"
-A PVEFW-smurflog -j DROP
-A PVEFW-smurflog -m comment --comment "PVESIG:k8rhuGB1IUidugKwAufSGGgKAZ4"
-A PVEFW-smurfs -s 0.0.0.0/32 -j RETURN
-A PVEFW-smurfs -m addrtype --src-type BROADCAST -g PVEFW-smurflog
-A PVEFW-smurfs -s 224.0.0.0/4 -g PVEFW-smurflog
-A PVEFW-smurfs -m comment --comment "PVESIG:HssVe5QCBXd5mc9kC88749+7fag"
-A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG FIN,PSH,URG -g PVEFW-logflags
-A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG NONE -g PVEFW-logflags
-A PVEFW-tcpflags -p tcp -m tcp --tcp-flags SYN,RST SYN,RST -g PVEFW-logflags
-A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN FIN,SYN -g PVEFW-logflags
-A PVEFW-tcpflags -p tcp -m tcp --sport 0 --tcp-flags FIN,SYN,RST,ACK SYN -g PVEFW-logflags
-A PVEFW-tcpflags -m comment --comment "PVESIG:CMFojwNPqllyqD67NeI5m+bP5mo"
-A fail2ban-ssh -s 122.194.229.7/32 -j REJECT --reject-with icmp-port-unreachable
-A fail2ban-ssh -s 116.31.116.37/32 -j REJECT --reject-with icmp-port-unreachable
-A fail2ban-ssh -s 221.148.126.18/32 -j REJECT --reject-with icmp-port-unreachable
-A fail2ban-ssh -j RETURN
COMMIT
# Completed on Tue Jan 31 21:01:58 2017


Now I am wondering where we may put our port forwarding rules. The documentation in the wiki doesn't say anything about that.

Feel free to improve the documentation and send patches ...

I don't see any registration button on the wiki page, thus can't login in and suggest changes. Seems to be a closed editors wiki.

 

[dietmar]

So I did some more experimenting with the proxmox pve firewall. From what I saw, establishes connections are not accepted atomatically by the firewall.

Really? What about this srule:

-A PVEFW-FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT

 

[dietmar]

I don't see any registration button on the wiki page, thus can't login in and suggest changes. Seems to be a closed editors wiki.

Please read:

https://pve.proxmox.com/wiki/Improving_the_Proxmox_VE_Documentation

I can create a wiki account for you if you really want to add wiki content.