[SOLVED] PVE Firewall not filtering anything

t.lamprecht

Proxmox Staff Member
Staff member
Jul 28, 2015
3,991
941
163
South Tyrol/Italy
shop.maurer-it.com
net.bridge.bridge-nf-call-arptables
net.bridge.bridge-nf-call-ip6tables
net.bridge.bridge-nf-call-iptables
Those are set by /etc/sysctl.d/pve.conf since 2012 https://git.proxmox.com/?p=pve-cluster.git;a=commitdiff;h=501839cac97f68d4dcba21df6fb3797b976e9e56 due to causing performance regressions on many guests, bridge separation should rather use a real separating technology like VLAN or VXLAN, see the following mail for some details
https://lists.proxmox.com/pipermail/pve-devel/2012-March/002418.html
(other distros like RHEL was taken as argument that this was accepted and working default behavior)

If you want to apply rules directly on the bridge, not the actual tapX or ethX devices like pve-firewall does, then you can just drop those lines from that config (or add an override in a lexical later sorted filename to set them to 1).

This could be mentioned in the docs, but besides that I see no issue here - the PVE firewall works after all and other do not suddenly stop working due to this, this is set on boot (so any simple test would show that the setting is off).
 

ozdjh

Member
Oct 8, 2019
64
15
8
@t.lamprecht, this is marked as solved but I don't know what the solution is.

I am seeing this same problem. We have a prod cluster and a dev cluster that we upgraded to 6.3 & Octopus about 5 weeks ago. We recently noticed spoofed traffic coming out of a prod cluster node and (with the help of support ticket #1879015) identified that net.bridge.bridge-nf-call-iptables was set to 0 and that none of the VM firewall rules were being enforced anymore. They had been working prior to the upgrade.

We checked the lab cluster but FW wasn't enabled at the datacenter level. We enabled it and can now see that PVE has set net.bridge.bridge-nf-call-iptables=1.

These are 2 totally standard installations, with FW rules configured solely through the web UI. Your support engineer says that net.bridge.bridge-nf-call-iptables should be 1 for the firewall to work as expected. I suggested editing /etc/sysctl.d/pve.conf but he said as we didn't know why PVE was disabling this we couldn't be sure it wouldn't do so again at some stage.

The iptables chains are called 'tapXiY' and are children of FORWARD. /etc/sysctl.d/pve.conf on both clusters sets the br-nf settings to 0. Yet on the dev cluster PVE has set net.bridge.bridge-nf-call-iptables to 1 while the prod cluster is leaving it at 0. Which is correct, and can you give me any idea what to look at to work out what in PVE is changing that value on one cluster but not the other?
 

shippj

New Member
May 22, 2021
2
0
1
41
This problem does NOT appear to be solved.

This seems like a CRITICAL SECURITY PROBLEM that has persisted for WAY too long!

I'm seeing this problem on 2 production machines. 6.3-4 and 6.4-4. I have the firewall enabled at the datacenter level and the host level. When I add a DROP or REJECT rule to the host, i'm still able to ping the host from the IP that is supposed to be dropped. The firewall also fails to work on VMs and containers. Nothing relevant in journalctl or syslog or the console. "pve-firewall restart" doesn't help. "pve-firewall status" show's running. Restarting the pve server doesn't help. I'm not using CEPH.

Every once in a while, I can get the firewall to work, but I'm unable to reproduce it. When it's working, changing the DROP rule to ALLOW and then back to DROP results in it continuing to allow. I'm waiting at least 60 seconds to make sure.
 
Last edited:

ozdjh

Member
Oct 8, 2019
64
15
8
The comments on this thread regarding a ceph upgrade without a reboot match our experience. We upgraded to 6.3 then after testing everything was OK we upgraded Ceph to Octopus. We rebooted during the PVE upgrade but did not reboot after the Ceph upgrade. That matches what others have said. Running "pve-firewall restart" sets things back to normal.

We've had a ticket open for this but that has been unproductive. We've spent too long on this with no interest from support about finding the root cause ('try upgrading to 6.4', 'it must be something else you've installed' etc). We've written a plugin for our monitoring system to alert if this happens again so we can manually restart the firewall. It's not ideal but we're going to leave it at that.
 

t.lamprecht

Proxmox Staff Member
Staff member
Jul 28, 2015
3,991
941
163
South Tyrol/Italy
shop.maurer-it.com
@t.lamprecht, this is marked as solved but I don't know what the solution is.
Note that this was not marked as solved by any staff here, but by the thread OP, so please do not take that thread prefix as "proxmox staff just ignores this", the original author had their issue deemed as solved and so marked their thread as such.

We plan to improve the situation by dropping the sysctl override from pve-cluster, which is quite outdated nowadays and can be better handled by an admin themselves. Also, we will enforce relevant settings more pro-actively and periodically from pve-firewall, as long as the FW is configured to be enabled.

Nothing relevant in journalctl or syslog or the console. "pve-firewall restart" doesn't help. "pve-firewall status" show's running. Restarting the pve server doesn't help. I'm not using CEPH.
Then you're probably not affected by this threads issue, as a restarted FW always updates the relevant sysctl. Ensure you actually enabled the FW on all levels, that the rules and settings are correct and that you do not open a connection to the target first so that connection tracking allows the other side even if it'd wouldn't be allowed else.
 

shippj

New Member
May 22, 2021
2
0
1
41
I think I just narrowed down the problem some. The firewall rules appear to work for tcp connections, but not icmp. When I add a firewall rule to block an IP, the tcp ports,such as ssh become unusable almost instantly, but icmp pings keep working no matter how hard I try to make the firewall block them.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!