[SOLVED] Firewall not working as Expected

[geffin]

Hello, I've implemented a firewall rule at the VM level on one of my Proxmox hosts, running version 8.
The rule is meant to block inbound ping requests to the VM. However, even after applying the rule, the VM still responds to ping requests.
Strangely, I've successfully tested the same firewall rule on another VM hosted on a different Proxmox host running version 7, and it works as intended.

I've ensured that the firewall configurations on both hosts are identical. I'm seeking guidance on what to investigate further from my end.
Any assistance would be greatly appreciated.

Thank you in advance.

 

[Moayad]

Hello,

Could you please share the exact firewall rule you've implemented to block ping requests? (Please ensure to get rid of any sensitive information before you share the rule).

Can you please also share the output with the following commands?

pve-firewall status
qm config <VMID>

 

[geffin]

Hi.

Below is a screenshot of the rule added:



Below is the results as requested:

root@pmx01:~# pve-firewall status
Status: enabled/running (pending changes)


root@pmx01:~# qm config 298
acpi: 1
agent: 1
boot: order=virtio0
cicustom: user=snippets:snippets/userconfig-1561.yaml
cipassword: **********
ciuser: root
cores: 2
cpu: host
ipconfig0: ip=******/**,gw=*****
kvm: 1
memory: 4096
meta: creation-qemu=7.1.0,ctime=1687264932
name: *********
net0: virtio=***********,bridge=vmbr0,firewall=1,tag=****
onboot: 1
ostype: l26
reboot: 1
sata1: MD0:298/vm-298-cloudinit.qcow2,media=cdrom,size=4M
scsihw: virtio-scsi-single
serial0: socket
smbios1: uuid=4e3b978d-9a7d-4bdd-9922-9ecf967d4b77
sockets: 1
vcpus: 2
vga: std
virtio0: MD0:298/vm-298-disk-0.qcow2,cache=none,discard=on,format=qcow2,iothread=1,size=50G
vmgenid: a47f7824-7682-4dee-b37d-e8fddb2deb1e

 

[Moayad]

Thank you for the outputs and screenshot!

Status: enabled/running (pending changes)

As you the above message (pending changes) indicates that firewall rules have been modified but not yet applied. In this case, I would try to restart the pve-firewall by issuing the following command:

pve-firewall restart

And then check the status again.

 

[geffin]

I executed the command pve-firewall restart, but the status remains the same:

root@pmx01:~# pve-firewall restart
root@pmx01:~# pve-firewall status
Status: enabled/running (pending changes)

 

[Moayad]

Thank you for the result!

May you try re-compile the firewall by issuing the following command:

pve-firewall compile

If that didn't fix the issue I would try to reload the pve-firewall by issuing the following:

pve-firewall reload

 

[geffin]

I have executed the pve-firewall compile command; however the status remains the same.

Attempted to execute the pve-firewall reload command and got the following:

root@pmx01:~# pve-firewall reload
ERROR: unknown command 'pve-firewall reload'
USAGE: pve-firewall <COMMAND> [ARGS] [OPTIONS]

pve-firewall help [<extra-args>] [OPTIONS]

pve-firewall compile
pve-firewall localnet
pve-firewall restart
pve-firewall simulate [OPTIONS]
pve-firewall start [OPTIONS]
pve-firewall status
pve-firewall stop

 

[Moayad]

sorry I meant to restart

pve-firewall restart

EDIT; if that didn't help can you please share the output of the following command?

systemctl status pve-firewall.service

 

[geffin]

Below is the results of the systemctl status pve-firewall.service command:

root@pmx01:~# systemctl status pve-firewall.service
● pve-firewall.service - Proxmox VE firewall
Loaded: loaded (/lib/systemd/system/pve-firewall.service; enabled; preset: enabled)
Active: active (running) since Mon 2023-08-14 16:34:32 SAST; 1 month 27 days ago
Process: 1140 ExecStartPre=/usr/bin/update-alternatives --set ebtables /usr/sbin/ebtables-legacy (code=exited, status=0/SUCCESS)
Process: 1143 ExecStartPre=/usr/bin/update-alternatives --set iptables /usr/sbin/iptables-legacy (code=exited, status=0/SUCCESS)
Process: 1144 ExecStartPre=/usr/bin/update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy (code=exited, status=0/SUCCESS)
Process: 1145 ExecStart=/usr/sbin/pve-firewall start (code=exited, status=0/SUCCESS)
Process: 3232873 ExecReload=/usr/sbin/pve-firewall restart (code=exited, status=0/SUCCESS)
Main PID: 1148 (pve-firewall)
Tasks: 1 (limit: 153501)
Memory: 88.3M
CPU: 29min 46.667s
CGroup: /system.slice/pve-firewall.service
└─1148 pve-firewall

Oct 11 15:06:18 pmx01 pve-firewall[1148]: status update error: unable to open file '/proc/sys/net/bridge/bridge-nf-call-iptables' - No such file or directory
Oct 11 15:06:28 pmx01 pve-firewall[1148]: status update error: unable to open file '/proc/sys/net/bridge/bridge-nf-call-iptables' - No such file or directory
Oct 11 15:06:38 pmx01 pve-firewall[1148]: status update error: unable to open file '/proc/sys/net/bridge/bridge-nf-call-iptables' - No such file or directory
Oct 11 15:06:48 pmx01 pve-firewall[1148]: status update error: unable to open file '/proc/sys/net/bridge/bridge-nf-call-iptables' - No such file or directory
Oct 11 15:06:58 pmx01 pve-firewall[1148]: status update error: unable to open file '/proc/sys/net/bridge/bridge-nf-call-iptables' - No such file or directory
Oct 11 15:07:08 pmx01 pve-firewall[1148]: status update error: unable to open file '/proc/sys/net/bridge/bridge-nf-call-iptables' - No such file or directory
Oct 11 15:07:18 pmx01 pve-firewall[1148]: status update error: unable to open file '/proc/sys/net/bridge/bridge-nf-call-iptables' - No such file or directory
Oct 11 15:07:28 pmx01 pve-firewall[1148]: status update error: unable to open file '/proc/sys/net/bridge/bridge-nf-call-iptables' - No such file or directory
Oct 11 15:07:38 pmx01 pve-firewall[1148]: status update error: unable to open file '/proc/sys/net/bridge/bridge-nf-call-iptables' - No such file or directory
Oct 11 15:07:48 pmx01 pve-firewall[1148]: status update error: unable to open file '/proc/sys/net/bridge/bridge-nf-call-iptables' - No such file or directory

 

[Moayad]

Can you please post the output of `pveversion -v` command?

 

[geffin]

Please see below:

root@pmx01:~# pveversion -v
proxmox-ve: 8.0.2 (running kernel: 6.4.0-060400-generic)
pve-manager: 8.0.4 (running version: 8.0.4/d258a813cfa6b390)
pve-kernel-6.2: 8.0.5
proxmox-kernel-helper: 8.0.3
proxmox-kernel-6.2.16-6-pve: 6.2.16-7
proxmox-kernel-6.2: 6.2.16-7
pve-kernel-6.2.16-3-pve: 6.2.16-3
ceph-fuse: 17.2.6-pve1+3
corosync: 3.1.7-pve3
criu: 3.17.1-2
glusterfs-client: 10.3-5
ifupdown2: 3.2.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-3
libknet1: 1.25-pve1
libproxmox-acme-perl: 1.4.6
libproxmox-backup-qemu0: 1.4.0
libproxmox-rs-perl: 0.3.1
libpve-access-control: 8.0.4
libpve-apiclient-perl: 3.3.1
libpve-common-perl: 8.0.7
libpve-guest-common-perl: 5.0.4
libpve-http-server-perl: 5.0.4
libpve-rs-perl: 0.8.5
libpve-storage-perl: 8.0.2
libspice-server1: 0.15.1-1
lvm2: 2.03.16-2
lxc-pve: 5.0.2-4
lxcfs: 5.0.3-pve3
novnc-pve: 1.4.0-2
proxmox-backup-client: 3.0.2-1
proxmox-backup-file-restore: 3.0.2-1
proxmox-kernel-helper: 8.0.3
proxmox-mail-forward: 0.2.0
proxmox-mini-journalreader: 1.4.0
proxmox-widget-toolkit: 4.0.6
pve-cluster: 8.0.3
pve-container: 5.0.4
pve-docs: 8.0.4
pve-edk2-firmware: 3.20230228-4
pve-firewall: 5.0.3
pve-firmware: 3.7-1
pve-ha-manager: 4.0.2
pve-i18n: 3.0.5
pve-qemu-kvm: 8.0.2-4
pve-xtermjs: 4.16.0-3
qemu-server: 8.0.6
smartmontools: 7.3-pve1
spiceterm: 3.3.0
swtpm: 0.8.0+pve1
vncterm: 1.8.0
zfsutils-linux: 2.1.12-pve1

 

[Moayad]

Thank you for the output!

May I ask you what the kernel running in your node?

proxmox-ve: 8.0.2 (running kernel: 6.4.0-060400-generic)

This not look like our kernel! [0]. Can you try to boot the node from the latest pve-kernel or the `pve-kernel-6.2: 8.0.5` one?

[0] https://pve.proxmox.com/wiki/Downlo...Proxmox_Virtual_Environment_8.x_to_latest_8.x

 

[geffin]

While those considerations are valid, it's important to note that we are encountering the same issue on a laboratory server. This server is running Proxmox version 8 but is utilizing the pve-kernel (pve-kernel-6.2: 8.0.5).

 

[geffin]

Any other suggestions as to what the issue could be?

 

[Moayad]

Hi,

Have you tried to boot from old pve-kernel?

Can you check if the `/proc/sys/net/bridge/bridge-nf-call-iptables' exist?

sudo ls -la /proc/sys/net/bridge/bridge-nf-call-iptables

Do you have specific parameters in the sysctl?

sysctl -p

 

[geffin]

Our lab environment is encountering a similar problem on Proxmox 8 with the pve-kernel. We haven't switched the other server to the old pve-kernel since it's a production server

The directory named 'bridge' does not exist in the /proc/sys/net/ path. Is it advisable for me to manually create this directory and include the required files?

There are no specific parameters specified in the sysctl configuration

 

[Moayad]

As said, your node is running on a non-Proxmox VE kernel (6.4.0-060400-generic). The available pve-kernel is `6.2.16-15-pve` or `6.2.16-10-pve`. I would recommend switching to the latest Proxmox VE kernel available and testing the pve-firewall.

Or you may try to load the kernel module `br_netfilter` manually

modprobe br_netfilter

 

[geffin]

The firewall rules are functioning correctly following the manual loading of the kernel module br_netfilter, as recommended by you.

 

[Moayad]

I'm glad to hear that manually loading the `br_netfilter` module helped!

Now you see only `Status: enabled/running` in the pve-firewall status right?

 

[geffin]

That is right, the (changes pending) is no longer present. Thanks again for all your help