Proxmox Firewall Doesn't seem to work and errors in log

T

TheDragon

Member
Jan 20, 2023
9
2
8
I'm trying to start making use of the Proxmox Firewall at Node/VM/NIC level. I've enabled the firewall at datacenter and node level initially, but the rules I've put in place don't seem to take effect, and I'm also getting these lines repeatedly in my PVE logs:

Jun 03 12:32:57 pve pve-firewall[3481]: status update error: ipset_restore_cmdlist: ipset v7.10: Error in line 32: Element cannot be added to the set: it's already added Jun 03 12:33:07 pve pve-firewall[3481]: status update error: ipset_restore_cmdlist: ipset v7.10: Error in line 14: Element cannot be added to the set: it's already added Jun 03 12:33:18 pve pve-firewall[3481]: status update error: ipset_restore_cmdlist: ipset v7.10: Error in line 14: Element cannot be added to the set: it's already added Jun 03 12:33:27 pve pve-firewall[3481]: status update error: ipset_restore_cmdlist: ipset v7.10: Error in line 14: Element cannot be added to the set: it's already added Jun 03 12:33:37 pve pve-firewall[3481]: status update error: ipset_restore_cmdlist: ipset v7.10: Error in line 32: Element cannot be added to the set: it's already added Jun 03 12:33:47 pve pve-firewall[3481]: status update error: ipset_restore_cmdlist: ipset v7.10: Error in line 4: Element cannot be added to the set: it's already added Jun 03 12:33:57 pve pve-firewall[3481]: status update error: ipset_restore_cmdlist: ipset v7.10: Error in line 32: Element cannot be added to the set: it's already added Jun 03 12:34:07 pve pve-firewall[3481]: status update error: ipset_restore_cmdlist: ipset v7.10: Error in line 22: Element cannot be added to the set: it's already added Jun 03 12:34:17 pve pve-firewall[3481]: status update error: ipset_restore_cmdlist: ipset v7.10: Error in line 27: Element cannot be added to the set: it's already added

Any assistance with this would be greatly appreciated!
 
spirit said:
can you send content of

cat /var/lib/pve-firewall/ipsetcmdlist2
cat /var/lib/pve-firewall/ipsetcmdlist1

?
Click to expand...
Thanks for the reply, see below the output from ipsetcmdlist1, ipsetcmdlist2 returned nothing.

destroy PVEFW-0-admin_devices-v4_swap create PVEFW-0-router_40-v4_swap hash:net family inet hashsize 64 maxelem 64 bucketsize 12 add PVEFW-0-router_40-v4_swap 10.0.40.1/24 swap PVEFW-0-router_40-v4_swap PVEFW-0-router_40-v4 flush PVEFW-0-router_40-v4_swap destroy PVEFW-0-router_40-v4_swap create PVEFW-0-router_20-v4_swap hash:net family inet hashsize 64 maxelem 64 bucketsize 12 add PVEFW-0-router_20-v4_swap 10.0.20.1/24 swap PVEFW-0-router_20-v4_swap PVEFW-0-router_20-v4 flush PVEFW-0-router_20-v4_swap destroy PVEFW-0-router_20-v4_swap create PVEFW-17B4F8F3_swap hash:net family inet hashsize 64 maxelem 64 bucketsize 12 add PVEFW-17B4F8F3_swap 10.0.0.1/24 add PVEFW-17B4F8F3_swap 10.0.20.1/24 add PVEFW-17B4F8F3_swap 10.0.30.1/24 add PVEFW-17B4F8F3_swap 10.0.40.1/24 swap PVEFW-17B4F8F3_swap PVEFW-17B4F8F3 flush PVEFW-17B4F8F3_swap destroy PVEFW-17B4F8F3_swap create PVEFW-0-router_10-v4_swap hash:net family inet hashsize 64 maxelem 64 bucketsize 12 add PVEFW-0-router_10-v4_swap 10.0.10.1/24 swap PVEFW-0-router_10-v4_swap PVEFW-0-router_10-v4 flush PVEFW-0-router_10-v4_swap destroy PVEFW-0-router_10-v4_swap create PVEFW-0-admin_devices-v4_swap hash:net family inet hashsize 64 maxelem 64 bucketsize 12 add PVEFW-0-admin_devices-v4_swap 10.0.20.50/24 add PVEFW-0-admin_devices-v4_swap 10.0.20.51/24 swap PVEFW-0-admin_devices-v4_swap PVEFW-0-admin_devices-v4 flush PVEFW-0-admin_devices-v4_swap destroy PVEFW-0-admin_devices-v4_swap create PVEFW-0-router_30-v4_swap hash:net family inet hashsize 64 maxelem 64 bucketsize 12 add PVEFW-0-router_30-v4_swap 10.0.30.1/24 swap PVEFW-0-router_30-v4_swap PVEFW-0-router_30-v4 flush PVEFW-0-router_30-v4_swap destroy PVEFW-0-router_30-v4_swap
 
I just ran into the same issue. I had added a couple of IP addresses using CIDR notation (x.x.x.x/24) as aliases. I was able to resolve the issue by changing all of them to IP only (x.x.x.x). So my issue is closed

But I am wondering : Would it make sense to improve the GUI to avoid this kind of issue? Add a validation that prevents invalid aliases in the Firewall section?
 
spirit said:
Hi,
if ips like:

"10.0.20.1/24"

are single ip, you shouldn't add /24.

you could use /32 ("10.0.20.1/32") or no netmask. ("10.0.20.1")

if you need to defined a full subnet , you should use netmask, like "10.0.20.0/24"
Click to expand...
My complete cluster.rule
Code: 
root@pve02:~# cat /etc/pve/firewall/cluster.fw
[OPTIONS]

enable: 1

[ALIASES]

all 0.0.0.0/0 # 0.0.0.0/0
along2 172.28.158.0/24 # along
chending251 172.22.251.0/24 # chending251
dropmaill 47.100.108.181
floor3 172.31.9.0/24 # 172.31.9.0/24
floor4 172.31.7.0/24 # 172.31.7.0/24
floor4_2 172.31.8.0/24 # 172.31.8.0/24
lab1 172.28.160.0/24
poolbj 172.27.128.54
pveself 172.28.188.0/24 # ziji
self2 172.28.189.0/24 # 172.28.189.0/24
vpn 192.168.210.0/24 # 192.168.210.0/24
zabbix 172.28.152.250

[IPSET blacklist] # 不允许访问

dc/all
dc/dropmaill

[IPSET whitelist] # 白名单列表

dc/along2
dc/chending251
dc/floor3
dc/pveself
dc/self2
dc/vpn
dc/zabbix

[group default] # 默认-禁止陌生访问

IN ACCEPT -source +dc/whitelist -dest +dc/whitelist -log info
IN DROP -source dc/all -log info
OUT DROP -dest +dc/blacklist -log info

[group hw] # 护网期间限制访问

IN DROP -source dc/all -dest dc/all -log info
OUT ACCEPT -source dc/all -dest dc/all -log info

[group rejectall] # 拒绝集群外访问

Code: 
Nov 04 10:57:08 pve02 pve-firewall[3343507]: status update error: ipset_restore_cmdlist: ipset v7.17: Error in line 2: The value of the CIDR parameter of the IP address is invalid
Nov 04 10:57:18 pve02 pve-firewall[3343507]: status update error: ipset_restore_cmdlist: ipset v7.17: Error in line 14: The value of the CIDR parameter of the IP address is invalid
Nov 04 10:57:28 pve02 pve-firewall[3343507]: status update error: ipset_restore_cmdlist: ipset v7.17: Error in line 3: The value of the CIDR parameter of the IP address is invalid

Code: 
root@pve02:~# cat /var/lib/pve-firewall/ipsetcmdlist1
destroy PVEFW-0-blacklist-v4_swap
create PVEFW-0-blacklist-v4_swap hash:net family inet hashsize 64 maxelem 64 bucketsize 12
add PVEFW-0-blacklist-v4_swap 0.0.0.0/0
add PVEFW-0-blacklist-v4_swap 47.100.108.181
swap PVEFW-0-blacklist-v4_swap PVEFW-0-blacklist-v4
flush PVEFW-0-blacklist-v4_swap
destroy PVEFW-0-blacklist-v4_swap
root@pve02:~# cat /var/lib/pve-firewall/ipsetcmdlist2
root@pve02:~#

Hello, could you please take a look at my error log?

To add to this, my cluster of 9 machines has three different kernel versions. I'm not sure if this is related. I'll try updating the kernel tonight.
Code: 
pve-manager/8.3.1/fb48e850ef9dde27 (running kernel: 6.8.12-5-pve)
pve-manager/8.3.1/fb48e850ef9dde27 (running kernel: 6.5.13-3-pve)
pve-manager/8.3.1/fb48e850ef9dde27 (running kernel: 6.1.0-18-amd64)
 
Last edited:
You must log in or register to reply here.
Top Bottom