closing file '/etc/pve/firewall/cluster.fw.tmp.63745' failed - File too large

[LeeS]

Is there an arbitrary filesize limit applied to Proxmox firewall rules? I seem to have hit it today? Can this be increased? Thanks!

 

[dietmar]

Is there an arbitrary filesize limit applied to Proxmox firewall rules? I seem to have hit it today? Can this be increased? Thanks!

Limit is 128KB.

 

[LeeS]

Thank you Dietmar. Can this be increased? Either per cluster, or generally in the next release?

 

[dietmar]

Thank you Dietmar. Can this be increased? Either per cluster, or generally in the next release?

Yes, we can increase that - but I wonder why you need such large files? It looks strange to me that you need such large
files for the firewall. I guess you store some kind of blacklist IPs? If so, I guess we can find a better way (I do not really want
to remove the files size limit).

 

[LeeS]

Yes, we can increase that - but I wonder why you need such large files? It looks strange to me that you need such large
files for the firewall. I guess you store some kind of blacklist IPs? If so, I guess we can find a better way (I do not really want
to remove the files size limit).

I hit the limit while including more sources for testing MAFIA (an automated blacklist firewall manager for proxmox - see this thread for details). It can be managed for now by just not activating all sources, but if another way can be achieved with either a higher limit - or a different/better way of handling the rules? - then that'd be awesome. When IPv6 takes off proper, the available space for rules in that 128kb will greatly diminish, so it couldn't hurt to think about it now.

Thank you for the fast and positive response!

 

[dietmar]

I hit the limit while including more sources for testing MAFIA

I think it makes no sense to store those blacklists on the distributed file system. pmxcfs is not designed to store
large data, because it keeps everything in RAM.

Instead, each node should store that locally. I guess we could extend the firewall to include those local blacklists.
Would that solve the problem?

 

[wbumiller]

Personally I think that if it fetches lists to block from external sources it shouldn't use the shared filesize-limited /etc/pve directory to store the lists. It could just manage an ipset for which you only need 1 firewall rule to block.
Of course if only one node in a cluster should fetch the blacklists then there has to be a way to communicate the data, then this becomes a little more problematic. (Though could be synced via ssh/rsync/...)
Anyway, if the lists are this huge a better strategy might be called for IMO.

 

[LeeS]

It would certainly make more sense for it to be included internally. MAFIA was just a product of "there's not an easy way to do this. Yet."

In todays multi-dozen-gigabyte RAM world though, a few hundred kb dedicated to firewall rules isn't a huge problem? Just thinking out loud here. You guys are the experts. I think it'd certainly be a massive benefit to see something MAFIA-esque included with Proxmox by default. The internet isn't getting any nicer.

Perhaps a default list of sources (for e.g. above) that could be handled separately from the traditional firewall rule propagation method. This would keep the distributed file system happy, and still greatly increase cluster security from known and unknown threats using publicly available data sets.

 

[LeeS]

The simplicity of the cluster firewall model was the driving force, really. It's an excellent way to do something once, and protect everything. I don't disagree that there is probably a better way though.

I'm not fussed about the future of my idea as a standalone. What I want is for PVE to be the go-to virtualisation platform. To my knowledge this isn't something anything else really has a feature, and dynamic adaptive security is always a massive feature.

 

[dietmar]

In todays multi-dozen-gigabyte RAM world though, a few hundred kb dedicated to firewall rules isn't a huge problem? Just thinking out loud here.

wasting RAM is always a bad idea.

 

[LeeS]

wasting RAM is always a bad idea.

I cannot argue there. But a few kb for added security isn't necessarily a waste.

Let me know if I can help with any ideas you come up with?

 

[webix]

Hello,
I am running the version 3.4-11 of proxmox and can't update to version 4.0 right now.
I need to block some countries subnets (yes the whole country) on proxmox. And i am limited with the 128Kb filesize.
I don't want to do it on each VPS but on the node itself to cover all vps's.

Can you please explain how-to change that limit? I need a minimum of 2Mb filesize to apply all rules and ipsets.

Regards

 

[webix]

I don't like to bump threads... but it's almost a week that i asked how to increase the limite of the filesize and i didn't had any answer.

 

[dietmar]

We will not increase that limit.

 

[webix]

hi dietmar. I am not asking you to increase that limit.
I am asking you to explain how can i increase the limit because i need it increased.

 

[cyfr]

Hate to revive an old thread but I feel I have to vote this up as well. Ipset's specific purpose is its ability to densify large indexes such as Country IP CIDRs for blocking actions. I'm making use of such lists currently on our telco switches. One primary reason I have interest in cluster.fw ipset is the ability to drop at the level, then on the VM switch level while viewing sngrep flows, it would be nice and clean. To accomplish this @webix we're probably better off running a separate VM for such purposes to layer fire-walling and proxy route. Just my two cents. Or, a simple to implement feature to cluster.fw is the addition of [includes] allowing reference to call such large files separately, no need in increasing its size. Outside of that the addition of leveraging hash:net maxelem when creating ipset containers in RAM before loading them. Take a look at projects firehol and iprange and deeper into ipset as they've made leaps of progress in using very little RAM for massive lists.

PS I discovered the limitation moments ago loading just the U.S. CIDR list alone.

 

[cyfr]

@webix try running lsof against your tmp file in the threads subject. It should point you to the application thats monitoring and closing the tmp file.