[TUTORIAL] PMG 7/Debian 11 with fail2ban

H

hata_ph

Well-Known Member
Nov 13, 2019
866
199
48
45
If you enable PMG's webUI access from the public internet, you may notice alot of authentication failure in /var/log/daemon.log.
Setup fail2ban to block/ban suspicious authentication failure attempts.

IMPORTANT: This setup was intended on a new setup of PMG 7.x. If you upgrade from previous version, double check the setup is using nftables or iptables or any old lingering fail2ban configuration.

Code: 
Sep 13 11:39:00 pmg pmgdaemon[32409]: authentication failure; rhost=::ffff:183.253.84.171 user=root@pam msg=auth failed: Authentication failure

1. Install fail2ban

Code: 
apt-get install fail2ban

2. Edit /etc/fail2ban/jail.d/defaults-debian.conf with below settings to enable fail2ban to use nftables instead of iptables as debian have shipped with nftables by default since version 10. Do this step if you want to use the default nftables instead of iptables, or else just ignore this step and install iptables on your PMG 7.x.

Code: 
[DEFAULT]
banaction = nftables-multiport
banaction_allports = nftables-allports

3. Create /etc/fail2ban/jail.d/pmg-web-auth.conf with below settings. Actually only port 8006 is needed. https/http is optional.

Code: 
[pmg-web-auth]
enabled = true
port = https,http,8006
filter = pmg-web-auth
logpath = /var/log/daemon.log
maxretry = 2
# 1 hour
bantime = 3600

4. Create /etc/fail2ban/filter.d/pmg-web-auth.conf with below settings.

Code: 
[Definition]
failregex = pmgdaemon\[.*authentication failure; rhost=<HOST> user=.* msg=.*
ignoreregex =

5. Run below command to test the filter.

Code: 
root@pmg:~# fail2ban-regex /var/log/daemon.log /etc/fail2ban/filter.d/pmg-web-auth.conf

Running tests
=============

Use   failregex filter file : pmg-web-auth, basedir: /etc/fail2ban
Use         log file : /var/log/daemon.log
Use         encoding : UTF-8


Results
=======

Failregex: 289 total
|-  #) [# of hits] regular expression
|   1) [289] pmgdaemon\[.*authentication failure; rhost=<HOST> user=.* msg=.*
`-

Ignoreregex: 0 total

Date template hits:
|- [# of hits] date format
|  [1904] {^LN-BEG}(?:DAY )?MON Day %k:Minute:Second(?:\.Microseconds)?(?: ExYear)?
`-

Lines: 1904 lines, 0 ignored, 289 matched, 1615 missed
[processed in 0.07 sec]

Missed line(s): too many to print.  Use --print-all-missed to print all 1615 lines
root@pmg:~#

6. Optional to enable recidive to ban recurrent IP. Add below settings to /etc/fail2ban/jail.d/defaults-debian.conf or create new /etc/fail2ban/jail.d/recidive.conf.

Code: 
[recidive]
enabled = true
maxretry = 3

7. Restart fail2ban services with the new settings.

Code: 
systemctl restart fail2ban

8. Run fail2ban-client to check the jail status.

Code: 
root@pmg:~# fail2ban-client status pmg-web-auth
Status for the jail: pmg-web-auth
|- Filter
|  |- Currently failed: 6
|  |- Total failed:     187
|  `- File list:        /var/log/daemon.log
`- Actions
   |- Currently banned: 0
   |- Total banned:     19
   `- Banned IP list:
root@pmg:~#
root@pmg:~# fail2ban-client status recidive
Status for the jail: recidive
|- Filter
|  |- Currently failed: 2
|  |- Total failed:     19
|  `- File list:        /var/log/fail2ban.log
`- Actions
   |- Currently banned: 67
   |- Total banned:     67
   `- Banned IP list:   103.247.23.241 110.137.36.141 110.169.33.171 114.34.143.126 116.4.97.240 118.70.169.219 122.116.169.87 122.116.226.223 124.90.112.180 125.70.238.161 128.201.97.48 138.201.30.76 139.91.58.2 14.154.30.97 148.251.128.232 168.197.154.149 168.90.211.32 170.78.40.35 170.78.96.97 170.83.16.221 175.138.106.77 175.44.42.19 177.156.170.10 177.170.20.12 177.35.242.45 178.217.216.117 179.159.214.35 183.253.70.11 185.230.76.44 186.23.236.4 187.189.5.102 188.124.228.14 189.39.103.150 189.85.150.143 193.33.88.181 196.75.2.149 200.112.210.165 201.139.170.153 202.138.244.37 209.141.53.116 210.240.163.93 222.69.254.18 223.70.201.178 223.73.88.191 31.27.213.108 36.66.190.197 36.91.14.2 45.179.177.254 45.181.230.54 5.185.17.11 58.32.32.133 71.199.58.165 77.37.250.109 81.192.169.51 82.202.176.41 82.202.176.42 82.202.176.43 83.110.17.121 87.255.198.188 87.5.193.144 87.5.6.99 87.92.54.24 14.154.31.171 179.176.73.149 183.253.70.154 196.75.21.130 81.248.68.120
root@pmg:~#

9. Check nftables rule status.

Code: 
root@pmg:~# nft list table inet f2b-table
table inet f2b-table {
        set addr-set-recidive {
                type ipv4_addr
                elements = { 5.185.17.11, 14.154.30.97,
                             14.154.31.171, 31.27.213.108,
                             36.66.190.197, 36.91.14.2,
                             45.179.177.254, 45.181.230.54,
                             58.32.32.133, 71.199.58.165,
                             77.37.250.109, 81.192.169.51,
                             81.248.68.120, 82.202.176.41,
                             82.202.176.42, 82.202.176.43,
                             83.110.17.121, 87.5.6.99,
                             87.5.193.144, 87.92.54.24,
                             87.255.198.188, 103.247.23.241,
                             110.137.36.141, 110.169.33.171,
                             114.34.143.126, 116.4.97.240,
                             118.70.169.219, 122.116.169.87,
                             122.116.226.223, 124.90.112.180,
                             125.70.238.161, 128.201.97.48,
                             138.201.30.76, 139.91.58.2,
                             148.251.128.232, 168.90.211.32,
                             168.197.154.149, 170.78.40.35,
                             170.78.96.97, 170.83.16.221,
                             175.44.42.19, 175.138.106.77,
                             177.35.242.45, 177.156.170.10,
                             177.170.20.12, 178.217.216.117,
                             179.159.214.35, 179.176.73.149,
                             183.253.70.11, 183.253.70.154,
                             185.230.76.44, 186.23.236.4,
                             187.189.5.102, 188.124.228.14,
                             189.39.103.150, 189.85.150.143,
                             193.33.88.181, 196.75.2.149,
                             196.75.21.130, 200.112.210.165,
                             201.139.170.153, 202.138.244.37,
                             209.141.53.116, 210.240.163.93,
                             222.69.254.18, 223.70.201.178,
                             223.73.88.191 }
        }

        set addr-set-pmg-web-auth {
                type ipv4_addr
        }

        chain f2b-chain {
                type filter hook input priority filter - 1; policy accept;
                meta l4proto { tcp } ip saddr @addr-set-recidive reject
                tcp dport { 80, 443, 8006 } ip saddr @addr-set-pmg-web-auth reject
        }
}
root@pmg:~#

10. To enable email notification, add below settings to /etc/fail2ban/jail.d/defaults-debian.conf or create new /etc/fail2ban/jail.d/mail-notification.conf. But there is a bug to use bsd-mailx's mail program with fail2ban, so you may need to use mailutils or manually patch the codes in /etc/fail2ban/action.d/mail-*.conf.
My suggestion is just create /etc/fail2ban/action.d/bsd-mailx-*.conf from /etc/fail2ban/action.d/mail-*.conf with the code changes.

Code: 
[DEFAULT]
mta = mail # use default mail program
#mta = bsd-mailx # use default mail program
destemail = root@localhost # Destination email address
sender = root@<fq-hostname> # Sender email address

https://pve.proxmox.com/wiki/Fail2ban
 
Last edited:
  • Like
Reactions: michabbs, mhaluska, guletz and 1 other person
Huge thanks for sharing this!

Without actively trying it - I'd say that having fail2ban setup correctly is a good idea if you have your WebUI accessible on the internet.

Alternatively you could also setup a nginx-proxy and limit access to it per/IP (and let pmgproxy only listen to localhost) - but of course if you need access from the complete public internet that still would need some kind of fail2ban protection.
 
  • Like
Reactions: hata_ph
hata_ph said:
with below settings to enable fail2ban to use nftables instead of iptables as debian have transition to nftables since version 10.
Click to expand...

Hi @hata_ph ,


In my opinion, I think that iptables/nftables, is not the best option. If you will have many bloked IPs, then your firewall rules could be huge.
It could be better, to use route(null route) option in fail2ban, insted of nftables/iptables. The main advantages is the fact that route table has cache, it will consume less resurces, and it is faster then any firewall. If a host will have a null route, then he will not receive any response, and will not need to be analysed by the firewall.

And if you have several host direct connected to the Internet(+fail2ban), you could use a dynamic routing protocol(like ospf) and export / import this "bad actors" on all of your systems, and not only on your PMG. Even better if you have a border router(ospf capable) ....


Good luck / Bafta!
 
  • Like
Reactions: hata_ph
Hi @guletz,

Thanks for your suggestion.

To use null route instead of nftables/iptables, configure /etc/fail2ban/jail.d/defaults-debian.conf with route as banaction.

Code: 
[DEFAULT]
banaction = route

But I do notice 2 disadvantage of using route instead nftable/iptables.

1. Cannot use recidive jail as it require banaction_allports setting and it only work with nftable/iptables.
2. Null route blocking is per IP. With nftables/iptables, I can block by ports too.
 
hata_ph said:
1. Cannot use recidive jail as it require banaction_allports setting and it only work with nftable/iptables.
Click to expand...
Agree with this.


hata_ph said:
2. Null route blocking is per IP. With nftables/iptables, I can block by ports too
Click to expand...

Block per IP, is the same with block any port for such IP. But block only some ports for a designated IP is a mistake in my opinion.

Good luck / Bafta !
 
Do you know why the port directive in the jail.local file includes https and http as well as 8006, which is the default port? I noticed it's the way the Proxmox wiki recommends it, but the reason is not stated there either.

port = https,http,8006

Also, the file /etc/fail2ban/jail.d/defaults-debian.conf in Debian Bullseye's fail2ban package includes these line:

Code: 
[sshd]
enabled = true

Do you know whether your default options to use nftables have to go first or whether it does not really matter?

Code: 
[DEFAULT]
banaction = nftables-multiport
banaction_allports = nftables-allports

Also regarding nftables, do you let fail2ban work using nftables while you keep Proxmox using iptables from the rules you add in the GUI, or does Proxmox 7 support nftables and I missed that one?

Thanks.
 
Last edited:
1. PMG's web use HTTPS with port 8006, so you need to monitor both.
2. fail2ban enable monitor sshd port by default. You can disable it if you not open sshd to internet.
3. PMG 7 only support debian 11 and debian use nftables firewall by default, so configure fail2ban with nftables. If you prefer iptables, you have to manually remove nftables and install iptables yourself. Btw, I don't believe PMG itself use iptables/nftables firewall.
 
  • Like
Reactions: jsabater
Hi all, I did a further testing with only monitoring port 8006 and the ban work too. So port https and http is not need in the pmg-web-auth jail.
Pls test it first before run it at your production server.
 
hata_ph said:
Hi all, I did a further testing with only monitoring port 8006 and the ban work too. So port https and http is not need in the pmg-web-auth jail.
Pls test it first before run it at your production server.
Click to expand...

Your tutorial is OK. But is not so good in our real world. In many cases, "bad actors" use let say "distributed brute force atack", and they are very smart, in few hours, they adjust the test/uniq IP, according with your ban-time. I agree that fail2ban is better then nothing, but is not so good.

Is much better is to restrict access to your "golden gate" like ssh, pmx-web-interface, and so on by IP, or ddns host name of your admin sistems(also a vpn is more secure then any IP/ddns ACL)

Good luck / Bafta !
 
Hi guys.
I can't even "apt-get install fail2ban", I receive this error:
Code: 
root@proxmoxmg1:~# apt-get update && apt-get install fail2ban
Hit:1 http://download.proxmox.com/debian/pmg bullseye InRelease
Reading package lists... Done
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
E: Unable to locate package fail2ban

Am I missing a repository?
Capture d’écran 2021-11-06 195113.png

Thanks.
 
cglmicro said:
Hi guys.
I can't even "apt-get install fail2ban", I receive this error:
Code: 
root@proxmoxmg1:~# apt-get update && apt-get install fail2ban
Hit:1 http://download.proxmox.com/debian/pmg bullseye InRelease
Reading package lists... Done
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
E: Unable to locate package fail2ban

Am I missing a repository?
View attachment 31149

Thanks.
Click to expand...
You need to enable the debian repository to install the packages.
 
cglmicro said:
Hi guys.
I can't even "apt-get install fail2ban", I receive this error:
Code: 
root@proxmoxmg1:~# apt-get update && apt-get install fail2ban
Hit:1 http://download.proxmox.com/debian/pmg bullseye InRelease
Reading package lists... Done
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
E: Unable to locate package fail2ban

Am I missing a repository?
View attachment 31149

Thanks.
Click to expand...

Your packages repositories are configured, but disabled. You need to tick each box. For your information, fail2ban is in the main section of the first one:

deb http://ftp.de.debian.org/debian bullseye main contrib non-free

But I'd definitely tick the three of them unless this is an isolated test scenario and you know what you are doing.
 
Thanks HATA_PH and JSABATER.
It was, for a long time, test machine that became a prod one. Can't even reme,ber why I disabled it in the first place.
It's working, thanks again.
 
Hi again.

My Fail2ban was logging IPs in it's table, but the command "nft list table inet f2b-table" was returning an error, and I was still able to see hacker trying to get in even if Fail2ban had added their IP.

For those who encounter the same problem, here is what was causing it for me: I did a copy of /etc/fail2ban/jail.conf to /etc/fail2ban/jail.local so I can tweak some values like the default bantime that sshd was referring to. By doing this, the instructions of this file was ignored:
/etc/fail2ban/jail.d/defaults-debian.conf
[DEFAULT]
banaction = nftables-multiport
banaction_allports = nftables-allports

On my newest slave that use nftables, I had to modify my jail.local to comment out:
#banaction = iptables-multiport
#banaction_allports = iptables-allports

On my oldest master that use iptables (was updated from debian buster), I had to keep iptables.

Thanks for this tutorial.
 
Last edited:
  • Like
Reactions: hata_ph
cglmicro said:
Hi again.

My Fail2ban was logging IPs in it's table, but the command "nft list table inet f2b-table" was returning an error, and I was still able to see hacker trying to get in even if Fail2ban had added their IP.

For those who encounter the same problem, here is what was causing it for me: I did a copy of /etc/fail2ban/jail.conf to /etc/fail2ban/jail.local so I can tweak some values like the default bantime that sshd was referring to. By doing this, the instructions of this file was ignored:
/etc/fail2ban/jail.d/defaults-debian.conf
[DEFAULT]
banaction = nftables-multiport
banaction_allports = nftables-allports

On my newest slave that use nftables, I had to modify my jail.local to comment out:
#banaction = iptables-multiport
#banaction_allports = iptables-allports

On my oldest master that use iptables (was updated from debian buster), I had to keep iptables.

Thanks for this tutorial.
Click to expand...
That is very disturbing news, @cgimicro. I have exactly the same content in my /etc/fail2ban/jail.d/defaults-debian.conf.

What version of fail2ban were you using? I am using Debian Bullseye with Proxmox 7.

Were you using the same or Debian Buster with Proxmox 6?

I have not been able to find any bug report regarding this. Do you know of any?
 
Hi jsabater.

If you only have /etc/fail2ban/jail.d/defaults-debian.conf and no /etc/fail2ban/jail.local (for advance customization), you are ok.

Fail2ban was not installed prior to my buster->bullseye and pmg6.x->pmg7.x upgrade.

I have the latest Fail2ban version.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back