Fine-Tuning of Spam Recognition

[prahn]

Hi!

I am using PMG for around 4 weeks now. After setup the recognition of spam was really good!
But now, after running it for a while, I have more and more spam coming through...
What can I do to improve the recognition of spam?? Is there any Howto available?

During the last week from Monday to Friday I had only 10 spam mails coming through to my personal mail account.
But on the weekend I had so much mails coming through... on Saturday it had been 22 mails and on Sunday even 62 mails!!

Is there any threshold which I can modify, to be more restrictive?
What else can I do, to make recognition better?

My settings and rules in PMG are almost the default settings, I really don't know where to start improving the settings.
Any help appreciated!

Here is a snippet from my mail.log of a spam mail coming through, which is obviously spam:

May 18 09:57:16 pmg postfix/smtpd[5216]: 0340B22132C: client=unknown[23.247.2.88]
May 18 09:57:16 pmg postfix/cleanup[5226]: 0340B22132C: message-id=<E-seqtRkB_bCVzJgvXOtnToBNoUZwEMgdyCHUSD2JfY.D4GS7eI7FosSrxW5jr9lHT8zkBmANItrZ6jGMhUAOJA@kidoffice.cam>
May 18 09:57:16 pmg postfix/qmgr[474]: 0340B22132C: from=<force@kidoffice.cam>, size=10478, nrcpt=1 (queue active)
May 18 09:57:16 pmg pmg-smtp-filter[4917]: 2021/05/18-09:57:16 CONNECT TCP Peer: "[127.0.0.1]:43846" Local: "[127.0.0.1]:10024"
May 18 09:57:16 pmg pmg-smtp-filter[4917]: 2215EE60A38F7C5B7B5: new mail message-id=<E-seqtRkB_bCVzJgvXOtnToBNoUZwEMgdyCHUSD2JfY.D4GS7eI7FosSrxW5jr9lHT8zkBmANItrZ6jGMhUAOJA@kidoffice.cam>#012
May 18 09:57:16 pmg postfix/smtpd[5216]: disconnect from unknown[23.247.2.88] ehlo=1 mail=1 rcpt=1 data=1 quit=1 commands=5
May 18 09:57:20 pmg pmg-smtp-filter[4917]: 2215EE60A38F7C5B7B5: SA score=2/5 time=4.029 bayes=undefined autolearn=no autolearn_force=no hits=BODY_ENHANCEMENT2(0.1),HTML_MESSAGE(0.001),HTML_MIME_NO_HTML_TAG(0.635),KAM_DMARC_STATUS(0.01),MIME_HTML_ONL
Y(0.1),MIME_QP_LONG_LINE(0.001),RDNS_NONE(1.274),SPF_HELO_PASS(-0.001),SPF_PASS(-0.001)
May 18 09:57:20 pmg postfix/smtpd[5231]: connect from localhost[127.0.0.1]
May 18 09:57:20 pmg postfix/smtpd[5231]: 6E2D92215FB: client=localhost[127.0.0.1], orig_client=unknown[23.247.2.88]
May 18 09:57:20 pmg postfix/cleanup[5226]: 6E2D92215FB: message-id=<E-seqtRkB_bCVzJgvXOtnToBNoUZwEMgdyCHUSD2JfY.D4GS7eI7FosSrxW5jr9lHT8zkBmANItrZ6jGMhUAOJA@kidoffice.cam>
May 18 09:57:20 pmg postfix/qmgr[474]: 6E2D92215FB: from=<force@kidoffice.cam>, size=11360, nrcpt=1 (queue active)
May 18 09:57:20 pmg pmg-smtp-filter[4917]: 2215EE60A38F7C5B7B5: accept mail to <xxx@xxx.de> (6E2D92215FB) (rule: default-accept)

 

[Stoiko Ivanov]

Hm - the one thing that stands out in that mail-log snippet is that the sending server does not have a PTR entry:

client=unknown[23.247.2.88]

enabling: GUI->Configuration->Mail Proxy->Options->Reject Unknown Clients would reject this mail (it translates to adding
'reject_unknown_client_hostname' to the appropriate postfix ACL - see http://www.postfix.org/postconf.5.html#reject_unknown_client_hostname)

If you enable this setting keep an eye on your logs - in certain environments legitimate e-mails can be blocked with this setting.

else the mail does not show any hits for SpamAssassin, which I would consider a good candidate for increasing the score (see section 4.8.3 in https://pmg.proxmox.com/pmg-docs/pmg-admin-guide.html#pmgconfig_spamdetector)

In general I can recommend checking the Getting Started page in the pmg-wiki:
https://pmg.proxmox.com/wiki/index.php/Getting_started_with_Proxmox_Mail_Gateway

I hope this helps!

 

[prahn]

Hi Stoiko!

Thanks for your reply. I enabled "Reject Unknown Clients" as you recommended.

Of course I read the Getting Started page right after setup and added three DNSBLs.
But here I can not find any other hint at all for improving Spam Detection?!

Yes, I also read section 4.8.3, but I don't have any idea how define any custom rule there.

Any more hints for me?

 

[hata_ph]

It would be better if you to provide what is your current spam detection strategy.
What is your mail filter rules?
Did you enable DNSBL and what is your DNSBL/postscreen setting?

 

[prahn]

I don't really have a strategy on my own, I am just using the default rules, that PMG provides.

I enabled DNSBL with these lists:
zen.spamhaus.org,psbl.surriel.com,b.barracudacentral.org

Where do I define postscreen settings?
What mail filter rules would your recommend?

 

[hata_ph]

have a look of some mail filter regex example...
You can always create rules to block/quarantine suspicious email address or domain using what and who object.

https://forum.proxmox.com/threads/mail-filter-example.71904/

 

[poetry]

Mail filtering is way more complex then just setting up something initially and then forgetting about it.
Bad actors will constantly try to pass the spam filter and you will have do constantly adapt and make changes to improve the detection rate.
It also highly depends on how much mail traffic you are filtering. If you are filtering email for just one domain/company then you can really fine tune the mail filter but if you filter for many then this gets really hard to filter without too many false positives.
Keep in mind that every exception you make can be your downfall so think hard how are you adding your exceptions if any. Adding domain name or email address as exception is quite a terrible idea as senders can spoof from address and you will receive dangerous email.
I have written a long post about this a while ago https://forum.proxmox.com/threads/s...x-filter-in-reply-to-field.80037/#post-354681 if you are interested and since then I have done a lot more changes but haven't shared much info here yet.
My advice is to make sure you analyze every spam email you receive and try to make sure you will never received similar email again in a scalable way (if you are adding domain name/ip address or similar to block list this is not scalable at all as bad actors can change domain names more quickly as you adding them to block lists - instead make sure you block email by using as many features as possible from proxmox mail gateway. I like DNSBL Sites as first line of defense and Configuration-Spam Detected - Custom Scores as second line of defense for my particular mail flow. Every spam email we get I check if I can add Custom Scores so every similar email will be blocked in the future.
I might do a big post about what I have done once I feel like it's working very well. There are always ways we can improve spam detection but will also need to make sure the right email will get to our costumers.

 

[prahn]

Thank you for your thoughts and the link to your posting. I read it very precisely.

However, today I have 2 mail-servers bugging our mail-gateway all the time, PMG always rejects their mails, but they continue to try over and over... any idea how to stop this? (besides putting the ip addresses on a deny-list in my firewall!)

This appears in the mail.log file every couple of seconds:

May 21 15:00:56 pmg postfix/smtpd[18047]: connect from unknown[190.210.44.145]
May 21 15:00:56 pmg postfix/smtpd[18047]: NOQUEUE: reject: RCPT from unknown[190.210.44.145]: 450 4.7.25 Client host rejected: cannot find your hostname, [190.210.44.145]; from=<salta@osssb.com.ar> to=<nnn@xxx.de> proto=ESMTP helo=<SRVOSSSBEDGE.osssb.com>
May 21 15:00:57 pmg postfix/smtpd[18047]: NOQUEUE: reject: RCPT from unknown[190.210.44.145]: 450 4.7.25 Client host rejected: cannot find your hostname, [190.210.44.145]; from=<salta@osssb.com.ar> to=<ggg@xxx.de> proto=ESMTP helo=<SRVOSSSBEDGE.osssb.com>
May 21 15:00:57 pmg postfix/smtpd[18047]: NOQUEUE: reject: RCPT from unknown[190.210.44.145]: 450 4.7.25 Client host rejected: cannot find your hostname, [190.210.44.145]; from=<salta@osssb.com.ar> to=<fff@xxx.de> proto=ESMTP helo=<SRVOSSSBEDGE.osssb.com>
May 21 15:00:57 pmg postfix/smtpd[18047]: NOQUEUE: reject: RCPT from unknown[190.210.44.145]: 450 4.7.25 Client host rejected: cannot find your hostname, [190.210.44.145]; from=<salta@osssb.com.ar> to=<eee@xxx.de> proto=ESMTP helo=<SRVOSSSBEDGE.osssb.com>
May 21 15:00:57 pmg postfix/smtpd[18047]: NOQUEUE: reject: RCPT from unknown[190.210.44.145]: 450 4.7.25 Client host rejected: cannot find your hostname, [190.210.44.145]; from=<salta@osssb.com.ar> to=<nnn@xxx.de> proto=ESMTP helo=<SRVOSSSBEDGE.osssb.com>
May 21 15:00:57 pmg postfix/smtpd[18047]: NOQUEUE: reject: RCPT from unknown[190.210.44.145]: 450 4.7.25 Client host rejected: cannot find your hostname, [190.210.44.145]; from=<salta@osssb.com.ar> to=<lll@xxx.de> proto=ESMTP helo=<SRVOSSSBEDGE.osssb.com>
May 21 15:00:58 pmg postfix/smtpd[18047]: NOQUEUE: reject: RCPT from unknown[190.210.44.145]: 450 4.7.25 Client host rejected: cannot find your hostname, [190.210.44.145]; from=<salta@osssb.com.ar> to=<ddd@xxx.de> proto=ESMTP helo=<SRVOSSSBEDGE.osssb.com>
May 21 15:00:58 pmg postfix/smtpd[18047]: NOQUEUE: reject: RCPT from unknown[190.210.44.145]: 450 4.7.25 Client host rejected: cannot find your hostname, [190.210.44.145]; from=<salta@osssb.com.ar> to=<info@xxx.de> proto=ESMTP helo=<SRVOSSSBEDGE.osssb.com>
May 21 15:00:58 pmg postfix/smtpd[18047]: disconnect from unknown[190.210.44.145] ehlo=1 mail=8 rcpt=0/8 rset=7 quit=1 commands=17/25

This IP address is not on any blacklist at all, according to https://mxtoolbox.com/blacklists.aspx

 

[hata_ph]

Is @osssb.com.ar your domain? Is 190.210.44.145 your pmg?

 

[prahn]

No, it's not any IP that I know... and also the domain does not belong to me!

 

[Stoiko Ivanov]

I would not worry too much about that - Some hosts will try to send spam to you - this is quite normal in today's internet.
As long as the mails are not accepted (or at least not delivered to your users) - this is perfectly fine and only causes some lines to be added to your logs.

However in that case the mail is rejected because your PMG cannot resolve the reverse pointer of 190.210.44.145 (and because you have enabled Reject Unknown Clients in your Mail Proxy options).

The IP does have a reverse pointer when I check it from here:

 dig +short -x 190.210.44.145 
mail.osssb.com.ar.

So I would check the DNS settings of your PMG

I hope this helps!

 

[hata_ph]

No, it's not any IP that I know... and also the domain does not belong to me!

Then no worry about it. Mostly is spam mails.

 

[prahn]

Then no worry about it. Mostly is spam mails.

Normally I wouldn't care, but when looking at the Tracking Center in PMG I only see "rejected messages" from this host and the message "aborted by limit (too many hits)". I thought PMG would have a function to block those over-active spammers!

 

[prahn]

The IP does have a reverse pointer when I check it from here:

  dig +short -x 190.210.44.145
mail.osssb.com.ar.

So I would check the DNS settings of your PMG

This is a little strange, when doing a "dig +short -x 190.210.44.145" on the CLI of PMG I get the same result. So DNS looks quite ok.
However, when I ask for the IP with dig "mail.osssb.com.ar" it resolves to a different IP: 190.210.44.147
Maybe this is the reason, why PMG reject the connection?

 

[Stoiko Ivanov]

Maybe this is the reason, why PMG reject the connection?

I don't think so - most likely the PTR record was not there/was not propagated through DNS yet when they tried to send mail to your PMG, but got added later (when we checked)