DNSBL not working (URIBL_BLACK & URIBL_DBL_SPAM not showing in headers)

[g00gle]

Hi All,

I've trouble with the DNSBL Settings in Mail Proxy -> Options.
My settings are as followed:

I'm running on PMG version 6.0-5

The problem is, that I'm receiving loads of spam from known "Blacklisted IP" senders. For example:

The X-SPAM-LEVEL information from the headers:

X-SPAM-LEVEL: Spam detection results:  2
    BAYES_05                 -0.5 Bayes spam probability is 1 to 5%
    DKIM_INVALID              0.1 DKIM or DK signature exists, but is not valid
    DKIM_SIGNED               0.1 Message has a DKIM or DK signature, not necessarily valid
    FROM_SUSPICIOUS_NTLD    0.499 From abused NTLD
    HTML_MESSAGE            0.001 HTML included in message
    KAM_ADVERT2              0.75 This is probably an unwanted commercial email...
    KAM_NUMSUBJECT            0.5 Subject ends in numbers excluding current years
    RDNS_NONE               0.793 Delivered to internal network by a host with no rDNS
    SPF_HELO_PASS          -0.001 SPF: HELO matches SPF record
    SPF_PASS               -0.001 SPF: sender matches SPF record
    T_REMOTE_IMAGE           0.01 Message contains an external image

But because the sending IP is listed at the DNSBL sites it shouldn't get through the mail proxy's. The spam level is below the level of 3. But It should be blocked anyway because the IP of the sending party is listed at zen.spamhaus.org for example.

root@mailgateway01:/etc# host 50.0.131.45.zen.spamhaus.org
50.0.131.45.zen.spamhaus.org has address 127.0.0.3

We use our own DNS servers, so we don't make use of google or etc.

As you can see the in the mail headers URIBL_BLACK & URIBL_DBL_SPAM are completely missing, So I think it's not working at all.
I hope someone can help me with some advise.

Thank you all!

 

[g00gle]

As addition to the post above. I see mails being blocked in the logs:

root@mailgateway01:/var/log# grep -nir 'Sep  3' mail.info | grep 'blocked using' | tail
51280:Sep  3 09:35:42 mailgateway01 postfix/postscreen[24833]: NOQUEUE: reject: RCPT from [144.217.19.18]:38631: 550 5.7.1 Service unavailable; client [144.217.19.18] blocked using zen.spamhaus.org; from=<elise@acrevi.store>, to=<info@<strip>.nl>, proto=ESMTP, helo=<yoga.acrevi.store>
51537:Sep  3 09:46:03 mailgateway01 postfix/postscreen[25904]: NOQUEUE: reject: RCPT from [103.139.208.21]:53369: 550 5.7.1 Service unavailable; client [103.139.208.21] blocked using zen.spamhaus.org; from=<mpbmdecidedly@3z.net>, to=<support@<strip>.nl>, proto=ESMTP, helo=<myanmarbeelive.com>
51701:Sep  3 10:00:41 mailgateway01 postfix/postscreen[27616]: NOQUEUE: reject: RCPT from [77.83.200.179]:40736: 550 5.7.1 Service unavailable; client [77.83.200.179] blocked using zen.spamhaus.org; from=<info@shaxiapercent.top>, to=<orders@<strip>.nl>, proto=ESMTP, helo=<shaxiapercent.top>
51724:Sep  3 10:01:20 mailgateway01 postfix/postscreen[27616]: NOQUEUE: reject: RCPT from [41.249.186.175]:53901: 550 5.7.1 Service unavailable; client [41.249.186.175] blocked using zen.spamhaus.org; from=<ckzcomparative@carecompanions.com>, to=<support@<strip>.nl>, proto=ESMTP, helo=<menara.ma>
51865:Sep  3 10:08:31 mailgateway01 postfix/postscreen[28744]: NOQUEUE: reject: RCPT from [91.93.73.148]:57413: 550 5.7.1 Service unavailable; client [91.93.73.148] blocked using b.barracudacentral.org; from=<RichardMillerbania@superonline.net>, to=<mrjr_93bwce@<strip>.nl>, proto=ESMTP, helo=<host-91-93-73-148.reverse.superonline.net>
52697:Sep  3 10:20:36 mailgateway01 postfix/postscreen[30338]: NOQUEUE: reject: RCPT from [2.153.101.250]:40469: 550 5.7.1 Service unavailable; client [2.153.101.250] blocked using zen.spamhaus.org; from=<rz@rosannazanetta.eu>, to=<support@<strip>.nl>, proto=ESMTP, helo=<2.153.101.250.dyn.user.ono.com>
52798:Sep  3 10:27:04 mailgateway01 postfix/postscreen[31113]: NOQUEUE: reject: RCPT from [195.66.207.18]:35821: 550 5.7.1 Service unavailable; client [195.66.207.18] blocked using zen.spamhaus.org; from=<RyanFowler@od.ua>, to=<support@<strip>.nl>, proto=ESMTP, helo=<229-123.sky.od.ua>
53083:Sep  3 10:41:18 mailgateway01 postfix/postscreen[706]: NOQUEUE: reject: RCPT from [81.177.73.26]:38181: 550 5.7.1 Service unavailable; client [81.177.73.26] blocked using zen.spamhaus.org; from=<TonyTorres@locus.it>, to=<info@<strip>.nl>, proto=ESMTP, helo=<locus.it>
53298:Sep  3 10:46:29 mailgateway01 postfix/postscreen[1250]: NOQUEUE: reject: RCPT from [89.24.119.126]:59981: 550 5.7.1 Service unavailable; client [89.24.119.126] blocked using zen.spamhaus.org; from=<WalterLanepvdyq@tmcz.cz>, to=<elfe@<strip>.nl>, proto=ESMTP, helo=<89-24-119-126.customers.tmcz.cz>
53429:Sep  3 10:57:25 mailgateway01 postfix/postscreen[2917]: NOQUEUE: reject: RCPT from [203.83.183.11]:42643: 550 5.7.1 Service unavailable; client [203.83.183.11] blocked using dnsrbl.org; from=<CharlesScottpuaof@dmic.org.bd>, to=<dex@<strip>.nl>, proto=ESMTP, helo=<lists.dmic.org.bd>

Also added 2 blacklist services:
dnsrbl.org and bl.blocklist.de

But before that the "blocked using <provider>" where already there. So I don't get it.

 

[Stoiko Ivanov]

Have you also enabled 'Use RBL checks' in 'Configuration' -> 'Spam Detector' -> 'Options'?
PMG uses RBLs on two levels:
* before-queue with postscreen (which is where the rejects you posted in your second post come from - so this should be working
* within the pmg-smtp-filter/spamdetector/rule-system - where SpamAssassin checks the mail against various RBLs (where the headers should be coming from, if it were active and an IP triggered the RBL-checks

An alternative explanation could be, that the ip was not listed yet, when you received the mail, but entered the RBL later (before you checked)

I hope this helps!

 

[g00gle]

Thank you for your answer @Stoiko Ivanov.

I've checked the RBL checks and it's enabled:

Thing is I can't find any email that does contain the 'URIBL_*' information in the headers. So based on that I think something is configured wrong or not working as supposed to.

On your alternative explanation, I thought about that as well, but it's really difficult to be sure about that. Because it's unknown on what time the IP's are added to the blacklists. Or is there a way?

The mail gateway's are working really good from the start we using it. And they seem to block loads off spam.. But since a few days/weeks we receive about 20 to 25 spam mails each day on some of our mail addresses and this was never the case before. I changed nothing except the usual upgrades. And this was already happening before the upgrade to v6.0 so I think thats not related.

 

[Stoiko Ivanov]

hmm - one specific of spamassassin is that it only uses the first configured DNS-server from /etc/resolv.conf (postfix and most other programs cope better with one not-functioning DNS (but it does slow things down as well) - does that work in your configuration?

 

[g00gle]

There are 2 internal nameservers configured that we run on powerdns-recursor. There are no settings made regarding caching, so this should be the default.

But if I get your suggestion correct I should add a not functioning DNS server to make postfix work better? It seems me to be a very odd solution for this issue.

And still I can't seem to find any DNSBL_* information in the headers of inbound mails. Any suggestions why this is? The settings seem correct to me.

 

[Stoiko Ivanov]

But if I get your suggestion correct I should add a not functioning DNS server to make postfix work better? It seems me to be a very odd solution for this issue.

No - not at all! if both nameservers do work correctly then this is not an issue.
If the first one does not work then spamassassin cannot use RBL-checks. (see the FAQ on https://cwiki.apache.org/confluence/display/spamassassin/DnsBlocklists)

if your first DNS-server entry in '/etc/resolv.conf' can resolv the RBL-queries (check with `drill` or `dig`) and RBL-checks are enabled for the spamdetector I can't see where the problem might come from

 

[g00gle]

I think it's working fine:

root@mailgateway01:~# dig 50.0.131.45.zen.spamhaus.org @10.0.0.71

; <<>> DiG 9.11.5-P4-5.1-Debian <<>> 50.0.131.45.zen.spamhaus.org @10.0.0.71
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 25797
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;50.0.131.45.zen.spamhaus.org.    IN    A

;; ANSWER SECTION:
50.0.131.45.zen.spamhaus.org. 300 IN    A    127.0.0.3

;; Query time: 1798 msec
;; SERVER: 10.0.0.71#53(10.0.0.71)
;; WHEN: Tue Sep 03 12:09:51 CEST 2019
;; MSG SIZE  rcvd: 73

root@mailgateway01:~# dig 50.0.131.45.zen.spamhaus.org @10.0.0.72

; <<>> DiG 9.11.5-P4-5.1-Debian <<>> 50.0.131.45.zen.spamhaus.org @10.0.0.72
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 5443
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;50.0.131.45.zen.spamhaus.org.    IN    A

;; ANSWER SECTION:
50.0.131.45.zen.spamhaus.org. 300 IN    A    127.0.0.3

;; Query time: 4867 msec
;; SERVER: 10.0.0.72#53(10.0.0.72)
;; WHEN: Tue Sep 03 12:10:00 CEST 2019
;; MSG SIZE  rcvd: 73

root@mailgateway01:~#

So you don't have any other suggestion where this might be going wrong?

 

[Stoiko Ivanov]

If those 2 servers are the only one's in your current '/etc/resolv.conf' (certain packages like dnsmasq, unbound, ... have hooks to rewrite 'resolv.conf' upon starting) - then I can't see why this is not working....

(you can use `spamassassin` with debug-flag on a mail (in .eml format) - to see if it does any DNS-checks and where it might not be working)

I hope this helps!

 

[g00gle]

Alright, I did run spamassassin -D < mail.eml on 2 of the mails that came trough this morning.
I can see the RBL checks in the logs and I also can see the spamscore getting high as 4 and 7 what should block them.

I will watch the system for the following days. I hope everything works better now with the 2 new dnsbl providers.
Maybe this is enough.

I think your suggestion:

An alternative explanation could be, that the ip was not listed yet, when you received the mail, but entered the RBL later (before you checked)

Is the most likely cause in this situation.

-------------

I also noticed the following, maybe this gives a clue of where to search for this issue.

In the spamassassin debugger:

X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on <strip>
X-Spam-Level: ****
X-Spam-Status: No, score=4.2 required=5.0 tests=AWL,BAYES_00,DKIM_INVALID,
    DKIM_SIGNED,FROM_FMBLA_NEWDOM,FROM_SUSPICIOUS_NTLD,HTML_EMBEDS,
    HTML_MESSAGE,MIME_QP_LONG_LINE,RCVD_IN_SBL_CSS,SPF_HELO_NONE,
    T_KAM_HTML_FONT_INVALID,T_REMOTE_IMAGE,URIBL_DBL_SPAM autolearn=no
    autolearn_force=no version=3.4.2

And in the email headers i'm completely missing this piece of information, I only have something similar to this:

X-SPAM-LEVEL: Spam detection results:  0
    BAYES_00                 -1.9 Bayes spam probability is 0 to 1%
    DKIM_INVALID              0.1 DKIM or DK signature exists, but is not valid
    DKIM_SIGNED               0.1 Message has a DKIM or DK signature, not necessarily valid
    FROM_SUSPICIOUS_NTLD    0.499 From abused NTLD
    HTML_MESSAGE            0.001 HTML included in message
    RDNS_NONE               0.793 Delivered to internal network by a host with no rDNS
    SPF_HELO_PASS          -0.001 SPF: HELO matches SPF record
    SPF_PASS               -0.001 SPF: sender matches SPF record
    T_KAM_HTML_FONT_INVALID   0.01 Test for Invalidly Named or Formatted Colors in HTML
    T_REMOTE_IMAGE           0.01 Message contains an external image

And still the RBL information is missing there.

I also noticed that the mail scanned with spamasssin in debug is not marked with a spam-flag (see above): X-Spam-Status: No
But the score is above spam level 3 as i've configured as treshhold for quarantine and spam marking:

So I would assume that spamassassin would mark this is a spam email ?? Or is there a difference here?

 

[Stoiko Ivanov]

the `spamassassin` commandline tool uses the Mail::Spamassassin perl module as does the `pmg-smtp-filter`.
The difference is that the configuration of your spam-levels and such happens only with `pmg-smtp-filter` - the score of 5 is just spamassassin's hardcoded default.

from a quick glance both pmg-smtp-filter and spamassassin had the same 'hits' (rules which triggered) - apart from 'URIBL_DBL_SPAM'

I hope this helps!