[SOLVED] global blacklist is not working default settings / after reboot rules working maybe a bug

redjohn · Sep 4, 2019

Hello,

I have create a global blacklist rule (default rule). Only move the message to qms and not block. Under "Who Objects" -> "Blacklist" I Add some E-Mail Adress. The Blacklist Rule is by default on the top with Prio 98. If I now send test E-Mail with blocked E-Mailadress, E-Mail will delivered but this is not correct.

I also disable Spam Detector -> Use autowhitelists -> no AND Use Bayesian filter -> no

Some log from Tracking Center:

Code:

Sep 4 22:07:45 mailgw pmg-smtp-filter[11659]: 210145D70198D63331: SA score=0/5 time=3.753 bayes=undefined autolearn=disabled hits=DKIM_SIGNED(0.1),DKIM_VALID(-0.1),DKIM_VALID_AU(-0.1),DKIM_VALID_EF(-0.1),HTML_MESSAGE(0.001),KAM_NUMSUBJECT(0.5),SPF_HELO_NONE(0.001),SPF_PASS(-0.001),URIBL_BLOCKED(0.001)

Sep 4 22:07:45 mailgw pmg-smtp-filter[11659]: 210145D70198D63331: accept mail to <my@domain.de> (F5F4455453) (rule: default-accept)

The Strange thing I see is rule:default-accept it that correct?

Regards,

Oliver

Stoiko Ivanov · Sep 5, 2019

please post the output of pmgdb dump (in code-tags) - if you anonymize data please make it in a consistent way.

redjohn · Sep 5, 2019

Hello,

you find pmgdb dump here:

Code:

Found RULE 29 (prio: 98, in, active): Blacklist
  FOUND FROM GROUP 44: Blacklist
    OBJECT 79: nomail@fromthisdomain.com
    OBJECT 116: myblock@emailadress.com
  FOUND ACTION GROUP 60: Block
    OBJECT 109: block message
Found RULE 28 (prio: 96, out, inactive): Virus Alert
  FOUND WHAT GROUP 51: Virus
    OBJECT 100: active
  FOUND ACTION GROUP 60: Block
    OBJECT 109: block message
  FOUND ACTION GROUP 62: Notify Admin
    OBJECT 111: notify __ADMIN__
  FOUND ACTION GROUP 63: Notify Sender
    OBJECT 112: notify __SENDER__
Found RULE 27 (prio: 96, in, active): Block Viruses
  FOUND WHAT GROUP 51: Virus
    OBJECT 100: active
  FOUND ACTION GROUP 61: Quarantine
    OBJECT 110: Move to quarantine.
  FOUND ACTION GROUP 62: Notify Admin
    OBJECT 111: notify __ADMIN__
Found RULE 26 (prio: 93, in, active): Block Dangerous Files
  FOUND WHAT GROUP 50: Dangerous Content
    OBJECT 94: content-type=application/javascript
    OBJECT 95: content-type=application/x-executable
    OBJECT 93: content-type=application/x-java
    OBJECT 92: content-type=application/x-ms-dos-executable
    OBJECT 96: content-type=application/x-ms-dos-executable
    OBJECT 97: content-type=message/partial
    OBJECT 98: filename=.*\.(vbs|pif|lnk|shs|shb)
    OBJECT 99: filename=.*\.\{.+\}
  FOUND ACTION GROUP 57: Remove attachments
    OBJECT 106: remove matching attachments
Found RULE 30 (prio: 90, in, active): Modify Header
  FOUND ACTION GROUP 55: Modify Spam Level
    OBJECT 104: modify field: X-SPAM-LEVEL:__SPAM_INFO__
Found RULE 37 (prio: 87, in+out, inactive): Block Multimedia Files
  FOUND WHAT GROUP 48: Multimedia
    OBJECT 83: content-type=audio/.*
    OBJECT 84: content-type=video/.*
  FOUND ACTION GROUP 57: Remove attachments
    OBJECT 106: remove matching attachments
Found RULE 31 (prio: 85, in, active): Whitelist
  FOUND FROM GROUP 45: Whitelist
    OBJECT 80: mail@fromthisdomain.com
  FOUND ACTION GROUP 59: Accept
    OBJECT 108: accept message
Found RULE 34 (prio: 82, in, active): Block Spam (Level 10)
  FOUND WHAT GROUP 54: Spam (Level 10)
    OBJECT 103: Level 10
  FOUND ACTION GROUP 61: Quarantine
    OBJECT 110: Move to quarantine.
  FOUND ACTION GROUP 62: Notify Admin
    OBJECT 111: notify __ADMIN__
Found RULE 33 (prio: 81, in, active): Quarantine/Mark Spam (Level 5)
  FOUND WHAT GROUP 53: Spam (Level 5)
    OBJECT 102: Level 5
  FOUND ACTION GROUP 61: Quarantine
    OBJECT 110: Move to quarantine.
  FOUND ACTION GROUP 56: Modify Spam Subject
    OBJECT 105: modify field: subject:***SPAM*** __SUBJECT__
Found RULE 32 (prio: 80, in, active): Quarantine/Mark Spam (Level 3)
  FOUND WHAT GROUP 52: Spam (Level 3)
    OBJECT 101: Level 3
  FOUND ACTION GROUP 56: Modify Spam Subject
    OBJECT 105: modify field: subject:***SPAM*** __SUBJECT__
Found RULE 35 (prio: 70, out, inactive): Block outgoing Spam
  FOUND WHAT GROUP 52: Spam (Level 3)
    OBJECT 101: Level 3
  FOUND ACTION GROUP 60: Block
    OBJECT 109: block message
  FOUND ACTION GROUP 62: Notify Admin
    OBJECT 111: notify __ADMIN__
  FOUND ACTION GROUP 63: Notify Sender
    OBJECT 112: notify __SENDER__
Found RULE 36 (prio: 60, out, inactive): Add Disclaimer
  FOUND ACTION GROUP 64: Disclaimer
    OBJECT 113: disclaimer

myblock@emailadress.com is the not blocked emailadress. If I send from this adress, email will delivered normaly but why? Rest rules are default btw.

Regards,

Oliver

Stoiko Ivanov · Sep 5, 2019

the rule looks ok!
just to be sure: the address you add here is not the 'From' header - but the envelop-sender /MAIL FROM address.

please post the complete log of a mail that got delivered from myblock@emailadress.com

redjohn · Sep 5, 2019

Hello,

here is the complete log:

Code:

Sep 5 14:05:53 mail1 postfix/smtpd[2884]: connect from mail-ed1-f50.google.com[209.85.208.50]
Sep 5 14:05:53 mail1 postfix/smtpd[2884]: Anonymous TLS connection established from mail-ed1-f50.google.com[209.85.208.50]: TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256
Sep 5 14:05:56 mail1 postfix/smtpd[2884]: D4A6D21041: client=mail-ed1-f50.google.com[209.85.208.50]
Sep 5 14:05:56 mail1 postfix/cleanup[2917]: D4A6D21041: message-id=<CANEQQO4bBtjKi84R4=Uzs5D+6A+S_dNYUM7KOzVem6fV+MRMNw@mail.gmail.com>
Sep 5 14:05:56 mail1 postfix/qmgr[360]: D4A6D21041: from=<myblock@emailadress.com>, size=2847, nrcpt=1 (queue active)
Sep 5 14:05:56 mail1 postfix/smtpd[2884]: disconnect from mail-ed1-f50.google.com[209.85.208.50] ehlo=2 starttls=1 mail=1 rcpt=1 data=1 quit=1 commands=7
Sep 5 14:05:56 mail1 pmg-smtp-filter[2166]: 210E75D70FA24E8099: new mail message-id=<CANEQQO4bBtjKi84R4=Uzs5D+6A+S_dNYUM7KOzVem6fV+MRMNw@mail.gmail.com>#012
Sep 5 14:05:57 mail1 pmg-smtp-filter[2166]: 210E75D70FA24E8099: SA score=0/5 time=0.855 bayes=undefined autolearn=no autolearn_force=no hits=AWL(0.221),DKIM_SIGNED(0.1),DKIM_VALID(-0.1),DKIM_VALID_AU(-0.1),DKIM_VALID_EF(-0.1),FREEMAIL_ENVFROM_END_DIGIT(0.25),FREEMAIL_FROM(0.001),HTML_MESSAGE(0.001),KAM_NUMSUBJECT(0.5),RCVD_IN_MSPIKE_H2(-0.001),SPF_HELO_NONE(0.001),SPF_PASS(-0.001)
Sep 5 14:05:57 mail1 postfix/smtpd[2943]: connect from localhost[127.0.0.1]
Sep 5 14:05:57 mail1 postfix/smtpd[2943]: CC7F921243: client=localhost[127.0.0.1], orig_client=mail-ed1-f50.google.com[209.85.208.50]
Sep 5 14:05:57 mail1 postfix/cleanup[2917]: CC7F921243: message-id=<CANEQQO4bBtjKi84R4=Uzs5D+6A+S_dNYUM7KOzVem6fV+MRMNw@mail.gmail.com>
Sep 5 14:05:57 mail1 postfix/qmgr[360]: CC7F921243: from=<myblock@emailadress.com>, size=4226, nrcpt=1 (queue active)
Sep 5 14:05:57 mail1 pmg-smtp-filter[2166]: 210E75D70FA24E8099: accept mail to <myrecipient@emailadress.com> (CC7F921243) (rule: default-accept)
Sep 5 14:05:57 mail1 postfix/smtpd[2943]: disconnect from localhost[127.0.0.1] ehlo=1 xforward=1 mail=1 rcpt=1 data=1 commands=5
Sep 5 14:05:57 mail1 pmg-smtp-filter[2166]: 210E75D70FA24E8099: processing time: 0.889 seconds (0.855, 0.018, 0)
Sep 5 14:05:57 mail1 postfix/lmtp[2919]: D4A6D21041: to=<myrecipient@emailadress.com>, relay=127.0.0.1[127.0.0.1]:10024, delay=4.1, delays=3.2/0.01/0.04/0.89, dsn=2.5.0, status=sent (250 2.5.0 OK (210E75D70FA24E8099))
Sep 5 14:05:57 mail1 postfix/qmgr[360]: D4A6D21041: removed
Sep 5 14:05:57 mail1 postfix/smtp[2944]: CC7F921243: to=<myrecipient@emailadress.com>, relay=10.10.10.1[10.10.10.1]:25, delay=0.09, delays=0/0.01/0.06/0.01, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as E180D413A2)
Sep 5 14:05:57 mail1 postfix/qmgr[360]: CC7F921243: removed

You see two adresses: myblock@emailadress.com and myrecipient@emailadress.com. Original I send some email from gmail.com. But E-Mail address is not blocked. Any ideas?

Regards

redjohn · Sep 5, 2019

Okay after i reboot the cluster (master and note) i send email again and its blocked:

Code:

Sep 5 15:14:45 mail1 postfix/smtpd[1131]: connect from mail-ed1-f46.google.com[209.85.208.46]
Sep 5 15:14:45 mail1 postfix/smtpd[1131]: Anonymous TLS connection established from mail-ed1-f46.google.com[209.85.208.46]: TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256
Sep 5 15:14:48 mail1 postfix/smtpd[1131]: C6E9D82911: client=mail-ed1-f46.google.com[209.85.208.46]
Sep 5 15:14:48 mail1 postfix/cleanup[1137]: C6E9D82911: message-id=<CANEQQO6CxP5YqdFQtOrWFPguUjQDabTHxucMY85py-aXUo5dAA@mail.gmail.com>
Sep 5 15:14:48 mail1 postfix/qmgr[353]: C6E9D82911: from=<myblock@emailadress.com>, size=2853, nrcpt=1 (queue active)
Sep 5 15:14:48 mail1 postfix/smtpd[1131]: disconnect from mail-ed1-f46.google.com[209.85.208.46] ehlo=2 starttls=1 mail=1 rcpt=1 data=1 quit=1 commands=7
Sep 5 15:14:48 mail1 pmg-smtp-filter[552]: 8294A5D710A48E129C: new mail message-id=<CANEQQO6CxP5YqdFQtOrWFPguUjQDabTHxucMY85py-aXUo5dAA@mail.gmail.com>#012
Sep 5 15:14:53 mail1 pmg-smtp-filter[552]: 8294A5D710A48E129C: SA score=0/5 time=4.469 bayes=undefined autolearn=ham autolearn_force=no hits=AWL(0.525),DKIM_SIGNED(0.1),DKIM_VALID(-0.1),DKIM_VALID_AU(-0.1),DKIM_VALID_EF(-0.1),FREEMAIL_ENVFROM_END_DIGIT(0.25),FREEMAIL_FROM(0.001),HTML_MESSAGE(0.001),SPF_HELO_NONE(0.001),SPF_PASS(-0.001)
Sep 5 15:14:53 mail1 pmg-smtp-filter[552]: 8294A5D710A48E129C: block mail to <myrecipient@emailadress.com> (rule: Blacklist)
Sep 5 15:14:53 mail1 pmg-smtp-filter[552]: 8294A5D710A48E129C: processing time: 4.497 seconds (4.469, 0.018, 0)
Sep 5 15:14:53 mail1 postfix/lmtp[1138]: C6E9D82911: to=<myrecipient@emailadress.com>, relay=127.0.0.1[10.10.10.1]:10024, delay=7.7, delays=3.2/0.01/0.04/4.5, dsn=2.7.0, status=sent (250 2.7.0 BLOCKED (8294A5D710A48E129C))
Sep 5 15:14:53 mail1 postfix/qmgr[353]: C6E9D82911: removed

But if i remove the adress from blacklist and send again address is also blocked but why? I have two nodes and one master server. it looks like that the change not report to the notes. but if i look on the nodes webinterface address is removed from blacklist and should not blocked.

Regards,

Oliver

Stoiko Ivanov · Sep 5, 2019

Ok - I missed the information that this is a cluster somehow - sorry!

do you send your test-emails to the master or to the node?
does it work after you restart the 'pmg-smtp-filter'?
it takes a while for the rule-db to get synced to the node from the master - this is not synchronous but happens regularly (every two minutes).

In any case I think I managed to reproduce the issue (changing an object in the rulesystem not being directly active on the (non-master) nodes)

Could I ask you to open a bug-report at https://bugzilla.proxmox.com

Thanks!

(The issue should not occur on the master-node)

redjohn · Sep 5, 2019

Stoiko Ivanov said:
Ok - I missed the information that this is a cluster somehow - sorry!

do you send your test-emails to the master or to the node?
does it work after you restart the 'pmg-smtp-filter'?
it takes a while for the rule-db to get synced to the node from the master - this is not synchronous but happens regularly (every two minutes).

In any case I think I managed to reproduce the issue (changing an object in the rulesystem not being directly active on the (non-master) nodes)

Could I ask you to open a bug-report at https://bugzilla.proxmox.com

Thanks!

(The issue should not occur on the master-node)

Thanks for fast reply. Ok it is not a problem if the change takes some minutes (2 minutes). But I have delete blacklist now and wait for 1 hour. And new send email is also blocked. On both nodes in web interface blacklisted address is delete but pmg block it without any rule!

After restart pmg-smtp-filter on the two nodes everything works fine but that is not a solution. Restart pmg-smtp-filter every 2 minutes?

I only use master pmg for configuration and spam not for send and receive emails. Did you have any fix? Because I'm not the only person with pmg cluster and not working changes on nodes.

Sync between master and node is not working, also not even after 2 minutes or 1 hour!

Stoiko Ivanov · Sep 5, 2019

OliverB said:
I only use master pmg for configuration and spam not for send and receive emails. Did you have any fix? Because I'm not the only person with pmg cluster and not working changes on nodes.

As said - please open a bug-report on https://bugzilla.proxmox.com - we'll look into it.

redjohn · Sep 5, 2019

Stoiko Ivanov said:
As said - please open a bug-report on https://bugzilla.proxmox.com - we'll look into it.

I also open a bug:

https://bugzilla.proxmox.com/show_bug.cgi?id=2360

This problem have everyone with a cluster now. How fast can it fixed?

redjohn · Sep 5, 2019

Okay I can reproduce this "Bug" (?) (I have one PMG Master and two PMG Node systems)

1. Creat some rule on master node e.g. Blacklist a test e-mailadress, now wait for some minutes (5 minutes)
2. Send some test message from the Blacklisted test e-mailadress
-> Look in Tracking Center and can see email is delivered without any problem and send to mailbox
3. Restart pmg-smtp-filter on both nodes via systemctl
4. Resend some test message from Blacklisted e-mailadress now and you can see email is blocked by both nodes

5. Now remove e-mailadress from blacklist and wait for some minutes (5 minutes)
6. Send some test message from not more blacklisted e-mailadress again
-> Look in Tracking Center and can see email is blocked but email is no more on blacklist!
7. Restart pmg-smtp-filter on both nodes via systemctl
8. Send test message again -> And message is now delivered without any problems.

in pmgdb dump on both nodes I can see the blacklist e-mailadress after some minutes ~2 minutes I think and if I remove the blacklist entire and look again with pmgdb dump on both nodes I can see blacklist e-mailadress is removed.

So I can say Sync between master and nodes is not working without restart pmg-smtp-filter and that's not correct, I create rules and this rules only affected if I restart the services on the nodes or restart the hole node.

I need some fix for this problem otherwise cluster function is not makes no sense!

Regards

mac.linux.free · Oct 25, 2019

blacklist is sadly not working for me

root@mgw2:~# pmgdb dump
Found RULE 6 (prio: 99, in, active): Whitelist
FOUND FROM GROUP 3: Whitelist
OBJECT 2: mail@fromthisdomain.com
OBJECT 38: xx.xx.xx.xx
FOUND ACTION GROUP 17: Accept
OBJECT 30: accept message
Found RULE 4 (prio: 98, in, active): Blacklist
FOUND FROM GROUP 2: Blacklist
OBJECT 39: block@email.com
OBJECT 1: nomail@fromthisdomain.com
OBJECT 37: city
FOUND TO GROUP 2: Blacklist
OBJECT 39: block@email.com
OBJECT 1: nomail@fromthisdomain.com
OBJECT 37: city
FOUND ACTION GROUP 18: Block
OBJECT 31: block message
Found RULE 2 (prio: 96, in, active): Block Viruses
FOUND WHAT GROUP 9: Virus
OBJECT 22: active
FOUND ACTION GROUP 19: Quarantine
OBJECT 32: Move to quarantine.
FOUND ACTION GROUP 20: Notify Admin
OBJECT 33: notify __ADMIN__
Found RULE 3 (prio: 96, out, active): Virus Alert
FOUND WHAT GROUP 9: Virus
OBJECT 22: active
FOUND ACTION GROUP 18: Block
OBJECT 31: block message
FOUND ACTION GROUP 20: Notify Admin
OBJECT 33: notify __ADMIN__
FOUND ACTION GROUP 21: Notify Sender
OBJECT 34: notify __SENDER__
Found RULE 1 (prio: 93, in, active): Block Dangerous Files
FOUND WHAT GROUP 8: Dangerous Content
OBJECT 16: content-type=application/javascript
OBJECT 17: content-type=application/x-executable
OBJECT 15: content-type=application/x-java
OBJECT 14: content-type=application/x-ms-dos-executable
OBJECT 18: content-type=application/x-ms-dos-executable
OBJECT 19: content-type=message/partial
OBJECT 20: filename=.*\.(vbs|pif|lnk|shs|shb)
OBJECT 21: filename=.*\.\{.+\}
FOUND ACTION GROUP 15: Remove attachments
OBJECT 28: remove matching attachments
Found RULE 5 (prio: 90, in, active): Modify Header
FOUND ACTION GROUP 13: Modify Spam Level
OBJECT 26: modify field: X-SPAM-LEVEL:__SPAM_INFO__
Found RULE 12 (prio: 87, in+out, inactive): Block Multimedia Files
FOUND WHAT GROUP 6: Multimedia
OBJECT 5: content-type=audio/.*
OBJECT 6: content-type=video/.*
FOUND ACTION GROUP 15: Remove attachments
OBJECT 28: remove matching attachments
Found RULE 9 (prio: 82, in, active): Block Spam (Level 10)
FOUND WHAT GROUP 12: Spam (Level 10)
OBJECT 25: Level 10
FOUND ACTION GROUP 18: Block
OBJECT 31: block message
Found RULE 8 (prio: 81, in, active): Quarantine/Mark Spam (Level 5)
FOUND WHAT GROUP 11: Spam (Level 5)
OBJECT 24: Level 5
FOUND ACTION GROUP 14: Modify Spam Subject
OBJECT 27: modify field: subject:SPAM: __SUBJECT__
FOUND ACTION GROUP 19: Quarantine
OBJECT 32: Move to quarantine.
Found RULE 7 (prio: 80, in, active): Quarantine/Mark Spam (Level 3)
FOUND WHAT GROUP 10: Spam (Level 3)
OBJECT 23: Level 3
FOUND ACTION GROUP 14: Modify Spam Subject
OBJECT 27: modify field: subject:SPAM: __SUBJECT__
FOUND ACTION GROUP 19: Quarantine
OBJECT 32: Move to quarantine.
Found RULE 10 (prio: 70, out, active): Block outgoing Spam
FOUND FROM GROUP 2: Blacklist
OBJECT 39: block@email.com
OBJECT 1: nomail@fromthisdomain.com
OBJECT 37: city
FOUND WHAT GROUP 10: Spam (Level 3)
OBJECT 23: Level 3
FOUND ACTION GROUP 18: Block
OBJECT 31: block message
FOUND ACTION GROUP 20: Notify Admin
OBJECT 33: notify __ADMIN__
FOUND ACTION GROUP 21: Notify Sender
OBJECT 34: notify __SENDER__
Found RULE 11 (prio: 60, out, inactive): Add Disclaimer
FOUND ACTION GROUP 22: Disclaimer
OBJECT 35: disclaimer

please help

I need black and whitelist functionality on port 25 and 26 though this is a gateway for multiple customers.

bwdavis · Apr 16, 2020

What is the status of this issue? Is it resolved, because I am seeing SPF_PASS(-0.001) on all emails even though it is sent from some other domain which is spoofing my domain. I am also on a master/node 1 of each cluster and am having problems with the blacklist being enforced.
We installed PMG 6.1 very recently. OK Forgive me, I see this issue is marked [SOLVED] at the top.

I will look for a thread addressing my SPF fail to block issue.

Bruce

Search

Search

[SOLVED] global blacklist is not working default settings / after reboot rules working maybe a bug

redjohn

Renowned Member

Stoiko Ivanov

Proxmox Staff Member

redjohn

Renowned Member

Stoiko Ivanov

Proxmox Staff Member

redjohn

Renowned Member

redjohn

Renowned Member

Stoiko Ivanov

Proxmox Staff Member

redjohn

Renowned Member

Stoiko Ivanov

Proxmox Staff Member

redjohn

Renowned Member

redjohn

Renowned Member

mac.linux.free

Renowned Member

bwdavis

Member