[SOLVED] How to filter subjects with UTF-Encoding

[mp_lbag]

Hello everyone,

I am facing this problem: I have a large list of keywords that I search for in the subject and then block; however, mails with a subject like this one still come through:

Subject: =?UTF-8?B?Umllc2lnZSBQcmVpc3JlZHVrdGlvbjogV2Vya3N0YXR0d2FnZW4gZsO8ciAyNDksLSBzdGF0dCA2OTksLSBFdXJvIQ==?= BPg/PTLut/NPPFikQD0wRodnrM4V4+vTpVJXUbfcKP/vGilPZVTyBnWxvsO K4eKHDI1+5oTX3LxlRfXIOBFeNtFjHgRa+tcaWK918OEi8HuavJ6c/fbDVTvgYpKFWUxhs0JBvQl 94PYWFQjmpmFGxKviKY=

this then resolves in plain text to (translated) "Huge price reduction: Workshop trolley for 249,- instead of 699,- Euro!" -> so my direct search term "workshop trolley" is not working at all.

Doesn´t PMG decode these lines during the check? Or how can I tell it to do so and THEN search for the terms? Or do I have to proceed in a completely different way ?

Filtering on the UTF-8 part in the subject (via regex) is more of a problem, because other mails that should go through have a subject like this, for example:

Subject: =?UTF-8?Q?LIVE-Webinar:_NIS2-Compliance_meister:?= =?UTF-8?Q?_Ein_=C3=9Cberblick_mit_Veeam_und_Manuel_Atug?=

and would then also be filtered.

At least, with these kinds of subjects i still have a "plain text" part with which I can work, but in the "bad" example above it is completely "encrypted" and not directly readable.

I would be very grateful for any tips on this!

Thank you very much,

Markus

 

[Stoiko Ivanov]

The match is done on the decoded header fields (not the one with mime-encoding) - so on "Workshop trolley for 249,- instead of 699,- Euro!" instead of "=?UTF-8?B?Umllc2lnZSBQcmVpc3JlZHVrdGlvbjogV2Vya3N0YXR0d2FnZW4gZsO8ciAyNDksLSBzdGF0dCA2OTksLSBFdXJvIQ=...."

if you share your actual regular expression and the actual Subject (and the logs of an affected mail) we might be able to see where the issue might be.

 

[mp_lbag]

Hi Stoiko, thanks for the reply.

here we go: (Hint: "Werkstattwagen" is the german word for "Workshop Trolley" )

i attached the complete Mail-Header and the log from PMG for this mail in the .txt-file.

The subject line

Subject: =?UTF-8?B?Umllc2lnZSBQcmVpc3JlZHVrdGlvbjogV2Vya3N0YXR0d2FnZW4gZsO8ciAyNDksLSBzdGF0dCA2OTksLSBFdXJvIQ==?=

decodes (UTF-8/Base64) to:

"Riesige Preisreduktion: Werkstattwagen für 249,- statt 699,- Euro!"

This should then fit to the "Match Field" / Subject-Value like here which i am using in the top-level Block-Rule.




Thanks and kind regards.

Markus

 

[Stoiko Ivanov]

Works here - I created a rule with:
* What Object - containing a Match Field-object that has subject as field and Werkstattwagen as value.
* Action Object - Block

Sent the Mail-Subject_log.txt as you sent it (it qualifies as an email with the headers on top) - and the mail got blocked as desired...

* How does your ruleset look like - how does the rule that contains the Match field object look like?
* does the direction of the rule match?
* is the rule active?

 

[mp_lbag]

Thats what i have:

a Rule named "Block Strings" with Direction "In", Active "yes", with Action "BLOCK" and What-Objects named "Strings" which is containing e.g. the clear words i want to filter , one by one. (one line for "Werkstattwagen" another for "KI = rich" and so on ...



Strings: (just an excerpt):



If you believe me or not: i just tested the whole thing once again, sending mails from internal hosts as well as my external test-machine: NOW (since about 13pm , no changes actually made on ruleset or so. just manually updated the spamassassin-files because i saw that it was available, now it works and the mails are beeing blocked:


2024-06-13T14:20:10.420464+02:00 mail pmg-smtp-filter[817409]: 2C16D1666AE3F54AFF0: SA score=3/5 time=5.061 bayes=0.01 autolearn=no autolearn_force=no hits=ALL_TRUSTED(-1),AWL(0.385),BAYES_05(-0.5),DATE_IN_PAST_96_XX(3.405),HEADER_FROM_DIFFERENT_DOMAINS(0.25),JMQ_SPF_NEUTRAL(0.5),KAM_DMARC_STATUS(0.01),MISSING_MID(0.497),SCC_BODY_SINGLE_WORD(0.166),T_SCC_BODY_TEXT_LINE(-0.01),T_SPF_PERMERROR(0.01)
2024-06-13T14:20:10.422197+02:00 mail pmg-smtp-filter[817409]: 2C16D1666AE3F54AFF0: block mail to <mp@l********k.de> (rule: Block Strings)

I´m really happy, that it is so now ... but: i really do not understand why ...

Kind regards,

Markus

 

[Stoiko Ivanov]

I´m really happy, that it is so now ... but: i really do not understand why ...

I would not understand that either - changes to the rulesystem (like adding a new what-object) are picked up with one of the next mails that are being processed.

just as a general recommendation - since the logs indicate that you have Bayes enabled - we currently recommend to keep it disabled:
https://pmg.proxmox.com/wiki/index.php/Roadmap#Proxmox_Mail_Gateway_8.0

 

[mp_lbag]

Thank you !

Did that.