[SOLVED] How to filter subjects with UTF-Encoding

[mp_lbag]

Hello everyone,

I am facing this problem: I have a large list of keywords that I search for in the subject and then block; however, mails with a subject like this one still come through:

Subject: =?UTF-8?B?Umllc2lnZSBQcmVpc3JlZHVrdGlvbjogV2Vya3N0YXR0d2FnZW4gZsO8ciAyNDksLSBzdGF0dCA2OTksLSBFdXJvIQ==?= BPg/PTLut/NPPFikQD0wRodnrM4V4+vTpVJXUbfcKP/vGilPZVTyBnWxvsO K4eKHDI1+5oTX3LxlRfXIOBFeNtFjHgRa+tcaWK918OEi8HuavJ6c/fbDVTvgYpKFWUxhs0JBvQl 94PYWFQjmpmFGxKviKY=

this then resolves in plain text to (translated) "Huge price reduction: Workshop trolley for 249,- instead of 699,- Euro!" -> so my direct search term "workshop trolley" is not working at all.

Doesn´t PMG decode these lines during the check? Or how can I tell it to do so and THEN search for the terms? Or do I have to proceed in a completely different way ?

Filtering on the UTF-8 part in the subject (via regex) is more of a problem, because other mails that should go through have a subject like this, for example:

Subject: =?UTF-8?Q?LIVE-Webinar:_NIS2-Compliance_meister:?= =?UTF-8?Q?_Ein_=C3=9Cberblick_mit_Veeam_und_Manuel_Atug?=

and would then also be filtered.

At least, with these kinds of subjects i still have a "plain text" part with which I can work, but in the "bad" example above it is completely "encrypted" and not directly readable.

I would be very grateful for any tips on this!

Thank you very much,

Markus

 

[Stoiko Ivanov]

The match is done on the decoded header fields (not the one with mime-encoding) - so on "Workshop trolley for 249,- instead of 699,- Euro!" instead of "=?UTF-8?B?Umllc2lnZSBQcmVpc3JlZHVrdGlvbjogV2Vya3N0YXR0d2FnZW4gZsO8ciAyNDksLSBzdGF0dCA2OTksLSBFdXJvIQ=...."

if you share your actual regular expression and the actual Subject (and the logs of an affected mail) we might be able to see where the issue might be.

 

[mp_lbag]

Hi Stoiko, thanks for the reply.

here we go: (Hint: "Werkstattwagen" is the german word for "Workshop Trolley" )

i attached the complete Mail-Header and the log from PMG for this mail in the .txt-file.

The subject line

Subject: =?UTF-8?B?Umllc2lnZSBQcmVpc3JlZHVrdGlvbjogV2Vya3N0YXR0d2FnZW4gZsO8ciAyNDksLSBzdGF0dCA2OTksLSBFdXJvIQ==?=

decodes (UTF-8/Base64) to:

"Riesige Preisreduktion: Werkstattwagen für 249,- statt 699,- Euro!"

This should then fit to the "Match Field" / Subject-Value like here which i am using in the top-level Block-Rule.




Thanks and kind regards.

Markus

 

[Stoiko Ivanov]

Works here - I created a rule with:
* What Object - containing a Match Field-object that has subject as field and Werkstattwagen as value.
* Action Object - Block

Sent the Mail-Subject_log.txt as you sent it (it qualifies as an email with the headers on top) - and the mail got blocked as desired...

* How does your ruleset look like - how does the rule that contains the Match field object look like?
* does the direction of the rule match?
* is the rule active?

 

[mp_lbag]

Thats what i have:

a Rule named "Block Strings" with Direction "In", Active "yes", with Action "BLOCK" and What-Objects named "Strings" which is containing e.g. the clear words i want to filter , one by one. (one line for "Werkstattwagen" another for "KI = rich" and so on ...



Strings: (just an excerpt):



If you believe me or not: i just tested the whole thing once again, sending mails from internal hosts as well as my external test-machine: NOW (since about 13pm , no changes actually made on ruleset or so. just manually updated the spamassassin-files because i saw that it was available, now it works and the mails are beeing blocked:


2024-06-13T14:20:10.420464+02:00 mail pmg-smtp-filter[817409]: 2C16D1666AE3F54AFF0: SA score=3/5 time=5.061 bayes=0.01 autolearn=no autolearn_force=no hits=ALL_TRUSTED(-1),AWL(0.385),BAYES_05(-0.5),DATE_IN_PAST_96_XX(3.405),HEADER_FROM_DIFFERENT_DOMAINS(0.25),JMQ_SPF_NEUTRAL(0.5),KAM_DMARC_STATUS(0.01),MISSING_MID(0.497),SCC_BODY_SINGLE_WORD(0.166),T_SCC_BODY_TEXT_LINE(-0.01),T_SPF_PERMERROR(0.01)
2024-06-13T14:20:10.422197+02:00 mail pmg-smtp-filter[817409]: 2C16D1666AE3F54AFF0: block mail to <mp@l********k.de> (rule: Block Strings)

I´m really happy, that it is so now ... but: i really do not understand why ...

Kind regards,

Markus

 

[Stoiko Ivanov]

I´m really happy, that it is so now ... but: i really do not understand why ...

I would not understand that either - changes to the rulesystem (like adding a new what-object) are picked up with one of the next mails that are being processed.

just as a general recommendation - since the logs indicate that you have Bayes enabled - we currently recommend to keep it disabled:
https://pmg.proxmox.com/wiki/index.php/Roadmap#Proxmox_Mail_Gateway_8.0

 

[mp_lbag]

Thank you !

Did that.

 

[larsen]

I was having the same problem and didn't know that actually nothing needed to be decoded as the "test string" feature showed "...did not match the text...".

Guess it would help many users if that feature could also decode the string automatically, so I have created a feature request:
https://bugzilla.proxmox.com/show_bug.cgi?id=5893