Interfacing PMG with a Neural Network for spam score alteration

A

AFrenchCompany

Member
Dec 6, 2021
37
3
13
47
Hi,

I have trained a small model with highly curated spam and ham from our own PMG instance.

The model is behaving extremely well and I would like to use it to alter the spam score from PMG's spamassassin slightly.

Currently,
* the model can be ran in CLI in a conda env
* it accepts an .eml file as the input
* answers with a float score for spamassassin [-2,+2].

What would be my best options to interface PMG with this model ?

Thanks !


edit: If that makes it easier, I could expose the inference via a web API, taking the .eml as body and answering with JSON or plaintext.
 
Last edited:
There seems to be a way by creating a SpamAssassin Perl plugin, which in turn would call the python script or API.

If no recommendations are made around here, I will try this method and update the thread.

Hopefully that could help others.
 
I used the Custom Check Interface feature of PMG.
I have a script that properly queries my model and returns a score.

But I cannot see that score anywhere in the X-SPAM-LEVEL.
Is this expected ?

Script is in /usr/local/bin/pmg-custom-check
`custom_check 1` has been changed to /etc/pmg/pmg.conf
and I can see that my API is being hit.

Edit: OK, something's wrong with my custom check apparently.
This is visible in the tracking center `pmg-smtp-filter[572]: 31D8366705279DBE8F custom check error: got unexpected output!`
 
Last edited:
The output seems aligned with what the documentation states.
Yet I still get custom check error: got unexpected output!

root@spamfilter:~# /usr/local/bin/pmg-custom-check v1 /var/spool/pmg/spam/A0/2F9A966700405B62A0
v1
SCORE: 1.91

Am I missing something regarding what the custom check is supposed to return ?

Edit: this is extremely odd, despite the errors in the logs, when emails are classified as spam I can see the CustomCheck score.
Edit2: the errors aren't systematic, some emails do get the score [...] autolearn=no autolearn_force=no hits=CustomCheck(1.37)

Screenshot from 2024-06-18 09-43-49.pngScreenshot from 2024-06-18 09-43-40.png
 
Last edited:
  • Like
Reactions: Stoiko Ivanov
AFrenchCompany said:
The output seems aligned with what the documentation states.
Yet I still get custom check error: got unexpected output!

root@spamfilter:~# /usr/local/bin/pmg-custom-check v1 /var/spool/pmg/spam/A0/2F9A966700405B62A0
v1
SCORE: 1.91

Am I missing something regarding what the custom check is supposed to return ?
Click to expand...
This should work - here a link to the relevant source-code:
https://git.proxmox.com/?p=pmg-api....0e2aa33e6c2b286ab92;hb=refs/heads/master#l381
could you show the logs of one e-mail where it fails?

does your script also provide negative values (with fractional parts)? - I think there might be a glitch there in the source...
 
  • Like
Reactions: AFrenchCompany
Thank you for investigating :)

Indeed, the script returns in the [-2,2] range which include negative values for emails it deems to be probably not spam.
 
I've applied the patch and can confirm that it works great.

Screenshot from 2024-06-21 08-30-58.png

If I may suggest something though, CustomCheck doesn't follow the same casing as other rules.
For consistency sake it could be more appropriate to display it as CUSTOM_CHECK
 
You must log in or register to reply here.
Top Bottom