As described in some articles here, I’ve put together a few custom solutions to get rid of as much spam as possible as early as possible — including a Postfix policy service (Policyguard).
The small number of emails (<< 5%) that ultimately end up with a SpamAssassin score between 2 and 7 are the ones I wanted to double-check using AI.
The given score is the sum of all other SA tests, except the custom score, which is added via PMG after the SA scan.
If the score is already high enough, the email will be rejected anyway — even without an AI check. If the score is very low, it’s probably not spam. However, there is one exception: if the domain has never been seen before, it could be a compromised domain — in that case, I still want to check it. For this, the service pulls greylisting statistics from the current version of Policyguard. But Policyguard itself is not required for the AI check to function, you just have no greylisting statistics.
A good fit here are the currently new, but very strong and inexpensive decision modells OpenRouter. I use Inception MercuryDecide
How does the service work?
The Perl module (.pm) is hooked into SpamAssassin as the “last check” via the .cf configuration.
This Perl service then launches a Python script and passes the email in EML format along with the current score.
The Python script, in turn, has its configuration (including the prompt) stored in a .toml file. This defines:
What data is transmitted?
Can a local model be used?
The current version only support decison modells and those are currently not working locally.
The decision modells are superseeding the chatbot models in price, speed and detection in an order of magnitude, so i gave up support for those.
What does it cost?
MercuryDecide costs currently $0.02 per million input tokens, which works out to approximately one cent per 750 emails.
Paste your API Key into ai-spamcheck-decide.toml
Remove the txt file ending from attached files and copy them to
Restart Spamassassin
Will i help to setup?
Yes, i am a freelancer.
The small number of emails (<< 5%) that ultimately end up with a SpamAssassin score between 2 and 7 are the ones I wanted to double-check using AI.
The given score is the sum of all other SA tests, except the custom score, which is added via PMG after the SA scan.
If the score is already high enough, the email will be rejected anyway — even without an AI check. If the score is very low, it’s probably not spam. However, there is one exception: if the domain has never been seen before, it could be a compromised domain — in that case, I still want to check it. For this, the service pulls greylisting statistics from the current version of Policyguard. But Policyguard itself is not required for the AI check to function, you just have no greylisting statistics.
A good fit here are the currently new, but very strong and inexpensive decision modells OpenRouter. I use Inception MercuryDecide
How does the service work?
The Perl module (.pm) is hooked into SpamAssassin as the “last check” via the .cf configuration.
This Perl service then launches a Python script and passes the email in EML format along with the current score.
The Python script, in turn, has its configuration (including the prompt) stored in a .toml file. This defines:
- The spam score range in which an AI scan is actually performed. There are 2x2 Parameters: min and max parameters for either the normal scan and the greylist scan. Scanning greylist mails below -5 does not make sense: The maximum output score of 9 won't change anything, if the input score is too low.
- The output score range: minimum and maximum score the AI is allowed to assign
- History file: if sender and subject is identical (the last 10 are stored by default), don’t re-check — the score should remain the same
- API key and model used
- Whether and how many lines of the email body should be included. The body significantly improves detection, but is sensitive from a data protection perspective
- The prompt template / noul question. This can be optimized depending on whether newsletters or fake applications, etc. should be filtered — or not
What data is transmitted?
- Sending server
- Sender
- Recipient
- Subject
- Optional: X lines of the extracted e-mail body text (sanitized from html code)
Can a local model be used?
The current version only support decison modells and those are currently not working locally.
The decision modells are superseeding the chatbot models in price, speed and detection in an order of magnitude, so i gave up support for those.
What does it cost?
MercuryDecide costs currently $0.02 per million input tokens, which works out to approximately one cent per 750 emails.
Code:
# Create Folder for Statistics
mkdir -p /t/
chmod -R 777 /t/Paste your API Key into ai-spamcheck-decide.toml
Remove the txt file ending from attached files and copy them to
Code:
/etc/mail/spamassassin/aicheck.cf
/usr/local/lib/AICheck.pm
/usr/local/bin/ai-spamcheck.py
/etc/ai-spamcheck-decide.toml
chmod +x /usr/local/bin/ai-spamcheck.pyRestart Spamassassin
Code:
systemctl restart pmg-smtp-filter.service
Code:
# Test with E-Mail with score between 2 and 9
spamassassin -t < test.eml
#or
/usr/local/bin/ai-spamcheck.py test.eml --debugWill i help to setup?
Yes, i am a freelancer.
Last edited:
