[TUTORIAL] Train SpamAsssassin with mail of an IMAP mailbox

[magenbrot]

Hi,

I've created a script which fetches mails from a special IMAP mailbox, where I forward spam and ham to, and feeds them to sa-learn as spam or ham. If anyone is interested to try out you can get all the details in the github repository: https://github.com/magenbrot/sa_learn_from_imap

Here's a more detailed description:

First up you'll need an IMAP mailbox where you and your customers redirect SPAM and HAM to. I've a mailbox called filter@mymails.org. Since I use postfix (I don't know what other MTAs do), I can use a + in my recipient address (see recipient_delimiter documentation of Postfix). So I'm forwarding all spam to filter+spam@mymailbox.org and all ham to filter+ham@mymailbox.org.

Furthermore I have a sieve-script running on my server, which will move the incoming mails by the recipient to the correct folder.

Now about training the SpamAssassin: The script has 3 required, positional arguments. The IMAP-host, -user and the -password. By default it assumes that your SPAM mails are sorted in a folder called 'spam' and your HAM mails are filed in 'ham'.

This can be changed with the parameters --spam-folder and --ham-folder. It needs a working directory, this is /tmp/sa-learn by default and can be changed with the --directory parameter. Finally you can specify the sa-learn script it should use with --sa-learn. This defaults to /usr/bin/sa-learn.

Please call the script with the user you're running SpamAssassin with.

Example (run with python3, either directly or in the venv from above):

./sa_learn_from_imap.py mail.mymails.org 'filter@mymails.org' 'password123$'

A cronjob could look like this, put this in /etc/cron.d/sa-learn-ham-spam:

#MAILTO="postmaster@mymails.org"
MAILTO=""

# run with bash, as sh doesn't support 'source'
SHELL=/bin/bash

15 * * * * root source /opt/sa_learn_from_imap/env/python3/bin/activate && \\
          /opt/sa_learn_from_imap/sa_learn_from_imap.py mail.mymails.org 'filter@mymails.org' 'password123$'

 

[killmasta93]

very interesting could you post your github repo?

 

[magenbrot]

very interesting could you post your github repo?

Oh, I forgot the important part Sorry, here's the link: https://github.com/magenbrot/sa_learn_from_imap

 

[killmasta93]

very nice, theres few things im somewhat lost correct me if im wrong.

1) So you used recipient_delimiter instead of filterspam@mymailbox.org and filterham@mymailbox.org? so when users have spam emails
they would forward that email to filterspam@mymailbox.org?
2) then you have a sieve-script which would be located on your postfix email server and not proxmox? how does it know what folder to put in? even if in the sieve filter says if header contains but doesn't the header change when the user forwards the email?


Thanks

 

[magenbrot]

1) So you used recipient_delimiter instead of filterspam@mymailbox.org and filterham@mymailbox.org? so when users have spam emails
they would forward that email to filterspam@mymailbox.org?

I've called the mailbox just filter@mydomain.com. With the delimiter you can add +ham or +spam for example to distinguish between Ham and Spam. So the complete address would be filter+spam@mydomain.com or filter+ham@mydomain.com.

You can use that for example with roundcube and the markasjunk plugin, so your users can select the mail and simply click the Ham or Spam button. markasjunk will then forward it to the address in the configuration.

2) then you have a sieve-script which would be located on your postfix email server and not proxmox? how does it know what folder to put in? even if in the sieve filter says if header contains but doesn't the header change when the user forwards the email?

Yes, the mailbox is on my "normal" IMAP server, not on proxmox. But the script runs on my proxmox gateway because there's my spamassassin installation. The script doesn't depend on PMG, it can be used with every SA based mail filter.

The sieve filter just matches if the recipient address contains +spam or +ham and moves the mail to the according folder.

if header :contains "Received" "+spam" {
  fileinto "spam";
  stop;
} elsif header :contains "Received" "+ham" {
  fileinto "ham";
  stop;
}

 

[killmasta93]

Thanks for the reply, now i get it, out of curiosity im guessing it only works for IMAP? and not POP? also if the user have IMAP could they use outlook or they have to use Roundcube for the plugin?

 

[magenbrot]

I don't know about outlook, but for Thunderbird is use the "Mail Redirect" (https://addons.thunderbird.net/de/thunderbird/addon/mailredirect/). I'm unsure if simply forwarding the mail is enough, as this will change the mail headers.

In the current state the script supports IMAP servers only.