Spam mail filtering proxmox

[sanojs]

Hi,

We are receiving many spam mails similar like below. Please suggest how we can block these emails with proxmox.

Spam score is showing as 1

From: "Gan" <eiral@af2100.com>
To: "'test@abc.com'"
Subject: <test subject>
MIME-Version: 1.0
Content-Type: multipart/related; boundary="--151115222387242367660988809474838"
Message-Id: <20200728004845.B39BC3053261F@webmail.ekonek.com>
X-TM-AS-GCONF: 00
X-TM-AS-Product-Ver: IMSVA-9.1.0.1600-8.2.0.1013-24456.001
X-TM-AS-Result: No--11.304-5.0-31-10
X-imss-scan-details: No--11.304-5.0-31-10
X-TMASE-Version: IMSVA-9.1.0.1600-8.2.1013-24456.001
X-TMASE-Result: 10--11.303600-10.000000
X-TMASE-MatchedRID: 4zFKYhkAdItOhD+LzkYc62yGdbpKa3ZsfMhjt3b5PP2w9hOhubFTzhFl
wJhVcSVOYij5Usy05aUx3CegaCP+SwwIsQZbtzZwxi0q1322bZz4qCLIu0mtIDAiPlVVBqli8J0
4GYfkt4BdQJ0TvOSfrgaCaro1qgVla6E59IxHGX/+OTCJja0mclPgO2JKQydYN2zkzesjSLy4gb
dQ73mtrY6OpPLjYsLhglWkBIw3X6ZI5aNiVPKgsZhyXgGCpXKTcbVmJQ7kUKWPaLJ/Ca3ST59P6
0UPZK0Rr4Q+BMkLsN0qhZrv4f7cO0VuR7z+3YYro9IWP1abi753s7Qz9VHmRx2dnS9wqrcnl7UY
Wab8Hr49bmp9hCIzGRpX2PbCcGI1JACUwtIIZFSkpKo35nd2nmZrKp1iUZhL28Ojb1C4Zhd+R6n
/w8goDcB6FcoCyupuPtQSvlp+oWH4miWUEaTQPnzKBttezamJpHurUsnxCrPEDKpuLqDeoEw+83
ILQ5eIOvT6eLNLlXT+2FaXuCi5+5XOthlVwS31iQy4MG/9StlZDL1gLmoa/FwwZNlQvtYaaNe4p
gaZVXYsafDVzVRS/3XgOhxJT3ykPsc39yjoFeSl/MtrTwS4UPyYqUhGd9RfufKSjpVRAY2limDi
gTbmdPlZCcg9XfwWPJuGM8tMGsU=
X-TMASE-SNAP-Result: 1.821001.0001-0-1-12:0,22:0,33:0,34:0-0

X-SPAM-LEVEL: Spam detection results: 1

BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DEAR_SOMETHING 1.973 Contains 'Dear (something)'
DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain
HTML_MESSAGE 0.001 HTML included in message
MIME_BOUND_DD_DIGITS 1.373 Spam tool pattern in MIME boundary
MIME_HTML_ONLY 0.1 Message only has text/html MIME parts
RCVD_IN_DNSWL_NONE -0.0001 Sender listed at https://www.dnswl.org/, no trust
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record

Thanks,




Thanks,
Sanoj

 

[Stoiko Ivanov]

BAYES_00 -1.9 Bayes spam probability is 0 to 1%

Seems your bayes filtering is wrong (gives wrong indication) - I would suggest turning it off in GUI->Configuration->Spam Detector->Options

I hope this helps

 

[ittk]

@Stoiko Ivanov

Seems also for me the same, always getting the BAYES_00 -1.9 Scoring, but have configured it to be on: and AWS have always positives values like this AWL(1.130) range starting from 0.7 i would say ...

These are my switches configured on GUI-side (PMG defaults)

use_awl: <boolean> (default = 1)
Use the Auto-Whitelist plugin.

use_bayes: <boolean> (default = 1)
Whether to use the naive-Bayesian-style classifier.

Here comes my listing for found bayes* filer on PMG, they are always fresh, but i did never do any manually sa-learn, but this won't be required, when auto-learning is on?

# ls -la /root/.spamassassin/
total 21148
drwx------ 2 root root 4096 Jul 31 11:47 .
drwx------ 6 root root 4096 Jul 31 09:05 ..
-rw------- 1 root root 5304320 Jul 31 11:48 auto-whitelist
-rw------- 1 root root 6 Jul 31 11:48 auto-whitelist.mutex
-rw------- 1 root root 74560 Jul 31 11:48 bayes_journal
-rw------- 1 root root 3054 Jul 31 11:47 bayes.mutex
-rw------- 1 root root 10522624 Jul 31 11:47 bayes_seen
-rw------- 1 root root 9007104 Jul 31 11:47 bayes_toks

File /etc/mail/spamassassin/local.cf has this content:

# dont use things by default
use_bayes 0
bayes_auto_expire 0
bayes_learn_to_journal 1


ok_languages all


envelope_sender_header X-Proxmox-Envelope-From

# use fast lock (non-nfs save)
lock_method flock




use_bayes 1




include /usr/share/spamassassin-extra/KAM.cf

So have a actually an Problem with my bayes learning on PMG? As told before have never used manually called sa-learn commands.

So maybe is there mssing an bayes_path configuration?

in /etc/spamassassin/local.cf and add the following line as proposed here? https://www.nerd-quickies.net/2015/10/02/spamassassin-bayes_00-1-90-although-sa-learn-runs-daily/

1 2 3

[...] bayes_path /root/.spamassassin/bayes

 

[ittk]

And here my "bayes" stripped results from running command sa-learn -D --dump magic:

Jul 31 12:06:59.227 [9507] dbg: bayes: tie-ing to DB file R/O /root/.spamassassin/bayes_toks
Jul 31 12:06:59.227 [9507] dbg: bayes: tie-ing to DB file R/O /root/.spamassassin/bayes_seen
Jul 31 12:06:59.228 [9507] dbg: bayes: found bayes db version 3
Jul 31 12:06:59.228 [9507] dbg: bayes: DB journal sync: last sync: 1596189937
Jul 31 12:06:59.228 [9507] dbg: config: score set 3 chosen.
Jul 31 12:06:59.230 [9507] dbg: dns: EDNS, UDP payload size 4096
Jul 31 12:06:59.230 [9507] dbg: dns: servers obtained from Net::DNS : [127.0.0.1]:53
Jul 31 12:06:59.230 [9507] dbg: dns: nameservers set to 127.0.0.1
Jul 31 12:06:59.231 [9507] dbg: dns: using socket module: IO::Socket::IP version 0.39
Jul 31 12:06:59.231 [9507] dbg: dns: is Net::DNS::Resolver available? yes
Jul 31 12:06:59.231 [9507] dbg: dns: Net::DNS version: 1.19
Jul 31 12:06:59.231 [9507] dbg: sa-learn: spamtest initialized
Jul 31 12:06:59.231 [9507] dbg: plugin: Mail::SpamAssassin::Plugin::Bayes=HASH(0x560f9ddac2c0) implements 'learner_dump_database', priority 0
0.000          0          3          0  non-token data: bayes db version
0.000          0        203          0  non-token data: nspam
0.000          0     106301          0  non-token data: nham
0.000          0     365108          0  non-token data: ntokens
0.000          0 1596077713          0  non-token data: oldest atime
0.000          0 1596189917          0  non-token data: newest atime
0.000          0 1596189937          0  non-token data: last journal sync atime
0.000          0 1596164444          0  non-token data: last expiry atime
0.000          0      86400          0  non-token data: last expire atime delta
0.000          0     204339          0  non-token data: last expire reduction count
Jul 31 12:06:59.232 [9507] dbg: bayes: untie-ing
Jul 31 12:06:59.232 [9507] dbg: plugin: Mail::SpamAssassin::Plugin::Bayes=HASH(0x560f9ddac2c0) implements 'learner_close', priority 0

 

[Stoiko Ivanov]

Seems also for me the same, always getting the BAYES_00 -1.9 Scoring, but have configured it to be on: and AWS have always positives values like this AWL(1.130) range starting from 0.7 i would say ...

seems your bayes filter is off - disable bayes (this clears the bayes db)
as for awl - if it biases in the right direction that should be ok - else disable it in the GUI as well

 

[ittk]

seems your bayes filter is off - disable bayes (this clears the bayes db)
as for awl - if it biases in the right direction that should be ok - else disable it in the GUI as well

Thanks for your reply.

No, as written both Bayes and awl are tuned on, like pmg default are...
So is manuell Execution of sa-learn required? Or will all handeled automatically?

 

[Stoiko Ivanov]

sorry I phrased that quite ambiguous - yes your bayes filtering is configured on - however it is biased quite wrong - i.e. it yields non-spam for spammails (and maybe the other way around) - disable it in the gui to clear the database - you can try re-enabling it afterwards (that would start it with a fresh database)

as written elsewhere pmg only implements bayes autolearning as spamassassin does it:
https://cwiki.apache.org/confluence/display/SPAMASSASSIN/AutolearningNotWorking

that's what's handled automatically - although a few of our users do maintain and train their bayes database with sa-learn manually.

 

[ittk]

sorry I phrased that quite ambiguous - yes your bayes filtering is configured on - however it is biased quite wrong - i.e. it yields non-spam for spammails (and maybe the other way around) - disable it in the gui to clear the database - you can try re-enabling it afterwards (that would start it with a fresh database)

as written elsewhere pmg only implements bayes autolearning as spamassassin does it:
https://cwiki.apache.org/confluence/display/SPAMASSASSIN/AutolearningNotWorking

that's what's handled automatically - although a few of our users do maintain and train their bayes database with sa-learn manually.

I must revise my answer a bit, that all accepted e-mails have BAYES_00. I have searched withing the tracking center for BAYES Strings and also found some matches for: So only SA scored mails having SA score of 3 oder more, have different BAYES_ WEIGHT Levels like this ones:

SA score=3/5 time=0.839 bayes=0.12 autolearn=no autolearn_force=no hits=BAYES_20(-0.001)
SA score=4/5 time=0.657 bayes=0.50 autolearn=no autolearn_force=no hits=BAYES_50(0.8)
SA score=12/5 time=3.999 bayes=1.00 autolearn=no autolearn_force=no hits=AWL(1.263),BAYES_99(3.5),BAYES_999(0.2)

So question is: Is my bayes working as it should? So is there no need for me to clear my bayes database? As i have autolearn=no autolearn_force=no for my above matches?

But also searched for some autolearn not no, like spam or ham status:

I got:

SA score=15/5 time=0.185 bayes=undefined autolearn=spam
SA score=0/5 time=0.397 bayes=undefined autolearn=ham

So i assume my autolearing works like a charm?

Sorry for the confusion and have the great weekend...

 

[sanojs]

Seems your bayes filtering is wrong (gives wrong indication) - I would suggest turning it off in GUI->Configuration->Spam Detector->Options

I hope this helps

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Thanks for your reply

I have applied the same thing you have mentioned above.

But we are getting spam mails with non-existing e-mail address end with our own domain, for example, our domain name is abc.com and getting mails with example.abc.com (which is an non-existing id)

Is this because of disabling bayes filtering, if not can you suggest what might be the reason for these type of spam mails.

 

[Stoiko Ivanov]

Is this because of disabling bayes filtering, if not can you suggest what might be the reason for these type of spam mails.

Bayes filtering performance depends on the data with which you train your bayes filter - so in theory it could be that bayes filtering caught all those mails - from my experience I would doubt that though.

Depending on how many different mails are used for this you could add them to the blacklist who object - then they would get blocked.

If this does not work for you - please provide some sample logs (anonymize them if necessary)

I hope this helps!

 

[sanojs]

Hi Stoiko,

Please find below the logs.

From: Support <itsupport@example.com>
To: abc@example.com
Subject mailbox: New found messages in quarantine: 4 of 4
Date: 02 Aug 2020 18:27:56 -0700
MIME-Version: 1.0
Content-Type: text/html;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
X-SPAM-LEVEL: Spam detection results: 1
AWL 0.350 Adjusted score from AWL reputation of From: address
HTML_MESSAGE 0.001 HTML included in message
KAM_NUMSUBJECT 0.5 Subject ends in numbers excluding current years
MIME_HTML_ONLY 0.1 Message only has text/html MIME parts
SPF_HELO_PASS -0.001 SPF: HELO matches SPF record
TO_EQ_FM_DOM_HTML_ONLY 0.599 To domain == From domain and HTML only

Thanks

 

[Stoiko Ivanov]

Please find below the logs.

These are not the logs but the headers of such a mail - not quite the same information.

In any case if this is the complete spam-level header as added by PMG it seems there is nothing particular about this mail which triggers spamassassin

please provide all lines related to that mail form /var/log/mail.log (or use the log-tracker in the GUI)