[TUTORIAL] Advancing Proxmox Mail Gateway (especially Spam and Virus Detection)

Let's ansibleize your advancements and put them into different ansible roles, so they are available to more people without the fear of doing something wrong.
 
  • Like
Reactions: velocity08
@heutger I was wondering if you were getting
Code: 
Nov 10 00:25:20 pmg clamd[778]: LibClamAV Error: yyerror(): /var/lib/clamav/antidebug_antivm.yar line 497 undefined identifier "pe"
Nov 10 00:25:20 pmg clamd[778]: LibClamAV Error: yyerror(): /var/lib/clamav/antidebug_antivm.yar line 512 undefined identifier "pe"
Nov 10 00:25:20 pmg clamd[778]: LibClamAV Error: yyerror(): /var/lib/clamav/antidebug_antivm.yar line 528 undefined identifier "pe"
Nov 10 00:25:20 pmg clamd[778]: LibClamAV Error: yyerror(): /var/lib/clamav/antidebug_antivm.yar line 544 undefined identifier "pe"
Nov 10 00:25:20 pmg clamd[778]: LibClamAV Error: yyerror(): /var/lib/clamav/antidebug_antivm.yar line 557 undefined identifier "pe"
Nov 10 00:25:20 pmg clamd[778]: LibClamAV Error: yyerror(): /var/lib/clamav/antidebug_antivm.yar line 603 undefined identifier "pe"
Nov 10 00:25:20 pmg clamd[778]: LibClamAV Error: yyerror(): /var/lib/clamav/antidebug_antivm.yar line 614 undefined identifier "pe"
Nov 10 00:25:20 pmg clamd[778]: LibClamAV Error: cli_loadyara: failed to parse rules file /var/lib/clamav/antidebug_antivm.yar, error count 7

recently check the logs and happens when clavmav scans
 
killmasta93 said:
@heutger I was wondering if you were getting
Code: 
Nov 10 00:25:20 pmg clamd[778]: LibClamAV Error: yyerror(): /var/lib/clamav/antidebug_antivm.yar line 497 undefined identifier "pe"
Nov 10 00:25:20 pmg clamd[778]: LibClamAV Error: yyerror(): /var/lib/clamav/antidebug_antivm.yar line 512 undefined identifier "pe"
Nov 10 00:25:20 pmg clamd[778]: LibClamAV Error: yyerror(): /var/lib/clamav/antidebug_antivm.yar line 528 undefined identifier "pe"
Nov 10 00:25:20 pmg clamd[778]: LibClamAV Error: yyerror(): /var/lib/clamav/antidebug_antivm.yar line 544 undefined identifier "pe"
Nov 10 00:25:20 pmg clamd[778]: LibClamAV Error: yyerror(): /var/lib/clamav/antidebug_antivm.yar line 557 undefined identifier "pe"
Nov 10 00:25:20 pmg clamd[778]: LibClamAV Error: yyerror(): /var/lib/clamav/antidebug_antivm.yar line 603 undefined identifier "pe"
Nov 10 00:25:20 pmg clamd[778]: LibClamAV Error: yyerror(): /var/lib/clamav/antidebug_antivm.yar line 614 undefined identifier "pe"
Nov 10 00:25:20 pmg clamd[778]: LibClamAV Error: cli_loadyara: failed to parse rules file /var/lib/clamav/antidebug_antivm.yar, error count 7

recently check the logs and happens when clavmav scans
Click to expand...

Hi,

as of the ClamAV update in the past, I posted here about, which did not ignore yararules failures any more, I‘m not sure, which solution you applied. One just fixed some of the failures, the other was to disable and remove yararules at all. I‘d choosen the last option to prevent continous bugfixing. As of that, I can’t confirm, my logs look fine, may be you should disable also the yararules at all. I gave a link on both solutions in my previous post about the ClamAV update.

Regards,
Christian
 
  • Like
Reactions: killmasta93
Just a very small add-on. I currently use the postfix header checks on my commercial test for some aggressive spammers. For sure, I believe, that once I can train bayes via the centralized spam and ham boxes, detection will get better and I can milter-reject some more mails, but sometimes I would prefer something like Scrollout F1 has, that you can run your own blacklist by catching aggressive spam. So what I mean are the typical "purchase this attendee or user list" spams. For sure, I could also block Gmail at all as most of such spam is coming from Google Mail Servers (valid signed with DKIM, coming from valid SPF servers, so all new techniques promoted by Google don't help, as if Google is then sending the spam themselves, as they seem to be lame on outgoing spam detection meanwhile propagating always new technologies and how well they perform for Google), but I'm afraid, that's no got idea. So here is my current postfix header_checks file (as adjusted from above):

Code: 
/^From:/ INFO
/^To:/ INFO
/^Subject:.*medtrade.*/ REJECT
/^Subject:.*user.*list.*/ REJECT
/^Subject:/ INFO
 
  • Like
Reactions: killmasta93
I still play around with additional blacklists and also wrote elsewhere in this forum on how I'm performing this, so refer to Relay Blacklist Optimization to check more out of my tests. How I'm performing:

I add to main.cf.in additional blacklists with warn_if_reject and then reject_rbl_client or reject_rhsbl_client for domain name blacklists and then check my logs (via tracker) for some time, usually minimum a week, up to one month. That's also why I already asked for and adjust as written above the postfix to perform header checks to add the sender, subject and recipient information given in the e-mail which may differ from the technical ones, how the mail is coming in. So called ESPs use the sender for detecting bounces and therefor don't give the later shown sender (e.g. in the mail client) but use a technical address. But if checking e.g. Amazon SES addresses for being rejected legit or not, it's hard if not being able to see the real sender, Amazon SES is used for. Same for Mailchimp and many others.
 
  • Like
Reactions: Calogero Lo Leggio and killmasta93
Quick question for the SMTP milter reject could apply for the blacklist? lets say i block a domain and if that domain sends an email instead of dropping it would reject telling them that they are blocked?
 
No, SMTP milter only works for the SpamAssassin part, so the ruleset of PMG is not integrated here. I would prefer, if Proxmox would overthink for the next release, how they integrated their ruleset, SpamAssassin and ClamAV virus scan and replace the current content filter way by a milter way. For the time being, SMTP milter can only reject (if you like and set this level) on reaching a particular score by SpamAssassin checks and always then state to the sender, that it's content blocked by SpamAssassin. If you want to reject because of a "blacklist", you may do that manually via postfix.
 
  • Like
Reactions: killmasta93
Thanks for the reply, dully noted. Also was implementing the EBL on msbl.org just got a cron error which was this

Code: 
/etc/cron.hourly/sa-update:
plugin: failed to parse plugin (from @INC): Can't locate Mail/SpamAssassin/Plugin/HashBL.pm in @INC (you may need to install the Mail::SpamAssassin::Plugin::HashBL module) (@INC contains: lib /usr/share/perl5 /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.24.1 /usr/local/share/perl/5.24.1 /usr/lib/x86_64-linux-gnu/perl5/5.24 /usr/lib/x86_64-linux-gnu/perl/5.24 /usr/share/perl/5.24 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at (eval 160) line 1.

plugin: failed to parse plugin (from @INC): Can't locate Mail/SpamAssassin/Plugin/HashBL.pm in @INC (you may need to install the Mail::SpamAssassin::Plugin::HashBL module) (@INC contains: lib /usr/share/perl5 /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.24.1 /usr/local/share/perl/5.24.1 /usr/lib/x86_64-linux-gnu/perl5/5.24 /usr/lib/x86_64-linux-gnu/perl/5.24 /usr/share/perl/5.24 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at (eval 233) line 1.

Thank you
 
killmasta93 said:
Thanks for the reply, dully noted. Also was implementing the EBL on msbl.org just got a cron error which was this

Code: 
/etc/cron.hourly/sa-update:
plugin: failed to parse plugin (from @INC): Can't locate Mail/SpamAssassin/Plugin/HashBL.pm in @INC (you may need to install the Mail::SpamAssassin::Plugin::HashBL module) (@INC contains: lib /usr/share/perl5 /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.24.1 /usr/local/share/perl/5.24.1 /usr/lib/x86_64-linux-gnu/perl5/5.24 /usr/lib/x86_64-linux-gnu/perl/5.24 /usr/share/perl/5.24 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at (eval 160) line 1.

plugin: failed to parse plugin (from @INC): Can't locate Mail/SpamAssassin/Plugin/HashBL.pm in @INC (you may need to install the Mail::SpamAssassin::Plugin::HashBL module) (@INC contains: lib /usr/share/perl5 /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.24.1 /usr/local/share/perl/5.24.1 /usr/lib/x86_64-linux-gnu/perl5/5.24 /usr/lib/x86_64-linux-gnu/perl/5.24 /usr/share/perl/5.24 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at (eval 233) line 1.

Thank you
Click to expand...

Looks like you don't have the plugin. If that's the issue, it seems like you didn't perform the latest updates from PMG as they brought a new SpamAssassin version, which include HashBL.pm.
 
  • Like
Reactions: killmasta93
Quick question any ideas howcome on the sa-update i keep getting a cron email from this
Code: 
channel: could not find working mirror, channel failed

This is the sa-update
Code: 
#!/bin/sh

# schaal @it
#
# Simple script to update SpamAssassin

SYSLOG_TAG=sa-update

compile=0

logger -d -t $SYSLOG_TAG "Start SA-Update"

sa-update --nogpg
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi


sa-update --nogpg --channel updates.spamassassin.org
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi

sa-update --nogpg --channel sa.zmi.at
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi

sa-update --nogpg --channel sa.schaal-it.net
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi

sa-update --nogpg --channel sought.rules.yerp.org
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi

sa-update --nogpg --channel spamassassin.heinlein-support.de
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi

if [ $compile -eq 1 ]; then
    logger -d -t $SYSLOG_TAG "SA-Update found"
    sa-compile --quiet 2>/dev/null
    systemctl restart pmg-smtp-filter.service
else
    logger -d -t $SYSLOG_TAG "No SA-Update found"
fi
 
killmasta93 said:
Quick question any ideas howcome on the sa-update i keep getting a cron email from this
Code: 
channel: could not find working mirror, channel failed

This is the sa-update
Code: 
#!/bin/sh

# schaal @it
#
# Simple script to update SpamAssassin

SYSLOG_TAG=sa-update

compile=0

logger -d -t $SYSLOG_TAG "Start SA-Update"

sa-update --nogpg
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi


sa-update --nogpg --channel updates.spamassassin.org
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi

sa-update --nogpg --channel sa.zmi.at
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi

sa-update --nogpg --channel sa.schaal-it.net
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi

sa-update --nogpg --channel sought.rules.yerp.org
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi

sa-update --nogpg --channel spamassassin.heinlein-support.de
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi

if [ $compile -eq 1 ]; then
    logger -d -t $SYSLOG_TAG "SA-Update found"
    sa-compile --quiet 2>/dev/null
    systemctl restart pmg-smtp-filter.service
else
    logger -d -t $SYSLOG_TAG "No SA-Update found"
fi
Click to expand...

At my updated documentation (and also mentioned some posts before) with new SA SHA256 signatures are mandantory. First also zmi failed, but has been fixed, however sought seems to be gone at all, so please remove to update for sought and it will work again.
 
  • Like
Reactions: killmasta93
heutger said:
At my updated documentation (and also mentioned some posts before) with new SA SHA256 signatures are mandantory. First also zmi failed, but has been fixed, however sought seems to be gone at all, so please remove to update for sought and it will work again.
Click to expand...
Thanks that did the trick for the 5.1 gave issue but the 5.0 was not giving any issue so i guess on the new box which had 5.1 had that issue

Thank you again
 
Update: So today i got a few cron error
Code: 
invalid regexp for rule SCHAALIT_HEADER_5752: /\Part num your Hacked phone!/: Can't find Unicode property definition "a" in regex; marked by <-- HERE in m/\Pa <-- HERE rt num your Hacked phone!/

config: warning: description exists for non-existent rule SCHAALIT_HEADER_5752

This is the SA-update

Code: 
#!/bin/sh

# schaal @it
#
# Simple script to update SpamAssassin

SYSLOG_TAG=sa-update

compile=0

logger -d -t $SYSLOG_TAG "Start SA-Update"

sa-update --nogpg
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi


sa-update --nogpg --channel updates.spamassassin.org
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi

sa-update --nogpg --channel sa.zmi.at
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi

sa-update --nogpg --channel sa.schaal-it.net
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi

sa-update --nogpg --channel spamassassin.heinlein-support.de
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi

if [ $compile -eq 1 ]; then
    logger -d -t $SYSLOG_TAG "SA-Update found"
    sa-compile --quiet 2>/dev/null
    systemctl restart pmg-smtp-filter
else
    logger -d -t $SYSLOG_TAG "No SA-Update found"
fi
 
killmasta93 said:
Update: So today i got a few cron error
Code: 
invalid regexp for rule SCHAALIT_HEADER_5752: /\Part num your Hacked phone!/: Can't find Unicode property definition "a" in regex; marked by <-- HERE in m/\Pa <-- HERE rt num your Hacked phone!/

config: warning: description exists for non-existent rule SCHAALIT_HEADER_5752

This is the SA-update

Code: 
#!/bin/sh

# schaal @it
#
# Simple script to update SpamAssassin

SYSLOG_TAG=sa-update

compile=0

logger -d -t $SYSLOG_TAG "Start SA-Update"

sa-update --nogpg
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi


sa-update --nogpg --channel updates.spamassassin.org
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi

sa-update --nogpg --channel sa.zmi.at
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi

sa-update --nogpg --channel sa.schaal-it.net
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi

sa-update --nogpg --channel spamassassin.heinlein-support.de
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi

if [ $compile -eq 1 ]; then
    logger -d -t $SYSLOG_TAG "SA-Update found"
    sa-compile --quiet 2>/dev/null
    systemctl restart pmg-smtp-filter
else
    logger -d -t $SYSLOG_TAG "No SA-Update found"
fi
Click to expand...

I got that too and already contacted the rule provider and he already solved the problem.
 
  • Like
Reactions: bhueske and killmasta93
DerDanilo said:
Let's ansibleize your advancements and put them into different ansible roles, so they are available to more people without the fear of doing something wrong.
Click to expand...
heutger said:
Will send you PM ;-)
Click to expand...

Hello,

First, I'd like to thank @heutger for his contribution to this. However, has there been any movement on this? So folks like me could benefit from this in a more reliable way? So far, I have a ad hoc, working setup, with postfixadmin, rspamd, dovecot, ClamAV, and Sieve. Although, I will experiment with vanilla Proxmox Mail Gateway nevertheless, I'd still like to give this a shot? The Guide here, the formatting, seems to cause more problems than it solves. :\
 
not sure if anyone else is getting this constant from the cron?

Code: 
channel: no 'mirrors.updates.spamassassin.org' record found, channel failed
channel: no 'mirrors.updates.spamassassin.org' record found, channel failed
channel: no 'mirrors.sa.zmi.at' record found, channel failed
channel: no 'mirrors.sa.schaal-it.net' record found, channel failed
channel: no 'mirrors.spamassassin.heinlein-support.de' record found, channel failed
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back