[TUTORIAL] Advancing Proxmox Mail Gateway (especially Spam and Virus Detection)

[heutger]

This post is split into three (3) parts, as I was not able to post it in one, so read all the next three posts to have the full overview.

 

[heutger]

I decide to create a new thread on my optimizations done to Proxmox Mail Gateway (PMG) with some explanations.

Steps done:

vi /etc/apt/sources.list.d/pmg-enterprise.list
apt-get update
apt-get upgrade
apt-get dist-upgrade

This steps changed to testing repository (should not be used for productive, a subscription should be ordered there), updated the apt-cache and performed all outstanding updates and upgrades.

apt-get install gcc
apt-get install make
apt-get install unzip
apt-get install vim
vi /etc/profile

Fetch all necessary packages for later steps, replace the horrible "neutered" vi version shipped with a more advanced version and add some aliases to /etc/profile and disable double entries in bash history.

vi /etc/pmg/pmg-api.pem
vi /etc/pmg/pmg-tls.pem
reboot

Replace the API certificate (also for the GUI) and the TLS certificate for STARTTLS with your own ones and reboot the system to take all changes effective (especially for sure changes on e.g. kernel packages by the updates and upgrades before).

mkdir .ssh
vi .ssh/authorized_keys

Place your public SSH key for faster and easier access to the PMG root shell.

apt-get install ufw
ufw enable
ufw default deny incoming
ufw default allow outgoing
ufw allow ssh
ufw allow smtp
ufw allow 8006
ufw status verbose

If PMG is not placed in a company DMZ (as it usually should and would be), enable the firewall, allow all outgoing, deny all incoming except ssh, smtp and the PMG GUI. ufw makes it easier to administer. For sure also outgoing connections could be fine-tuned, but usually it's not necessary.

apt-get install unbound
vi /etc/systemd/timesyncd.conf
systemctl restart systemd-timesyncd

As well, if not placed in a company DMZ with a companywide own DNS server (without forwarding to ISP resolvers) and/or time server, a local DNS server should be installed to prevent to be blocked at the RBL and URIBL lists (making them unusable) as well it's a good idea to set the timeserver to your countries nameserver pool.

mkdir -p /etc/pmg/templates
cp /var/lib/pmg/templates/* /etc/pmg/templates/.
cd /etc/pmg/templates
vi main.cf.in
vi /etc/postfix/header_checks

PMG uses templates for configuration files, so just adjust this templates. Tweak postfix a bit (details below) and add header checks for getting the given from, to and subject of mails been accepted as many senders use bounce-detection addresses, which prevent from successful interpreting the real message behavior.

apt-get install pyzor
cd /tmp
wget http://www.dcc-servers.net/dcc/source/dcc.tar.Z
tar xzvf dcc.tar.Z
cd dcc-*
./configure
make
make install
vi /lib/systemd/system/dcc.service
vi /var/dcc/dcc_conf
systemctl enable dcc
systemctl start dcc

Add pyzor and DCC (in daemon mode) to improve spam detection, be aware of the license requirements, e.g. that's the reason why PMG is not shipped directly with DCC.

apt-get install re2c
cd /etc/cron.hourly && wget sa.schaal-it.net/sa-update && chown root.root sa-update && chmod 755 sa-update
vi sa-update

Add additional rules to PMG from trusted sources (may or may not work for you) and let them update hourly. Let updates invoke sa-comile to get compiled rulesets, which improve performance of SpamAssassin.

cd /tmp
wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz
wget http://geolite.maxmind.com/download/geoip/database/GeoIPv6.dat.gz
gunzip GeoIP.dat.gz
gunzip GeoIPv6.dat.gz
mkdir /usr/share/GeoIP
mv GeoIP.dat /usr/share/GeoIP/
mv GeoIPv6.dat /usr/share/GeoIP/
vi /etc/pmg/templates/init.pre.in
vi /etc/pmg/templates/v320.pre.in
vi /etc/mail/spamassassin/custom.cf
/etc/cron.hourly/sa-update
pmgconfig sync --restart 1
spamassassin -D --lint
echo "test" | spamassassin -D pyzor 2>&1 | less

Add GeoIP data to improve bayes filter as well as adding rules to add scores for countries been passed by mails (may or may not work for you), enable pyzor and dcc, add blacklists for scoring (e.g. barracudacentral has been removed from SpamAssassin some time ago because of requirement to register IPs to use BRBL add their website, heise nix spam started to increase false positives, consider to check back with http://dnsbl.inps.de/index.cgi?lang=de&site=00009 and https://www.intra2net.com/en/support/antispam/ meanwhile last one shows rare false positives, I can't agree with), sync the changed templates, check SpamAssassin configuration for errors, check Pyzor, if it's working and perform the first ruleset update and rules compilation. After being finished, restart PMG SpamAssassin implementation to take sure, everything is running as expected.

wget https://github.com/extremeshok/clamav-unofficial-sigs/archive/master.zip
unzip master.zip
cp clamav-unofficial-sigs-master/clamav-unofficial-sigs.sh /usr/local/sbin/
chmod 755 /usr/local/sbin/clamav-unofficial-sigs.sh
mkdir /etc/clamav-unofficial-sigs
cp clamav-unofficial-sigs-master/config/* /etc/clamav-unofficial-sigs/
mkdir /var/log/clamav-unofficial-sigs
cd /etc/clamav-unofficial-sigs
cat /etc/*release*
mv os.debian9.conf os.conf
vi user.conf
/usr/local/sbin/clamav-unofficial-sigs.sh --install-cron
/usr/local/sbin/clamav-unofficial-sigs.sh --install-logrotate
/usr/local/sbin/clamav-unofficial-sigs.sh --install-man
/usr/local/sbin/clamav-unofficial-sigs.sh
cp /tmp/clamav-unofficial-sigs-master/systemd/* /etc/systemd/
clamscan --debug 2>&1 /dev/null | grep "loaded"

Install additional signatures for ClamAV to improve detection. Check the licenses first, free subscriptions are limited, on contribution e.g. MalwarePatrol give better subscriptions for free.

cd /tmp
wget https://github.com/Angristan/OpenVPN-install/archive/master.zip
unzip master.zip
cd OpenVPN-install-master
chmod +x openvpn-install.sh
./openvpn-install.sh
vi /etc/openvpn/server.conf
service openvpn restart
vi /root/client.ovpn
ufw allow openvpn
ufw delete allow 8006
ufw allow from 10.8.0.0/24 to any port 8006
ufw delete allow ssh
ufw allow from 10.8.0.0/24 to any port ssh

This steps are only required, if PMG is outside the company DMZ and you want to restrict access to PMGs GUI and SSH server only via OpenVPN. Setup is easy, however usually meant for road warrior setups, so need a few adjustments (mentioned below).

apt-get install lftp
mkdir /backup
mkdir /backup/node
mkdir /root/scripts
vi /root/scripts/backup.sh
vi /root/scripts/upload.sh
chmod +x /root/scripts/backup.sh
chmod +x /root/scripts/upload.sh
/root/scripts/backup.sh
vi /etc/crontab

Same for this backup setup, if PMG is running in the company environment, you may already have other backup solutions like Veeam if running as virtual machine on ESXi, otherwise, this scripts would help to backup most important data, for sure, it's not the whole system and it's no rsync, but it also allows rotation.

/etc/apt/sources.list.d/pmg-enterprise.list (for testing updates):

deb http://download.proxmox.com/debian/pmg stretch pmg-no-subscription

/etc/profile (for history control improvements and aliases):

# /etc/profile: system-wide .profile file for the Bourne shell (sh(1))
# and Bourne compatible shells (bash(1), ksh(1), ash(1), ...).

if [ "`id -u`" -eq 0 ]; then
  PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
else
  PATH="/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games"
fi
export PATH

HISTSIZE=1000
if [ "$HISTCONTROL" = "ignorespace" ] ; then
  export HISTCONTROL=ignoreboth
else
  export HISTCONTROL=ignoredups
fi
export HISTSIZE HISTCONTROL

alias "ls=ls -al"
alias "rm=rm --preserve-root"
alias "cd..=cd .."
alias "dir=ls -al"

if [ "${PS1-}" ]; then
  if [ "${BASH-}" ] && [ "$BASH" != "/bin/sh" ]; then
    # The file bash.bashrc already sets the default PS1.
    # PS1='\h:\w\$ '
    if [ -f /etc/bash.bashrc ]; then
      . /etc/bash.bashrc
    fi
  else
    if [ "`id -u`" -eq 0 ]; then
      PS1='# '
    else
      PS1='$ '
    fi
  fi
fi

if [ -d /etc/profile.d ]; then
  for i in /etc/profile.d/*.sh; do
    if [ -r $i ]; then
      . $i
    fi
  done
  unset i
fi

/etc/systemd/timesyncd.conf (for using local timeserver pool, here pool for Germany):

#  This file is part of systemd.
#
#  systemd is free software; you can redistribute it and/or modify it
#  under the terms of the GNU Lesser General Public License as published by
#  the Free Software Foundation; either version 2.1 of the License, or
#  (at your option) any later version.
#
# Entries in this file show the compile time defaults.
# You can change settings by editing this file.
# Defaults can be restored by simply deleting this file.
#
# See timesyncd.conf(5) for details.

[Time]
NTP=de.pool.ntp.org

/etc/pmg/templates/main.cf.in (for improved postfix setup):

# auto-generated by proxmox

compatibility_level = 2
command_directory = /usr/sbin
daemon_directory = /usr/lib/postfix/sbin
data_directory = /var/lib/postfix

# appending .domain is the MUA's job.
append_dot_mydomain = yes

smtpd_banner = $myhostname [% pmg.mail.banner %]
biff = no

[% IF pmg.mail.dwarning %]
delay_warning_time = [% pmg.mail.dwarning %]h
[% END %]

best_mx_transport = local
message_size_limit = [% pmg.mail.maxsize %]
mailbox_size_limit = [% ((pmg.mail.maxsize*2 > 51200000) ? pmg.mail.maxsize*2 : 51200000) %]

mydomain = [% dns.domain %]
myhostname = [% dns.hostname %].[% dns.domain %]

parent_domain_matches_subdomains = debug_peer_list,fast_flush_domains,mynetworks,permit_mx_backup_networks,qmqpd_authorized_clients,smtpd_access_maps

alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
mydestination = localhost, $myhostname
mynetworks = [% postfix.mynetworks %]

relay_domains = hash:/etc/pmg/domains

transport_maps = hash:/etc/pmg/transport

[% IF pmg.mail.relay %]
[% IF pmg.mail.relaynomx %]
relay_transport = smtp:[[% pmg.mail.relay %]]:[% pmg.mail.relayport %]
[% ELSE %]
relay_transport = smtp:[% pmg.mail.relay %]:[% pmg.mail.relayport %]
[% END %]
[% END %]

[% IF pmg.mail.smarthost %]
default_transport = smtp:[% pmg.mail.smarthost %]:[% pmg.mail.smarthostport %]
[% END %]

content_filter=scan:127.0.0.1:10024

mail_name = Proxmox

[% IF pmg.mail.helotests %]
smtpd_helo_required = yes
smtpd_helo_restrictions = permit_mynetworks reject_non_fqdn_helo_hostname reject_invalid_helo_hostname reject_rhsbl_helo dbl.spamhaus.org reject_rhsbl_helo dbl.abuse.ro
[% ELSE %]
smtpd_helo_restrictions =
[% END %]

Adding additional HELO checks, e.g. checking against Spamhaus and Abuse.ro domain blacklist.

postscreen_access_list =
        permit_mynetworks,
        cidr:/etc/postfix/postscreen_access

[% IF postfix.dnsbl_sites %]
postscreen_dnsbl_sites = [% postfix.dnsbl_sites %]
postscreen_dnsbl_threshold = [% postfix.dnsbl_threshold %]
[% END %]

postscreen_dnsbl_action = enforce
postscreen_greet_action = enforce

Using the given DNS blacklists with weights and a threshold of 2 (Update: Can now be set via GUI) let's still use barracudacentral and nixspam, although barracudacentral had some false-positives and questionable promoting paid emailreg service, a bit similar to uceprotect, nixspam recently started blocking services like getpocket, facebook, instragram, pinterest etc., so it's not usable anymore as fully trusted blacklist, but in combination it's reasonable. May or may not work for you, maybe other lists work better for you. Lateron added senderscore, spameatingmonkey, realtimeblacklist and dronebl as "level 1" and gbudb truncate as well as blocklist.de as "level 2".

smtpd_sender_restrictions =
        permit_mynetworks
        reject_non_fqdn_sender
        check_client_access     cidr:/etc/postfix/clientaccess
        check_sender_access     regexp:/etc/postfix/senderaccess
        check_recipient_access  regexp:/etc/postfix/rcptaccess
[%- IF pmg.mail.rejectunknown %] reject_unknown_client_hostname[% END %]
[%- IF pmg.mail.rejectunknownsender %] reject_unknown_sender_domain[% END %]
        reject_rhsbl_client dbl.spamhaus.org
        reject_rhsbl_reverse_client dbl.spamhaus.org
        reject_rhsbl_sender dbl.spamhaus.org
        reject_rhsbl_client dbl.abuse.ro
        reject_rhsbl_reverse_client dbl.abuse.ro
        reject_rhsbl_sender dbl.abuse.ro

Same as above, Spamhaus and Abuse.ro domain blacklist could be additional used.

smtpd_recipient_restrictions =
        permit_mynetworks
        reject_unauth_destination
        reject_non_fqdn_recipient
        check_recipient_access  regexp:/etc/postfix/rcptaccess
[%- IF postfix.usepolicy %] check_sender_access  regexp:/etc/postfix/senderaccess[% END %]
[%- IF postfix.usepolicy %] check_client_access  cidr:/etc/postfix/clientaccess[% END %]
[%- IF postfix.usepolicy %] check_policy_service inet:127.0.0.1:10022[% END %]
[%- IF pmg.mail.verifyreceivers %] reject_unknown_recipient_domain[% END %]
[%- IF pmg.mail.verifyreceivers %] reject_unverified_recipient[% END %]

smtpd_data_restrictions = reject_unauth_pipelining

A small but good tweak.

[% IF pmg.mail.rejectunknownsender %]
unknown_address_reject_code = 550
[% ELSE %]
[% IF pmg.mail.verifyreceivers %]
unknown_address_reject_code = 550
[% END %]
[% END %]

[% IF pmg.mail.rejectunknown %]
unknown_client_reject_code = 550
[% END %]

[% IF pmg.mail.verifyreceivers %]
unverified_recipient_reject_code = [% pmg.mail.verifyreceivers %]
[% END %]

Changing additional 450 to 550 reject codes.

smtpd_client_connection_count_limit = [% pmg.mail.conn_count_limit %]
smtpd_client_connection_rate_limit = [% pmg.mail.conn_rate_limit %]
smtpd_client_message_rate_limit = [% pmg.mail.message_rate_limit %]

[% IF pmg.mail.tls %]
smtp_tls_security_level = may
smtp_tls_policy_maps = hash:/etc/pmg/tls_policy
smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt
smtpd_tls_security_level = may
smtpd_tls_cert_file = /etc/pmg/pmg-tls.pem
smtpd_tls_key_file = $smtpd_tls_cert_file
[% IF pmg.mail.tlslog %]
smtpd_tls_loglevel = 1
smtp_tls_loglevel = 1
[% END %]
[% IF pmg.mail.tlsheader %]
smtpd_tls_received_header = yes
[% END %]
[% END %]

header_checks = regexp:/etc/postfix/header_checks

Adding header checks mentioned above, explained below.

smtp_tls_session_cache_database = btree:/var/lib/postfix/smtp_tls_session_cache
smtpd_tls_session_cache_database = btree:/var/lib/postfix/smtpd_tls_session_cache

[% IF pmg.mail.hide_received %]
unverified_recipient_reject_reason = Recipient address lookup failed
[% END %]


default_destination_concurrency_limit = 40
lmtp_destination_concurrency_limit = 20
relay_destination_concurrency_limit = 20
smtp_destination_concurrency_limit = 20
virtual_destination_concurrency_limit = 20

recipient_delimiter = +

This part left unchanged.

 

[heutger]

/etc/postfix/header_checks:

/^From:/ INFO
/^To:/ INFO
/^Subject:/ INFO

Adds given from string and address, to string and address as well as subject as Info-Text into the mail.log and as been shown there into the Tracking Center as well.

/lib/systemd/system/dcc.service (lets dcc run as daemon):

[Unit]
Description=DCC (Distributed Checksum Clearinghouses) interface daemon
After=remote-fs.target systemd-journald-dev-log.socket

[Service]
Type=forking
PermissionsStartOnly=true
RuntimeDirectory=dcc
ExecStart=/var/dcc/libexec/dccifd
User=root
Group=root
Nice=1

#DCC writes pid file with "-" at the beginning which confuses systemd
#PIDFile=/run/dcc/dccifd.pid

[Install]
WantedBy=multi-user.target

/var/dcc/dcc_conf (enables dccifd, the only somehow important" file not being backed up in the backup script below):

#! /bin/sh

# set parameters for DCC start, stop, and cron scripts

# This file is parsed by /bin/sh, and so parameters must be appropriately quoted
#   Start your own comment lines with something other than "#" such as "##"
#   so that they will be preserved when installing a new version.


# from Rhyolite Software DCC 1.3.163-1.88 $Revision$
DCC_CONF_VERSION=4

# don't set DCC_HOMEDIR since if we got here, it must be set
DCC_LIBEXEC=/var/dcc/libexec
DCC_RUNDIR=/var/run/dcc

# DCC user name
DCCUID=root


DCCD_ENABLE=off
# DCC server-IDs must be globally unique.
SRVR_ID=
# BRAND can be any short alphanumeric string that suggests the identity
#   of the server.
BRAND=
# args used to start dccd
DCCD_ARGS=


# GREY_CLIENT_ARGS contains "on", "-GnoIP", etc. to turn on greylisting
#    in the dccm and dccifd DCC clients.
#   Also turns on the local greylist dccd server unless GREY_ENABLE=off
GREY_CLIENT_ARGS=
# GREY_ENABLE turns local greylist server 'on' or 'off',
#    but does not effect dccm, dccifd
GREY_ENABLE=

# GREY_SRVR_ID DCC server-IDs must be globally unique, but greylisting dccd
#   servers are usually isolated.  If you have more than one greylist server,
#   ensure that they use distinct server-IDs and that they flood each other
#   with entries in /var/dcc/flod
GREY_SRVR_ID=$SRVR_ID
# Start dccd for grey listing or set server options such as -Gweak-IP.
#   See also GREY_ENABLE.
GREY_DCCD_ARGS=

# dccm and dccifd client reputation parameters such as -tREP,20
REP_ARGS=-tREP,20

# DNS blacklist -B parameters for dccifd and dccm
#   For example, this checks SMTP client IP addresses for any value Spamhaus'
#   ZEN.  It also looks for for values between 127.0.0.0 and 127.0.0.8 for SMTP
#   envelope sender mail_host or mail_host domain names, URLs in mail message
#   bodies, MX servers, DNS (NS) servers.
#   It then checks URLs in the message body in Spamhaus' DBL.
#   It also whitelists with the "0 = none" values of dnswl.org.
# DNSBL_ARGS="'-Bset:rej-msg=5.7.1 550 %ID %BTYPE https://www.spamhaus.org/query/bl?ip=%BTGT' \
#    -Bzen.spamhaus.org,127.0.0.0-127.0.0.8 \
#    -Bset:no-mail_host -Bset:no-URL -Bset:no-NS -Bzen.spamhaus.org \
#    -Bset:URL -Bset:no-client -Bdbl.spamhaus.org,127.0.1.1-127.0.1.59,name \
#    -Bset:white -Bset:client -Bset:no-URL -Bset:no-mail_host \
#    -Bset:no-NS -Bset:no-MX '-Blist.dnswl.org,127.0.0.3&255.255.0.255'"
DNSBL_ARGS=


DCCM_ENABLE=off
#
# used to start dccm
#   a common value is
#    DCCM_ARGS="-SHELO -Smail_host -SSender -SList-ID"
#   Note the use of single quotes in
#    DCCM_ARGS="-SHELO '-r5.7.1 550 mail %s from %s rejected with DCC'"
#   Put greylist parameters in GREY_CLIENT_ARGS but not in DCCM_ARGS.
DCCM_ARGS="-SHELO -Smail_host -SSender -SList-ID"
# The log directories should be cleaned with the /var/dcc/libexec/cron-dccd
#   nightly cron job.  DCCM_LOGDIR can start with D? H? or M? as in
#   DCCM_LOGDIR='D?log'  See -l in the man page.
#   Set DCCM_LOGDIR to the empty or null string to disable logging.
DCCM_LOGDIR="D?log"
DCCM_WHITECLNT=whiteclnt
DCCM_USERDIRS=userdirs
# set DCCM_LOG_AT to a number that determines "bulk mail" for your situation.
#   50 is a typical value.
# Leave DCCM_REJECT_AT blank until you are confident that most sources of
#   solicited bulk mail have been whitelisted.  Then set it to the number
#   that defines "bulk mail" for your site.  This rejection or "bulk" threshold
#   does not affect the blacklisting of the DCCM_WHITECLNT whitelist file.
# Add '-aIGNORE' to DCCM_ARGS to ignore the bulkiness of mail except to
#   add X-DCC headers.
DCCM_LOG_AT=5
DCCM_REJECT_AT=
# override basic list of DCC server checksums controlling rejections or logging
DCCM_CKSUMS=
# additional DCC server checksums worthy of rejections or logging
DCCM_XTRA_CKSUMS=


DCCIFD_ENABLE=on
#
# used to start dccifd
#   a common value is
#   DCCIFD_ARGS="-SHELO -Smail_host -SSender -SList-ID"
DCCIFD_ARGS="-SHELO -Smail_host -SSender -SList-ID"
DCCIFD_LOGDIR="$DCCM_LOGDIR"
DCCIFD_WHITECLNT="$DCCM_WHITECLNT"
#   When both dccm and dccifd are used it may be necessary to set
#   DCCIFD_USERDIRS="$DCCM_USERDIRS/local"
DCCIFD_USERDIRS="$DCCM_USERDIRS"
DCCIFD_LOG_AT="$DCCM_LOG_AT"
DCCIFD_REJECT_AT="$DCCM_REJECT_AT"
# override basic list of checksums controlling rejections or logging
DCCIFD_CKSUMS="$DCCM_CKSUMS"
# additional DCC server checksums worthy of rejections or logging
DCCIFD_XTRA_CKSUMS="$DCCM_XTRA_CKSUMS"

# days to keep files in DCC log directories
DBCLEAN_LOGDAYS=14
# used to start dbclean, including -H, -u, -e, and -E
DBCLEAN_ARGS=


# optionally set to something like "local5" or "local5.notice" for
#   dccd, dbclean, dccifd, and dccm.  The defaults are equivalent to
#   DCC_INFO_LOG_FACILITY=MAIL.NOTICE and DCC_ERROR_LOG_FACILITY=MAIL.ERR
DCC_INFO_LOG_FACILITY=
DCC_ERROR_LOG_FACILITY=


# ensure that the log facilities include levels and that $DCC_LOGGER
#   has a default.
if test -n "$DCC_INFO_LOG_FACILITY"; then
    if expr "X$DCC_INFO_LOG_FACILITY" : 'X.*\..*' >/dev/null; then
    :
    else
    DCC_INFO_LOG_FACILITY="$DCC_INFO_LOG_FACILITY.notice"
    fi
    DCC_LOG_ARGS="$DCC_LOG_ARGS -Linfo,$DCC_INFO_LOG_FACILITY"
fi
if test -z "$DCC_ERROR_LOG_FACILITY"; then
    # for $DCC_LOGGER
    DCC_ERROR_LOG_FACILITY=mail.err
else
    if expr "X$DCC_ERROR_LOG_FACILITY" : 'X.*\..*' >/dev/null; then
    :
    else
    DCC_ERROR_LOG_FACILITY="$DCC_ERROR_LOG_FACILITY.err"
    fi
    DCC_LOG_ARGS="$DCC_LOG_ARGS -Lerror,$DCC_ERROR_LOG_FACILITY"
fi
DCC_LOGGER="logger -s -p ${DCC_ERROR_LOG_FACILITY-mail.err} -t ${LOGGER_TAG-DCC}"


# capture ./configure values for make-dcc_conf
Configure_DCC_LIBEXEC=/var/dcc/libexec
Configure_DCC_RUNDIR=/var/run/dcc
Configure_DCCUID=root
Configure_DCC_LOGGER="logger -s -p ${DCC_ERROR_LOG_FACILITY-mail.err} -t ${LOGGER_TAG-DCC}"

/etc/cron.hourly/sa-update (adding rules from zmi, schaal-it and heinlein):

#!/bin/sh

# schaal @it
#
# Simple script to update SpamAssassin

SYSLOG_TAG=sa-update

compile=0

logger -d -t $SYSLOG_TAG "Start SA-Update"

sa-update --nogpg
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi


sa-update --nogpg --channel updates.spamassassin.org
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi

sa-update --nogpg --channel sa.zmi.at
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi

sa-update --nogpg --channel sa.schaal-it.net
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi

sa-update --nogpg --channel spamassassin.heinlein-support.de
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi

if [ $compile -eq 1 ]; then
    logger -d -t $SYSLOG_TAG "SA-Update found"
    sa-compile --quiet 2>/dev/null
    systemctl restart pmg-smtp-filter
else
    logger -d -t $SYSLOG_TAG "No SA-Update found"
fi

/etc/pmg/templates/init.pre.in (enabling Pyzor, DCC and GeoData):

# This is the right place to customize your installation of SpamAssassin.
#
# See 'perldoc Mail::SpamAssassin::Conf' for details of what can be
# tweaked.
#
# This file contains plugin activation commands for plugins included
# in SpamAssassin 3.0.x releases.  It will not be installed if you
# already have a file in place called "init.pre".
#
###########################################################################

# RelayCountry - add metadata for Bayes learning, marking the countries
# a message was relayed through
#
# loadplugin Mail::SpamAssassin::Plugin::RelayCountry

[% IF pmg.spam.rbl_checks %]
loadplugin Mail::SpamAssassin::Plugin::URIDNSBL
[% END %]

# Hashcash - perform hashcash verification.
#
loadplugin Mail::SpamAssassin::Plugin::Hashcash

[% IF pmg.spam.rbl_checks %]
loadplugin Mail::SpamAssassin::Plugin::SPF
[% END %]

# always load dkim to improve accuracy
loadplugin Mail::SpamAssassin::Plugin::DKIM

loadplugin Mail::SpamAssassin::Plugin::Pyzor
use_pyzor 1

loadplugin Mail::SpamAssassin::Plugin::DCC
dcc_path /usr/local/bin/dccproc
dcc_home /var/dcc
dcc_dccifd_path /var/dcc/dccifd
dcc_body_max 999999
dcc_fuz1_max 999999
dcc_fuz2_max 999999
use_dcc 1
dcc_timeout 10

loadplugin Mail::SpamAssassin::Plugin::RelayCountry

/etc/pmg/templates/v320.pre.in (enabling compiled rulesets):

# This is the right place to customize your installation of SpamAssassin.
#
# See 'perldoc Mail::SpamAssassin::Conf' for details of what can be
# tweaked.
#
# This file was installed during the installation of SpamAssassin 3.2.0,
# and contains plugin loading commands for the new plugins added in that
# release.  It will not be overwritten during future SpamAssassin installs,
# so you can modify it to enable some disabled-by-default plugins below,
# if you so wish.
#
###########################################################################

# Check - Provides main check functionality
#
loadplugin Mail::SpamAssassin::Plugin::Check

# HTTPSMismatch - find URI mismatches between href and anchor text
#
loadplugin Mail::SpamAssassin::Plugin::HTTPSMismatch

# URIDetail - test URIs using detailed URI information
#
loadplugin Mail::SpamAssassin::Plugin::URIDetail

# Shortcircuit - stop evaluation early if high-accuracy rules fire
#
# loadplugin Mail::SpamAssassin::Plugin::Shortcircuit

# Plugins which used to be EvalTests.pm
# broken out into separate plugins
loadplugin Mail::SpamAssassin::Plugin::Bayes
loadplugin Mail::SpamAssassin::Plugin::BodyEval
loadplugin Mail::SpamAssassin::Plugin::DNSEval
loadplugin Mail::SpamAssassin::Plugin::HTMLEval
loadplugin Mail::SpamAssassin::Plugin::HeaderEval
loadplugin Mail::SpamAssassin::Plugin::MIMEEval
loadplugin Mail::SpamAssassin::Plugin::RelayEval
loadplugin Mail::SpamAssassin::Plugin::URIEval
loadplugin Mail::SpamAssassin::Plugin::WLBLEval

# VBounce - anti-bounce-message rules, see rules/20_vbounce.cf
#
loadplugin Mail::SpamAssassin::Plugin::VBounce

# Rule2XSBody - speedup by compilation of ruleset to native code
#
loadplugin Mail::SpamAssassin::Plugin::Rule2XSBody

# ASN - Look up the Autonomous System Number of the connecting IP
# and create a header containing ASN data for bayes tokenization.
# See plugin's POD docs for usage info.
#
# loadplugin Mail::SpamAssassin::Plugin::ASN

# ImageInfo - rules to match metadata of image attachments
#
loadplugin Mail::SpamAssassin::Plugin::ImageInfo

/etc/mail/spamassassin/custom.cf (adding country filters as well as additional backlist scores, xxx, xxx24 and xxxuri should be your own invaluement subscription domains):

ifplugin Mail::SpamAssassin::Plugin::RelayCountry
add_header all Relay-Country _RELAYCOUNTRY_
header RELAYCOUNTRY_BAD X-Relay-Countries =~ /(CN|RU|UA|RO|VN)/
describe RELAYCOUNTRY_BAD Relayed through spammy country at some point
score RELAYCOUNTRY_BAD 2.0
header RELAYCOUNTRY_GOOD X-Relay-Countries =~ /^(DE|AT|CH)/
describe RELAYCOUNTRY_GOOD First untrusted GW is DE, AT or CH
score RELAYCOUNTRY_GOOD -0.5
endif # Mail::SpamAssassin::Plugin::RelayCountry

header RCVD_IN_BRBL eval:check_rbl('brbl-lastexternal', 'b.barracudacentral.org.', '127.0.0.2')
describe RCVD_IN_BRBL Received via a relay in Barracuda RBL
tflags RCVD_IN_BRBL net
score RCVD_IN_BRBL 1.4

header RCVD_IN_NIX_SPAM eval:check_rbl('nix-spam-lastexternal', 'ix.dnsbl.manitu.net.')
describe RCVD_IN_NIX_SPAM Listed in NiX Spam DNSBL (heise.de)
tflags RCVD_IN_NIX_SPAM net
score RCVD_IN_NIX_SPAM 1.4

header RCVD_IN_GBUDB eval:check_rbl('gbudb-lastexternal', 'truncate.gbudb.net.', '127.0.0.2')
describe RCVD_IN_GBUDB Listed in GBUdb Truncate
tflags RCVD_IN_GBUDB net
score RCVD_IN_GBUDB 1.4

header RCVD_IN_BLOCKLIST eval:check_rbl('blocklist-lastexternal', 'bl.blocklist.de.')
describe RCVD_IN_BLOCKLIST Listed in Blocklist DNSRBL
tflags RCVD_IN_BLOCKLIST net
score RCVD_IN_BLOCKLIST 1.4

header RCVD_IN_IVMSIP eval:check_rbl('ivmsip-lastexternal', 'xxx')
describe RCVD_IN_IVMSIP Listed in ivmSIP
tflags RCVD_IN_IVMSIP net
score RCVD_IN_IVMSIP 1.4

header RCVD_IN_IVMSIP24 eval:check_rbl('ivmsip-lastexternal', 'xxx24')
describe RCVD_IN_IVMSIP24 Listed in ivmSIP/24
tflags RCVD_IN_IVMSIP24 net
score RCVD_IN_IVMSIP24 1

ifplugin Mail::SpamAssassin::Plugin::URIDNSBL
urirhssub URIBL_IVMURI xxxuri A 2
body URIBL_IVMURI eval:check_uridnsbl('URIBL_IVMURI')
describe URIBL_IVMURI Listed in ivmURI
tflags URIBL_IVMURI net
score URIBL_IVMURI 1.4
reuse URIBL_IVMURI
endif

/etc/clamav-unofficial-sigs/user.conf (enabling additional signatures, xxx needs to be replaced by your license keys):

# This file contains user configuration settings for clamav-unofficial-sigs.sh
###################
# This is property of eXtremeSHOK.com
# You are free to use, modify and distribute, however you may not remove this notice.
# Copyright (c) Adrian Jon Kriel :: admin@extremeshok.com
# License: BSD (Berkeley Software Distribution)
##################
#
# Script updates can be found at: https://github.com/extremeshok/clamav-unofficial-sigs
#
##################
#
# NOT COMPATIBLE WITH VERSION 3.XX / 4.XX CONFIG
#
################################################################################
# SEE MASTER.CONF FOR CONFIG EXPLAINATIONS
################################################################################

# Values in this file will always override those in the master.conf and os.conf files.
# This is useful to specify your authorisation/receipt codes and to always force certain options.
# Please note, it is your responsibility to manage the contents of this file.
# Values provided here are just examples, feel free to use any values from the main config file.

malwarepatrol_receipt_code="xxxxx"
malwarepatrol_product_code="32"
malwarepatrol_list="clamav_basic" # clamav_basic or clamav_ext
malwarepatrol_free="no"

securiteinfo_authorisation_signature="xxxxx"

# Default dbs rating
# valid rating: LOW, MEDIUM, HIGH
#default_dbs_rating="MEDIUM"

# Per Database
# These ratings will override the global rating for the specific database
# valid rating: LOW, MEDIUM, HIGH, DISABLE
#sanesecurity_dbs_rating=""
#securiteinfo_dbs_rating=""
#linuxmalwaredetect_dbs_rating=""
#yararulesproject_dbs_rating=""

# =========================
# Additional signature databases
# =========================
#declare -a additional_dbs=(
#   ftp://ftp.example.net/pub/sigs.ndb
#   http://www.example.org/sigs.ldb
#) #END ADDITIONAL DATABASES

# Uncomment the following line to enable the script
user_configuration_complete="yes"

# https://eXtremeSHOK.com ######################################################

 

[heutger]

/etc/openvpn/server.conf (need to disable road warrior setup, no forwarding of traffic, no pushing DNS servers, xxx needs to be replaced by your cert files):

port 1194
proto udp
dev tun
user nobody
group nogroup
persist-key
persist-tun
keepalive 10 120
topology subnet
server 10.8.0.0 255.255.255.0
ifconfig-pool-persist ipp.txt
crl-verify crl.pem
ca ca.crt
cert xxx.crt
key xxx.key
tls-crypt tls-auth.key
dh none
ecdh-curve secp384r1
auth SHA384
cipher AES-256-GCM
tls-server
tls-version-min 1.2
tls-cipher TLS-ECDHE-RSA-WITH-AES-256-GCM-SHA384
status openvpn.log
verb 3
duplicate-cn

/root/client.ovpn (same here, xxx should be your host, ... will be your certs and keys):

client
proto udp
remote xxx 1194
dev tun
resolv-retry infinite
nobind
persist-key
persist-tun
remote-cert-tls server
verify-x509-name xxx name
auth SHA384
auth-nocache
cipher AES-256-GCM
tls-client
tls-version-min 1.2
tls-cipher TLS-ECDHE-RSA-WITH-AES-256-GCM-SHA384
verb 3
...

/root/scripts/backup.sh:

#!/bin/bash

echo [`date +'%d.%m.%Y %H:%M'`] Starte Backup ...

echo [`date +'%d.%m.%Y %H:%M'`] Exportiere aktuelle Konfiguration [1/4] ...
rm /var/lib/pmg/backup/*
pmgbackup backup

# Zu sicherndes Verzeichnisse, z.B. /etc /root /var
sourcevrz='/etc /lib/systemd/system /root /usr/local /var/dcc /var/lib/pmg/backup /var/log'

# Backupverzeichnis, z.B. /backup/node
backupvrz=/backup/node

# Variablendefinition
datum=$(date +'%Y%m%d')
dateiname=$backupvrz/backup$datum.tar

# -----------------------------------------------------
function f_delFiles()
# -----------------------------------------------------
# $1 Backupverzeichnis
{
  loeschdatum=$(date --date='7 days ago' +'%Y%m%d')
  rm $1/backup$loeschdatum.tar
}

echo [`date +'%d.%m.%Y %H:%M'`] Lösche Files in $backupvrz, die älter als 7 Tage sind [2/4] ...
f_delFiles $backupvrz

echo [`date +'%d.%m.%Y %H:%M'`] Sichere $sourcevrz nach $dateiname [3/4] ...
tar Pcf $dateiname $sourcevrz

echo [`date +'%d.%m.%Y %H:%M'`] Synchronisiere mit Onlinespeicher [4/4] ...
bash /root/scripts/upload.sh

echo [`date +'%d.%m.%Y %H:%M'`] Fertig!

/root/scripts/upload.sh (uploads to your FTP storage, passive mode and starttls preferred and used if available, xxx should be your host, credentials, folders etc.):

#!/bin/bash
HOST='xxx'
USER='xxx'
PASS='xxx'
TARGETFOLDER='/xxx'
SOURCEFOLDER='/backup'

lftp -e "
open $HOST
user $USER $PASS
lcd $SOURCEFOLDER
mirror --reverse --delete --use-cache --verbose $SOURCEFOLDER $TARGETFOLDER
bye
"

/etc/crontab (you should anyhow cron your backup, I prefer my own settings, but could also be done via cron.daily or whatever you like, I also let the backup send me the output, but it's also your decision, xxx should be your address, x should be your minute, hour, day etc. settings):

# /etc/crontab: system-wide crontab
# Unlike any other crontab you don't have to run the `crontab'
# command to install the new version when you edit this file
# and files in /etc/cron.d. These files also have username fields,
# that none of the other crontabs do.

SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

# m h dom mon dow user    command
  x x *   *   *   root  /root/scripts/backup.sh 2>&1 | /usr/bin/mail -r xxx@xxx -s "backup script output" xxx@xxx

x *    * * *    root    cd / && run-parts --report /etc/cron.hourly
x x    * * *    root    test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )
x x    * * 7    root    test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly )
x x    1 * *    root    test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly )
#

pmgconfig dump (tighter setup, xxx should be your domain data, xxx and xxx24 later-on are the invaluement lists, take your own subscription):

composed.wl_bounce_relays = xxx
dns.domain = xxx
dns.hostname = xxx
ipconfig.int_ip = xxx
pmg.admin.advfilter = 0
pmg.admin.avast = 0
pmg.admin.clamav = 1
pmg.admin.dailyreport = 1
pmg.admin.demo = 0
pmg.admin.email = xxx
pmg.admin.http_proxy =
pmg.admin.statlifetime = 7
pmg.clamav.archiveblockencrypted = 0
pmg.clamav.archivemaxfiles = 1000
pmg.clamav.archivemaxrec = 5
pmg.clamav.archivemaxsize = 25000000
pmg.clamav.dbmirror = database.clamav.net
pmg.clamav.maxcccount = 0
pmg.clamav.maxscansize = 100000000
pmg.clamav.safebrowsing = 1
pmg.mail.banner = ESMTP Proxmox
pmg.mail.conn_count_limit = 50
pmg.mail.conn_rate_limit = 0
pmg.mail.dnsbl_sites =
zen.spamhaus.org*2,bl.spamcop.net*2,psbl.surriel.com*2,spamrbl.imp.ch*2,noptr.spamrats.com*2,escalations.dnsbl.sorbs.net*2,bl.score.senderscore.com*2,bl.spameatingmonkey.net*2,rbl.realtimeblacklist.com*2,dnsbl.dronebl.org*2,ix.dnsbl.manitu.net,b.barracudacentral.org,truncate.gbudb.net,bl.blocklist.de,xxx,xxx24
pmg.mail.dwarning = 4
pmg.mail.ext_port = 25
pmg.mail.greylist = 0
pmg.mail.helotests = 1
pmg.mail.hide_received = 0
pmg.mail.int_port = 26
pmg.mail.max_filters = 14
pmg.mail.max_policy = 5
pmg.mail.max_smtpd_in = 96
pmg.mail.max_smtpd_out = 96
pmg.mail.maxsize = 104857600
pmg.mail.message_rate_limit = 0
pmg.mail.rejectunknown = 1
pmg.mail.rejectunknownsender = 1
pmg.mail.relay = xxx
pmg.mail.relaynomx = 0
pmg.mail.relayport = 25
pmg.mail.smarthost =
pmg.mail.spf = 1
pmg.mail.tls = 1
pmg.mail.tlsheader = 1
pmg.mail.tlslog = 1
pmg.mail.verifyreceivers = 550
pmg.spam.bounce_score = 0
pmg.spam.clamav_heuristic_score = 3
pmg.spam.languages = all
pmg.spam.maxspamsize = 262144
pmg.spam.rbl_checks = 1
pmg.spam.use_awl = 1
pmg.spam.use_bayes = 1
pmg.spam.use_razor = 1
pmg.spam.wl_bounce_relays =
pmg.spamquar.allowhrefs = 1
pmg.spamquar.authmode = ticket
pmg.spamquar.hostname =
pmg.spamquar.lifetime = 7
pmg.spamquar.mailfrom =
pmg.spamquar.port = 8006
pmg.spamquar.protocol = https
pmg.spamquar.reportstyle = verbose
pmg.spamquar.viewimages = 1
pmg.virusquar.allowhrefs = 1
pmg.virusquar.lifetime = 7
pmg.virusquar.viewimages = 1
postfix.dnsbl_sites =
zen.spamhaus.org*2,bl.spamcop.net*2,psbl.surriel.com*2,spamrbl.imp.ch*2,noptr.spamrats.com*2,escalations.dnsbl.sorbs.net*2,bl.score.senderscore.com*2,bl.spameatingmonkey.net*2,rbl.realtimeblacklist.com*2,dnsbl.dronebl.org*2,ix.dnsbl.manitu.net,b.barracudacentral.org,truncate.gbudb.net,bl.blocklist.de,xxx,xxx24
postfix.int_ip = xxx
postfix.mynetworks = 127.0.0.0/8 [::1]/128 xxx
postfix.transportnets =
postfix.usepolicy = 1

pmgdb dump (as well tighter setup):

Found RULE 4: Blacklist
  FOUND FROM GROUP 2: Blacklist
    OBJECT 1: nomail@fromthisdomain.com
  FOUND ACTION GROUP 18: Block
    OBJECT 31: block message
Found RULE 3: Virus Alert
  FOUND WHAT GROUP 9: Virus
    OBJECT 22: active
  FOUND ACTION GROUP 18: Block
    OBJECT 31: block message
  FOUND ACTION GROUP 20: Notify Admin
    OBJECT 33: notify __ADMIN__
  FOUND ACTION GROUP 21: Notify Sender
    OBJECT 34: notify __SENDER__
Found RULE 2: Remove Viruses
  FOUND WHAT GROUP 9: Virus
    OBJECT 22: active
  FOUND ACTION GROUP 15: Remove attachments
    OBJECT 28: remove matching attachments
  FOUND ACTION GROUP 20: Notify Admin
    OBJECT 33: notify __ADMIN__
  FOUND ACTION GROUP 23: Modify Virus Subject
    OBJECT 37: modify field: subject:VIRUS: __SUBJECT__
Found RULE 1: Remove Dangerous Files
  FOUND WHAT GROUP 8: Dangerous Content
    OBJECT 16: content-type=application/javascript
    OBJECT 17: content-type=application/x-executable
    OBJECT 15: content-type=application/x-java
    OBJECT 14: content-type=application/x-ms-dos-executable
    OBJECT 18: content-type=application/x-ms-dos-executable
    OBJECT 19: content-type=message/partial
    OBJECT 20: filename=.*\.(vbs|pif|lnk|shs|shb)
    OBJECT 21: filename=.*\.\{.+\}
  FOUND ACTION GROUP 15: Remove attachments
    OBJECT 28: remove matching attachments
  FOUND ACTION GROUP 20: Notify Admin
    OBJECT 33: notify __ADMIN__
  FOUND ACTION GROUP 24: Modify Dangerous Subject
    OBJECT 38: modify field: subject:DANGEROUS: __SUBJECT__
Found RULE 5: Modify Header
  FOUND ACTION GROUP 13: Modify Spam Level
    OBJECT 26: modify field: X-SPAM-LEVEL:__SPAM_INFO__
Found RULE 12: Block Multimedia Files
  FOUND WHAT GROUP 6: Multimedia
    OBJECT 5: content-type=audio/.*
    OBJECT 6: content-type=video/.*
  FOUND ACTION GROUP 15: Remove attachments
    OBJECT 28: remove matching attachments
Found RULE 6: Whitelist
  FOUND FROM GROUP 3: Whitelist
    OBJECT 2: mail@fromthisdomain.com
  FOUND ACTION GROUP 17: Accept
    OBJECT 30: accept message
Found RULE 9: Block Spam (Level 10)
  FOUND WHAT GROUP 12: Spam (Level 10)
    OBJECT 25: Level 10
  FOUND ACTION GROUP 18: Block
    OBJECT 31: block message
Found RULE 8: Quarantine/Mark Spam (Level 5)
  FOUND WHAT GROUP 11: Spam (Level 5)
    OBJECT 24: Level 5
  FOUND ACTION GROUP 14: Modify Spam Subject
    OBJECT 27: modify field: subject:SPAM: __SUBJECT__
  FOUND ACTION GROUP 19: Quarantine
    OBJECT 32: Move to quarantine.
Found RULE 7: Mark Spam (Level 6)
  FOUND WHAT GROUP 10: Spam (Level 6)
    OBJECT 23: Level 6
  FOUND ACTION GROUP 14: Modify Spam Subject
    OBJECT 27: modify field: subject:SPAM: __SUBJECT__
Found RULE 10: Block outgoing Spam
  FOUND WHAT GROUP 10: Spam (Level 6)
    OBJECT 23: Level 6
  FOUND ACTION GROUP 18: Block
    OBJECT 31: block message
  FOUND ACTION GROUP 20: Notify Admin
    OBJECT 33: notify __ADMIN__
  FOUND ACTION GROUP 21: Notify Sender
    OBJECT 34: notify __SENDER__
Found RULE 11: Add Disclaimer
  FOUND ACTION GROUP 22: Disclaimer
    OBJECT 35: disclaimer

Or alternative:

pmgconfig dump (loser setup):

composed.wl_bounce_relays = xxx
dns.domain = xxx
dns.hostname = xxx
ipconfig.int_ip = xxx
pmg.admin.advfilter = 0
pmg.admin.avast = 0
pmg.admin.clamav = 1
pmg.admin.dailyreport = 1
pmg.admin.demo = 0
pmg.admin.email = xxx
pmg.admin.http_proxy =
pmg.admin.statlifetime = 7
pmg.clamav.archiveblockencrypted = 0
pmg.clamav.archivemaxfiles = 1000
pmg.clamav.archivemaxrec = 5
pmg.clamav.archivemaxsize = 25000000
pmg.clamav.dbmirror = database.clamav.net
pmg.clamav.maxcccount = 0
pmg.clamav.maxscansize = 100000000
pmg.clamav.safebrowsing = 1
pmg.mail.banner = ESMTP Proxmox
pmg.mail.conn_count_limit = 50
pmg.mail.conn_rate_limit = 0
pmg.mail.dnsbl_sites = zzen.spamhaus.org*2,bl.spamcop.net*2,psbl.surriel.com*2,spamrbl.imp.ch*2,noptr.spamrats.com*2,escalations.dnsbl.sorbs.net*2,bl.score.senderscore.com*2,bl.spameatingmonkey.net*2,rbl.realtimeblacklist.com*2,dnsbl.dronebl.org*2,ix.dnsbl.manitu.net,b.barracudacentral.org,truncate.gbudb.net,bl.blocklist.de,xxx,xxx24
pmg.mail.dwarning = 4
pmg.mail.ext_port = 25
pmg.mail.greylist = 0
pmg.mail.helotests = 1
pmg.mail.hide_received = 0
pmg.mail.int_port = 26
pmg.mail.max_filters = 30
pmg.mail.max_policy = 5
pmg.mail.max_smtpd_in = 100
pmg.mail.max_smtpd_out = 100
pmg.mail.maxsize = 104857600
pmg.mail.message_rate_limit = 0
pmg.mail.rejectunknown = 1
pmg.mail.rejectunknownsender = 1
pmg.mail.relay = xxx
pmg.mail.relaynomx = 0
pmg.mail.relayport = 25
pmg.mail.smarthost =
pmg.mail.spf = 0
pmg.mail.tls = 1
pmg.mail.tlsheader = 1
pmg.mail.tlslog = 1
pmg.mail.verifyreceivers = 550
pmg.spam.bounce_score = 0
pmg.spam.clamav_heuristic_score = 3
pmg.spam.languages = all
pmg.spam.maxspamsize = 262144
pmg.spam.rbl_checks = 1
pmg.spam.use_awl = 1
pmg.spam.use_bayes = 1
pmg.spam.use_razor = 1
pmg.spam.wl_bounce_relays =
pmg.spamquar.allowhrefs = 1
pmg.spamquar.authmode = ticket
pmg.spamquar.hostname =
pmg.spamquar.lifetime = 7
pmg.spamquar.mailfrom =
pmg.spamquar.port = 8006
pmg.spamquar.protocol = https
pmg.spamquar.reportstyle = verbose
pmg.spamquar.viewimages = 1
pmg.virusquar.allowhrefs = 1
pmg.virusquar.lifetime = 7
pmg.virusquar.viewimages = 1
postfix.dnsbl_sites =
zen.spamhaus.org*2,bl.spamcop.net*2,psbl.surriel.com*2,spamrbl.imp.ch*2,noptr.spamrats.com*2,escalations.dnsbl.sorbs.net*2,bl.score.senderscore.com*2,bl.spameatingmonkey.net*2,rbl.realtimeblacklist.com*2,dnsbl.dronebl.org*2,ix.dnsbl.manitu.net,b.barracudacentral.org,truncate.gbudb.net,bl.blocklist.de,xxx,xxx24
postfix.int_ip = xxx
postfix.mynetworks = 127.0.0.0/8 [::1]/128 xxx
postfix.transportnets =
postfix.usepolicy = 0

pmgdb dump (loser setup):

Found RULE 4: Blacklist
  FOUND FROM GROUP 2: Blacklist
    OBJECT 1: nomail@fromthisdomain.com
  FOUND ACTION GROUP 18: Block
    OBJECT 31: block message
Found RULE 3: Virus Alert
  FOUND WHAT GROUP 9: Virus
    OBJECT 22: active
  FOUND ACTION GROUP 18: Block
    OBJECT 31: block message
  FOUND ACTION GROUP 20: Notify Admin
    OBJECT 33: notify __ADMIN__
  FOUND ACTION GROUP 21: Notify Sender
    OBJECT 34: notify __SENDER__
Found RULE 2: Remove Viruses
  FOUND WHAT GROUP 9: Virus
    OBJECT 22: active
  FOUND ACTION GROUP 15: Remove attachments
    OBJECT 28: remove matching attachments
  FOUND ACTION GROUP 20: Notify Admin
    OBJECT 33: notify __ADMIN__
  FOUND ACTION GROUP 23: Modify Virus Subject
    OBJECT 37: modify field: subject:VIRUS: __SUBJECT__
Found RULE 1: Remove Dangerous Files
  FOUND WHAT GROUP 8: Dangerous Content
    OBJECT 16: content-type=application/javascript
    OBJECT 17: content-type=application/x-executable
    OBJECT 15: content-type=application/x-java
    OBJECT 14: content-type=application/x-ms-dos-executable
    OBJECT 18: content-type=application/x-ms-dos-executable
    OBJECT 19: content-type=message/partial
    OBJECT 20: filename=.*\.(vbs|pif|lnk|shs|shb)
    OBJECT 21: filename=.*\.\{.+\}
  FOUND ACTION GROUP 15: Remove attachments
    OBJECT 28: remove matching attachments
  FOUND ACTION GROUP 20: Notify Admin
    OBJECT 33: notify __ADMIN__
  FOUND ACTION GROUP 24: Modify Dangerous Subject
    OBJECT 38: modify field: subject:DANGEROUS: __SUBJECT__
Found RULE 5: Modify Header
  FOUND ACTION GROUP 13: Modify Spam Level
    OBJECT 26: modify field: X-SPAM-LEVEL:__SPAM_INFO__
Found RULE 12: Block Multimedia Files
  FOUND WHAT GROUP 6: Multimedia
    OBJECT 5: content-type=audio/.*
    OBJECT 6: content-type=video/.*
  FOUND ACTION GROUP 15: Remove attachments
    OBJECT 28: remove matching attachments
Found RULE 6: Whitelist
  FOUND FROM GROUP 3: Whitelist
    OBJECT 2: mail@fromthisdomain.com
  FOUND ACTION GROUP 17: Accept
    OBJECT 30: accept message
Found RULE 9: Block Spam (Level 10)
  FOUND WHAT GROUP 12: Spam (Level 10)
    OBJECT 25: Level 10
  FOUND ACTION GROUP 18: Block
    OBJECT 31: block message
Found RULE 8: Quarantine/Mark Spam (Level 5)
  FOUND WHAT GROUP 11: Spam (Level 5)
    OBJECT 24: Level 5
  FOUND ACTION GROUP 14: Modify Spam Subject
    OBJECT 27: modify field: subject:SPAM: __SUBJECT__
  FOUND ACTION GROUP 19: Quarantine
    OBJECT 32: Move to quarantine.
Found RULE 7: Mark Spam (Level 6)
  FOUND WHAT GROUP 10: Spam (Level 6)
    OBJECT 23: Level 6
  FOUND ACTION GROUP 14: Modify Spam Subject
    OBJECT 27: modify field: subject:SPAM: __SUBJECT__
Found RULE 10: Block outgoing Spam
  FOUND WHAT GROUP 10: Spam (Level 6)
    OBJECT 23: Level 6
  FOUND ACTION GROUP 18: Block
    OBJECT 31: block message
  FOUND ACTION GROUP 20: Notify Admin
    OBJECT 33: notify __ADMIN__
  FOUND ACTION GROUP 21: Notify Sender
    OBJECT 34: notify __SENDER__
Found RULE 11: Add Disclaimer
  FOUND ACTION GROUP 22: Disclaimer
    OBJECT 35: disclaimer

Missing in the dumps is the whitelisting of the network 194.25.134.0/24 as the German Telecom is often listed on usually trusted blacklists.

 

[heutger]

Loser vs. tighter setup has the following differences:

SPF disabled as many server admins are not managing their SPF records correct or may have set them once and then forgot about them (had the same with our commercial outgoing secure mail gateway were we now - regarding GDPR - encrypt all mails, if there are keys available. The gateway tries to be "intelligent" and fetches keys from ldap and keyserver URLs, which are well known, result in many complaints of users, which recently played around with PGP/GPG, uploaded keys to the keyservers and now get encrypted mails, they can't read).

 

[heutger]

Here comes something new! Bayes is the major weapon against Spam at content filter stage. But it needs to be trained (autolearn is fine, however it takes a long time to start and it's still an auto-filter with all its advantages and disadvantages, so that's why it takes so long to be usable, SpamAssassin is very conservative in learning spam and it needs a good ratio of ham and spam to start tagging).

I have a Plesk Server behind PMG and I use Roundcube there to access my mail copy. I now used this mail copy to create a folder called Ham and sorted spam to Spam and ham to Ham. So spam are all the mails tagged with SPAM: which are really spam as well as spam, which got not recognized as spam (really rare, so no hard work, but it's most important as this should be recognized in future, confirming spam is also fine, but this spam brings much more quality to the filter), also maybe VIRUS: and DANGEROUS: and all ham, which has been tagged with SPAM: (again it's very important to learn this as ham to improve the filter). After that, I created SCP access to the machine containing the folders and a script to download the spam and ham folders, learn them as spam or ham and then remove the folders again.

The steps done are below and the script as well:

ssh-keygen -t rsa -b 4096
cat .ssh/id_rsa.pub
vi /root/scripts/cs-learn.sh
chmod +x /root/scripts/cs-learn.sh
/root/scripts/cs-learn.sh

/root/scripts/cs-learn.sh (for sure, again you need to give your details at xxx):

#!/bin/bash
HOST='xxx'
USER='xxx'
DOMAIN='xxx'
ADDRESS='xxx'

echo Fetching Spam [1/5] ...
scp -q -r $USER@$HOST:/var/qmail/mailnames/$DOMAIN/$ADDRESS/Maildir/.Spam .

echo Fetching Ham [2/5] ...
scp -q -r $USER@$HOST:/var/qmail/mailnames/$DOMAIN/$ADDRESS/Maildir/.Ham .

echo Learning Spam [3/5] ...
sa-learn --spam .Spam/cur/* .Spam/new/*

echo Learning Ham [4/5] ...
sa-learn --ham .Ham/cur/* .Ham/new/*

echo Cleanup [5/5] ...
rm -Rf .Spam
rm -Rf .Ham

 

[heutger]

Loser vs. tighter setup enhanced:

I already corrected above, that I switched from bind9 to unbound, which solved all the 450 tempfail on sender address and client host checking as well as the 550 false-positives. I'm quite unsure, why bind9 is working such bad, but it's solved now, so I won't think about it anymore.

I also lowered score from 6 to 5 now on both test installations (loser as well as tighter), to not confuse with the shipped Level 5 action, I won't adjust my dumps above, feel free to use them as 6, adjust them to 5 or use the 5 action instead and adjust it to your need.

Most important change, I also won't change above as it would result in an extra main.cf.in, would confuse thread readers and maybe others won't like to change it that way: I decided to change in the loser configuration the really "hard" FCrDNS check (as many companies are not yet ready to fulfill) reject_unknown_client_hostname (which is enabled via GUI) to reject_unknown_reverse_client_hostname. So what it does is not checking any more if client IP address resolve to a hostname and this hostname resolve to an IP and the IP it resolve to is the same as the client IP given, but it now just checks if client IP address resolve to a hostname, that's it. That's a requirement, which could be expected in the late 2010s, but I see currently FCrDNS not available by every system. It's a good compromise between disabling the option at all (which really kick out a good amount of spammers) versus rejecting some not very well administered servers of (potential) customers.

 

[heutger]

Once more something new! As update mails are only sent with subscription, I decide to add update mails also to testing as well as adding auto-update. Everyone can decide by himself, if auto-update is something desirable, but for my private(!) environment, I prefer auto-update, for sure, I wouldn't recommend for a commercial environment, there should be a subscription, no testing repositories and updates should be performed with existing patch management processes or procedures, I strongly advice against automatic updates in productive environments, it's a security risk. However, you can use the settings below to get more information by mail and/or let the system prefetch packages for getting updated, perform regular autoremove or autoclean, however last two options I also would never recommend in productive environments.

Steps performed:

apt-get install apticron
apt-get install unattended-upgrades
vi /etc/apticron/apticron.conf
apt-cache policy
vi /etc/apt/apt.conf.d/50unattended-upgrades
rm /etc/apt/apt.conf.d/20auto-upgrades
vi /etc/apt/apt.conf.d/02periodic
vi /etc/apt/listchanges.conf
unattended-upgrade -d

/etc/apticron/apticron.conf (xxx again is your address and domain):

# apticron.conf
#
# set EMAIL to a space separated list of addresses which will be notified of
# impending updates
#
EMAIL="root"

#
# Set DIFF_ONLY to "1" to only output the difference of the current run
# compared to the last run (ie. only new upgrades since the last run). If there
# are no differences, no output/email will be generated. By default, apticron
# will output everything that needs to be upgraded.
#
# DIFF_ONLY="1"

#
# Set LISTCHANGES_PROFILE if you would like apticron to invoke apt-listchanges
# with the --profile option. You should add a corresponding profile to
# /etc/apt/listchanges.conf
#
# LISTCHANGES_PROFILE="apticron"

#
# From hostname manpage: "Displays  all FQDNs of the machine. This option
# enumerates all configured network addresses on all configured network inter‐
# faces, and translates them to DNS domain names. Addresses that cannot be
# translated (i.e. because they do not have an appro‐ priate  reverse DNS
# entry) are skipped. Note that different addresses may resolve to the same
# name, therefore the output may contain duplicate entries. Do not make any
# assumptions about the order of the output."
#
# ALL_FQDNS="1"

#
# Set SYSTEM if you would like apticron to use something other than the output
# of "hostname -f" for the system name in the mails it generates. This option
# overrides the ALL_FQDNS above.
#
# SYSTEM="foobar.example.com"

#
# Set IPADDRESSNUM if you would like to configure the maximal number of IP
# addresses apticron displays. The default is to display 1 address of each
# family type (inet, inet6), if available.
#
# IPADDRESSNUM="1"

#
# Set IPADDRESSES to a whitespace separated list of reachable addresses for
# this system. By default, apticron will try to work these out using the
# "ip" command
#
# IPADDRESSES="192.0.2.1 2001:db8:1:2:3::1"

#
# Set NOTIFY_HOLDS="0" if you don't want to be notified about new versions of
# packages on hold in your system. The default behavior is downloading and
# listing them as any other package.
#
# NOTIFY_HOLDS="0"

#
# Set NOTIFY_NEW="0" if you don't want to be notified about packages which
# are not installed in your system. Yes, it's possible! There are some issues
# related to systems which have mixed stable/unstable sources. In these cases
# apt-get will consider for example that packages with "Priority:
# required"/"Essential: yes" in unstable but not in stable should be installed,
# so they will be listed in dist-upgrade output. Please take a look at
# http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=531002#44
#
# NOTIFY_NEW="0"

#
# Set NOTIFY_NO_UPDATES="0" if you don't want to be notified when there is no
# new versions. Set to 1 could assure you that apticron works well.
#
# NOTIFY_NO_UPDATES="0"

#
# Set CUSTOM_SUBJECT if you want to replace the default subject used in
# the notification e-mails. This may help filtering/sorting client-side e-mail.
# If you want to use internal vars please use single quotes here. Ex:
# $CUSTOM_SUBJECT='[apticron] $SYSTEM: $NUM_PACKAGES package update(s)'
#
# CUSTOM_SUBJECT=""

# Set CUSTOM_NO_UPDATES_SUBJECT if you want to replace the default subject used
# in the no update notification e-mails. This may help filtering/sorting
# client-side e-mail.
# If you want to use internal vars please use single quotes here. Ex:
# $CUSTOM_NO_UPDATES_SUBJECT='[apticron] $SYSTEM: no updates'
#
# CUSTOM_NO_UPDATES_SUBJECT=""

#
# Set CUSTOM_FROM if you want to replace the default sender by changing the
# 'From:' field used in the notification e-mails. Your default sender will
# be something like root@xxx.
#
CUSTOM_FROM="xxx@xxx"

/etc/apt/apt.conf.d/50unattended-upgrades:

// Unattended-Upgrade::Origins-Pattern controls which packages are
// upgraded.
//
// Lines below have the format format is "keyword=value,...".  A
// package will be upgraded only if the values in its metadata match
// all the supplied keywords in a line.  (In other words, omitted
// keywords are wild cards.) The keywords originate from the Release
// file, but several aliases are accepted.  The accepted keywords are:
//   a,archive,suite (eg, "stable")
//   c,component     (eg, "main", "contrib", "non-free")
//   l,label         (eg, "Debian", "Debian-Security")
//   o,origin        (eg, "Debian", "Unofficial Multimedia Packages")
//   n,codename      (eg, "jessie", "jessie-updates")
//     site          (eg, "http.debian.net")
// The available values on the system are printed by the command
// "apt-cache policy", and can be debugged by running
// "unattended-upgrades -d" and looking at the log file.
//
// Within lines unattended-upgrades allows 2 macros whose values are
// derived from /etc/debian_version:
//   ${distro_id}            Installed origin.
//   ${distro_codename}      Installed codename (eg, "jessie")
Unattended-Upgrade::Origins-Pattern {
        // Codename based matching:
        // This will follow the migration of a release through different
        // archives (e.g. from testing to stable and later oldstable).
//      "o=Debian,n=jessie";
//      "o=Debian,n=jessie-updates";
//      "o=Debian,n=jessie-proposed-updates";
//      "o=Debian,n=jessie,l=Debian-Security";

        // Archive or Suite based matching:
        // Note that this will silently match a different release after
        // migration to the specified archive (e.g. testing becomes the
        // new stable).
//      "o=Debian,a=stable";
//      "o=Debian,a=stable-updates";
//      "o=Debian,a=proposed-updates";
//      "origin=Debian,codename=${distro_codename},label=Debian-Security";

        "o=Debian,a=stable,n=stretch";
        "o=Debian,a=stable-updates,n=stretch-updates";
        "o=Debian,a=stable,n=stretch,l=Debian-Security";
        "o=Proxmox,a=stable,n=stretch";
};

// List of packages to not update (regexp are supported)
Unattended-Upgrade::Package-Blacklist {
//    "vim";
//    "libc6";
//    "libc6-dev";
//    "libc6-i686";
};

// This option allows you to control if on a unclean dpkg exit
// unattended-upgrades will automatically run
//   dpkg --force-confold --configure -a
// The default is true, to ensure updates keep getting installed
Unattended-Upgrade::AutoFixInterruptedDpkg "false";

// Split the upgrade into the smallest possible chunks so that
// they can be interrupted with SIGUSR1. This makes the upgrade
// a bit slower but it has the benefit that shutdown while a upgrade
// is running is possible (with a small delay)
//Unattended-Upgrade::MinimalSteps "true";

// Install all unattended-upgrades when the machine is shuting down
// instead of doing it in the background while the machine is running
// This will (obviously) make shutdown slower
//Unattended-Upgrade::InstallOnShutdown "true";

// Send email to this address for problems or packages upgrades
// If empty or unset then no email is sent, make sure that you
// have a working mail setup on your system. A package that provides
// 'mailx' must be installed. E.g. "user@example.com"
Unattended-Upgrade::Mail "root";

// Set this value to "true" to get emails only on errors. Default
// is to always send a mail if Unattended-Upgrade::Mail is set
//Unattended-Upgrade::MailOnlyOnError "true";

// Do automatic removal of new unused dependencies after the upgrade
// (equivalent to apt-get autoremove)
Unattended-Upgrade::Remove-Unused-Dependencies "false";

// Automatically reboot *WITHOUT CONFIRMATION* if
//  the file /var/run/reboot-required is found after the upgrade
Unattended-Upgrade::Automatic-Reboot "false";

// Automatically reboot even if there are users currently logged in.
//Unattended-Upgrade::Automatic-Reboot-WithUsers "true";

// If automatic reboot is enabled and needed, reboot at the specific
// time instead of immediately
//  Default: "now"
//Unattended-Upgrade::Automatic-Reboot-Time "02:00";

// Use apt bandwidth limit feature, this example limits the download
// speed to 70kb/sec
//Acquire::http::Dl-Limit "70";

// Enable logging to syslog. Default is False
// Unattended-Upgrade::SyslogEnable "false";

// Specify syslog facility. Default is daemon
// Unattended-Upgrade::SyslogFacility "daemon";

/etc/apt/apt.conf.d/02periodic:

#  Dir "/";
#  - RootDir for all configuration files
#
#  Dir::Cache "var/cache/apt/";
#  - Set apt package cache directory
#
#  Dir::Cache::Archives "archives/";
#  - Set package archive directory
#
APT::Periodic::Enable "1";
#  - Enable the update/upgrade script (0=disable)
#
#  APT::Periodic::BackupArchiveInterval "0";
#  - Backup after n-days if archive contents changed.(0=disable)
#
#  APT::Periodic::BackupLevel "3";
#  - Backup level.(0=disable), 1 is invalid.
#
#  Dir::Cache::Backup "backup/";
#  - Set periodic package backup directory
#
#  APT::Archives::MaxAge "0"; (old, deprecated)
#  APT::Periodic::MaxAge "0"; (new)
#  - Set maximum allowed age of a cache package file. If a cache
#    package file is older it is deleted (0=disable)
#
#  APT::Archives::MinAge "2"; (old, deprecated)
#  APT::Periodic::MinAge "2"; (new)
#  - Set minimum age of a package file. If a file is younger it
#    will not be deleted (0=disable). Useful to prevent races
#    and to keep backups of the packages for emergency.
#
#  APT::Archives::MaxSize "0"; (old, deprecated)
#  APT::Periodic::MaxSize "0"; (new)
#  - Set maximum size of the cache in MB (0=disable). If the cache
#    is bigger, cached package files are deleted until the size
#    requirement is met (the oldest packages will be deleted
#    first).
#
APT::Periodic::Update-Package-Lists "1";
#  - Do "apt-get update" automatically every n-days (0=disable)
#   
APT::Periodic::Download-Upgradeable-Packages "1";
#  - Do "apt-get upgrade --download-only" every n-days (0=disable)
#
#  APT::Periodic::Download-Upgradeable-Packages-Debdelta "1";
#  - Use debdelta-upgrade to download updates if available (0=disable)
#
APT::Periodic::Unattended-Upgrade "1";
#  - Run the "unattended-upgrade" security upgrade script
#    every n-days (0=disabled)
#    Requires the package "unattended-upgrades" and will write
#    a log in /var/log/unattended-upgrades
#
#  APT::Periodic::AutocleanInterval "0";
#  - Do "apt-get autoclean" every n-days (0=disable)
#
APT::Periodic::Verbose "1";
#  - Send report mail to root
#      0:  no report             (or null string)
#      1:  progress report       (actually any string)
#      2:  + command outputs     (remove -qq, remove 2>/dev/null, add -d)
#      3:  + trace on

/etc/apt/listchanges.conf:

[apt]
frontend=pager
confirm=false
email_address=root
save_seen=/var/lib/apt/listchanges.db
which=both

 

[heutger]

Another update is coming in! I realized, that there are five kinds of mails: Junk Mail, which is already blocked by postscreen and/or DNSRBL blacklists; Bounce Mail, which is not deliverable, Clean Mail, which has usually scores around 0 (hopefully most) up to 4 (some rare issues), Usual Spam, which has score around 5 to 9, including some False-Positives, Newsletters etc. and really Sure Stupid Spam, which has score from 10 and much more. This stupid spam takes a good value of mail volume against the clean mails meanwhile the usual spam is ignorable. Great, if it's possible to reject such spam, PMG is only able to block this one, but it's like firewall rules, reject means refuse meanwhile block means drop. In Germany last one is illegal as it suppress mails, that's not allowed by law. So I added another layer of spam checking with the possibility to reject. The current drawback, I try to find a solution for (help is welcome): The tracking center doesn't show such mails as well as they won't be included in the statistics. Reason is, that the rejects, which are handled by PMG look for reject: NOQUEUE, but this rejects are milter-rejects: END-OF-MESSAGE. I found the source, where this is handled, but I can't code, so I can't add this mails to handling.

Steps performed:

apt-get --no-install-recommends install spamass-milter
mkdir /var/lib/spamass-milter
vi /etc/default/spamassassin
vi /lib/systemd/system/spamassassin.service
systemctl enable spamassassin
systemctl start spamassassin
vi /etc/default/spamass-milter
systemctl restart spamass-milter
vi /etc/pmg/templates/main.cf.in
vi /etc/cron.hourly/sa-update
vi /etc/cron.daily/spamass-milter
chmod +x /etc/cron.daily/spamass-milter
/etc/cron.daily/spamass-milter
chown -R spamass-milter:spamass-milter /var/lib/spamass-milter
pmgconfig sync --restart 1

/etc/default/spamassassin (this file is usually included with SpamAssassin installation from the Debian sources, but as the SpamAssassin package is installed and adjusted by PMG, the file is not available. The following file is the original one from the SpamAssassin package with adjusted options):

# /etc/default/spamassassin
# Duncan Findlay

# WARNING: please read README.spamd before using.
# There may be security risks.

# If you're using systemd (default for jessie), the ENABLED setting is
# not used. Instead, enable spamd by issuing:
# systemctl enable spamassassin.service
# Change to "1" to enable spamd on systems using sysvinit:
ENABLED=0

# Options
# See man spamd for possible options. The -d option is automatically added.

# SpamAssassin uses a preforking model, so be careful! You need to
# make sure --max-children is not set to anything higher than 5,
# unless you know what you're doing.

OPTIONS="--username spamass-milter --nouser-config --max-children 5 --helper-home-dir"

# Pid file
# Where should spamd write its PID to file? If you use the -u or
# --username option above, this needs to be writable by that user.
# Otherwise, the init script will not be able to shut spamd down.
PIDFILE="/var/run/spamd.pid"

# Set nice level of spamd
#NICE="--nicelevel 15"

# Cronjob
# Set to anything but 0 to enable the cron job to automatically update
# spamassassin's rules on a nightly basis
CRON=0

/lib/systemd/system/spamassassin.service (the same as above applies for this one; as PMG provides its own SpamAssassin package, this package also includes spamd, but at a different path, so file has been adjusted not to use /usr/sbin/spamd but /usr/bin/spamd, which differs from the original by using /etc/mail/spamassassin instead of /etc/spamassassin as well as some SSL socket adjustments, which - I believe - can be ignored for local installations):

[Unit]
Description=Perl-based spam filter using text analysis
After=syslog.target network.target

[Service]
Type=forking
PIDFile=/var/run/spamd.pid
EnvironmentFile=-/etc/default/spamassassin
ExecStart=/usr/bin/spamd -d --pidfile=/var/run/spamd.pid $OPTIONS
ExecReload=/bin/kill -HUP $MAINPID
StandardOutput=null
StandardError=null
Restart=always

[Install]
WantedBy=multi-user.target

/etc/default/spamass-milter (that's the "magic" of this setup and I would prefer PMG would integrate SpamAssassin that way by default, so this workaround is not required as it for sure takes additional performance for extra scan and some more RAM and a few bytes more harddisk and it's not integrated in tracking center or statistics, but currently it's the only way. I also tried to call pmg-smtp-filter here, but it did not work. The options required are the last two ones, first settings is to reject mails with a score above 10, so the high level spam, second is to hide all the work done by spamass-milter as otherwise the second SpamAssassin run will result in different scores):

# spamass-milt startup defaults

# OPTIONS are passed directly to spamass-milter.
# man spamass-milter for details

# Non-standard configuration notes:
# See README.Debian if you use the -x option with sendmail
# You should not pass the -d option in OPTIONS; use SOCKET for that.

# Default, use the spamass-milter user as the default user, ignore
# messages from localhost
OPTIONS="-u spamass-milter -i 127.0.0.1"

# Reject emails with spamassassin scores > 15.
OPTIONS="${OPTIONS} -r 10"

# Do not modify Subject:, Content-Type: or body.
OPTIONS="${OPTIONS} -m"

######################################
# If /usr/sbin/postfix is executable, the following are set by
# default. You can override them by uncommenting and changing them
# here.
######################################
# SOCKET="/var/spool/postfix/spamass/spamass.sock"
# SOCKETOWNER="postfix:postfix"
# SOCKETMODE="0660"
######################################

Then it needs to be integrated in Postfix, therefor the following lines have been added to the end of /etc/pmg/templates/main.cf.in:

smtpd_milters = unix:/var/spool/postfix/spamass/spamass.sock
milter_connect_macros = j {daemon_name} v {if_name} _
milter_default_action = accept

I don't post here my new main.cf.in as it currently includes additional DNSRBL blacklists, I test. I will post here, once I've done with my tests.

/etc/cron.hourly/sa-update (script needs to be adjusted to reload spamassassin on new rulesets too):

#!/bin/sh

# schaal @it
#
# Simple script to update SpamAssassin

SYSLOG_TAG=sa-update

compile=0

logger -d -t $SYSLOG_TAG "Start SA-Update"

sa-update --nogpg
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi


sa-update --nogpg --channel updates.spamassassin.org
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi

sa-update --nogpg --channel sa.zmi.at
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi

sa-update --nogpg --channel sa.schaal-it.net
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi

sa-update --nogpg --channel spamassassin.heinlein-support.de
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi

if [ $compile -eq 1 ]; then
    logger -d -t $SYSLOG_TAG "SA-Update found"
    sa-compile --quiet 2>/dev/null
    systemctl restart pmg-smtp-filter
    systemctl restart spamassassin
else
    logger -d -t $SYSLOG_TAG "No SA-Update found"
fi

/etc/cron.daily/spamass-milter (AWL, Bayes etc. from SpamAssassin as been run by pmg-smtp-filter needs to be synced with SpamAssassin been run by spamass-milter, especially as manual sa-learn calls will be performed against SpamAassassin as been run by pmg-smtp-filter, it's required to run a daily job to copy the files from /root/.spamassassin to /var/lib/spamass-milter/.spamassasin):

#!/bin/sh

cp -R /root/.spamassassin/* /var/lib/spamass-milter/.spamassassin/.

 

[killmasta93]

really good stuff i still have not tried the Bayes, The dcc,pyzor and unoficial Clam Av amazing work. and the only thing about Bayes is that spam that comes into the company rather then your inbox somewhat complicated to manage

 

[Sommer]

Really nice job @heutger
I will wait for @proxmox team to implement all your optimizations and release it on the next version

 

[heutger]

really good stuff i still have not tried the Bayes, The dcc,pyzor and unoficial Clam Av amazing work. and the only thing about Bayes is that spam that comes into the company rather then your inbox somewhat complicated to manage

Thanks. I believe, bayes will have much impact on the quality of the spam filter. For sure, it's much harder to manage, if the mail server itself is not the gateway learning spam or ham. There are great explanations on how to use like https://thomas-leister.de/mailserver-unter-ubuntu-16.04/ (however, with new setup they use rspamd, however I'm currently not too much fan of rspamd anymore), which are really great invoking sa-learn on movements from spam to ham or vice versa. However, I have my cs-learn script above to learn in my private setup with a Plesk server behind (you may have wondered, why cs-learn, it's standing for cloud server as my private server has the name cs for cloud server), but I currently work for the company on a solution on how to learn spam from Exchange if team should forward spam mails as attachment to a special mailbox (and maybe the same with ham) looking at https://kuther.net/howtos/howto-receive-mail-and-save-attachment-fetchmail-procmail-and-metamail and https://dump.4network.org/2015/06/1...achment-with-fetchmail-procmail-and-uudeview/ how that's working, but need to check, if I can use it with two different accounts, one for spam and one for ham. In addition I'm looking for a new ticketing system and also will then try to have a button invoking an API call or sth. similar to learn spam or ham from tickets to PMG, then it would be great and flying.

 

[heutger]

Really nice job @heutger
I will wait for @proxmox team to implement all your optimizations and release it on the next version

Thanks! Would be great, however, if I could provide a wishlist (and also provided a bit like that), first of all I would be very happy, if PMG would changed from SpamAssassin content_filter integration to milter as this would make things much easier. Firewall and OpenVPN would only be required for outside of a DMZ placed systems, which are not recommended, my private one is and the test as well, however, productive setup will be inside, backup could be improved, but additional backup script is not required if inside a DMZ on a virtual host and the node performs a valid backup, but all the options of rulesets, blacklists, signatures etc. could be optional choosable and adjustable, some like DCC could be shipped with an easy setup script (as DCC is not allowed to be shipped without license) or maybe include some licenses (with special discounted agreements with the vendors) with subscriptions to have more features instead of "only" productive repository and support options.

However, feel free to use my adjustments and also Proxmox can use them as they like.

My new mission currently is to get bayes working (by getting enough spam and ham into the database) and to provide additional ways to feed sa-learn. I currently test some last blacklists and will also recheck heinlein, schaal and zmi rulesets as recently they had less impact on spam scores for spam mails but much impact on spam score for ham false-positives. I will keep you posted.

 

[heutger]

Urgent: ClamAV-Update 0.100.0 with recent updates (e.g. Debian 9.5 update without subscription) doesn't ignore broken/wrong rules any more, but clamav-daemon then dies. So it's important to temporary apply the workaround https://github.com/extremeshok/clamav-unofficial-sigs/issues/203#issuecomment-400211109 until clamav-unofficial-sigs gets updated. Currently it seems not to be updated any more, there are many open issues.

 

[killmasta93]

Urgent: ClamAV-Update 0.100.0 with recent updates (e.g. Debian 9.5 update without subscription) doesn't ignore broken/wrong rules any more, but clamav-daemon then dies. So it's important to temporary apply the workaround https://github.com/extremeshok/clamav-unofficial-sigs/issues/203#issuecomment-400211109 until clamav-unofficial-sigs gets updated. Currently it seems not to be updated any more, there are many open issues.

So after running your guide i would have to configure this?

Set in /etc/clamav-unofficial-sigs/master.conf
yararulesproject_enabled="no"
enable_yararules="no"
And delete *.yar and *.yara from /var/lib/clamav/

 

[killmasta93]

Yippie, Bayes started and it's ... great!

as for the Bayes on has to put email that comes in for all of the company to a certain folder which updates though IMAP? Thats what i had before on scrolloutf1 the only issue if that it was very manually

 

[heutger]

So after running your guide i would have to configure this?

Set in /etc/clamav-unofficial-sigs/master.conf
yararulesproject_enabled="no"
enable_yararules="no"
And delete *.yar and *.yara from /var/lib/clamav/

That's right. Alternative it *should* also work to apply https://github.com/extremeshok/clamav-unofficial-sigs/pull/193 but you may encounter https://github.com/extremeshok/clamav-unofficial-sigs/issues/204, so I currently prefer to disable yara at all. That's also why I prefer to use the script, there is still some support, the sanesecurity website looks a bit of unsupported. I would prefer, if it would be easier to change/add antivirus scanner to PMG and be much more open in decision, Sophos or Avira would be my preferred candidates. Sophos we already use wide spread (however, it would be great to have another one, as Sophos is already running as endpoint protection, so there would be no advantage to scan in the mail flow), so Avira would be great as well.

 

[heutger]

as for the Bayes on has to put email that comes in for all of the company to a certain folder which updates though IMAP? Thats what i had before on scrolloutf1 the only issue if that it was very manually

I already provided a script above for Plesk, where Mails are sorted into Spam or Ham for one mailbox (could be improved for more mailboxes) and can be used to feed bayes database. However, all the users then are required to use any webmail provided by Plesk or IMAP access to the mailboxes as been provided by Plesk and always use Spam folder for confirmed Spam or (most interesting and important) undetected Spam (so false-negatives) as well as Ham folder for false-positives. It's also possible to configure Plesk to run SpamAssassin by itself and write RegEx to automatically move Spam to Spam (however, also false-positives will then be sorted there). It's also possible to use and adjust the setup from Thomas Leister to perform sa-learn on movement to Spam or Ham, just need to be adjusted, that this sa-learn is not performed locally on the Plesk server but on the PMG server. Maybe Proxmox will provide in future a Proxmox Mail Archive with direct IMAP access (without option to delete => would perform an archive) and SMTP AUTH access to send mails and integrate it with PMG, then all this adjustments may already been done and make life easier for many subscribers.

As written above, I currently try to do the same with Exchange (may be performed similar with every mail server setup) and global mailboxes spam and ham, employees need then to forward (as attachment) spam to spam and false-positives to ham, for a minimum of time to reach the 200 spams and 200 hams, lateron it could only be performed on annoying spam, once bayes is running, for sure would be better to continue to forward spam and ham as the milter-reject scores could then be adjusted better and "safe spam" could be rejected already on lower scores. However, beside being father and husband, I currently prepare a training, which take all my spare time.

 

[killmasta93]

That's right. Alternative it *should* also work to apply https://github.com/extremeshok/clamav-unofficial-sigs/pull/193 but you may encounter https://github.com/extremeshok/clamav-unofficial-sigs/issues/204, so I currently prefer to disable yara at all. That's also why I prefer to use the script, there is still some support, the sanesecurity website looks a bit of unsupported. I would prefer, if it would be easier to change/add antivirus scanner to PMG and be much more open in decision, Sophos or Avira would be my preferred candidates. Sophos we already use wide spread (however, it would be great to have another one, as Sophos is already running as endpoint protection, so there would be no advantage to scan in the mail flow), so Avira would be great as well.

Gotcha did not know so far its been great i would hope one day to incorporate the free Bitdefender Antivirus.

 

[killmasta93]

I already provided a script above for Plesk, where Mails are sorted into Spam or Ham for one mailbox (could be improved for more mailboxes) and can be used to feed bayes database. However, all the users then are required to use any webmail provided by Plesk or IMAP access to the mailboxes as been provided by Plesk and always use Spam folder for confirmed Spam or (most interesting and important) undetected Spam (so false-negatives) as well as Ham folder for false-positives. It's also possible to configure Plesk to run SpamAssassin by itself and write RegEx to automatically move Spam to Spam (however, also false-positives will then be sorted there). It's also possible to use and adjust the setup from Thomas Leister to perform sa-learn on movement to Spam or Ham, just need to be adjusted, that this sa-learn is not performed locally on the Plesk server but on the PMG server. Maybe Proxmox will provide in future a Proxmox Mail Archive with direct IMAP access (without option to delete => would perform an archive) and SMTP AUTH access to send mails and integrate it with PMG, then all this adjustments may already been done and make life easier for many subscribers.

As written above, I currently try to do the same with Exchange (may be performed similar with every mail server setup) and global mailboxes spam and ham, employees need then to forward (as attachment) spam to spam and false-positives to ham, for a minimum of time to reach the 200 spams and 200 hams, lateron it could only be performed on annoying spam, once bayes is running, for sure would be better to continue to forward spam and ham as the milter-reject scores could then be adjusted better and "safe spam" could be rejected already on lower scores. However, beside being father and husband, I currently prepare a training, which take all my spare time.

Forgot to ask has to be IMAP right?