Advancing Proxmox Mail Gateway (especially Spam and Virus Detection)

Discussion in 'Mail Gateway: Installation and configuration' started by heutger, May 29, 2018.

  1. heutger

    heutger Active Member

    Joined:
    Apr 25, 2018
    Messages:
    331
    Likes Received:
    78
    This post is split into three (3) parts, as I was not able to post it in one, so read all the next three posts to have the full overview.
     
    #1 heutger, May 29, 2018
    Last edited: May 29, 2018
  2. heutger

    heutger Active Member

    Joined:
    Apr 25, 2018
    Messages:
    331
    Likes Received:
    78
    I decide to create a new thread on my optimizations done to Proxmox Mail Gateway (PMG) with some explanations.

    Steps done:

    Code:
    vi /etc/apt/sources.list.d/pmg-enterprise.list
    apt-get update
    apt-get upgrade
    apt-get dist-upgrade
    This steps changed to testing repository (should not be used for productive, a subscription should be ordered there), updated the apt-cache and performed all outstanding updates and upgrades.

    Code:
    apt-get install gcc
    apt-get install make
    apt-get install unzip
    apt-get install vim
    vi /etc/profile
    Fetch all necessary packages for later steps, replace the horrible "neutered" vi version shipped with a more advanced version and add some aliases to /etc/profile and disable double entries in bash history.

    Code:
    vi /etc/pmg/pmg-api.pem
    vi /etc/pmg/pmg-tls.pem
    reboot
    Replace the API certificate (also for the GUI) and the TLS certificate for STARTTLS with your own ones and reboot the system to take all changes effective (especially for sure changes on e.g. kernel packages by the updates and upgrades before).

    Code:
    mkdir .ssh
    vi .ssh/authorized_keys
    Place your public SSH key for faster and easier access to the PMG root shell.

    Code:
    apt-get install ufw
    ufw enable
    ufw default deny incoming
    ufw default allow outgoing
    ufw allow ssh
    ufw allow smtp
    ufw allow 8006
    ufw status verbose
    If PMG is not placed in a company DMZ (as it usually should and would be), enable the firewall, allow all outgoing, deny all incoming except ssh, smtp and the PMG GUI. ufw makes it easier to administer. For sure also outgoing connections could be fine-tuned, but usually it's not necessary.

    Code:
    apt-get install unbound
    vi /etc/systemd/timesyncd.conf
    systemctl restart systemd-timesyncd
    As well, if not placed in a company DMZ with a companywide own DNS server (without forwarding to ISP resolvers) and/or time server, a local DNS server should be installed to prevent to be blocked at the RBL and URIBL lists (making them unusable) as well it's a good idea to set the timeserver to your countries nameserver pool.

    Code:
    mkdir -p /etc/pmg/templates
    cp /var/lib/pmg/templates/* /etc/pmg/templates/.
    cd /etc/pmg/templates
    vi main.cf.in
    vi /etc/postfix/header_checks
    PMG uses templates for configuration files, so just adjust this templates. Tweak postfix a bit (details below) and add header checks for getting the given from, to and subject of mails been accepted as many senders use bounce-detection addresses, which prevent from successful interpreting the real message behavior.

    Code:
    apt-get install pyzor
    cd /tmp
    wget http://www.dcc-servers.net/dcc/source/dcc.tar.Z
    tar xzvf dcc.tar.Z
    cd dcc-*
    ./configure
    make
    make install
    vi /lib/systemd/system/dcc.service
    vi /var/dcc/dcc_conf
    systemctl enable dcc
    systemctl start dcc
    Add pyzor and DCC (in daemon mode) to improve spam detection, be aware of the license requirements, e.g. that's the reason why PMG is not shipped directly with DCC.

    Code:
    apt-get install re2c
    cd /etc/cron.hourly && wget sa.schaal-it.net/sa-update && chown root.root sa-update && chmod 755 sa-update
    vi sa-update
    Add additional rules to PMG from trusted sources (may or may not work for you) and let them update hourly. Let updates invoke sa-comile to get compiled rulesets, which improve performance of SpamAssassin.

    Code:
    cd /tmp
    wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz
    wget http://geolite.maxmind.com/download/geoip/database/GeoIPv6.dat.gz
    gunzip GeoIP.dat.gz
    gunzip GeoIPv6.dat.gz
    mkdir /usr/share/GeoIP
    mv GeoIP.dat /usr/share/GeoIP/
    mv GeoIPv6.dat /usr/share/GeoIP/
    vi /etc/pmg/templates/init.pre.in
    vi /etc/pmg/templates/v320.pre.in
    vi /etc/mail/spamassassin/custom.cf
    /etc/cron.hourly/sa-update
    pmgconfig sync --restart 1
    spamassassin -D --lint
    echo "test" | spamassassin -D pyzor 2>&1 | less
    Add GeoIP data to improve bayes filter as well as adding rules to add scores for countries been passed by mails (may or may not work for you), enable pyzor and dcc, add blacklists for scoring (e.g. barracudacentral has been removed from SpamAssassin some time ago because of requirement to register IPs to use BRBL add their website, heise nix spam started to increase false positives, consider to check back with http://dnsbl.inps.de/index.cgi?lang=de&site=00009 and https://www.intra2net.com/en/support/antispam/ meanwhile last one shows rare false positives, I can't agree with), sync the changed templates, check SpamAssassin configuration for errors, check Pyzor, if it's working and perform the first ruleset update and rules compilation. After being finished, restart PMG SpamAssassin implementation to take sure, everything is running as expected.

    Code:
    wget https://github.com/extremeshok/clamav-unofficial-sigs/archive/master.zip
    unzip master.zip
    cp clamav-unofficial-sigs-master/clamav-unofficial-sigs.sh /usr/local/sbin/
    chmod 755 /usr/local/sbin/clamav-unofficial-sigs.sh
    mkdir /etc/clamav-unofficial-sigs
    cp clamav-unofficial-sigs-master/config/* /etc/clamav-unofficial-sigs/
    mkdir /var/log/clamav-unofficial-sigs
    cd /etc/clamav-unofficial-sigs
    cat /etc/*release*
    mv os.debian9.conf os.conf
    vi user.conf
    /usr/local/sbin/clamav-unofficial-sigs.sh --install-cron
    /usr/local/sbin/clamav-unofficial-sigs.sh --install-logrotate
    /usr/local/sbin/clamav-unofficial-sigs.sh --install-man
    /usr/local/sbin/clamav-unofficial-sigs.sh
    cp /tmp/clamav-unofficial-sigs-master/systemd/* /etc/systemd/
    clamscan --debug 2>&1 /dev/null | grep "loaded"
    Install additional signatures for ClamAV to improve detection. Check the licenses first, free subscriptions are limited, on contribution e.g. MalwarePatrol give better subscriptions for free.

    Code:
    cd /tmp
    wget https://github.com/Angristan/OpenVPN-install/archive/master.zip
    unzip master.zip
    cd OpenVPN-install-master
    chmod +x openvpn-install.sh
    ./openvpn-install.sh
    vi /etc/openvpn/server.conf
    service openvpn restart
    vi /root/client.ovpn
    ufw allow openvpn
    ufw delete allow 8006
    ufw allow from 10.8.0.2 to any port 8006
    ufw delete allow ssh
    ufw allow from 10.8.0.2 to any port ssh
    This steps are only required, if PMG is outside the company DMZ and you want to restrict access to PMGs GUI and SSH server only via OpenVPN. Setup is easy, however usually meant for road warrior setups, so need a few adjustments (mentioned below).

    Code:
    apt-get install lftp
    mkdir /backup
    mkdir /backup/node
    mkdir /root/scripts
    vi /root/scripts/backup.sh
    vi /root/scripts/upload.sh
    chmod +x /root/scripts/backup.sh
    chmod +x /root/scripts/upload.sh
    /root/scripts/backup.sh
    vi /etc/crontab
    Same for this backup setup, if PMG is running in the company environment, you may already have other backup solutions like Veeam if running as virtual machine on ESXi, otherwise, this scripts would help to backup most important data, for sure, it's not the whole system and it's no rsync, but it also allows rotation.

    /etc/apt/sources.list.d/pmg-enterprise.list (for testing updates):
    Code:
    deb http://download.proxmox.com/debian/pmg stretch pmg-no-subscription
    /etc/profile (for history control improvements and aliases):
    Code:
    # /etc/profile: system-wide .profile file for the Bourne shell (sh(1))
    # and Bourne compatible shells (bash(1), ksh(1), ash(1), ...).
    
    if [ "`id -u`" -eq 0 ]; then
      PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
    else
      PATH="/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games"
    fi
    export PATH
    
    HISTSIZE=1000
    if [ "$HISTCONTROL" = "ignorespace" ] ; then
      export HISTCONTROL=ignoreboth
    else
      export HISTCONTROL=ignoredups
    fi
    export HISTSIZE HISTCONTROL
    
    alias "ls=ls -al"
    alias "rm=rm --preserve-root"
    alias "cd..=cd .."
    alias "dir=ls -al"
    
    if [ "${PS1-}" ]; then
      if [ "${BASH-}" ] && [ "$BASH" != "/bin/sh" ]; then
        # The file bash.bashrc already sets the default PS1.
        # PS1='\h:\w\$ '
        if [ -f /etc/bash.bashrc ]; then
          . /etc/bash.bashrc
        fi
      else
        if [ "`id -u`" -eq 0 ]; then
          PS1='# '
        else
          PS1='$ '
        fi
      fi
    fi
    
    if [ -d /etc/profile.d ]; then
      for i in /etc/profile.d/*.sh; do
        if [ -r $i ]; then
          . $i
        fi
      done
      unset i
    fi
    /etc/systemd/timesyncd.conf (for using local timeserver pool, here pool for Germany):
    Code:
    #  This file is part of systemd.
    #
    #  systemd is free software; you can redistribute it and/or modify it
    #  under the terms of the GNU Lesser General Public License as published by
    #  the Free Software Foundation; either version 2.1 of the License, or
    #  (at your option) any later version.
    #
    # Entries in this file show the compile time defaults.
    # You can change settings by editing this file.
    # Defaults can be restored by simply deleting this file.
    #
    # See timesyncd.conf(5) for details.
    
    [Time]
    NTP=de.pool.ntp.org
    /etc/pmg/templates/main.cf.in (for improved postfix setup):
    Code:
    # auto-generated by proxmox
    
    compatibility_level = 2
    command_directory = /usr/sbin
    daemon_directory = /usr/lib/postfix/sbin
    data_directory = /var/lib/postfix
    
    # appending .domain is the MUA's job.
    append_dot_mydomain = yes
    
    smtpd_banner = $myhostname [% pmg.mail.banner %]
    biff = no
    
    [% IF pmg.mail.dwarning %]
    delay_warning_time = [% pmg.mail.dwarning %]h
    [% END %]
    
    best_mx_transport = local
    message_size_limit = [% pmg.mail.maxsize %]
    mailbox_size_limit = [% ((pmg.mail.maxsize*2 > 51200000) ? pmg.mail.maxsize*2 : 51200000) %]
    
    mydomain = [% dns.domain %]
    myhostname = [% dns.hostname %].[% dns.domain %]
    
    parent_domain_matches_subdomains = debug_peer_list,fast_flush_domains,mynetworks,permit_mx_backup_networks,qmqpd_authorized_clients,smtpd_access_maps
    
    alias_maps = hash:/etc/aliases
    alias_database = hash:/etc/aliases
    mydestination = localhost, $myhostname
    mynetworks = [% postfix.mynetworks %]
    
    relay_domains = hash:/etc/pmg/domains
    
    transport_maps = hash:/etc/pmg/transport
    
    [% IF pmg.mail.relay %]
    [% IF pmg.mail.relaynomx %]
    relay_transport = smtp:[[% pmg.mail.relay %]]:[% pmg.mail.relayport %]
    [% ELSE %]
    relay_transport = smtp:[% pmg.mail.relay %]:[% pmg.mail.relayport %]
    [% END %]
    [% END %]
    
    [% IF pmg.mail.smarthost %]
    default_transport = smtp:[% pmg.mail.smarthost %]:[% pmg.mail.smarthostport %]
    [% END %]
    
    content_filter=scan:127.0.0.1:10024
    
    mail_name = Proxmox
    
    [% IF pmg.mail.helotests %]
    smtpd_helo_required = yes
    smtpd_helo_restrictions = permit_mynetworks reject_non_fqdn_helo_hostname reject_invalid_helo_hostname reject_rhsbl_helo dbl.spamhaus.org reject_rhsbl_helo dbl.abuse.ro
    [% ELSE %]
    smtpd_helo_restrictions =
    [% END %]
    Adding additional HELO checks, e.g. checking against Spamhaus and Abuse.ro domain blacklist.

    Code:
    postscreen_access_list =
            permit_mynetworks,
            cidr:/etc/postfix/postscreen_access
    
    [% IF postfix.dnsbl_sites %]
    postscreen_dnsbl_sites = [% postfix.dnsbl_sites %]
    postscreen_dnsbl_threshold = [% postfix.dnsbl_threshold %]
    [% END %]
    
    postscreen_dnsbl_action = enforce
    postscreen_greet_action = enforce
    Using the given DNS blacklists with weights and a threshold of 2 (Update: Can now be set via GUI) let's still use barracudacentral and nixspam, although barracudacentral had some false-positives and questionable promoting paid emailreg service, a bit similar to uceprotect, nixspam recently started blocking services like getpocket, facebook, instragram, pinterest etc., so it's not usable anymore as fully trusted blacklist, but in combination it's reasonable. May or may not work for you, maybe other lists work better for you. Lateron added senderscore, spameatingmonkey, realtimeblacklist and dronebl as "level 1" and gbudb truncate as well as blocklist.de as "level 2".

    Code:
    smtpd_sender_restrictions =
            permit_mynetworks
            reject_non_fqdn_sender
            check_client_access     cidr:/etc/postfix/clientaccess
            check_sender_access     regexp:/etc/postfix/senderaccess
            check_recipient_access  regexp:/etc/postfix/rcptaccess
    [%- IF pmg.mail.rejectunknown %] reject_unknown_client_hostname[% END %]
    [%- IF pmg.mail.rejectunknownsender %] reject_unknown_sender_domain[% END %]
            reject_rhsbl_client dbl.spamhaus.org
            reject_rhsbl_reverse_client dbl.spamhaus.org
            reject_rhsbl_sender dbl.spamhaus.org
            reject_rhsbl_client dbl.abuse.ro
            reject_rhsbl_reverse_client dbl.abuse.ro
            reject_rhsbl_sender dbl.abuse.ro
    Same as above, Spamhaus and Abuse.ro domain blacklist could be additional used.

    Code:
    smtpd_recipient_restrictions =
            permit_mynetworks
            reject_unauth_destination
            reject_non_fqdn_recipient
            check_recipient_access  regexp:/etc/postfix/rcptaccess
    [%- IF postfix.usepolicy %] check_sender_access  regexp:/etc/postfix/senderaccess[% END %]
    [%- IF postfix.usepolicy %] check_client_access  cidr:/etc/postfix/clientaccess[% END %]
    [%- IF postfix.usepolicy %] check_policy_service inet:127.0.0.1:10022[% END %]
    [%- IF pmg.mail.verifyreceivers %] reject_unknown_recipient_domain[% END %]
    [%- IF pmg.mail.verifyreceivers %] reject_unverified_recipient[% END %]
    
    smtpd_data_restrictions = reject_unauth_pipelining
    A small but good tweak.

    Code:
    [% IF pmg.mail.rejectunknownsender %]
    unknown_address_reject_code = 550
    [% ELSE %]
    [% IF pmg.mail.verifyreceivers %]
    unknown_address_reject_code = 550
    [% END %]
    [% END %]
    
    [% IF pmg.mail.rejectunknown %]
    unknown_client_reject_code = 550
    [% END %]
    
    [% IF pmg.mail.verifyreceivers %]
    unverified_recipient_reject_code = [% pmg.mail.verifyreceivers %]
    [% END %]
    Changing additional 450 to 550 reject codes.

    Code:
    smtpd_client_connection_count_limit = [% pmg.mail.conn_count_limit %]
    smtpd_client_connection_rate_limit = [% pmg.mail.conn_rate_limit %]
    smtpd_client_message_rate_limit = [% pmg.mail.message_rate_limit %]
    
    [% IF pmg.mail.tls %]
    smtp_tls_security_level = may
    smtp_tls_policy_maps = hash:/etc/pmg/tls_policy
    smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt
    smtpd_tls_security_level = may
    smtpd_tls_cert_file = /etc/pmg/pmg-tls.pem
    smtpd_tls_key_file = $smtpd_tls_cert_file
    [% IF pmg.mail.tlslog %]
    smtpd_tls_loglevel = 1
    smtp_tls_loglevel = 1
    [% END %]
    [% IF pmg.mail.tlsheader %]
    smtpd_tls_received_header = yes
    [% END %]
    [% END %]
    
    header_checks = regexp:/etc/postfix/header_checks
    Adding header checks mentioned above, explained below.

    Code:
    smtp_tls_session_cache_database = btree:/var/lib/postfix/smtp_tls_session_cache
    smtpd_tls_session_cache_database = btree:/var/lib/postfix/smtpd_tls_session_cache
    
    [% IF pmg.mail.hide_received %]
    unverified_recipient_reject_reason = Recipient address lookup failed
    [% END %]
    
    
    default_destination_concurrency_limit = 40
    lmtp_destination_concurrency_limit = 20
    relay_destination_concurrency_limit = 20
    smtp_destination_concurrency_limit = 20
    virtual_destination_concurrency_limit = 20
    
    recipient_delimiter = +
    This part left unchanged.
     
    #2 heutger, May 29, 2018
    Last edited: Oct 24, 2018
    DerDanilo and killmasta93 like this.
  3. heutger

    heutger Active Member

    Joined:
    Apr 25, 2018
    Messages:
    331
    Likes Received:
    78
    /etc/postfix/header_checks:
    Code:
    /^From:/ INFO
    /^To:/ INFO
    /^Subject:/ INFO
    Adds given from string and address, to string and address as well as subject as Info-Text into the mail.log and as been shown there into the Tracking Center as well.

    /lib/systemd/system/dcc.service (lets dcc run as daemon):
    Code:
    [Unit]
    Description=DCC (Distributed Checksum Clearinghouses) interface daemon
    After=remote-fs.target systemd-journald-dev-log.socket
    
    [Service]
    Type=forking
    PermissionsStartOnly=true
    RuntimeDirectory=dcc
    ExecStart=/var/dcc/libexec/dccifd
    User=root
    Group=root
    Nice=1
    
    #DCC writes pid file with "-" at the beginning which confuses systemd
    #PIDFile=/run/dcc/dccifd.pid
    
    [Install]
    WantedBy=multi-user.target
    /var/dcc/dcc_conf (enables dccifd, the only somehow important" file not being backed up in the backup script below):
    Code:
    #! /bin/sh
    
    # set parameters for DCC start, stop, and cron scripts
    
    # This file is parsed by /bin/sh, and so parameters must be appropriately quoted
    #   Start your own comment lines with something other than "#" such as "##"
    #   so that they will be preserved when installing a new version.
    
    
    # from Rhyolite Software DCC 1.3.163-1.88 $Revision$
    DCC_CONF_VERSION=4
    
    # don't set DCC_HOMEDIR since if we got here, it must be set
    DCC_LIBEXEC=/var/dcc/libexec
    DCC_RUNDIR=/var/run/dcc
    
    # DCC user name
    DCCUID=root
    
    
    DCCD_ENABLE=off
    # DCC server-IDs must be globally unique.
    SRVR_ID=
    # BRAND can be any short alphanumeric string that suggests the identity
    #   of the server.
    BRAND=
    # args used to start dccd
    DCCD_ARGS=
    
    
    # GREY_CLIENT_ARGS contains "on", "-GnoIP", etc. to turn on greylisting
    #    in the dccm and dccifd DCC clients.
    #   Also turns on the local greylist dccd server unless GREY_ENABLE=off
    GREY_CLIENT_ARGS=
    # GREY_ENABLE turns local greylist server 'on' or 'off',
    #    but does not effect dccm, dccifd
    GREY_ENABLE=
    
    # GREY_SRVR_ID DCC server-IDs must be globally unique, but greylisting dccd
    #   servers are usually isolated.  If you have more than one greylist server,
    #   ensure that they use distinct server-IDs and that they flood each other
    #   with entries in /var/dcc/flod
    GREY_SRVR_ID=$SRVR_ID
    # Start dccd for grey listing or set server options such as -Gweak-IP.
    #   See also GREY_ENABLE.
    GREY_DCCD_ARGS=
    
    # dccm and dccifd client reputation parameters such as -tREP,20
    REP_ARGS=-tREP,20
    
    # DNS blacklist -B parameters for dccifd and dccm
    #   For example, this checks SMTP client IP addresses for any value Spamhaus'
    #   ZEN.  It also looks for for values between 127.0.0.0 and 127.0.0.8 for SMTP
    #   envelope sender mail_host or mail_host domain names, URLs in mail message
    #   bodies, MX servers, DNS (NS) servers.
    #   It then checks URLs in the message body in Spamhaus' DBL.
    #   It also whitelists with the "0 = none" values of dnswl.org.
    # DNSBL_ARGS="'-Bset:rej-msg=5.7.1 550 %ID %BTYPE https://www.spamhaus.org/query/bl?ip=%BTGT' \
    #    -Bzen.spamhaus.org,127.0.0.0-127.0.0.8 \
    #    -Bset:no-mail_host -Bset:no-URL -Bset:no-NS -Bzen.spamhaus.org \
    #    -Bset:URL -Bset:no-client -Bdbl.spamhaus.org,127.0.1.1-127.0.1.59,name \
    #    -Bset:white -Bset:client -Bset:no-URL -Bset:no-mail_host \
    #    -Bset:no-NS -Bset:no-MX '-Blist.dnswl.org,127.0.0.3&255.255.0.255'"
    DNSBL_ARGS=
    
    
    DCCM_ENABLE=off
    #
    # used to start dccm
    #   a common value is
    #    DCCM_ARGS="-SHELO -Smail_host -SSender -SList-ID"
    #   Note the use of single quotes in
    #    DCCM_ARGS="-SHELO '-r5.7.1 550 mail %s from %s rejected with DCC'"
    #   Put greylist parameters in GREY_CLIENT_ARGS but not in DCCM_ARGS.
    DCCM_ARGS="-SHELO -Smail_host -SSender -SList-ID"
    # The log directories should be cleaned with the /var/dcc/libexec/cron-dccd
    #   nightly cron job.  DCCM_LOGDIR can start with D? H? or M? as in
    #   DCCM_LOGDIR='D?log'  See -l in the man page.
    #   Set DCCM_LOGDIR to the empty or null string to disable logging.
    DCCM_LOGDIR="D?log"
    DCCM_WHITECLNT=whiteclnt
    DCCM_USERDIRS=userdirs
    # set DCCM_LOG_AT to a number that determines "bulk mail" for your situation.
    #   50 is a typical value.
    # Leave DCCM_REJECT_AT blank until you are confident that most sources of
    #   solicited bulk mail have been whitelisted.  Then set it to the number
    #   that defines "bulk mail" for your site.  This rejection or "bulk" threshold
    #   does not affect the blacklisting of the DCCM_WHITECLNT whitelist file.
    # Add '-aIGNORE' to DCCM_ARGS to ignore the bulkiness of mail except to
    #   add X-DCC headers.
    DCCM_LOG_AT=5
    DCCM_REJECT_AT=
    # override basic list of DCC server checksums controlling rejections or logging
    DCCM_CKSUMS=
    # additional DCC server checksums worthy of rejections or logging
    DCCM_XTRA_CKSUMS=
    
    
    DCCIFD_ENABLE=on
    #
    # used to start dccifd
    #   a common value is
    #   DCCIFD_ARGS="-SHELO -Smail_host -SSender -SList-ID"
    DCCIFD_ARGS="-SHELO -Smail_host -SSender -SList-ID"
    DCCIFD_LOGDIR="$DCCM_LOGDIR"
    DCCIFD_WHITECLNT="$DCCM_WHITECLNT"
    #   When both dccm and dccifd are used it may be necessary to set
    #   DCCIFD_USERDIRS="$DCCM_USERDIRS/local"
    DCCIFD_USERDIRS="$DCCM_USERDIRS"
    DCCIFD_LOG_AT="$DCCM_LOG_AT"
    DCCIFD_REJECT_AT="$DCCM_REJECT_AT"
    # override basic list of checksums controlling rejections or logging
    DCCIFD_CKSUMS="$DCCM_CKSUMS"
    # additional DCC server checksums worthy of rejections or logging
    DCCIFD_XTRA_CKSUMS="$DCCM_XTRA_CKSUMS"
    
    # days to keep files in DCC log directories
    DBCLEAN_LOGDAYS=14
    # used to start dbclean, including -H, -u, -e, and -E
    DBCLEAN_ARGS=
    
    
    # optionally set to something like "local5" or "local5.notice" for
    #   dccd, dbclean, dccifd, and dccm.  The defaults are equivalent to
    #   DCC_INFO_LOG_FACILITY=MAIL.NOTICE and DCC_ERROR_LOG_FACILITY=MAIL.ERR
    DCC_INFO_LOG_FACILITY=
    DCC_ERROR_LOG_FACILITY=
    
    
    # ensure that the log facilities include levels and that $DCC_LOGGER
    #   has a default.
    if test -n "$DCC_INFO_LOG_FACILITY"; then
        if expr "X$DCC_INFO_LOG_FACILITY" : 'X.*\..*' >/dev/null; then
        :
        else
        DCC_INFO_LOG_FACILITY="$DCC_INFO_LOG_FACILITY.notice"
        fi
        DCC_LOG_ARGS="$DCC_LOG_ARGS -Linfo,$DCC_INFO_LOG_FACILITY"
    fi
    if test -z "$DCC_ERROR_LOG_FACILITY"; then
        # for $DCC_LOGGER
        DCC_ERROR_LOG_FACILITY=mail.err
    else
        if expr "X$DCC_ERROR_LOG_FACILITY" : 'X.*\..*' >/dev/null; then
        :
        else
        DCC_ERROR_LOG_FACILITY="$DCC_ERROR_LOG_FACILITY.err"
        fi
        DCC_LOG_ARGS="$DCC_LOG_ARGS -Lerror,$DCC_ERROR_LOG_FACILITY"
    fi
    DCC_LOGGER="logger -s -p ${DCC_ERROR_LOG_FACILITY-mail.err} -t ${LOGGER_TAG-DCC}"
    
    
    # capture ./configure values for make-dcc_conf
    Configure_DCC_LIBEXEC=/var/dcc/libexec
    Configure_DCC_RUNDIR=/var/run/dcc
    Configure_DCCUID=root
    Configure_DCC_LOGGER="logger -s -p ${DCC_ERROR_LOG_FACILITY-mail.err} -t ${LOGGER_TAG-DCC}"
    
    /etc/cron.hourly/sa-update (adding rules from zmi, schaal-it and heinlein):
    Code:
    #!/bin/sh
    
    # schaal @it
    #
    # Simple script to update SpamAssassin
    
    SYSLOG_TAG=sa-update
    
    compile=0
    
    logger -d -t $SYSLOG_TAG "Start SA-Update"
    
    sa-update --nogpg
    retval="$?"
    if [ $retval -eq 0 ]; then compile=1; fi
    
    
    sa-update --nogpg --channel updates.spamassassin.org
    retval="$?"
    if [ $retval -eq 0 ]; then compile=1; fi
    
    sa-update --nogpg --channel sa.zmi.at
    retval="$?"
    if [ $retval -eq 0 ]; then compile=1; fi
    
    sa-update --nogpg --channel sa.schaal-it.net
    retval="$?"
    if [ $retval -eq 0 ]; then compile=1; fi
    
    sa-update --nogpg --channel spamassassin.heinlein-support.de
    retval="$?"
    if [ $retval -eq 0 ]; then compile=1; fi
    
    if [ $compile -eq 1 ]; then
        logger -d -t $SYSLOG_TAG "SA-Update found"
        sa-compile --quiet 2>/dev/null
        systemctl restart pmg-smtp-filter
    else
        logger -d -t $SYSLOG_TAG "No SA-Update found"
    fi
    /etc/pmg/templates/init.pre.in (enabling Pyzor, DCC and GeoData):
    Code:
    # This is the right place to customize your installation of SpamAssassin.
    #
    # See 'perldoc Mail::SpamAssassin::Conf' for details of what can be
    # tweaked.
    #
    # This file contains plugin activation commands for plugins included
    # in SpamAssassin 3.0.x releases.  It will not be installed if you
    # already have a file in place called "init.pre".
    #
    ###########################################################################
    
    # RelayCountry - add metadata for Bayes learning, marking the countries
    # a message was relayed through
    #
    # loadplugin Mail::SpamAssassin::Plugin::RelayCountry
    
    [% IF pmg.spam.rbl_checks %]
    loadplugin Mail::SpamAssassin::Plugin::URIDNSBL
    [% END %]
    
    # Hashcash - perform hashcash verification.
    #
    loadplugin Mail::SpamAssassin::Plugin::Hashcash
    
    [% IF pmg.spam.rbl_checks %]
    loadplugin Mail::SpamAssassin::Plugin::SPF
    [% END %]
    
    # always load dkim to improve accuracy
    loadplugin Mail::SpamAssassin::Plugin::DKIM
    
    loadplugin Mail::SpamAssassin::Plugin::Pyzor
    use_pyzor 1
    
    loadplugin Mail::SpamAssassin::Plugin::DCC
    dcc_path /usr/local/bin/dccproc
    dcc_home /var/dcc
    dcc_dccifd_path /var/dcc/dccifd
    dcc_body_max 999999
    dcc_fuz1_max 999999
    dcc_fuz2_max 999999
    use_dcc 1
    dcc_timeout 10
    
    loadplugin Mail::SpamAssassin::Plugin::RelayCountry
    /etc/pmg/templates/v320.pre.in (enabling compiled rulesets):
    Code:
    # This is the right place to customize your installation of SpamAssassin.
    #
    # See 'perldoc Mail::SpamAssassin::Conf' for details of what can be
    # tweaked.
    #
    # This file was installed during the installation of SpamAssassin 3.2.0,
    # and contains plugin loading commands for the new plugins added in that
    # release.  It will not be overwritten during future SpamAssassin installs,
    # so you can modify it to enable some disabled-by-default plugins below,
    # if you so wish.
    #
    ###########################################################################
    
    # Check - Provides main check functionality
    #
    loadplugin Mail::SpamAssassin::Plugin::Check
    
    # HTTPSMismatch - find URI mismatches between href and anchor text
    #
    loadplugin Mail::SpamAssassin::Plugin::HTTPSMismatch
    
    # URIDetail - test URIs using detailed URI information
    #
    loadplugin Mail::SpamAssassin::Plugin::URIDetail
    
    # Shortcircuit - stop evaluation early if high-accuracy rules fire
    #
    # loadplugin Mail::SpamAssassin::Plugin::Shortcircuit
    
    # Plugins which used to be EvalTests.pm
    # broken out into separate plugins
    loadplugin Mail::SpamAssassin::Plugin::Bayes
    loadplugin Mail::SpamAssassin::Plugin::BodyEval
    loadplugin Mail::SpamAssassin::Plugin::DNSEval
    loadplugin Mail::SpamAssassin::Plugin::HTMLEval
    loadplugin Mail::SpamAssassin::Plugin::HeaderEval
    loadplugin Mail::SpamAssassin::Plugin::MIMEEval
    loadplugin Mail::SpamAssassin::Plugin::RelayEval
    loadplugin Mail::SpamAssassin::Plugin::URIEval
    loadplugin Mail::SpamAssassin::Plugin::WLBLEval
    
    # VBounce - anti-bounce-message rules, see rules/20_vbounce.cf
    #
    loadplugin Mail::SpamAssassin::Plugin::VBounce
    
    # Rule2XSBody - speedup by compilation of ruleset to native code
    #
    loadplugin Mail::SpamAssassin::Plugin::Rule2XSBody
    
    # ASN - Look up the Autonomous System Number of the connecting IP
    # and create a header containing ASN data for bayes tokenization.
    # See plugin's POD docs for usage info.
    #
    # loadplugin Mail::SpamAssassin::Plugin::ASN
    
    # ImageInfo - rules to match metadata of image attachments
    #
    loadplugin Mail::SpamAssassin::Plugin::ImageInfo
    /etc/mail/spamassassin/custom.cf (adding country filters as well as additional backlist scores, xxx, xxx24 and xxxuri should be your own invaluement subscription domains):
    Code:
    ifplugin Mail::SpamAssassin::Plugin::RelayCountry
    add_header all Relay-Country _RELAYCOUNTRY_
    header RELAYCOUNTRY_BAD X-Relay-Countries =~ /(CN|RU|UA|RO|VN)/
    describe RELAYCOUNTRY_BAD Relayed through spammy country at some point
    score RELAYCOUNTRY_BAD 2.0
    header RELAYCOUNTRY_GOOD X-Relay-Countries =~ /^(DE|AT|CH)/
    describe RELAYCOUNTRY_GOOD First untrusted GW is DE, AT or CH
    score RELAYCOUNTRY_GOOD -0.5
    endif # Mail::SpamAssassin::Plugin::RelayCountry
    
    header RCVD_IN_BRBL eval:check_rbl('brbl-lastexternal', 'b.barracudacentral.org.', '127.0.0.2')
    describe RCVD_IN_BRBL Received via a relay in Barracuda RBL
    tflags RCVD_IN_BRBL net
    score RCVD_IN_BRBL 1.4
    
    header RCVD_IN_NIX_SPAM eval:check_rbl('nix-spam-lastexternal', 'ix.dnsbl.manitu.net.')
    describe RCVD_IN_NIX_SPAM Listed in NiX Spam DNSBL (heise.de)
    tflags RCVD_IN_NIX_SPAM net
    score RCVD_IN_NIX_SPAM 1.4
    
    header RCVD_IN_GBUDB eval:check_rbl('gbudb-lastexternal', 'truncate.gbudb.net.', '127.0.0.2')
    describe RCVD_IN_GBUDB Listed in GBUdb Truncate
    tflags RCVD_IN_GBUDB net
    score RCVD_IN_GBUDB 1.4
    
    header RCVD_IN_BLOCKLIST eval:check_rbl('blocklist-lastexternal', 'bl.blocklist.de.')
    describe RCVD_IN_BLOCKLIST Listed in Blocklist DNSRBL
    tflags RCVD_IN_BLOCKLIST net
    score RCVD_IN_BLOCKLIST 1.4
    
    header RCVD_IN_IVMSIP eval:check_rbl('ivmsip-lastexternal', 'xxx')
    describe RCVD_IN_IVMSIP Listed in ivmSIP
    tflags RCVD_IN_IVMSIP net
    score RCVD_IN_IVMSIP 1.4
    
    header RCVD_IN_IVMSIP24 eval:check_rbl('ivmsip-lastexternal', 'xxx24')
    describe RCVD_IN_IVMSIP24 Listed in ivmSIP/24
    tflags RCVD_IN_IVMSIP24 net
    score RCVD_IN_IVMSIP24 1
    
    ifplugin Mail::SpamAssassin::Plugin::URIDNSBL
    urirhssub URIBL_IVMURI xxxuri A 2
    body URIBL_IVMURI eval:check_uridnsbl('URIBL_IVMURI')
    describe URIBL_IVMURI Listed in ivmURI
    tflags URIBL_IVMURI net
    score URIBL_IVMURI 1.4
    reuse URIBL_IVMURI
    endif
    /etc/clamav-unofficial-sigs/user.conf (enabling additional signatures, xxx needs to be replaced by your license keys):
    Code:
    # This file contains user configuration settings for clamav-unofficial-sigs.sh
    ###################
    # This is property of eXtremeSHOK.com
    # You are free to use, modify and distribute, however you may not remove this notice.
    # Copyright (c) Adrian Jon Kriel :: admin@extremeshok.com
    # License: BSD (Berkeley Software Distribution)
    ##################
    #
    # Script updates can be found at: https://github.com/extremeshok/clamav-unofficial-sigs
    #
    ##################
    #
    # NOT COMPATIBLE WITH VERSION 3.XX / 4.XX CONFIG
    #
    ################################################################################
    # SEE MASTER.CONF FOR CONFIG EXPLAINATIONS
    ################################################################################
    
    # Values in this file will always override those in the master.conf and os.conf files.
    # This is useful to specify your authorisation/receipt codes and to always force certain options.
    # Please note, it is your responsibility to manage the contents of this file.
    # Values provided here are just examples, feel free to use any values from the main config file.
    
    malwarepatrol_receipt_code="xxxxx"
    malwarepatrol_product_code="32"
    malwarepatrol_list="clamav_basic" # clamav_basic or clamav_ext
    malwarepatrol_free="no"
    
    securiteinfo_authorisation_signature="xxxxx"
    
    # Default dbs rating
    # valid rating: LOW, MEDIUM, HIGH
    #default_dbs_rating="MEDIUM"
    
    # Per Database
    # These ratings will override the global rating for the specific database
    # valid rating: LOW, MEDIUM, HIGH, DISABLE
    #sanesecurity_dbs_rating=""
    #securiteinfo_dbs_rating=""
    #linuxmalwaredetect_dbs_rating=""
    #yararulesproject_dbs_rating=""
    
    # =========================
    # Additional signature databases
    # =========================
    #declare -a additional_dbs=(
    #   ftp://ftp.example.net/pub/sigs.ndb
    #   http://www.example.org/sigs.ldb
    #) #END ADDITIONAL DATABASES
    
    # Uncomment the following line to enable the script
    user_configuration_complete="yes"
    
    # https://eXtremeSHOK.com ######################################################
     
    #3 heutger, May 29, 2018
    Last edited: Oct 23, 2018
    DerDanilo and killmasta93 like this.
  4. heutger

    heutger Active Member

    Joined:
    Apr 25, 2018
    Messages:
    331
    Likes Received:
    78
    /etc/openvpn/server.conf (need to disable road warrior setup, no forwarding of traffic, no pushing DNS servers, xxx needs to be replaced by your cert files):
    Code:
    port 1194
    proto udp
    dev tun
    user nobody
    group nogroup
    persist-key
    persist-tun
    keepalive 10 120
    topology subnet
    server 10.8.0.0 255.255.255.0
    ifconfig-pool-persist ipp.txt
    crl-verify crl.pem
    ca ca.crt
    cert xxx.crt
    key xxx.key
    tls-auth tls-auth.key 0
    dh dh.pem
    auth SHA256
    cipher AES-128-CBC
    tls-server
    tls-version-min 1.2
    tls-cipher TLS-DHE-RSA-WITH-AES-128-GCM-SHA256
    status openvpn.log
    verb 3
    /root/client.ovpn (same here, xxx should be your host, ... will be your certs and keys):
    Code:
    client
    proto udp
    remote xxx 1194
    dev tun
    resolv-retry infinite
    nobind
    persist-key
    persist-tun
    remote-cert-tls server
    verify-x509-name xxx name
    auth SHA256
    auth-nocache
    cipher AES-128-CBC
    tls-client
    tls-version-min 1.2
    tls-cipher TLS-DHE-RSA-WITH-AES-128-GCM-SHA256
    verb 3
    ...
    /root/scripts/backup.sh:
    Code:
    #!/bin/bash
    
    echo [`date +'%d.%m.%Y %H:%M'`] Starte Backup ...
    
    echo [`date +'%d.%m.%Y %H:%M'`] Exportiere aktuelle Konfiguration [1/4] ...
    rm /var/lib/pmg/backup/*
    pmgbackup backup
    
    # Zu sicherndes Verzeichnisse, z.B. /etc /root /var
    sourcevrz='/etc /lib/systemd/system /root /usr/local /var/dcc /var/lib/pmg/backup /var/log'
    
    # Backupverzeichnis, z.B. /backup/node
    backupvrz=/backup/node
    
    # Variablendefinition
    datum=$(date +'%Y%m%d')
    dateiname=$backupvrz/backup$datum.tar
    
    # -----------------------------------------------------
    function f_delFiles()
    # -----------------------------------------------------
    # $1 Backupverzeichnis
    {
      loeschdatum=$(date --date='7 days ago' +'%Y%m%d')
      rm $1/backup$loeschdatum.tar
    }
    
    echo [`date +'%d.%m.%Y %H:%M'`] Lösche Files in $backupvrz, die älter als 7 Tage sind [2/4] ...
    f_delFiles $backupvrz
    
    echo [`date +'%d.%m.%Y %H:%M'`] Sichere $sourcevrz nach $dateiname [3/4] ...
    tar Pcf $dateiname $sourcevrz
    
    echo [`date +'%d.%m.%Y %H:%M'`] Synchronisiere mit Onlinespeicher [4/4] ...
    bash /root/scripts/upload.sh
    
    echo [`date +'%d.%m.%Y %H:%M'`] Fertig!
    /root/scripts/upload.sh (uploads to your FTP storage, passive mode and starttls preferred and used if available, xxx should be your host, credentials, folders etc.):
    Code:
    #!/bin/bash
    HOST='xxx'
    USER='xxx'
    PASS='xxx'
    TARGETFOLDER='/xxx'
    SOURCEFOLDER='/backup'
    
    lftp -e "
    open $HOST
    user $USER $PASS
    lcd $SOURCEFOLDER
    mirror --reverse --delete --use-cache --verbose $SOURCEFOLDER $TARGETFOLDER
    bye
    "
    /etc/crontab (you should anyhow cron your backup, I prefer my own settings, but could also be done via cron.daily or whatever you like, I also let the backup send me the output, but it's also your decision, xxx should be your address, x should be your minute, hour, day etc. settings):
    Code:
    # /etc/crontab: system-wide crontab
    # Unlike any other crontab you don't have to run the `crontab'
    # command to install the new version when you edit this file
    # and files in /etc/cron.d. These files also have username fields,
    # that none of the other crontabs do.
    
    SHELL=/bin/sh
    PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
    
    # m h dom mon dow user    command
      x x *   *   *   root  /root/scripts/backup.sh 2>&1 | /usr/bin/mail -r xxx@xxx -s "backup script output" xxx@xxx
    
    x *    * * *    root    cd / && run-parts --report /etc/cron.hourly
    x x    * * *    root    test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )
    x x    * * 7    root    test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly )
    x x    1 * *    root    test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly )
    #
    pmgconfig dump (tighter setup, xxx should be your domain data, xxx and xxx24 later-on are the invaluement lists, take your own subscription):
    Code:
    composed.wl_bounce_relays = xxx
    dns.domain = xxx
    dns.hostname = xxx
    ipconfig.int_ip = xxx
    pmg.admin.advfilter = 0
    pmg.admin.avast = 0
    pmg.admin.clamav = 1
    pmg.admin.dailyreport = 1
    pmg.admin.demo = 0
    pmg.admin.email = xxx
    pmg.admin.http_proxy =
    pmg.admin.statlifetime = 7
    pmg.clamav.archiveblockencrypted = 0
    pmg.clamav.archivemaxfiles = 1000
    pmg.clamav.archivemaxrec = 5
    pmg.clamav.archivemaxsize = 25000000
    pmg.clamav.dbmirror = database.clamav.net
    pmg.clamav.maxcccount = 0
    pmg.clamav.maxscansize = 100000000
    pmg.clamav.safebrowsing = 1
    pmg.mail.banner = ESMTP Proxmox
    pmg.mail.conn_count_limit = 50
    pmg.mail.conn_rate_limit = 0
    pmg.mail.dnsbl_sites =
    zen.spamhaus.org*2,bl.spamcop.net*2,psbl.surriel.com*2,spamrbl.imp.ch*2,noptr.spamrats.com*2,escalations.dnsbl.sorbs.net*2,bl.score.senderscore.com*2,bl.spameatingmonkey.net*2,rbl.realtimeblacklist.com*2,dnsbl.dronebl.org*2,ix.dnsbl.manitu.net,b.barracudacentral.org,truncate.gbudb.net,bl.blocklist.de,xxx,xxx24
    pmg.mail.dwarning = 4
    pmg.mail.ext_port = 25
    pmg.mail.greylist = 0
    pmg.mail.helotests = 1
    pmg.mail.hide_received = 0
    pmg.mail.int_port = 26
    pmg.mail.max_filters = 14
    pmg.mail.max_policy = 5
    pmg.mail.max_smtpd_in = 96
    pmg.mail.max_smtpd_out = 96
    pmg.mail.maxsize = 104857600
    pmg.mail.message_rate_limit = 0
    pmg.mail.rejectunknown = 1
    pmg.mail.rejectunknownsender = 1
    pmg.mail.relay = xxx
    pmg.mail.relaynomx = 0
    pmg.mail.relayport = 25
    pmg.mail.smarthost =
    pmg.mail.spf = 1
    pmg.mail.tls = 1
    pmg.mail.tlsheader = 1
    pmg.mail.tlslog = 1
    pmg.mail.verifyreceivers = 550
    pmg.spam.bounce_score = 0
    pmg.spam.clamav_heuristic_score = 3
    pmg.spam.languages = all
    pmg.spam.maxspamsize = 262144
    pmg.spam.rbl_checks = 1
    pmg.spam.use_awl = 1
    pmg.spam.use_bayes = 1
    pmg.spam.use_razor = 1
    pmg.spam.wl_bounce_relays =
    pmg.spamquar.allowhrefs = 1
    pmg.spamquar.authmode = ticket
    pmg.spamquar.hostname =
    pmg.spamquar.lifetime = 7
    pmg.spamquar.mailfrom =
    pmg.spamquar.port = 8006
    pmg.spamquar.protocol = https
    pmg.spamquar.reportstyle = verbose
    pmg.spamquar.viewimages = 1
    pmg.virusquar.allowhrefs = 1
    pmg.virusquar.lifetime = 7
    pmg.virusquar.viewimages = 1
    postfix.dnsbl_sites =
    zen.spamhaus.org*2,bl.spamcop.net*2,psbl.surriel.com*2,spamrbl.imp.ch*2,noptr.spamrats.com*2,escalations.dnsbl.sorbs.net*2,bl.score.senderscore.com*2,bl.spameatingmonkey.net*2,rbl.realtimeblacklist.com*2,dnsbl.dronebl.org*2,ix.dnsbl.manitu.net,b.barracudacentral.org,truncate.gbudb.net,bl.blocklist.de,xxx,xxx24
    postfix.int_ip = xxx
    postfix.mynetworks = 127.0.0.0/8 [::1]/128 xxx
    postfix.transportnets =
    postfix.usepolicy = 1
    pmgdb dump (as well tighter setup):
    Code:
    Found RULE 4: Blacklist
      FOUND FROM GROUP 2: Blacklist
        OBJECT 1: nomail@fromthisdomain.com
      FOUND ACTION GROUP 18: Block
        OBJECT 31: block message
    Found RULE 3: Virus Alert
      FOUND WHAT GROUP 9: Virus
        OBJECT 22: active
      FOUND ACTION GROUP 18: Block
        OBJECT 31: block message
      FOUND ACTION GROUP 20: Notify Admin
        OBJECT 33: notify __ADMIN__
      FOUND ACTION GROUP 21: Notify Sender
        OBJECT 34: notify __SENDER__
    Found RULE 2: Remove Viruses
      FOUND WHAT GROUP 9: Virus
        OBJECT 22: active
      FOUND ACTION GROUP 15: Remove attachments
        OBJECT 28: remove matching attachments
      FOUND ACTION GROUP 20: Notify Admin
        OBJECT 33: notify __ADMIN__
      FOUND ACTION GROUP 23: Modify Virus Subject
        OBJECT 37: modify field: subject:VIRUS: __SUBJECT__
    Found RULE 1: Remove Dangerous Files
      FOUND WHAT GROUP 8: Dangerous Content
        OBJECT 16: content-type=application/javascript
        OBJECT 17: content-type=application/x-executable
        OBJECT 15: content-type=application/x-java
        OBJECT 14: content-type=application/x-ms-dos-executable
        OBJECT 18: content-type=application/x-ms-dos-executable
        OBJECT 19: content-type=message/partial
        OBJECT 20: filename=.*\.(vbs|pif|lnk|shs|shb)
        OBJECT 21: filename=.*\.\{.+\}
      FOUND ACTION GROUP 15: Remove attachments
        OBJECT 28: remove matching attachments
      FOUND ACTION GROUP 20: Notify Admin
        OBJECT 33: notify __ADMIN__
      FOUND ACTION GROUP 24: Modify Dangerous Subject
        OBJECT 38: modify field: subject:DANGEROUS: __SUBJECT__
    Found RULE 5: Modify Header
      FOUND ACTION GROUP 13: Modify Spam Level
        OBJECT 26: modify field: X-SPAM-LEVEL:__SPAM_INFO__
    Found RULE 12: Block Multimedia Files
      FOUND WHAT GROUP 6: Multimedia
        OBJECT 5: content-type=audio/.*
        OBJECT 6: content-type=video/.*
      FOUND ACTION GROUP 15: Remove attachments
        OBJECT 28: remove matching attachments
    Found RULE 6: Whitelist
      FOUND FROM GROUP 3: Whitelist
        OBJECT 2: mail@fromthisdomain.com
      FOUND ACTION GROUP 17: Accept
        OBJECT 30: accept message
    Found RULE 9: Block Spam (Level 10)
      FOUND WHAT GROUP 12: Spam (Level 10)
        OBJECT 25: Level 10
      FOUND ACTION GROUP 18: Block
        OBJECT 31: block message
    Found RULE 8: Quarantine/Mark Spam (Level 5)
      FOUND WHAT GROUP 11: Spam (Level 5)
        OBJECT 24: Level 5
      FOUND ACTION GROUP 14: Modify Spam Subject
        OBJECT 27: modify field: subject:SPAM: __SUBJECT__
      FOUND ACTION GROUP 19: Quarantine
        OBJECT 32: Move to quarantine.
    Found RULE 7: Mark Spam (Level 6)
      FOUND WHAT GROUP 10: Spam (Level 6)
        OBJECT 23: Level 6
      FOUND ACTION GROUP 14: Modify Spam Subject
        OBJECT 27: modify field: subject:SPAM: __SUBJECT__
    Found RULE 10: Block outgoing Spam
      FOUND WHAT GROUP 10: Spam (Level 6)
        OBJECT 23: Level 6
      FOUND ACTION GROUP 18: Block
        OBJECT 31: block message
      FOUND ACTION GROUP 20: Notify Admin
        OBJECT 33: notify __ADMIN__
      FOUND ACTION GROUP 21: Notify Sender
        OBJECT 34: notify __SENDER__
    Found RULE 11: Add Disclaimer
      FOUND ACTION GROUP 22: Disclaimer
        OBJECT 35: disclaimer
    Or alternative:

    pmgconfig dump (loser setup):
    Code:
    composed.wl_bounce_relays = xxx
    dns.domain = xxx
    dns.hostname = xxx
    ipconfig.int_ip = xxx
    pmg.admin.advfilter = 0
    pmg.admin.avast = 0
    pmg.admin.clamav = 1
    pmg.admin.dailyreport = 1
    pmg.admin.demo = 0
    pmg.admin.email = xxx
    pmg.admin.http_proxy =
    pmg.admin.statlifetime = 7
    pmg.clamav.archiveblockencrypted = 0
    pmg.clamav.archivemaxfiles = 1000
    pmg.clamav.archivemaxrec = 5
    pmg.clamav.archivemaxsize = 25000000
    pmg.clamav.dbmirror = database.clamav.net
    pmg.clamav.maxcccount = 0
    pmg.clamav.maxscansize = 100000000
    pmg.clamav.safebrowsing = 1
    pmg.mail.banner = ESMTP Proxmox
    pmg.mail.conn_count_limit = 50
    pmg.mail.conn_rate_limit = 0
    pmg.mail.dnsbl_sites = zzen.spamhaus.org*2,bl.spamcop.net*2,psbl.surriel.com*2,spamrbl.imp.ch*2,noptr.spamrats.com*2,escalations.dnsbl.sorbs.net*2,bl.score.senderscore.com*2,bl.spameatingmonkey.net*2,rbl.realtimeblacklist.com*2,dnsbl.dronebl.org*2,ix.dnsbl.manitu.net,b.barracudacentral.org,truncate.gbudb.net,bl.blocklist.de,xxx,xxx24
    pmg.mail.dwarning = 4
    pmg.mail.ext_port = 25
    pmg.mail.greylist = 0
    pmg.mail.helotests = 1
    pmg.mail.hide_received = 0
    pmg.mail.int_port = 26
    pmg.mail.max_filters = 30
    pmg.mail.max_policy = 5
    pmg.mail.max_smtpd_in = 100
    pmg.mail.max_smtpd_out = 100
    pmg.mail.maxsize = 104857600
    pmg.mail.message_rate_limit = 0
    pmg.mail.rejectunknown = 1
    pmg.mail.rejectunknownsender = 1
    pmg.mail.relay = xxx
    pmg.mail.relaynomx = 0
    pmg.mail.relayport = 25
    pmg.mail.smarthost =
    pmg.mail.spf = 0
    pmg.mail.tls = 1
    pmg.mail.tlsheader = 1
    pmg.mail.tlslog = 1
    pmg.mail.verifyreceivers = 550
    pmg.spam.bounce_score = 0
    pmg.spam.clamav_heuristic_score = 3
    pmg.spam.languages = all
    pmg.spam.maxspamsize = 262144
    pmg.spam.rbl_checks = 1
    pmg.spam.use_awl = 1
    pmg.spam.use_bayes = 1
    pmg.spam.use_razor = 1
    pmg.spam.wl_bounce_relays =
    pmg.spamquar.allowhrefs = 1
    pmg.spamquar.authmode = ticket
    pmg.spamquar.hostname =
    pmg.spamquar.lifetime = 7
    pmg.spamquar.mailfrom =
    pmg.spamquar.port = 8006
    pmg.spamquar.protocol = https
    pmg.spamquar.reportstyle = verbose
    pmg.spamquar.viewimages = 1
    pmg.virusquar.allowhrefs = 1
    pmg.virusquar.lifetime = 7
    pmg.virusquar.viewimages = 1
    postfix.dnsbl_sites =
    zen.spamhaus.org*2,bl.spamcop.net*2,psbl.surriel.com*2,spamrbl.imp.ch*2,noptr.spamrats.com*2,escalations.dnsbl.sorbs.net*2,bl.score.senderscore.com*2,bl.spameatingmonkey.net*2,rbl.realtimeblacklist.com*2,dnsbl.dronebl.org*2,ix.dnsbl.manitu.net,b.barracudacentral.org,truncate.gbudb.net,bl.blocklist.de,xxx,xxx24
    postfix.int_ip = xxx
    postfix.mynetworks = 127.0.0.0/8 [::1]/128 xxx
    postfix.transportnets =
    postfix.usepolicy = 0
    pmgdb dump (loser setup):
    Code:
    Found RULE 4: Blacklist
      FOUND FROM GROUP 2: Blacklist
        OBJECT 1: nomail@fromthisdomain.com
      FOUND ACTION GROUP 18: Block
        OBJECT 31: block message
    Found RULE 3: Virus Alert
      FOUND WHAT GROUP 9: Virus
        OBJECT 22: active
      FOUND ACTION GROUP 18: Block
        OBJECT 31: block message
      FOUND ACTION GROUP 20: Notify Admin
        OBJECT 33: notify __ADMIN__
      FOUND ACTION GROUP 21: Notify Sender
        OBJECT 34: notify __SENDER__
    Found RULE 2: Remove Viruses
      FOUND WHAT GROUP 9: Virus
        OBJECT 22: active
      FOUND ACTION GROUP 15: Remove attachments
        OBJECT 28: remove matching attachments
      FOUND ACTION GROUP 20: Notify Admin
        OBJECT 33: notify __ADMIN__
      FOUND ACTION GROUP 23: Modify Virus Subject
        OBJECT 37: modify field: subject:VIRUS: __SUBJECT__
    Found RULE 1: Remove Dangerous Files
      FOUND WHAT GROUP 8: Dangerous Content
        OBJECT 16: content-type=application/javascript
        OBJECT 17: content-type=application/x-executable
        OBJECT 15: content-type=application/x-java
        OBJECT 14: content-type=application/x-ms-dos-executable
        OBJECT 18: content-type=application/x-ms-dos-executable
        OBJECT 19: content-type=message/partial
        OBJECT 20: filename=.*\.(vbs|pif|lnk|shs|shb)
        OBJECT 21: filename=.*\.\{.+\}
      FOUND ACTION GROUP 15: Remove attachments
        OBJECT 28: remove matching attachments
      FOUND ACTION GROUP 20: Notify Admin
        OBJECT 33: notify __ADMIN__
      FOUND ACTION GROUP 24: Modify Dangerous Subject
        OBJECT 38: modify field: subject:DANGEROUS: __SUBJECT__
    Found RULE 5: Modify Header
      FOUND ACTION GROUP 13: Modify Spam Level
        OBJECT 26: modify field: X-SPAM-LEVEL:__SPAM_INFO__
    Found RULE 12: Block Multimedia Files
      FOUND WHAT GROUP 6: Multimedia
        OBJECT 5: content-type=audio/.*
        OBJECT 6: content-type=video/.*
      FOUND ACTION GROUP 15: Remove attachments
        OBJECT 28: remove matching attachments
    Found RULE 6: Whitelist
      FOUND FROM GROUP 3: Whitelist
        OBJECT 2: mail@fromthisdomain.com
      FOUND ACTION GROUP 17: Accept
        OBJECT 30: accept message
    Found RULE 9: Block Spam (Level 10)
      FOUND WHAT GROUP 12: Spam (Level 10)
        OBJECT 25: Level 10
      FOUND ACTION GROUP 18: Block
        OBJECT 31: block message
    Found RULE 8: Quarantine/Mark Spam (Level 5)
      FOUND WHAT GROUP 11: Spam (Level 5)
        OBJECT 24: Level 5
      FOUND ACTION GROUP 14: Modify Spam Subject
        OBJECT 27: modify field: subject:SPAM: __SUBJECT__
      FOUND ACTION GROUP 19: Quarantine
        OBJECT 32: Move to quarantine.
    Found RULE 7: Mark Spam (Level 6)
      FOUND WHAT GROUP 10: Spam (Level 6)
        OBJECT 23: Level 6
      FOUND ACTION GROUP 14: Modify Spam Subject
        OBJECT 27: modify field: subject:SPAM: __SUBJECT__
    Found RULE 10: Block outgoing Spam
      FOUND WHAT GROUP 10: Spam (Level 6)
        OBJECT 23: Level 6
      FOUND ACTION GROUP 18: Block
        OBJECT 31: block message
      FOUND ACTION GROUP 20: Notify Admin
        OBJECT 33: notify __ADMIN__
      FOUND ACTION GROUP 21: Notify Sender
        OBJECT 34: notify __SENDER__
    Found RULE 11: Add Disclaimer
      FOUND ACTION GROUP 22: Disclaimer
        OBJECT 35: disclaimer
    
    Missing in the dumps is the whitelisting of the network 194.25.134.0/24 as the German Telecom is often listed on usually trusted blacklists.
     
    #4 heutger, May 29, 2018
    Last edited: Jul 18, 2018
    killmasta93 likes this.
  5. heutger

    heutger Active Member

    Joined:
    Apr 25, 2018
    Messages:
    331
    Likes Received:
    78
    Loser vs. tighter setup has the following differences:

    SPF disabled as many server admins are not managing their SPF records correct or may have set them once and then forgot about them (had the same with our commercial outgoing secure mail gateway were we now - regarding GDPR - encrypt all mails, if there are keys available. The gateway tries to be "intelligent" and fetches keys from ldap and keyserver URLs, which are well known, result in many complaints of users, which recently played around with PGP/GPG, uploaded keys to the keyservers and now get encrypted mails, they can't read).
     
    #5 heutger, May 30, 2018
    Last edited: Jun 8, 2018
    DerDanilo and killmasta93 like this.
  6. heutger

    heutger Active Member

    Joined:
    Apr 25, 2018
    Messages:
    331
    Likes Received:
    78
    Here comes something new! Bayes is the major weapon against Spam at content filter stage. But it needs to be trained (autolearn is fine, however it takes a long time to start and it's still an auto-filter with all its advantages and disadvantages, so that's why it takes so long to be usable, SpamAssassin is very conservative in learning spam and it needs a good ratio of ham and spam to start tagging).

    I have a Plesk Server behind PMG and I use Roundcube there to access my mail copy. I now used this mail copy to create a folder called Ham and sorted spam to Spam and ham to Ham. So spam are all the mails tagged with SPAM: which are really spam as well as spam, which got not recognized as spam (really rare, so no hard work, but it's most important as this should be recognized in future, confirming spam is also fine, but this spam brings much more quality to the filter), also maybe VIRUS: and DANGEROUS: and all ham, which has been tagged with SPAM: (again it's very important to learn this as ham to improve the filter). After that, I created SCP access to the machine containing the folders and a script to download the spam and ham folders, learn them as spam or ham and then remove the folders again.

    The steps done are below and the script as well:

    Code:
    ssh-keygen -t rsa -b 4096
    cat .ssh/id_rsa.pub
    vi /root/scripts/cs-learn.sh
    chmod +x /root/scripts/cs-learn.sh
    /root/scripts/cs-learn.sh
    /root/scripts/cs-learn.sh (for sure, again you need to give your details at xxx):
    Code:
    #!/bin/bash
    HOST='xxx'
    USER='xxx'
    DOMAIN='xxx'
    ADDRESS='xxx'
    
    echo Fetching Spam [1/5] ...
    scp -q -r $USER@$HOST:/var/qmail/mailnames/$DOMAIN/$ADDRESS/Maildir/.Spam .
    
    echo Fetching Ham [2/5] ...
    scp -q -r $USER@$HOST:/var/qmail/mailnames/$DOMAIN/$ADDRESS/Maildir/.Ham .
    
    echo Learning Spam [3/5] ...
    sa-learn --spam .Spam/cur/* .Spam/new/*
    
    echo Learning Ham [4/5] ...
    sa-learn --ham .Ham/cur/* .Ham/new/*
    
    echo Cleanup [5/5] ...
    rm -Rf .Spam
    rm -Rf .Ham
     
    #6 heutger, Jun 5, 2018
    Last edited: Jun 8, 2018
  7. heutger

    heutger Active Member

    Joined:
    Apr 25, 2018
    Messages:
    331
    Likes Received:
    78
    Loser vs. tighter setup enhanced:

    I already corrected above, that I switched from bind9 to unbound, which solved all the 450 tempfail on sender address and client host checking as well as the 550 false-positives. I'm quite unsure, why bind9 is working such bad, but it's solved now, so I won't think about it anymore.

    I also lowered score from 6 to 5 now on both test installations (loser as well as tighter), to not confuse with the shipped Level 5 action, I won't adjust my dumps above, feel free to use them as 6, adjust them to 5 or use the 5 action instead and adjust it to your need.

    Most important change, I also won't change above as it would result in an extra main.cf.in, would confuse thread readers and maybe others won't like to change it that way: I decided to change in the loser configuration the really "hard" FCrDNS check (as many companies are not yet ready to fulfill) reject_unknown_client_hostname (which is enabled via GUI) to reject_unknown_reverse_client_hostname. So what it does is not checking any more if client IP address resolve to a hostname and this hostname resolve to an IP and the IP it resolve to is the same as the client IP given, but it now just checks if client IP address resolve to a hostname, that's it. That's a requirement, which could be expected in the late 2010s, but I see currently FCrDNS not available by every system. It's a good compromise between disabling the option at all (which really kick out a good amount of spammers) versus rejecting some not very well administered servers of (potential) customers.
     
    #7 heutger, Jun 8, 2018
    Last edited: Jun 9, 2018
    DerDanilo and killmasta93 like this.
  8. heutger

    heutger Active Member

    Joined:
    Apr 25, 2018
    Messages:
    331
    Likes Received:
    78
    Once more something new! As update mails are only sent with subscription, I decide to add update mails also to testing as well as adding auto-update. Everyone can decide by himself, if auto-update is something desirable, but for my private(!) environment, I prefer auto-update, for sure, I wouldn't recommend for a commercial environment, there should be a subscription, no testing repositories and updates should be performed with existing patch management processes or procedures, I strongly advice against automatic updates in productive environments, it's a security risk. However, you can use the settings below to get more information by mail and/or let the system prefetch packages for getting updated, perform regular autoremove or autoclean, however last two options I also would never recommend in productive environments.

    Steps performed:
    Code:
    apt-get install apticron
    apt-get install unattended-upgrades
    vi /etc/apticron/apticron.conf
    apt-cache policy
    vi /etc/apt/apt.conf.d/50unattended-upgrades
    rm /etc/apt/apt.conf.d/20auto-upgrades
    vi /etc/apt/apt.conf.d/02periodic
    vi /etc/apt/listchanges.conf
    unattended-upgrade -d
    /etc/apticron/apticron.conf (xxx again is your address and domain):
    Code:
    # apticron.conf
    #
    # set EMAIL to a space separated list of addresses which will be notified of
    # impending updates
    #
    EMAIL="root"
    
    #
    # Set DIFF_ONLY to "1" to only output the difference of the current run
    # compared to the last run (ie. only new upgrades since the last run). If there
    # are no differences, no output/email will be generated. By default, apticron
    # will output everything that needs to be upgraded.
    #
    # DIFF_ONLY="1"
    
    #
    # Set LISTCHANGES_PROFILE if you would like apticron to invoke apt-listchanges
    # with the --profile option. You should add a corresponding profile to
    # /etc/apt/listchanges.conf
    #
    # LISTCHANGES_PROFILE="apticron"
    
    #
    # From hostname manpage: "Displays  all FQDNs of the machine. This option
    # enumerates all configured network addresses on all configured network inter‐
    # faces, and translates them to DNS domain names. Addresses that cannot be
    # translated (i.e. because they do not have an appro‐ priate  reverse DNS
    # entry) are skipped. Note that different addresses may resolve to the same
    # name, therefore the output may contain duplicate entries. Do not make any
    # assumptions about the order of the output."
    #
    # ALL_FQDNS="1"
    
    #
    # Set SYSTEM if you would like apticron to use something other than the output
    # of "hostname -f" for the system name in the mails it generates. This option
    # overrides the ALL_FQDNS above.
    #
    # SYSTEM="foobar.example.com"
    
    #
    # Set IPADDRESSNUM if you would like to configure the maximal number of IP
    # addresses apticron displays. The default is to display 1 address of each
    # family type (inet, inet6), if available.
    #
    # IPADDRESSNUM="1"
    
    #
    # Set IPADDRESSES to a whitespace separated list of reachable addresses for
    # this system. By default, apticron will try to work these out using the
    # "ip" command
    #
    # IPADDRESSES="192.0.2.1 2001:db8:1:2:3::1"
    
    #
    # Set NOTIFY_HOLDS="0" if you don't want to be notified about new versions of
    # packages on hold in your system. The default behavior is downloading and
    # listing them as any other package.
    #
    # NOTIFY_HOLDS="0"
    
    #
    # Set NOTIFY_NEW="0" if you don't want to be notified about packages which
    # are not installed in your system. Yes, it's possible! There are some issues
    # related to systems which have mixed stable/unstable sources. In these cases
    # apt-get will consider for example that packages with "Priority:
    # required"/"Essential: yes" in unstable but not in stable should be installed,
    # so they will be listed in dist-upgrade output. Please take a look at
    # http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=531002#44
    #
    # NOTIFY_NEW="0"
    
    #
    # Set NOTIFY_NO_UPDATES="0" if you don't want to be notified when there is no
    # new versions. Set to 1 could assure you that apticron works well.
    #
    # NOTIFY_NO_UPDATES="0"
    
    #
    # Set CUSTOM_SUBJECT if you want to replace the default subject used in
    # the notification e-mails. This may help filtering/sorting client-side e-mail.
    # If you want to use internal vars please use single quotes here. Ex:
    # $CUSTOM_SUBJECT='[apticron] $SYSTEM: $NUM_PACKAGES package update(s)'
    #
    # CUSTOM_SUBJECT=""
    
    # Set CUSTOM_NO_UPDATES_SUBJECT if you want to replace the default subject used
    # in the no update notification e-mails. This may help filtering/sorting
    # client-side e-mail.
    # If you want to use internal vars please use single quotes here. Ex:
    # $CUSTOM_NO_UPDATES_SUBJECT='[apticron] $SYSTEM: no updates'
    #
    # CUSTOM_NO_UPDATES_SUBJECT=""
    
    #
    # Set CUSTOM_FROM if you want to replace the default sender by changing the
    # 'From:' field used in the notification e-mails. Your default sender will
    # be something like root@xxx.
    #
    CUSTOM_FROM="xxx@xxx"
    /etc/apt/apt.conf.d/50unattended-upgrades:
    Code:
    // Unattended-Upgrade::Origins-Pattern controls which packages are
    // upgraded.
    //
    // Lines below have the format format is "keyword=value,...".  A
    // package will be upgraded only if the values in its metadata match
    // all the supplied keywords in a line.  (In other words, omitted
    // keywords are wild cards.) The keywords originate from the Release
    // file, but several aliases are accepted.  The accepted keywords are:
    //   a,archive,suite (eg, "stable")
    //   c,component     (eg, "main", "contrib", "non-free")
    //   l,label         (eg, "Debian", "Debian-Security")
    //   o,origin        (eg, "Debian", "Unofficial Multimedia Packages")
    //   n,codename      (eg, "jessie", "jessie-updates")
    //     site          (eg, "http.debian.net")
    // The available values on the system are printed by the command
    // "apt-cache policy", and can be debugged by running
    // "unattended-upgrades -d" and looking at the log file.
    //
    // Within lines unattended-upgrades allows 2 macros whose values are
    // derived from /etc/debian_version:
    //   ${distro_id}            Installed origin.
    //   ${distro_codename}      Installed codename (eg, "jessie")
    Unattended-Upgrade::Origins-Pattern {
            // Codename based matching:
            // This will follow the migration of a release through different
            // archives (e.g. from testing to stable and later oldstable).
    //      "o=Debian,n=jessie";
    //      "o=Debian,n=jessie-updates";
    //      "o=Debian,n=jessie-proposed-updates";
    //      "o=Debian,n=jessie,l=Debian-Security";
    
            // Archive or Suite based matching:
            // Note that this will silently match a different release after
            // migration to the specified archive (e.g. testing becomes the
            // new stable).
    //      "o=Debian,a=stable";
    //      "o=Debian,a=stable-updates";
    //      "o=Debian,a=proposed-updates";
    //      "origin=Debian,codename=${distro_codename},label=Debian-Security";
    
            "o=Debian,a=stable,n=stretch";
            "o=Debian,a=stable-updates,n=stretch-updates";
            "o=Debian,a=stable,n=stretch,l=Debian-Security";
            "o=Proxmox,a=stable,n=stretch";
    };
    
    // List of packages to not update (regexp are supported)
    Unattended-Upgrade::Package-Blacklist {
    //    "vim";
    //    "libc6";
    //    "libc6-dev";
    //    "libc6-i686";
    };
    
    // This option allows you to control if on a unclean dpkg exit
    // unattended-upgrades will automatically run
    //   dpkg --force-confold --configure -a
    // The default is true, to ensure updates keep getting installed
    Unattended-Upgrade::AutoFixInterruptedDpkg "false";
    
    // Split the upgrade into the smallest possible chunks so that
    // they can be interrupted with SIGUSR1. This makes the upgrade
    // a bit slower but it has the benefit that shutdown while a upgrade
    // is running is possible (with a small delay)
    //Unattended-Upgrade::MinimalSteps "true";
    
    // Install all unattended-upgrades when the machine is shuting down
    // instead of doing it in the background while the machine is running
    // This will (obviously) make shutdown slower
    //Unattended-Upgrade::InstallOnShutdown "true";
    
    // Send email to this address for problems or packages upgrades
    // If empty or unset then no email is sent, make sure that you
    // have a working mail setup on your system. A package that provides
    // 'mailx' must be installed. E.g. "user@example.com"
    Unattended-Upgrade::Mail "root";
    
    // Set this value to "true" to get emails only on errors. Default
    // is to always send a mail if Unattended-Upgrade::Mail is set
    //Unattended-Upgrade::MailOnlyOnError "true";
    
    // Do automatic removal of new unused dependencies after the upgrade
    // (equivalent to apt-get autoremove)
    Unattended-Upgrade::Remove-Unused-Dependencies "false";
    
    // Automatically reboot *WITHOUT CONFIRMATION* if
    //  the file /var/run/reboot-required is found after the upgrade
    Unattended-Upgrade::Automatic-Reboot "false";
    
    // Automatically reboot even if there are users currently logged in.
    //Unattended-Upgrade::Automatic-Reboot-WithUsers "true";
    
    // If automatic reboot is enabled and needed, reboot at the specific
    // time instead of immediately
    //  Default: "now"
    //Unattended-Upgrade::Automatic-Reboot-Time "02:00";
    
    // Use apt bandwidth limit feature, this example limits the download
    // speed to 70kb/sec
    //Acquire::http::Dl-Limit "70";
    
    // Enable logging to syslog. Default is False
    // Unattended-Upgrade::SyslogEnable "false";
    
    // Specify syslog facility. Default is daemon
    // Unattended-Upgrade::SyslogFacility "daemon";
    /etc/apt/apt.conf.d/02periodic:
    Code:
    #  Dir "/";
    #  - RootDir for all configuration files
    #
    #  Dir::Cache "var/cache/apt/";
    #  - Set apt package cache directory
    #
    #  Dir::Cache::Archives "archives/";
    #  - Set package archive directory
    #
    APT::Periodic::Enable "1";
    #  - Enable the update/upgrade script (0=disable)
    #
    #  APT::Periodic::BackupArchiveInterval "0";
    #  - Backup after n-days if archive contents changed.(0=disable)
    #
    #  APT::Periodic::BackupLevel "3";
    #  - Backup level.(0=disable), 1 is invalid.
    #
    #  Dir::Cache::Backup "backup/";
    #  - Set periodic package backup directory
    #
    #  APT::Archives::MaxAge "0"; (old, deprecated)
    #  APT::Periodic::MaxAge "0"; (new)
    #  - Set maximum allowed age of a cache package file. If a cache
    #    package file is older it is deleted (0=disable)
    #
    #  APT::Archives::MinAge "2"; (old, deprecated)
    #  APT::Periodic::MinAge "2"; (new)
    #  - Set minimum age of a package file. If a file is younger it
    #    will not be deleted (0=disable). Useful to prevent races
    #    and to keep backups of the packages for emergency.
    #
    #  APT::Archives::MaxSize "0"; (old, deprecated)
    #  APT::Periodic::MaxSize "0"; (new)
    #  - Set maximum size of the cache in MB (0=disable). If the cache
    #    is bigger, cached package files are deleted until the size
    #    requirement is met (the oldest packages will be deleted
    #    first).
    #
    APT::Periodic::Update-Package-Lists "1";
    #  - Do "apt-get update" automatically every n-days (0=disable)
    #   
    APT::Periodic::Download-Upgradeable-Packages "1";
    #  - Do "apt-get upgrade --download-only" every n-days (0=disable)
    #
    #  APT::Periodic::Download-Upgradeable-Packages-Debdelta "1";
    #  - Use debdelta-upgrade to download updates if available (0=disable)
    #
    APT::Periodic::Unattended-Upgrade "1";
    #  - Run the "unattended-upgrade" security upgrade script
    #    every n-days (0=disabled)
    #    Requires the package "unattended-upgrades" and will write
    #    a log in /var/log/unattended-upgrades
    #
    #  APT::Periodic::AutocleanInterval "0";
    #  - Do "apt-get autoclean" every n-days (0=disable)
    #
    APT::Periodic::Verbose "1";
    #  - Send report mail to root
    #      0:  no report             (or null string)
    #      1:  progress report       (actually any string)
    #      2:  + command outputs     (remove -qq, remove 2>/dev/null, add -d)
    #      3:  + trace on
    /etc/apt/listchanges.conf:
    Code:
    [apt]
    frontend=pager
    confirm=false
    email_address=root
    save_seen=/var/lib/apt/listchanges.db
    which=both
     
    killmasta93 likes this.
  9. heutger

    heutger Active Member

    Joined:
    Apr 25, 2018
    Messages:
    331
    Likes Received:
    78
    Another update is coming in! I realized, that there are five kinds of mails: Junk Mail, which is already blocked by postscreen and/or DNSRBL blacklists; Bounce Mail, which is not deliverable, Clean Mail, which has usually scores around 0 (hopefully most) up to 4 (some rare issues), Usual Spam, which has score around 5 to 9, including some False-Positives, Newsletters etc. and really Sure Stupid Spam, which has score from 10 and much more. This stupid spam takes a good value of mail volume against the clean mails meanwhile the usual spam is ignorable. Great, if it's possible to reject such spam, PMG is only able to block this one, but it's like firewall rules, reject means refuse meanwhile block means drop. In Germany last one is illegal as it suppress mails, that's not allowed by law. So I added another layer of spam checking with the possibility to reject. The current drawback, I try to find a solution for (help is welcome): The tracking center doesn't show such mails as well as they won't be included in the statistics. Reason is, that the rejects, which are handled by PMG look for reject: NOQUEUE, but this rejects are milter-rejects: END-OF-MESSAGE. I found the source, where this is handled, but I can't code, so I can't add this mails to handling.

    Steps performed:
    Code:
    apt-get --no-install-recommends install spamass-milter
    mkdir /var/lib/spamass-milter
    vi /etc/default/spamassassin
    vi /lib/systemd/system/spamassassin.service
    systemctl enable spamassassin
    systemctl start spamassassin
    vi /etc/default/spamass-milter
    systemctl restart spamass-milter
    vi /etc/pmg/templates/main.cf.in
    vi /etc/cron.hourly/sa-update
    vi /etc/cron.daily/spamass-milter
    chmod +x /etc/cron.daily/spamass-milter
    /etc/cron.daily/spamass-milter
    chown -R spamass-milter:spamass-milter /var/lib/spamass-milter
    pmgconfig sync --restart 1
    /etc/default/spamassassin (this file is usually included with SpamAssassin installation from the Debian sources, but as the SpamAssassin package is installed and adjusted by PMG, the file is not available. The following file is the original one from the SpamAssassin package with adjusted options):
    Code:
    # /etc/default/spamassassin
    # Duncan Findlay
    
    # WARNING: please read README.spamd before using.
    # There may be security risks.
    
    # If you're using systemd (default for jessie), the ENABLED setting is
    # not used. Instead, enable spamd by issuing:
    # systemctl enable spamassassin.service
    # Change to "1" to enable spamd on systems using sysvinit:
    ENABLED=0
    
    # Options
    # See man spamd for possible options. The -d option is automatically added.
    
    # SpamAssassin uses a preforking model, so be careful! You need to
    # make sure --max-children is not set to anything higher than 5,
    # unless you know what you're doing.
    
    OPTIONS="--username spamass-milter --nouser-config --max-children 5 --helper-home-dir"
    
    # Pid file
    # Where should spamd write its PID to file? If you use the -u or
    # --username option above, this needs to be writable by that user.
    # Otherwise, the init script will not be able to shut spamd down.
    PIDFILE="/var/run/spamd.pid"
    
    # Set nice level of spamd
    #NICE="--nicelevel 15"
    
    # Cronjob
    # Set to anything but 0 to enable the cron job to automatically update
    # spamassassin's rules on a nightly basis
    CRON=0
    /lib/systemd/system/spamassassin.service (the same as above applies for this one; as PMG provides its own SpamAssassin package, this package also includes spamd, but at a different path, so file has been adjusted not to use /usr/sbin/spamd but /usr/bin/spamd, which differs from the original by using /etc/mail/spamassassin instead of /etc/spamassassin as well as some SSL socket adjustments, which - I believe - can be ignored for local installations):
    Code:
    [Unit]
    Description=Perl-based spam filter using text analysis
    After=syslog.target network.target
    
    [Service]
    Type=forking
    PIDFile=/var/run/spamd.pid
    EnvironmentFile=-/etc/default/spamassassin
    ExecStart=/usr/bin/spamd -d --pidfile=/var/run/spamd.pid $OPTIONS
    ExecReload=/bin/kill -HUP $MAINPID
    StandardOutput=null
    StandardError=null
    Restart=always
    
    [Install]
    WantedBy=multi-user.target
    /etc/default/spamass-milter (that's the "magic" of this setup and I would prefer PMG would integrate SpamAssassin that way by default, so this workaround is not required as it for sure takes additional performance for extra scan and some more RAM and a few bytes more harddisk and it's not integrated in tracking center or statistics, but currently it's the only way. I also tried to call pmg-smtp-filter here, but it did not work. The options required are the last two ones, first settings is to reject mails with a score above 10, so the high level spam, second is to hide all the work done by spamass-milter as otherwise the second SpamAssassin run will result in different scores):
    Code:
    # spamass-milt startup defaults
    
    # OPTIONS are passed directly to spamass-milter.
    # man spamass-milter for details
    
    # Non-standard configuration notes:
    # See README.Debian if you use the -x option with sendmail
    # You should not pass the -d option in OPTIONS; use SOCKET for that.
    
    # Default, use the spamass-milter user as the default user, ignore
    # messages from localhost
    OPTIONS="-u spamass-milter -i 127.0.0.1"
    
    # Reject emails with spamassassin scores > 15.
    OPTIONS="${OPTIONS} -r 10"
    
    # Do not modify Subject:, Content-Type: or body.
    OPTIONS="${OPTIONS} -m"
    
    ######################################
    # If /usr/sbin/postfix is executable, the following are set by
    # default. You can override them by uncommenting and changing them
    # here.
    ######################################
    # SOCKET="/var/spool/postfix/spamass/spamass.sock"
    # SOCKETOWNER="postfix:postfix"
    # SOCKETMODE="0660"
    ######################################
    Then it needs to be integrated in Postfix, therefor the following lines have been added to the end of /etc/pmg/templates/main.cf.in:
    Code:
    smtpd_milters = unix:/var/spool/postfix/spamass/spamass.sock
    milter_connect_macros = j {daemon_name} v {if_name} _
    milter_default_action = accept
    I don't post here my new main.cf.in as it currently includes additional DNSRBL blacklists, I test. I will post here, once I've done with my tests.

    /etc/cron.hourly/sa-update (script needs to be adjusted to reload spamassassin on new rulesets too):
    Code:
    #!/bin/sh
    
    # schaal @it
    #
    # Simple script to update SpamAssassin
    
    SYSLOG_TAG=sa-update
    
    compile=0
    
    logger -d -t $SYSLOG_TAG "Start SA-Update"
    
    sa-update --nogpg
    retval="$?"
    if [ $retval -eq 0 ]; then compile=1; fi
    
    
    sa-update --nogpg --channel updates.spamassassin.org
    retval="$?"
    if [ $retval -eq 0 ]; then compile=1; fi
    
    sa-update --nogpg --channel sa.zmi.at
    retval="$?"
    if [ $retval -eq 0 ]; then compile=1; fi
    
    sa-update --nogpg --channel sa.schaal-it.net
    retval="$?"
    if [ $retval -eq 0 ]; then compile=1; fi
    
    sa-update --nogpg --channel spamassassin.heinlein-support.de
    retval="$?"
    if [ $retval -eq 0 ]; then compile=1; fi
    
    if [ $compile -eq 1 ]; then
        logger -d -t $SYSLOG_TAG "SA-Update found"
        sa-compile --quiet 2>/dev/null
        systemctl restart pmg-smtp-filter
        systemctl restart spamassassin
    else
        logger -d -t $SYSLOG_TAG "No SA-Update found"
    fi
    /etc/cron.daily/spamass-milter (AWL, Bayes etc. from SpamAssassin as been run by pmg-smtp-filter needs to be synced with SpamAssassin been run by spamass-milter, especially as manual sa-learn calls will be performed against SpamAassassin as been run by pmg-smtp-filter, it's required to run a daily job to copy the files from /root/.spamassassin to /var/lib/spamass-milter/.spamassasin):
    Code:
    #!/bin/sh
    
    cp -R /root/.spamassassin/* /var/lib/spamass-milter/.spamassassin/.
     
    #9 heutger, Jun 26, 2018
    Last edited: Oct 23, 2018
    IEM and killmasta93 like this.
  10. killmasta93

    killmasta93 Member

    Joined:
    Aug 13, 2017
    Messages:
    255
    Likes Received:
    8
    really good stuff i still have not tried the Bayes, The dcc,pyzor and unoficial Clam Av amazing work. and the only thing about Bayes is that spam that comes into the company rather then your inbox somewhat complicated to manage
     
  11. Sommer

    Sommer Member

    Joined:
    Jun 7, 2018
    Messages:
    45
    Likes Received:
    5
    Really nice job @heutger
    I will wait for @proxmox team to implement all your optimizations and release it on the next version :D
     
  12. heutger

    heutger Active Member

    Joined:
    Apr 25, 2018
    Messages:
    331
    Likes Received:
    78
    Thanks. I believe, bayes will have much impact on the quality of the spam filter. For sure, it's much harder to manage, if the mail server itself is not the gateway learning spam or ham. There are great explanations on how to use like https://thomas-leister.de/mailserver-unter-ubuntu-16.04/ (however, with new setup they use rspamd, however I'm currently not too much fan of rspamd anymore), which are really great invoking sa-learn on movements from spam to ham or vice versa. However, I have my cs-learn script above to learn in my private setup with a Plesk server behind (you may have wondered, why cs-learn, it's standing for cloud server as my private server has the name cs for cloud server), but I currently work for the company on a solution on how to learn spam from Exchange if team should forward spam mails as attachment to a special mailbox (and maybe the same with ham) looking at https://kuther.net/howtos/howto-receive-mail-and-save-attachment-fetchmail-procmail-and-metamail and https://dump.4network.org/2015/06/1...achment-with-fetchmail-procmail-and-uudeview/ how that's working, but need to check, if I can use it with two different accounts, one for spam and one for ham. In addition I'm looking for a new ticketing system and also will then try to have a button invoking an API call or sth. similar to learn spam or ham from tickets to PMG, then it would be great and flying.
     
  13. heutger

    heutger Active Member

    Joined:
    Apr 25, 2018
    Messages:
    331
    Likes Received:
    78
    Thanks! Would be great, however, if I could provide a wishlist (and also provided a bit like that), first of all I would be very happy, if PMG would changed from SpamAssassin content_filter integration to milter as this would make things much easier. Firewall and OpenVPN would only be required for outside of a DMZ placed systems, which are not recommended, my private one is and the test as well, however, productive setup will be inside, backup could be improved, but additional backup script is not required if inside a DMZ on a virtual host and the node performs a valid backup, but all the options of rulesets, blacklists, signatures etc. could be optional choosable and adjustable, some like DCC could be shipped with an easy setup script (as DCC is not allowed to be shipped without license) or maybe include some licenses (with special discounted agreements with the vendors) with subscriptions to have more features instead of "only" productive repository and support options.

    However, feel free to use my adjustments and also Proxmox can use them as they like.

    My new mission currently is to get bayes working (by getting enough spam and ham into the database) and to provide additional ways to feed sa-learn. I currently test some last blacklists and will also recheck heinlein, schaal and zmi rulesets as recently they had less impact on spam scores for spam mails but much impact on spam score for ham false-positives. I will keep you posted.
     
  14. heutger

    heutger Active Member

    Joined:
    Apr 25, 2018
    Messages:
    331
    Likes Received:
    78
    killmasta93 likes this.
  15. killmasta93

    killmasta93 Member

    Joined:
    Aug 13, 2017
    Messages:
    255
    Likes Received:
    8
    So after running your guide i would have to configure this?

    Code:
    Set in /etc/clamav-unofficial-sigs/master.conf
    yararulesproject_enabled="no"
    enable_yararules="no"
    And delete *.yar and *.yara from /var/lib/clamav/
     
  16. killmasta93

    killmasta93 Member

    Joined:
    Aug 13, 2017
    Messages:
    255
    Likes Received:
    8
    as for the Bayes on has to put email that comes in for all of the company to a certain folder which updates though IMAP? Thats what i had before on scrolloutf1 the only issue if that it was very manually
     
  17. heutger

    heutger Active Member

    Joined:
    Apr 25, 2018
    Messages:
    331
    Likes Received:
    78
    That's right. Alternative it *should* also work to apply https://github.com/extremeshok/clamav-unofficial-sigs/pull/193 but you may encounter https://github.com/extremeshok/clamav-unofficial-sigs/issues/204, so I currently prefer to disable yara at all. That's also why I prefer to use the script, there is still some support, the sanesecurity website looks a bit of unsupported. I would prefer, if it would be easier to change/add antivirus scanner to PMG and be much more open in decision, Sophos or Avira would be my preferred candidates. Sophos we already use wide spread (however, it would be great to have another one, as Sophos is already running as endpoint protection, so there would be no advantage to scan in the mail flow), so Avira would be great as well.
     
    killmasta93 likes this.
  18. heutger

    heutger Active Member

    Joined:
    Apr 25, 2018
    Messages:
    331
    Likes Received:
    78
    I already provided a script above for Plesk, where Mails are sorted into Spam or Ham for one mailbox (could be improved for more mailboxes) and can be used to feed bayes database. However, all the users then are required to use any webmail provided by Plesk or IMAP access to the mailboxes as been provided by Plesk and always use Spam folder for confirmed Spam or (most interesting and important) undetected Spam (so false-negatives) as well as Ham folder for false-positives. It's also possible to configure Plesk to run SpamAssassin by itself and write RegEx to automatically move Spam to Spam (however, also false-positives will then be sorted there). It's also possible to use and adjust the setup from Thomas Leister to perform sa-learn on movement to Spam or Ham, just need to be adjusted, that this sa-learn is not performed locally on the Plesk server but on the PMG server. Maybe Proxmox will provide in future a Proxmox Mail Archive with direct IMAP access (without option to delete => would perform an archive) and SMTP AUTH access to send mails and integrate it with PMG, then all this adjustments may already been done and make life easier for many subscribers.

    As written above, I currently try to do the same with Exchange (may be performed similar with every mail server setup) and global mailboxes spam and ham, employees need then to forward (as attachment) spam to spam and false-positives to ham, for a minimum of time to reach the 200 spams and 200 hams, lateron it could only be performed on annoying spam, once bayes is running, for sure would be better to continue to forward spam and ham as the milter-reject scores could then be adjusted better and "safe spam" could be rejected already on lower scores. However, beside being father and husband, I currently prepare a training, which take all my spare time.
     
  19. killmasta93

    killmasta93 Member

    Joined:
    Aug 13, 2017
    Messages:
    255
    Likes Received:
    8
    Gotcha did not know so far its been great i would hope one day to incorporate the free Bitdefender Antivirus.
     
  20. killmasta93

    killmasta93 Member

    Joined:
    Aug 13, 2017
    Messages:
    255
    Likes Received:
    8
    Forgot to ask has to be IMAP right?
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice