[TUTORIAL] Advancing Proxmox Mail Gateway (especially Spam and Virus Detection)

Discussion in 'Mail Gateway: Installation and configuration' started by heutger, May 29, 2018.

  1. heutger

    heutger Active Member

    Joined:
    Apr 25, 2018
    Messages:
    462
    Likes Received:
    107
    Getting somehow from all lists, I would expect a DNS problem, also stating "... record not found" looks a bit like your DNS server is not able to resolve the names. Try nslookup on any domain, you're sure, it should resolve. Also check the name servers stated in the response, maybe they are unusable or down. My recommendation is to set up your own DNS server on PMG directly, if you have nothing in your internal zone already, e.g. unbound as stated above. For private I'm doing so, for commercial, we use our centralized own DNS server on the Firewall device, same for the time server.
     
  2. heutger

    heutger Active Member

    Joined:
    Apr 25, 2018
    Messages:
    462
    Likes Received:
    107
    Not yet, will get it sorted out, but currently I just copy my configuration into this post. I use(d) the code element, so it should work out to copy directly from here. Setting up additional systems for testing purposes, I also used the code here for copying, it worked out well yet for me. However, you're welcome to create a GitHub repo for the adjustments or use ansible playbook. Would be happy to get mentioned, but currently I have no spare time to do it by myself (little 14 month old daughter...).
     
  3. killmasta93

    killmasta93 Member

    Joined:
    Aug 13, 2017
    Messages:
    348
    Likes Received:
    10
    Thanks for the reply, your right i was looking into it because on the other boxes of proxmox did not show then i realized the DNS which is running on a windows server was turned off because were doing a migration, during that week saw some weird cron emails from proxmox which went away eventually
     
  4. Petr Stepanek

    Petr Stepanek New Member

    Joined:
    Nov 1, 2018
    Messages:
    18
    Likes Received:
    2
    Did anybody try to use new GeoIP2 format for RelayCountry? GeoIP is deprecated but I wasn't succesfull with new "mmdb" format. I have tried to download "GeoIP2-Country.mmdb" and use "country_db_type GeoIP2" atribut in "custom.cf" file but with no luck. I received "RelayCountry: not found" If I tried to run:
    Code:
    spamassassin -D --lint
    I have tried install CPAN module "GeoIP2" also but the result is same.

    Btw. thank you heutger for great thread. I found lot of really great information here! Could I ask you, why you are not using DMARC?
     
  5. heutger

    heutger Active Member

    Joined:
    Apr 25, 2018
    Messages:
    462
    Likes Received:
    107
    Any help on GeoIP2 is welcome. Seems, that GeoIP2 is very fresh includes with SA, so similar to hashbl, there may be still some problems.

    Regarding DMARC, it’s SPF and DKIM combined. Honestly, I see many new technologies currently coming up, which are not adopted well, sometimes because they are misused for wrong ideas or they are adopted only by a few ones. So, e.g. DANE was a great idea, but as it depends on the worse designed DNSSEC and got misused to promote own signed ssl certs should be same level as ones done by CAs, it went the wrong direction (same for DV and mass DV via Let’s Encrypt and deprecate EV in e.g. Chrome and Firefox (that’s why I don’t use any of them any more), I warned of damaging security in the internet and it came as expected, most phishing sites are now running with DV certs leaving the normal user alone and loosing money because they are not able to distinguish any more between valid sites and invalid ones). Same for SPF, idea was great, but it forgot about cloud services coming up same time. As of such cloud services using a mail provider you never know, if they would change their IP blocks, so you won’t be able to set your SPF record well or may have a wrong one, as some set SPF long time ago, forgot about it and so never updated. Same for DKIM, many won’t have control on outgoing mail signing and also don’t understand on how to set the records or keep them updated. So I saw most messages using DKIM are spam mails, as they know, DKIM is a signal for some antispam solutions for legit mails, as spammers success depends on getting messages through, they are the ones, who adopt such solutions meanwhile the legit ones won’t or won’t be able to do so. That’s why I don’t use this techniques at all.
     
  6. Petr Stepanek

    Petr Stepanek New Member

    Joined:
    Nov 1, 2018
    Messages:
    18
    Likes Received:
    2
    Yes, it's true that SPF with cloud services is little bit tricky but good cloud providers should maintain their SPF record and you only include this parameter to your TXT record.

    In Czech Republic are some freemail providers who reject emails without DKIM sign (if you send more than some amount in one day), it's the reason why I had to implement DKIM. And I like DMARC because I receive feedback who tried to send email from my domain and I don't need to implement any special technology on my side (only TXT record in DNS). But you are right, that lot of email servers implement it in wrong way so I use it only for spam scoring not for rejecting.
     
  7. heutger

    heutger Active Member

    Joined:
    Apr 25, 2018
    Messages:
    462
    Likes Received:
    107
    Also for spam scoring I won’t use, otherwise all this GMail, Outlook.com/Hotmail, Yahoo spam would outrate my content rules as if I remember right, almost all of them use DKIM. Same for all this med, software and other (attendee) lists, I get spam about, I saw almost all of them DKIM signed, meanwhile my customers, business partners, ... don’t use DKIM at all. On my private installation I also can’t use, I asked about, but my Hosted Exchange provider, one of the largest in Germany, doesn’t support DKIM and I can’t redirect outgoing mail through my gateway.
     
  8. Petr Stepanek

    Petr Stepanek New Member

    Joined:
    Nov 1, 2018
    Messages:
    18
    Likes Received:
    2
    I agree that DKIM is more suitable for inhouse solution.
     
  9. Petr Stepanek

    Petr Stepanek New Member

    Joined:
    Nov 1, 2018
    Messages:
    18
    Likes Received:
    2
    After a lot of tests and RelayCountry.pm script debugging I found final solution for GeoIP2. It is really easy but lack of documentations makes it a lot more demanding.

    I added GeoIP2 CPAN module:
    Code:
    perl -MCPAN -e shell
    install GeoIP2::Database::Reader
    Edited `custom.cf`:
    Code:
    nano /etc/mail/spamassassin/custom.cf
    And enabled GeoIP2:
    Code:
    country_db_type GeoIP2
    country_db_path /usr/share/GeoIP/GeoLite2-Country.mmdb
    After all it is better to test it:
    Code:
    spamassassin -D --lint 2>&1 | grep -i --color relaycountry
    Does somebody have idea how to show RelayCountry header in emails? It looks like SA implementation in PMG is not able to add header at all.
     
    #69 Petr Stepanek, Dec 30, 2018
    Last edited: Dec 31, 2018
    heutger likes this.
  10. Petr Stepanek

    Petr Stepanek New Member

    Joined:
    Nov 1, 2018
    Messages:
    18
    Likes Received:
    2
    I forgot to add script for automatic db download:
    Code:
    #!/bin/sh
    
    GEOIPDIR="/usr/share/GeoIP"
    
    cd $GEOIPDIR
    wget http://geolite.maxmind.com/download/geoip/database/GeoLite2-Country.tar.gz
    
    
    tar zxvf GeoLite2-Country.tar.gz --strip-components=1 --wildcards GeoLite2-Country*/GeoLite2-Country.mmdb
    rm GeoLite2-Country.tar.gz
     
    heutger likes this.
  11. heutger

    heutger Active Member

    Joined:
    Apr 25, 2018
    Messages:
    462
    Likes Received:
    107
    PMG-SMTP-Filter (SpamAssassin and more) add header information to mails. Please double-check.
     
  12. Petr Stepanek

    Petr Stepanek New Member

    Joined:
    Nov 1, 2018
    Messages:
    18
    Likes Received:
    2
    I have tried to add
    to
    but with no luck. I receive
    If I run
    but I don't see anything in received email.
     
  13. Petr Stepanek

    Petr Stepanek New Member

    Joined:
    Nov 1, 2018
    Messages:
    18
    Likes Received:
    2
    Btw. same behavior for fresh PMG install.
     
  14. heutger

    heutger Active Member

    Joined:
    Apr 25, 2018
    Messages:
    462
    Likes Received:
    107
    I'm sorry, but you may then have any kind of misconfiguration. My headers look like this (for the notify of your response to this thread):

    Code:
    X-Spam-Status: No, score=-2.0 required=5.0 tests=AWL,BAYES_00,HTML_MESSAGE,
     RCVD_IN_DNSWL_NONE,RELAYCOUNTRY_GOOD,SPF_PASS,T_REMOTE_IMAGE
     autolearn=ham autolearn_force=no version=3.4.2
    X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on xx.xxxx.xxx
    X-SPAM-LEVEL: Spam detection results:  0
     AWL 0.376 Adjusted score from AWL reputation of From: address
     BAYES_00                 -1.9 Bayes spam probability is 0 to 1%
     HTML_MESSAGE            0.001 HTML included in message
     RCVD_IN_DNSWL_NONE     -0.0001 Sender listed at http://www.dnswl.org/, no trust
     RELAYCOUNTRY_GOOD        -0.5 First untrusted GW is DE, AT or CH
     SPF_PASS               -0.001 SPF: sender matches SPF record
     T_REMOTE_IMAGE           0.01 Message contains an external image
     
  15. Petr Stepanek

    Petr Stepanek New Member

    Joined:
    Nov 1, 2018
    Messages:
    18
    Likes Received:
    2
    Thank you for your time and that you are trying to help me.

    After some tests I found that the problem was in my misunderstanding. I thought that if I add
    to "custom.cf" I will see "X-SPAM_Relay-Country" in email headers but I will see only "RELAYCOUNTRY_GOOD" or "RELAYCOUNTRY_BAD" if it meet right policy.

    So "add_header all" rule will be never shown in email header?

    Edit: And one really important thing is that I had to restart "pmg-smtp-filter" service everytime when I edited "custom.cf" file. Problem was that I tested every change by "spamassassin -D" command which show every change in "custom.cf" immediately, instead of "pmg-smtp-filter" service which need to be restarted.
     
    #75 Petr Stepanek, Jan 1, 2019
    Last edited: Jan 1, 2019
  16. heutger

    heutger Active Member

    Joined:
    Apr 25, 2018
    Messages:
    462
    Likes Received:
    107
    That’s right, if you use my milter setup, you also need to restart spamassassin service as well. However, most of the headers are set already, so maybe, you won’t see any difference. Also custom.cf may be the wrong direction, if settings are already done in any .pre file. However, if spamassassin -D shows the output meanwhile PMG-SMTP-Filter doesn’t, it seems to be only the requirement to reload.
     
  17. heutger

    heutger Active Member

    Joined:
    Apr 25, 2018
    Messages:
    462
    Likes Received:
    107
    Just a small update: I consider to switch from ClamAV to Avast. As you may read in other threads, I tried to get other antivirus software as Bitdefender, ESET, Kaspersk, F-Secure, Avira and/or Sophos integrated, but they are either too expensive, not daemonized or not available for integration with PMG. So I stay with Avast now and took sure to be able to offer Avast licenses as well. The setup with which Avast came was about to scan with both, ClamAV and Avast. But as ClamAVs detection rates are worse without additional signatures and the additional signatures in my tests brought more false-positives than I would accept, I consider to switch totally. So I also won’t support any ClamAV adjustments any more. I will keep them on my private installation, commercial will be redone soon and I will then also purchase a license of PMG therefor, so it also won’t be good any more for „early tests“ of new versions, so I may see some upcoming problems and try to solve them, but I can’t guarantee therefor. If you encounter any problems, please leave a note here and I will try to figure out.
     
    #77 heutger, Jan 2, 2019
    Last edited: Jan 15, 2019
    killmasta93 likes this.
  18. heutger

    heutger Active Member

    Joined:
    Apr 25, 2018
    Messages:
    462
    Likes Received:
    107
    Here we go! I now switched from ClamAV to Avast. See below the steps I performed:

    Code:
    wget https://files.avast.com/files/resellers/linux/avast.gpg
    echo "deb http://deb.avast.com/lin/repo debian release" >> /etc/apt/sources.list
    apt-key add avast.gpg
    apt-get update
    apt-get install avast
    vi /etc/avast/license.avastlic
    /etc/init.d/avast start
    pmgsh set /config/admin --avast 1
    pmgsh set /config/admin --clamav 0
    No more file contents here, as for sure, I won't post my license. :D
     
    killmasta93 likes this.
  19. killmasta93

    killmasta93 Member

    Joined:
    Aug 13, 2017
    Messages:
    348
    Likes Received:
    10
    any ideas how much per license it cost for avast? did you also uninstalled ClamAV?
    are the license a specific ones? or any avast license will do?
     
  20. heutger

    heutger Active Member

    Joined:
    Apr 25, 2018
    Messages:
    462
    Likes Received:
    107
    Current pricing is something about US$150 (original pricing is in €) per server per year, with good discounts on multiple years and multiple servers. Core Security for Linux is enough and best fitting license. Via partners there are additional discounts possible. You can PM me for details.
     
    killmasta93 likes this.
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice