[TUTORIAL] Advancing Proxmox Mail Gateway (especially Spam and Virus Detection)

[velocity08]

Thank you this makes perfect sense to me now.

Let me see if I can help find a better way to maintain the code and updates.

Normally I like to read and write all items to be inline to keep the work flow together.

The explanations supplied are good but would make more sense if all the code was in 1 code block to be copied and pasted into 1 file.

This would save time trying to find the code snippet changes to be inserted.

Then the the code snippets to be listed separately and explanation of each code snippet inline.

This way the workflow stays the same for each section with all the code as it should be for each file that needs editing.

Personally I like to use CODA on local machine to manage my files then synchronise changes to destination server, it’s a little like a GIT repository just stored on my local machine.

Let me have a think of the easiest way to manage for you and will update ASAP.

Appreciate you taking the time to clearly explain.

“”Cheers
G

 

[velocity08]

Hi Heutger

can you please clarify in a little more detail what is meant by the following statement:

adding country filters as well as additional backlist scores, xxx, xxx24 and xxxuri should be your own invaluement subscription domains
​

""Cheers
G

 

[velocity08]

Hay Team

seeing this one in the spam-assassin debug output.

Jun 9 08:30:34.795 [7269] dbg: diag: [...] module not installed: Digest::SHA1 ('require' failed)​

is this valid or depreciated issue?

""Cheers
G

 

[heutger]

Thank you this makes perfect sense to me now.

Let me see if I can help find a better way to maintain the code and updates.

Normally I like to read and write all items to be inline to keep the work flow together.

The explanations supplied are good but would make more sense if all the code was in 1 code block to be copied and pasted into 1 file.

This would save time trying to find the code snippet changes to be inserted.

Then the the code snippets to be listed separately and explanation of each code snippet inline.

This way the workflow stays the same for each section with all the code as it should be for each file that needs editing.

Personally I like to use CODA on local machine to manage my files then synchronise changes to destination server, it’s a little like a GIT repository just stored on my local machine.

Let me have a think of the easiest way to manage for you and will update ASAP.

Appreciate you taking the time to clearly explain.

“”Cheers
G

Mainly all code for 1 file is in 1 block, but it's many files to adjust.

I already set up a Git Repository yet, but be unsure on how to fill. So changes are "easy" as they are only files to be stored there, but what's about scripts need to be installed, downloaded, settings been done etc.? So I'm afraid, there is some scripting work required. Also I wonder, if I could zip, tar.gz or whatever all necessary files and can extract them to my Git Repository? I never worked with Git really, so I know, it exists, however, it's usually for coding, not for adjustments.

 

[heutger]

Hi Heutger

can you please clarify in a little more detail what is meant by the following statement:

adding country filters as well as additional backlist scores, xxx, xxx24 and xxxuri should be your own invaluement subscription domains
​

""Cheers
G

I added the country filters feature of SpamAssassin by uncommenting the lines, using it, installing GeoIP database to maintain the country data and added some blacklists, I usually won't see blocking but hitting to provide additional scores in SpamAssassin. Meanwhile I now use all the lists as hitting lists, I should also add the other lists to SpamAssassin, however, the probability that they won't hit together with another list and may provide an urgent score is relative low, so need to consider.

I also used three extra lists here and this ones are paid ones, so I can't provide you with the list names, once it won't work, if your IPs are not whitelisted by the service provider as you subscribed to their paid lists, second I try to protect his firewall from trying ones just copying my list set, so you should sign up for his lists and provide their the real domain name or leave them out.

 

[heutger]

Hay Team

seeing this one in the spam-assassin debug output.

Jun 9 08:30:34.795 [7269] dbg: diag: [...] module not installed: Digest::SHA1 ('require' failed)​

is this valid or depreciated issue?

""Cheers
G

Looks like anything is using the deprecated SHA1 signatures. Do you have additional information on what is throwing this debug line?

 

[velocity08]

Morning Heutger

When running the /etc/cron.hourly/sa-update script this is part of the debug line output.

Doing a little googling just saw some solutions advising on how to install it for perl and some other posts advising its depreciated.

So not looking like they advising why its needed just how to install and fix the errors.

Let me know if there’s something you would like me to run to generate more information?

“”Cheers
G

 

[velocity08]

Mainly all code for 1 file is in 1 block, but it's many files to adjust.


I already set up a Git Repository yet, but be unsure on how to fill. So changes are "easy" as they are only files to be stored there, but what's about scripts need to be installed, downloaded, settings been done etc.? So I'm afraid, there is some scripting work required. Also I wonder, if I could zip, tar.gz or whatever all necessary files and can extract them to my Git Repository? I never worked with Git really, so I know, it exists, however, it's usually for coding, not for adjustments.

Morning heutger

For ease of updates it would be best to just provide changed files for download.
Original source config files are kept up to date with changes in GitHub which can be downloaded in a zip.
The install instructions can be put into a simple script to install all the required packages in 1 go.

I.e.

Install core packages

#apt-get install bcc make unzip vim pyzor re2c

Install Optional

... Clam AV
... GeoIP
... etc

Configuration file downloads.

Link to most recent downloadable config file to be placed into directories can be stored in GitHub

Config file locations ...

Copy/ paste config files in locations below:

/etc/...


For me the hardest part was following the inserted code that where config into files that had a lot of data and i needed to go through each config file line by line find and make any changes.

This turned a quick task into hours of checking each line of text and comparing.

Another option is to highlight in a different colour the added/ changed items in the CODE block, this would speed up the process immensely when reading the CODE block edits for config files.

Or if colours are not an option set

######### before

New or changed code here

######### after

The changed/ updated code so its easier to find.

Any hoo just some suggestions on how it can make the tutorial easier to follow and for any updates to config files.

Because this is in a forum there are going to be loads of comments and changes as the post grows which also makes it hard to follow and keep up to date, i would suggest moving this into a BLOG post in WP or even just in GitHub as it will make changes easier to maintain.

Even if none of this is scripted its easier to maintain in a BLOG or GitHub.

O forgot to mention both WP and GitHub have a form of version control so its easier to go back and see previous versions of config files etc to review changes over time.

Just my 2 cents.

“”Cheers
G

 

[heutger]

An update again! As playing around to improve my encryption, I also decided to have a quick view on my OpenVPN configuration and if it's still up to date. The big issue with my OpenVPN setup is, that the script based on is fine and always get improved, however, once configured, it does not provide updating an existing installation (just be able to remove and reinstall, already opened a feature request). So what did I change and merged to configuration in the first posts:

On server.conf I changed tls-auth to the more modern tls-crypt (and removed direction as not required anymore), I changed auth from SHA256 to SHA384, I changed cipher from AES-128-CBC to the more modern and robust perfect forward secrecy AES-256-GCM, I changed the tls-cipher from TLS-DHE-RSA-WITH-AES-128-GCM-SHA256 to the more modern and faster TLS-ECDHE-RSA-WITH-AES-256-GCM-SHA384 and I changed .

On client.ovpn for sure it's required to do the same to keep the connection been able to be established and need to be rolled out again to all clients, most important, my code snippet above does not show the extra lines with the key, cert, recent tls-auth and now tls-crypt key, so you need to find this extra lines key-direction with value 1 and remove this line as well as change tls-auth to tls-crypt.

Alternative you could remove and reinstall the OpenVPN settings via the script, you could then also change the keys to 4096 bit keys and DH key to 4096 bit key, something I now didn't perform, maybe in future. You may also choose EC keys, however, the iOS client currently does not support EC, so I stay with RSA and for the time being I stay with 2048 bit keys.

If someone likes to followup my selections on the script run recently, you can find in my first post and there all this adjustments came from (a feature request post) here: #31

Addendum: I changed my setting a bit more. I now allow with duplicate-cn a client to connect on multiple machines (with same cert/key combination) and also therefore allow now the full 10.8.0.0/24 network to connect to SSH and Port 8006 once connected via VPN. Config has been updated on the pages before.

Addendum 2: I also changed DH to ECDH, so no need to upgrade DH key, as I only use ECDH with 384 bit curve.

 

[heutger]

Morning Heutger

When running the /etc/cron.hourly/sa-update script this is part of the debug line output.

Doing a little googling just saw some solutions advising on how to install it for perl and some other posts advising its depreciated.

So not looking like they advising why its needed just how to install and fix the errors.

Let me know if there’s something you would like me to run to generate more information?

“”Cheers
G

Strange, don't see such behavior here on my installations. However, you may comment out all sa-update lists and then uncomment list by list to find which one arise the error.

 

[heutger]

Morning heutger

For ease of updates it would be best to just provide changed files for download.
Original source config files are kept up to date with changes in GitHub which can be downloaded in a zip.
The install instructions can be put into a simple script to install all the required packages in 1 go.

I.e.

Install core packages

#apt-get install bcc make unzip vim pyzor re2c

Install Optional

... Clam AV
... GeoIP
... etc

Configuration file downloads.

Link to most recent downloadable config file to be placed into directories can be stored in GitHub

Config file locations ...

Copy/ paste config files in locations below:

/etc/...


For me the hardest part was following the inserted code that where config into files that had a lot of data and i needed to go through each config file line by line find and make any changes.

This turned a quick task into hours of checking each line of text and comparing.

Another option is to highlight in a different colour the added/ changed items in the CODE block, this would speed up the process immensely when reading the CODE block edits for config files.

Or if colours are not an option set

######### before

New or changed code here

######### after

The changed/ updated code so its easier to find.

Any hoo just some suggestions on how it can make the tutorial easier to follow and for any updates to config files.

Because this is in a forum there are going to be loads of comments and changes as the post grows which also makes it hard to follow and keep up to date, i would suggest moving this into a BLOG post in WP or even just in GitHub as it will make changes easier to maintain.

Even if none of this is scripted its easier to maintain in a BLOG or GitHub.

O forgot to mention both WP and GitHub have a form of version control so its easier to go back and see previous versions of config files etc to review changes over time.

Just my 2 cents.

“”Cheers
G

Sounds good. For my rspamd adjustments I used a Wordpress article as my previous one has been removed here, however, I found that showing code lines/changes look too bad on my Wordpress theme, so I would not continue to use Wordpress on such topics. However, as I just documented my tests with rspamd but stopped any further investigation, I won't bring that to Github.

Would need help on a script, as I could remove all adjustments for ClamAV on my Github repository, so many xxx options will be gone, just on some rare topics like OpenVPN I can't provide my real configuration files but if there would be a script to ask some questions and then hook installation of e.g. OpenVPN would be fine.

For the first, yes, I could provide all my files without folder structure and add explanation on where to place and extra explanation e.g. on Github entry page and comment changes on releases.

I'm unsure, when I'm able to start this project, but will do soon as seems to be a good idea, as I also see this thread increasing from time to time and being unable to handle all my adjustments easy.

 

[Thomas k.]

Here we go! I now switched from ClamAV to Avast. See below the steps I performed:

wget files.avast.com/files/resellers/linux/avast.gpg
echo "deb deb.avast.com/lin/repo debian release" >> /etc/apt/sources.list
apt-key add avast.gpg
apt-get update
apt-get install avast
vi /etc/avast/license.avastlic
/etc/init.d/avast start
pmgsh set /config/admin --avast 1
pmgsh set /config/admin --clamav 0

No more file contents here, as for sure, I won't post my license.

Hi,

with Proxmox Mailgateway 5.2 you need to use the "debian-stretch" repo.

To bad that avast changed the path to the executable from "/bin/scan" to "/usr/bin/scan". Proxmox Team needs to adjust the path to the executable here:

pmg-smtp-filter[1026]: 4C0A425CFE8F1E329AF: can't exec avast scan: No such file or directory : ERROR at /usr/share/perl5/PMG/Utils.pm line 462.

Can be fixed with a symlink in the meantime:

ln -s /usr/bin/scan /bin/scan

Regards,
Thomas

 

[Kenny Huynh]

ifplugin Mail::SpamAssassin:lugin::RelayCountry add_header all Relay-Country _RELAYCOUNTRY_ header RELAYCOUNTRY_BAD X-Relay-Countries =~ /(CN|RU|UA|RO|VN)/ describe RELAYCOUNTRY_BAD Relayed through spammy country at some point score RELAYCOUNTRY_BAD 2.0 header RELAYCOUNTRY_GOOD X-Relay-Countries =~ /^(DE|AT|CH)/ describe RELAYCOUNTRY_GOOD First untrusted GW is DE, AT or CH score RELAYCOUNTRY_GOOD -0.5 endif # Mail::SpamAssassin:lugin::RelayCountry

Hello Heutger,

I have deleted VN in this config but all mail send from VN are mark the score RELAYCONTRY_BAD 2.0. Do I mistake some config?

Regards.

 

[heutger]

Hi,

with Proxmox Mailgateway 5.2 you need to use the "debian-stretch" repo.

To bad that avast changed the path to the executable from "/bin/scan" to "/usr/bin/scan". Proxmox Team needs to adjust the path to the executable here:

pmg-smtp-filter[1026]: 4C0A425CFE8F1E329AF: can't exec avast scan: No such file or directory : ERROR at /usr/share/perl5/PMG/Utils.pm line 462.

Can be fixed with a symlink in the meantime:

ln -s /usr/bin/scan /bin/scan

Regards,
Thomas

Hi,

thanks for your input. Can you provide the correct repo (what need to be adjusted)?

In addition, thanks for feedback on the changed path. Can you provide your information to the PMG Development Mailing list, so the code get updated or best provide the changed file to the PMG Development Mailing list to be integrated? Alternative (but less good) can you open a bug report at PMG bug tracker?

Regards,
Christian

 

[heutger]

Hello Heutger,

I have deleted VN in this config but all mail send from VN are mark the score RELAYCONTRY_BAD 2.0. Do I mistake some config?

Regards.

Hi,

did you reload pmg-smtp-filter after your change and if used also the miltered SpamAssassin? Otherwise the changes won't get adopted.

Regards,
Christian

 

[Kenny Huynh]

Hello,

Yes, I have reloaded pmg-smtp-filter and I do not install spamass-milter. I have also changed score for DCC and Pyzor in /usr/share/spamassassin/50_scores.cf and reload pmg-smtp-filter but It not working.

Please help me (

 

[heutger]

Hello,

Yes, I have reloaded pmg-smtp-filter and I do not install spamass-milter. I have also changed score for DCC and Pyzor in /usr/share/spamassassin/50_scores.cf and reload pmg-smtp-filter but It not working.

Please help me (

Hi,

so you say the value are not adopted? Honestly, it's confusing, but without being able to see the system, I can't help any further. Especially, if the adjusted scores are not taken, they may have also been set elsewhere and override from there, but if you removed VN from the bad countries, which you took from my tutorial on how to activate and choose the countries, it makes really no sense, why it's not adopted. A hard variant would be to try to reboot PMG, maybe any other process doesn't allow to take the values, otherwise, it could only be solved with a view on the system.

Regards,
Christian

 

[heutger]

Hi,

I'm now a bit off, but afterwards I plan to setup a new PMG on a VM on my Notebook and recreate all the changes I did and will then setup a GitHub repository with that. I will leave off the ClamAV adjustments but I will do all the rest. I'm not yet sure on some topics, so looking for your advice here:

1. I'm unsure, if I should do with example.com and set up a local DNS to be able to work and play with
2. Or if I should register a test domain to do with
3. Or should use a subdomain of an existing domain of my.

In addition, I run two setups currently, the more loser commercial test setup and the tighter private setup. So which one should I cover there. I for sure could document the differences, but need to follow "one line".

If doing with any domain above (beside one of my own) I could also use as less xxx as possible, so I could setup VPN and provide my keys, could setup SSL and provide the certs, I just need to leave off the two paid lists (maybe, I could also place them in the code, but they won't work for ones, which are not registered with this lists).

So any recommendations on how to handle? I believe, there were no extra setups or registrations (as they were with ClamAV) which should be kept secret.

Regards,
Christian

 

[velocity08]

Hi Christian

I'm unsure, if I should do with example.com and set up a local DNS to be able to work and play with

Yes this is fine.

You can define in the Read.me notes that DNS and other items in config files marked with ###comments### should be changed.

Or if I should register a test domain to do with

Don't waste your money on a test domain just use example.com and set a static entry in your .hosts file to point to example .com locally.

In addition, I run two setups currently, the more loser commercial test setup and the tighter private setup. So which one should I cover there. I for sure could document the differences, but need to follow "one line".

I think for the time being just do 1 the default method of setting up in a DMZ or local network behind a firewall and tune this over time.
Can do a second setup for commercial at a later stage as it will just be the addition of VPN, More FW rules and can be a bolt on to the original document structure.

If doing with any domain above (beside one of my own) I could also use as less xxx as possible, so I could setup VPN and provide my keys, could setup SSL and provide the certs, I just need to leave off the two paid lists (maybe, I could also place them in the code, but they won't work for ones, which are not registered with this lists).

Use the KISS methodology.

KISS - Keep IT Simple Successful.


Base default deployment first then add on options later.

Optional config could be for example:

• VPN
• Free SSL
• etc

just my 2 cents, hope the above helps.

""Cheers
G

 

[Zwankie]

Here we go! I now switched from ClamAV to Avast. See below the steps I performed:

wget https://files.avast.com/files/resellers/linux/avast.gpg
echo "deb http://deb.avast.com/lin/repo debian release" >> /etc/apt/sources.list
apt-key add avast.gpg
apt-get update
apt-get install avast
vi /etc/avast/license.avastlic
/etc/init.d/avast start
pmgsh set /config/admin --avast 1
pmgsh set /config/admin --clamav 0

No more file contents here, as for sure, I won't post my license.

For Debian Stretch the source is:
deb http://deb.avast.com/lin/repo debian-stretch release