Information for a good start with PMG

[Andy_red]

Hi everyone,

i have some questions, because i am a new proxmox user and would like to move my antispam from plesk to a proxy.

1. Which DNS Blacklist do you recommend?
   • (In the webui field, do I have to separate the URLs with a comma or a semicolon?)
   • I saw that there are DNSBLs that use scores like Mailspike, are they automatically recognized by PMG?
2. I would like to install a cluster of 2 servers, do you have a solution (better if you have a guide to recommend) for the cache of requests to the various DNSBL servers?
   So to be too limited by the maximum of queries per day.
3. Can you tell me if this guide is still the best in terms of adding signatures to ClamAV, (having a cluster I read that I don't need to do anything on the slave server by changing the template)?
4. is it possible to create email addresses like spam@pmg.mydomain.ch or ham@pmg.mydomain.ch that train spamassasin automatically?
   • I don't know if there is an action to be set with the mail filter, which allows you to perform this action.
5. I wanted to ask if proxmox slows down a lot with big blacklists or if it remains performing.
6. Is this post still updated? or are there more recent guides?

 

[heutger]

Hi everyone,

i have some questions, because i am a new proxmox user and would like to move my antispam from plesk to a proxy.

1. Which DNS Blacklist do you recommend?
   • (In the webui field, do I have to separate the URLs with a comma or a semicolon?)
   • I saw that there are DNSBLs that use scores like Mailspike, are they automatically recognized by PMG?
2. I would like to install a cluster of 2 servers, do you have a solution (better if you have a guide to recommend) for the cache of requests to the various DNSBL servers?
   So to be too limited by the maximum of queries per day.
3. Can you tell me if this guide is still the best in terms of adding signatures to ClamAV, (having a cluster I read that I don't need to do anything on the slave server by changing the template)?
4. is it possible to create email addresses like spam@pmg.mydomain.ch or ham@pmg.mydomain.chthat train spamassasin automatically?
   • I don't know if there is an action to be set with the mail filter, which allows you to perform this action.
5. I wanted to ask if proxmox slows down a lot with big blacklists or if it remains performing.
6. Is this post still updated? or are there more recent guides?

For number 6, I recently had spare time because of personal issues, however, I plan to create the repository soon with my adjustments based on PMG 6.2.

 

[Stoiko Ivanov]

Check out the getting started guide, we posted recently - that should get you started:
https://pmg.proxmox.com/wiki/index.php/Getting_started_with_Proxmox_Mail_Gateway

is it possible to create email addresses like spam@pmg.mydomain.ch or ham@pmg.mydomain.ch that train spamassasin automatically?

• I don't know if there is an action to be set with the mail filter, which allows you to perform this action.

bayes training is currently not implemented (and leads to problems in quite a few installations which tried it) - you can train your filter manually though

I hope this helps!

 

[Andy_red]

Thank you @Stoiko Ivanov ,
for the advice.

There is a section of the forum or your repositories where the "Service Configuration Templates" created by the community are shared, maybe for some script that uses Tensorflow or other algorithms for small things like NSFW detection?

 

[Stoiko Ivanov]

There is a section of the forum or your repositories where the "Service Configuration Templates" created by the community are shared, maybe for some script that uses Tensorflow or other algorithms for small things like NSFW detection?

no dedicated section - basically if someone creates something they want to share they post it in the forum and prefix the thread with 'TUTORIAL'

as for the Tensorflow detection - not that I'm aware of someone having done that (would probably be best integrated via the custom_check_script functionality : https://pmg.proxmox.com/pmg-docs/pmg-admin-guide.html#pmgconfig_custom_check)

 

[heutger]

Check out the getting started guide, we posted recently - that should get you started:
https://pmg.proxmox.com/wiki/index.php/Getting_started_with_Proxmox_Mail_Gateway


bayes training is currently not implemented (and leads to problems in quite a few installations which tried it) - you can train your filter manually though

I hope this helps!

Hi @Stoiko Ivanov , which problems occurred? I now know many which had direct contact or forum contact with me, which as myself use bayes with very huge success.

As for Tenderflow and custom scripts, for contributions wouldn’t it be great to create a forum corner? E.g. the ones who integrate Eset Antivirus.

As of a recent conservation, DCC (and also Pyzor) has also a big weight on spam detection (content). Maybe you could built extra subscription packages to avoid licensing issues. For blacklists I also beliebe Rob Ewan from invaluement would be open for a subscription option/bundling offer of their lists to your customers as beside the open lists I use (which some also have subscriptions for commercial use you could offer as an option) it has a great value as well. Last but not least maybe PMG could consider as Scrollout to provide their own list based on spam/ham quarantine decision of PMG users?

 

[Andy_red]

@heutger
I think pyzor can be integrated with the "Custom Check Interface" instead of in SA, it should make it easier to maintain.

 

[ittk]

@heutger
I think pyzor can be integrated with the "Custom Check Interface" instead of in SA, it should make it easier to maintain.

Don't think so and as SA Scoring are used with....

 

[Andy_red]

Don't think so and as SA Scoring are used with....

From what I have read, the "Custom Check Interface" supports these outputs:

the expected output need to be printed on STDOUT and consists of two lines:

• the api-version (currently v1) - see above
• one of the following 3 results:
  • OK - email is ok
  • VIRUS: <virusdescription> - email is treated as if it contained a virus (the virus description is logged and added to the email’s headers)
  • SCORE: <number> - <number> is added (negative numbers are also possible) to the email’s spamscore

so I think the score is taken into account

 

[ittk]

That's the completely wrong place for it. Pyzor and so on are interacting (PLUGINS for SA), go and read more on it e. g. here: https://cwiki.apache.org/confluence/display/SPAMASSASSIN/UsingPyzor

So there is really no need for anything else... My recommendation is: before you change anything behind the PMG default installation, collect experiance with PMG and configure it to your needs, before you do such advanced things which may also can lead to issues or broken PMG systems. Doing too much and especially too much changes at the same time, will not ease the troubleshooting side

 

[heutger]

That's the completely wrong place for it. Pyzor and so on are interacting (PLUGINS for SA), go and read more on it e. g. here: https://cwiki.apache.org/confluence/display/SPAMASSASSIN/UsingPyzor

So there is really no need for anything else... My recommendation is: before you change anything behind the PMG default installation, collect experiance with PMG and configure it to your needs, before you do such advanced things which may also can lead to issues or broken PMG systems. Doing too much and especially too much changes at the same time, will not ease the troubleshooting side

Custom Checks are built especially for integrating antivirus (or similar malware tools) but not for integrating spam checks. Spam checks are also already provided and deeply built in SpamAssassin, so although another way may work, why don't use the original way they are built for (and would take most advances).

However, I would welcome, if some adjustments (like already pre-queue) would be taken to PMG, e.g. Pyzor and DCC is really great help, however, it may depend on licensing issues, but then I would recommend to build either a script to invoke by self or offer an optional subscription option.

 

[Andy_red]

My concept is that of simplification, "Custom Checks" allow you to add and remove spam scores.
So using pyzor via CC would be possible and probably easier (once the script is created) to manage and install pyzor on the servers.
Same thing for NSFW filters, or any other software that wants to be adapted to PMG.

I am sure that if proxmox installs the pyzor module without activating it, leaving us the choice of the server to connect to, no license is needed (pyzor code is under GPL-2.0 License), and it would help us save time and complicated implementations.

 

[ittk]

My concept is that of simplification, "Custom Checks" allow you to add and remove spam scores.
So using pyzor via CC would be possible and probably easier (once the script is created) to manage and install pyzor on the servers.
Same thing for NSFW filters, or any other software that wants to be adapted to PMG.

I am sure that if proxmox installs the pyzor module without activating it, leaving us the choice of the server to connect to, no license is needed (pyzor code is under GPL-2.0 License), and it would help us save time and complicated implementations.

Therefore you can use command-line or also GUI-based method, you think too complicated for things already there, no need to use other mechanisms not mainly intended for.

https://pmg.proxmox.com/pmg-docs/pmg-admin-guide.html#pmgconfig_spamdetector_customscores

4.9. Custom SpamAssassin configuration
This is only for advanced users. SpamAssassin™'s rules and their associated scores get updated regularly and are trained on a huge corpus, which gets classified by experts. In most cases adding a rule for matching a particular keyword is the wrong approach, leading to many false positives. Usually bad detection rates are better addressed by properly setting up DNS than by adding a custom rule - watch out for matches to URIBL_BLOCKED in the logs or spam-headers - see the SpamAssassin DNSBL documentation.
To add or change the Proxmox SpamAssassin™ configuration please login to the console via SSH. Change to the /etc/mail/spamassassin/ directory. In this directory there are several files (init.pre, local.cf, …) - do not change them, as init.pre, v310.pre, v320.pre, local.cf will be overwritten by the template engine, while the others can get updated by any SpamAssassin™ package upgrade.
To add your custom configuration, you have to create a new file and name it custom.cf (in this directory), then add your configuration there. Make sure to use the correct SpamAssassin™ syntax, and test it with:
# spamassassin -D --lint
If you run a cluster, the custom.cf file is synchronized from the master node to all cluster members automatically.
To adjust the score assigned to a particular rule you can also use the Custom Rule Score settings in the GUI.

 

[Andy_red]

The issue is the leaks of a "store"/repository where u can download and install ezly custom files for CC our SA.

 

[ittk]

The issue is the leaks of a "store"/repository where u can download and install ezly custom files for CC our SA.

That's why it is called Custom so you can do whatever you need, if you want to, but you must be willing to do it... so just feel free to do it....

 

[heutger]

Therefore you can use command-line or also GUI-based method, you think too complicated for things already there, no need to use other mechanisms not mainly intended for.

https://pmg.proxmox.com/pmg-docs/pmg-admin-guide.html#pmgconfig_spamdetector_customscores

4.9. Custom SpamAssassin configuration
This is only for advanced users. SpamAssassin™'s rules and their associated scores get updated regularly and are trained on a huge corpus, which gets classified by experts. In most cases adding a rule for matching a particular keyword is the wrong approach, leading to many false positives. Usually bad detection rates are better addressed by properly setting up DNS than by adding a custom rule - watch out for matches to URIBL_BLOCKED in the logs or spam-headers - see the SpamAssassin DNSBL documentation.
To add or change the Proxmox SpamAssassin™ configuration please login to the console via SSH. Change to the /etc/mail/spamassassin/ directory. In this directory there are several files (init.pre, local.cf, …) - do not change them, as init.pre, v310.pre, v320.pre, local.cf will be overwritten by the template engine, while the others can get updated by any SpamAssassin™ package upgrade.
To add your custom configuration, you have to create a new file and name it custom.cf (in this directory), then add your configuration there. Make sure to use the correct SpamAssassin™ syntax, and test it with:
# spamassassin -D --lint
If you run a cluster, the custom.cf file is synchronized from the master node to all cluster members automatically.
To adjust the score assigned to a particular rule you can also use the Custom Rule Score settings in the GUI.

As you can read here, CC are not intend to be used for SpamAssassin modifications, therefor a different solution exist. However, the just one not really nice issue with PMG is, that it really split all stages from each other and so it provide multiple points to adjust. On mail flow it's Postfix, SpamAssasin and the rule system, additional malware and custom checks on the score flow. However, it's your decision on how you want to adjust. For Pyzor as well as DCC great rulesets are already provided by SpamAssassin, so it makes sense to use them, but you can decide different. Ok, if Pyzor is GPL, I'm wondering, why it's not shipped by default, however DCC had license issues.

 

[Andy_red]

@heutger
What I'm wondering is if to integrate pyzor with PMG just add a file to a folder /etc/mail/spamassassin/ shouldn't all these configuration files be grouped together in a repository to make setup easier?

the biggest problem i am having as sysadmin that has never dealt with mail servers in depth and especially antispam is that PMG does not have a simple documentation and the information is scattered in the forum

 

[heutger]

@heutger
What I'm wondering is if to integrate pyzor with PMG just add a file to a folder /etc/mail/spamassassin/ shouldn't all these configuration files be grouped together in a repository to make setup easier?

the biggest problem i am having as sysadmin that has never dealt with mail servers in depth and especially antispam is that PMG does not have a simple documentation and the information is scattered in the forum

Proxmox decided a setup which may fit for all. If you would like to adjust, it’s your job and a job to be done manually. It’s not like Plesk, install and forget, but also with Plesk if you want to optimize, you need also to go deep into the system and may not trust all assistants and options, e.g. Wordpress Toolkit never worked for me but broken installations. However, there are many people out there providing additional support. My thread e.g. answers many questions and Proxmox Team always looks into adjustments and adopt some of them. Also myself I learned many about what I won’t do in future any more. E.g. I would never anymore put any effort in improving ClamAV, it’s worse and additional signature won’t improve it enough, they will also bring false-positives, so although Avast still has negative publicity, it’s the best option to choose. Why Pyzor isn’t integrated by default, you need to ask Proxmox team, for DCC I got known, it’s a license issue. For Pyzor you already find all required files on your system for SA, but need to install Pyzor itself, just follow my Advancing Tread, it’s not updated to 6.2, but all adjustments (beside ClamAV) should still work. And for sure, if you like to adjust as sysadmin, you need to deep dive in mail systems, sorry for that.

 

[heutger]

@Stoiko Ivanov may be you can explain, why PMG doesn’t use Pyzor by default as well. DCC I read was about license issues (if shipped with software, you need to pay fees, however could be an option in your subscriptions or you could provide an easy install script which need to be invoked manually), but I don’t see any reasons for Pyzor and Pyzor has also good scores and influence on PMG spam detection quality.

 

[ittk]

That ar

@Stoiko Ivanov may be you can explain, why PMG doesn’t use Pyzor by default as well. DCC I read was about license issues (if shipped with software, you need to pay fees, however could be an option in your subscriptions or you could provide an easy install script which need to be invoked manually), but I don’t see any reasons for Pyzor and Pyzor has also good scores and influence on PMG spam detection quality.

Regarding Pyzor:

https://pyzor.readthedocs.io/en/release-1-0-0/introduction.html

Since the entire system is released under the GPL,...

License
The project is licensed under the GNU GPLv2 license.

History
Pyzor initially started out to be merely a Python implementation of Razor, but due to the protocol and the fact that Razor’s server is not Open Source or software libre, Frank Tobin decided to implement Pyzor with a new protocol and release the entire system as Open Source and software libre.