[TUTORIAL] Advancing Proxmox Mail Gateway (especially Spam and Virus Detection)

[heutger]

I guess my next question is, won't this render the GUI useless for making config changes? Is there already a way to sync these files or do we need to do it manually?

The GUI has many possibilities to set, however, it's lack of all things someone may need. So I opened feature requests for most of my adjustments and hope, they will be integrated later-on.

No, I don't know of any possibility, from time to time therefor I check back, if the files change and merge the changes.

 

[swarnat]

Hy,

After some months of optimizations, based on your Starting Guide, I still wasn't able to solve some questions I had. Probably you had already solved or also tried without success.

Generally your Guide was a very helpful step to start with PMG.

1- Mail::SpamAssassin:: Plugin:: Phishing
Since Spamassassin 3.4.2 there is a Plugin which should catch Phishing URL's from two "public managed" phishing lists.
This theoretically sounds clever and I would like to to run a test with this. But regardless what I did and how long I wait, there are no matches. Either this plugin is completely useless, because URL's are too unique, what I don't expect, or there is something I don't recognize.
Plugin is correctly loaded in Debug Output and don't show error. Lists are downloaded and configuration applied, like in official documentation. (I'm not allowed to posts links)

2- TxRep:
Recently I started to switch from AWL to the TxRep method of reputation scores. This is perfectly working.
I cannot say if it improves the spam filter a lot, but also don't downgrade them. And I hope they is getting better with more time.
But I don't found a solution to add X-Spam-Report Header into Tracking Center and don't fully know the Scores txRep applied. I only see, it was applied.
The Header is correctly added to Mail, but not logged to Maillog. I think because of special "pmg-smtp-filter" handling of logging.

Thanks!
Stefan

Edit: Smilies in Pluign name removed

 

[heutger]

Hy,

After some months of optimizations, based on your Starting Guide, I still wasn't able to solve some questions I had. Probably you had already solved or also tried without success.

Generally your Guide was a very helpful step to start with PMG.

1- Mail::SpamAssassin:: Plugin:: Phishing
Since Spamassassin 3.4.2 there is a Plugin which should catch Phishing URL's from two "public managed" phishing lists.
This theoretically sounds clever and I would like to to run a test with this. But regardless what I did and how long I wait, there are no matches. Either this plugin is completely useless, because URL's are too unique, what I don't expect, or there is something I don't recognize.
Plugin is correctly loaded in Debug Output and don't show error. Lists are downloaded and configuration applied, like in official documentation. (I'm not allowed to posts links)

2- TxRep:
Recently I started to switch from AWL to the TxRep method of reputation scores. This is perfectly working.
I cannot say if it improves the spam filter a lot, but also don't downgrade them. And I hope they is getting better with more time.
But I don't found a solution to add X-Spam-Report Header into Tracking Center and don't fully know the Scores txRep applied. I only see, it was applied.
The Header is correctly added to Mail, but not logged to Maillog. I think because of special "pmg-smtp-filter" handling of logging.

Thanks!
Stefan

Edit: Smilies in Pluign name removed

Hi,

I did not use the Phishing plugin yet, however, if debug output shows the plugin been loaded and used, it looks like it's really too unique. For sure, I won't believe (similar to foreign spam import to bayes, which doesn't really work), that phish is the same as others phish, so in Germany you would get completely different phishing campaigns (Telekom, Vodafone, Post, VR-Bank, Sparkasse, ...) than in other countries, where other companies exist. Maybe you could try to send a mail with a phishing URL from the list and try, what PMG does with that.

The tracking center should contain lines like

Mar 23 09:29:36 mg pmg-smtp-filter[22131]: 20A655C95EE6F580CC: SA score=4/5 time=0.779 bayes=undefined autolearn=no autolearn_force=no hits=DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,HASHBL_EMAIL,HTML_MESSAGE,RCVD_IN_BRBL,RELAYCOUNTRY_BAD,SPF_HELO_PASS,SPF_PASS,URIBL_RED

but I'm unsure, if TxRep writes there (much). If AWL is been used, AWL is in the line of hits, if TxRep is writing something different to X-Spam-Report Header, you really won't see in tracking center. You may check your mail.log, if it's written there, you could adjust the tracking parser (however it's not really easy as it counts lines and columns to present in the tracking center, a bit of "dirty" code), if it's only in the headers, you won't have a chance (I know).

Regards,
Christian

 

[luke5sky]

in PMMG Version 5.2-1 the Folder /etc/pmg/templates is gone, so Header_checks are not working anymore

 

[Stoiko Ivanov]

in PMMG Version 5.2-1 the Folder /etc/pmg/templates is gone, so Header_checks are not working anymore

the directory is not removed in pmg 5.2 - and nothing in the new release changes the way the templateing engine works.

Why do you thing that it is gone?
(in a fresh default install the directory is not present - you need to create it, if you wish to overwrite templates)

 

[luke5sky]

because i upgraded my 2 PMMG proxies today (from Version 5.1-4 i think to 5.2-1) and after the upgrade the template folders are gone.

 

[luke5sky]

Maybe my path to the templates Directory is incorrect, should it be /etc/pmg/templates or /var/lib/pmg/templates?

 

[luke5sky]

Ah i remember now, /var/lib/pmg/templates are the Default templates. i copied and modified the main.cf.in (months ago) to /etc/pmg/templates/ and after the update today (over webinterface) the Folder /etc/pmg/templates is not present anymore

 

[heutger]

Ah i remember now, /var/lib/pmg/templates are the Default templates. i copied and modified the main.cf.in (months ago) to /etc/pmg/templates/ and after the update today (over webinterface) the Folder /etc/pmg/templates is not present anymore

Yes, that's the way you got/get /etc/pmg/templates. However, please check with your backup, if the folder really has been removed by the upgrade as that's not the typical behavior. I also checked my test installations and none of them the folder has been removed. As you're running a cluster, maybe there was a sync problem?

 

[heutger]

Hi,

I also did a quick check now, the templates themselves seems not to have been changed with 5.2, so no need to adjust the copies in /etc/pmg/templates beside your own changes, you did (or took from this tutorial) to get all new features running which may have been introduced to the GUI and require template variables, which did not exist before.

Regards,
Christian

 

[luke5sky]

I could not find the folder in the backups, that means it has to be my fault, maybe i edited the file directly and forgot to copy it to the templates folder, i am very sorry for the inconvenience

 

[heutger]

Here comes something new. ;-) After getting my log files rotating with default settings, I was required to check tracking center every week as well as statistics section mostly got useless with values of just one week, I decided to change log rotation behavior to keep records for one (typical) month (30 days).

Steps performed:

vi /etc/logrotate.d/rsyslog

/etc/logrorate.d/rsyslog looks like this:

/var/log/syslog
{
    rotate 30
    daily
    missingok
    notifempty
    delaycompress
    compress
    postrotate
        invoke-rc.d rsyslog rotate > /dev/null
    endscript
}

/var/log/mail.info
/var/log/mail.warn
/var/log/mail.err
/var/log/mail.log
/var/log/daemon.log
/var/log/kern.log
/var/log/auth.log
/var/log/user.log
/var/log/lpr.log
/var/log/cron.log
/var/log/debug
/var/log/messages
{
    rotate 4
    weekly
    missingok
    notifempty
    compress
    delaycompress
    sharedscripts
    postrotate
        invoke-rc.d rsyslog rotate > /dev/null
    endscript
}

 

[killmasta93]

so on the track center keeps 30 days? and the spam?

 

[heutger]

so on the track center keeps 30 days? and the spam?

Right, I do not use quarantine, doesn't change anything to that, it's just how long logs will be available for review. As last time on holidays I was required to check also there for the performance, I wanted to change to be required to check once a month or be able to delay up to 1 month and keep statistics also for one month.

 

[heutger]

As I tested another pre-queue setup, I changed behavior on spam above level 5 (block/silent drop instead of tagging, although already rejected by milter, if reaching level 5) and forgot about on changing back to recent setup (as it did not work out, I talk about smtpd-proxy instead of milter). I just recognized 3 blocks since this change and ask myself (and you), how to further run the setup. Silent drop (block) normally in Germany is illegal, however, I currently have this set on my private setup and it just blocked now 3 messages because of this messages were too big, so the miltered SpamAssassin rejected to scan such messages. I also won't increase scan limit as in the mail flow it could take too much time, load, bandwidth and connections. So should I keep and recommend to keep as usual no valid messages will be such big and hit spam level of 5 or should I revert to tag so this messages may get through in future again?

 

[killmasta93]

i recommend the tag better i have seen really good emails just because it has capitals or signatures with photo it will get hit above a 4 i currently have 5 higher then that goes to spam and gets tagged. still using the unofficial clamav, currently trying to find a way to implement gdata seems a really good antivirus not only for proxmox but for a business environment also which did some trial. I just not sure how i could implement on proxmox

 

[heutger]

i recommend the tag better i have seen really good emails just because it has capitals or signatures with photo it will get hit above a 4 i currently have 5 higher then that goes to spam and gets tagged. still using the unofficial clamav, currently trying to find a way to implement gdata seems a really good antivirus not only for proxmox but for a business environment also which did some trial. I just not sure how i could implement on proxmox

You're right, it's then better to revert to tagging.

Recent PMG release seems to be able to integrate each antivirus software. However, I checked many of them yet and now use the avast solution as been provided first and it works well. I had no false positives any more and a good detection rate. Most important on any solution is the availability of a command line scanner first, the possibility to run as a daemon second and a pricing based on machines instead of user third. And for sure a still supported product and nothing EOL/EOS. Finally beside avast there is a big lack of alternatives.

 

[heutger]

i recommend the tag better i have seen really good emails just because it has capitals or signatures with photo it will get hit above a 4 i currently have 5 higher then that goes to spam and gets tagged. still using the unofficial clamav, currently trying to find a way to implement gdata seems a really good antivirus not only for proxmox but for a business environment also which did some trial. I just not sure how i could implement on proxmox

Did you found any possibility to run G DATA via command line? I was not able to find anything yet. However, if you're still looking for the most affordable solution, you may give Dr.Web anti-virus for Linux a trial. I found a manual, which looks like there is a command line scanner as well as the possibility to daemonize and the pricing is really attractive (aggressive): 26€ per server. However, I'm not aware of any scan quality. @Davide Bozzelli, @Stoiko Ivanov maybe an option to look for?

 

[killmasta93]

well currently im still giving it go will post back if i get it working. the dr web seems interesting, its just that i really dont like avast like it seems way to mainstream

 

[heutger]

well currently im still giving it go will post back if i get it working. the dr web seems interesting, its just that i really dont like avast like it seems way to mainstream

Sometimes mainstream is not the worst, e.g. mainstream antivirus may have more access to pattern and may be able to detect viruses much faster. Dr.Web I know from my Plesk times, it's cheap, but maybe it's also cheap on quality.