Feature Requests

heutger

Active Member
Apr 25, 2018
700
191
43
Fulda, Hessen, Germany
www.heutger.net
Your solution looks fine and I test it for installation in our company (then for sure with license). However, I recognized, that some things are missing:

- IPv6 is not fetched on installation and setting need restart?
- No time server could be set!
- Cert handling would be fine via UI, but is ok via command line
- Why no own bind server shipped with installation (prevent from blocks like URIBL with common nameservers)
- Why no firewall setup / settings to run Proxmox also "in the internet"
- Additional VPN setup would be fine to restrict access to the system
- Missing Pyzor and DCC (if because of license perhaps providing assistance/a script/... to install DCC)
- Missing additional postfix restrictions against RFC ignorant spam (smtpd_data_restrictions = reject_unauth_pipelining, unknown_address_reject_code = 550, unknown_client_reject_code = 550)
- Missing additional blacklist possibilities (reject_rhsbl_helo dbl.spamhaus.org, reject_rhsbl_sender dbl.spamhaus.org, reject_rhsbl_client dbl.spamhaus.org
- Missing additional content list possibilities (add additional blacklists to spamassassin like Barracuda RBL (has some false positives, so better for content), RBLDNS.RU (has more false positives, same for following ones), SPFBL, s5h RBL, JunkEmailFilter with Black, White and Brown, DNSRBL, JustSpam, inps DNSBL, GBUdb Truncate, V4BL/FREE, maybe also UCEPROTECT Level 1 and BACKSCATTERER Zone)
- Tracking Center may also show subject and especially the spam level reached, should be filterable for Status (maybe more Status options for rejected ones) and searchable for log content
- Statistic graphs don't include spam mails and there is no clear definition in difference of rejected is Junk and content scanned maybe Spam

So the solution looks very good with room for improvements. I believe, open sourcing the solution has the idea to find help from users, where to improve to kick all competitor products away.
 
Feb 6, 2018
76
6
8
46
I suggest to contribute by your own to the product by creating an howto here or by submitting patches to devs.
 
Last edited:

heutger

Active Member
Apr 25, 2018
700
191
43
Fulda, Hessen, Germany
www.heutger.net
I'm unsure, if I did things correct, but what I did yet:

1 vi /etc/apt/sources.list.d/pmg-enterprise.list => changing from enterprise repo to no-subscription repo for first tests

deb ... (external links not allowed) stretch pmg-no-subscription

2 apt-get update
3 apt-get upgrade
4 apt-get dist-upgrade

to update/upgrade with the "new" repo

5 vi /etc/pmg/pmg-api.pem
6 vi /etc/pmg/pmg-tls.pem

adding a valid certificate (first private key, then endpoint cert, then chain certs (between endpoint and root, but not root itself)

7 apt-get install pyzor

to install pyzor

8 apt-get install gcc
9 apt-get install make

to have the prerequirements to install dcc

10 cd /tmp
11 wget ... (external links not allowed) dcc-dccproc.tar.Z
12 tar xzvf dcc-dccproc.tar.Z
13 cd dcc-dccproc-*
14 ./configure
15 make
16 make install
17 ln -s /var/dcc/libexec/dccifd /usr/local/bin/dccifd

to install dcc

18 vi /etc/mail/spamassassin/custom.cf => to activate both

loadplugin Mail::SpamAssassin::plugin::pyzor
use_pyzor 1
loadplugin Mail::SpamAssassin::plugin::DCC
use_dcc 1
dcc_path /usr/local/bin/dccproc

19 spamassassin -D --lint

to test valid config

20 echo "test" | spamassassin -D pyzor 2>&1 | less

to test pyzor (did not found a way to test DCC yet)

21 apt-get install bind9
22 systemctl start bind9

to have a local nameserver to prevent blockings at blocklists because of too much enquiries from common (caching) nameservers, after that chaning nameserver to 127.0.0.1 in the PMG GUI

I found a bug (or somehow an unhappy setting) if changing the search domain to . (as I don't want non fqdn domains to be checked by adding search domain) that the same settings are used for postfix mydomain and myhostname resulting in illegal settings here and postfix to stop working

23 apt-get install ufw

for easier firewall configuration

24 ufw enable
25 ufw default deny incoming
26 ufw default allow outgoing
27 ufw allow ssh
28 ufw allow smtp
29 ufw allow 8006

to set the firewall up with all required ports from outside (if only using incoming filtering, if also using outgoing filtering, port 26 is needed as well)

30 ufw status verbose

to check firewall config

31 apt-get install ntp

to install timeserver update

32 vi /etc/ntp.conf => set a better time source to sync with

server de.pool.ntp.org

33 systemctl start ntp
34 ntpq -p

to adopt the new setting and check, if timeserver is fetching the right time source

I delayed VPN setup until I'm sure, I will keep PMG

I did not set the additional postfix restrictions, as main.cf seems to be written by PMG and as there is the custom.cf for spamassassin custom settings, I did not found any mentions on how to do with postfix.

I can't do GUI changes and contribute them, as I don't do web development any more.

For the additional lists in spamassassin, I will now first check out with running URIBL, Pyzor and DCC and will consider to add additional blacklists lateron. I will update this thread with additional custom.cf settings then and also with VPN setup, if done.

Oh, and I whitlisted 194.25.134.0/24 (as German Telecom always fails not to get their servers blacklisted) with the following PMG adjustments:

I removed quarantine or blocks and changed them to deliver with subject spam prefix or removing infected attachments, I adjusted level 3 because of too much false positive to level 4, I added the blocklists ix.dnsbl.manitu.net;sbl-xbl.spamhaus.org;bl.spamcop.net;psbl.surriel.com;spamrbl.imp.ch;noptr.spamrats.com;escalations.dnsbl.sorbs.net;singular.ttk.pte.hu because this set worked well for me (sbl-xbl instead of zen because of dialup-IPs and my home systems however also send mails using dialup-IPs, singular is on testing currently, checking rejects from singular currently, all the others are running well for yours, barracuda recently had false positives, so I don't use it here any more, same for GBUdb, you may check inps.de and/or intra2net.com spam statistics, but I don't agree with their counts for my setup(s)), I changed maximum message to 104857600 bytes, enabled reject unknown clients, senders and smtp helo checks, disabled greylisting and enabled spf as well as all TLS options.

Oh, and I installed a vi version, which is usable, not this stupid version preinstalled:

35 apt-get install vim
 
  • Like
Reactions: DerDanilo
Feb 6, 2018
76
6
8
46
For postfix conf the procedure to modify conf files is documented in the admin guide.
Basically mkdir -p /etc/pmg/templates and copy here global conf files which lives in /var/lib/pmg/templates.

Then make your own modifications and do
Pmgconfig sync
 
Feb 6, 2018
76
6
8
46
I'm unsure, if I did things correct, but what I did yet:

to have the prerequirements to install dcc

10 cd /tmp
11 wget ... (external links not allowed) dcc-dccproc.tar.Z
12 tar xzvf dcc-dccproc.tar.Z
13 cd dcc-dccproc-*
14 ./configure
15 make
16 make install
17 ln -s /var/dcc/libexec/dccifd /usr/local/bin/dccifd
I prefer running dcc as service (dccifd) to avoid on each spamassassin scan to spawn a new dccproc.
 

heutger

Active Member
Apr 25, 2018
700
191
43
Fulda, Hessen, Germany
www.heutger.net
I prefer running dcc as service (dccifd) to avoid on each spamassassin scan to spawn a new dccproc.
Sounds much better and I believe, therefore I need to create the symlink, however, I don't know on how to do that. Can you help?

I currently try to improve clamav detection, as it looks not so good (the mails I currently try and filter are filtered afterwards by two additional antispam service providers, so I can see, what they do with "the rest" and they recently found a virus, which clamav did not detect). I try SecurityInfo and SaneSecurity and will report then.

I just found another "bug", I will post bugs in an extra thread. It's about SPF seems to have problems with forwarded mails although the forwarding server seems to have SRS enabled.

And I need to remove ntpd again, I just found, that a time server service is already running, it's just not adjustable via GUI but via command line, I also will report on this.
 

heutger

Active Member
Apr 25, 2018
700
191
43
Fulda, Hessen, Germany
www.heutger.net
Updated tasks done:

31 vi /etc/systemd/timesyncd.conf

[Time]
NTP=de.pool.ntp.org

32 systemctl restart systemd-timesyncd

33 apt-get install vim

vi is catastrophic, vim is a bit too much, I prefer CentOS and vim-enhanced there, but vim looks most similar

34 mkdir .ssh
35 vi .ssh/authorized_keys

I prefer to use a real terminal and it's much easier to be authorized by a ssh key (for sure I won't post it here)

36 vi /etc/profile => I usually set some aliases and I hate spaces and duplicates in history, so I add this one

HISTSIZE=1000
if [ "$HISTCONTROL" = "ignorespace" ] ; then
export HISTCONTROL=ignoreboth
else
export HISTCONTROL=ignoredups
fi
export HISTSIZE HISTCONTROL

alias "ls=ls -al"
alias "rm=rm --preserve-root"
alias "cd..=cd .."
alias "dir=ls -al"

Now I will check for clamav improvements
 

heutger

Active Member
Apr 25, 2018
700
191
43
Fulda, Hessen, Germany
www.heutger.net
I did your recommendation, many thanks:

37 mkdir -p /etc/pmg/templates
38 cp /var/lib/pmg/templates/* /etc/pmg/templates/.
39 cd /etc/pmg/templates
40 vi main.cf.in

config follows below

41 pmgconfig sync --restart 1
 

heutger

Active Member
Apr 25, 2018
700
191
43
Fulda, Hessen, Germany
www.heutger.net
# auto-generated by proxmox


compatibility_level = 2

command_directory = /usr/sbin

daemon_directory = /usr/lib/postfix/sbin

data_directory = /var/lib/postfix


# appending .domain is the MUA's job.

append_dot_mydomain = yes


smtpd_banner = $myhostname [% pmg.mail.banner %]

biff = no


[% IF pmg.mail.dwarning %]

delay_warning_time = [% pmg.mail.dwarning %]h

[% END %]


best_mx_transport = local

message_size_limit = [% pmg.mail.maxsize %]

mailbox_size_limit = [% ((pmg.mail.maxsize*2 > 51200000) ? pmg.mail.maxsize*2 : 51200000) %]


mydomain = [% dns.domain %]

myhostname = [% dns.hostname %].[% dns.domain %]


parent_domain_matches_subdomains = debug_peer_list,fast_flush_domains,mynetworks,permit_mx_backup_networks,qmqpd_authorized_clients,smtpd_access_maps


alias_maps = hash:/etc/aliases

alias_database = hash:/etc/aliases

mydestination = localhost, $myhostname

mynetworks = [% postfix.mynetworks %]


relay_domains = hash:/etc/pmg/domains


transport_maps = hash:/etc/pmg/transport


[% IF pmg.mail.relay %]

[% IF pmg.mail.relaynomx %]

relay_transport = smtp:[[% pmg.mail.relay %]]:[% pmg.mail.relayport %]

[% ELSE %]

relay_transport = smtp:[% pmg.mail.relay %]:[% pmg.mail.relayport %]

[% END %]

[% END %]


[% IF pmg.mail.smarthost %]

default_transport = smtp:[% pmg.mail.smarthost %]

[% END %]


content_filter=scan:127.0.0.1:10024


mail_name = Proxmox


[% IF pmg.mail.helotests %]

smtpd_helo_required = yes

smtpd_helo_restrictions = permit_mynetworks reject_non_fqdn_helo_hostname reject_invalid_helo_hostname reject_rhsbl_helo dbl.spamhaus.org

[% ELSE %]

smtpd_helo_restrictions =

[% END %]


postscreen_access_list =

permit_mynetworks,

cidr:/etc/postfix/postscreen_access


[% IF postfix.dnsbl_sites %]

postscreen_dnsbl_sites = [% postfix.dnsbl_sites %]

[% END %]


postscreen_dnsbl_action = enforce

postscreen_greet_action = enforce


smtpd_sender_restrictions =

permit_mynetworks

reject_non_fqdn_sender

check_client_access cidr:/etc/postfix/clientaccess

check_sender_access regexp:/etc/postfix/senderaccess

check_recipient_access regexp:/etc/postfix/rcptaccess

[%- IF pmg.mail.rejectunknown %] reject_unknown_client_hostname[% END %]

[%- IF pmg.mail.rejectunknownsender %] reject_unknown_sender_domain[% END %]

reject_rhsbl_client dbl.spamhaus.org

reject_rhsbl_sender dbl.spamhaus.org


smtpd_recipient_restrictions =

permit_mynetworks

reject_unauth_destination

reject_non_fqdn_recipient

check_recipient_access regexp:/etc/postfix/rcptaccess

[%- IF postfix.usepolicy %] check_sender_access regexp:/etc/postfix/senderaccess[% END %]

[%- IF postfix.usepolicy %] check_client_access cidr:/etc/postfix/clientaccess[% END %]

[%- IF postfix.usepolicy %] check_policy_service inet:127.0.0.1:10022[% END %]

[%- IF pmg.mail.verifyreceivers %] reject_unknown_recipient_domain[% END %]

[%- IF pmg.mail.verifyreceivers %] reject_unverified_recipient[% END %]


smtpd_data_restrictions = reject_unauth_pipelining


[% IF pmg.mail.rejectunknownsender %]

unknown_address_reject_code = 550

[% ELSE %]

[% IF pmg.mail.verifyreceivers %]

unknown_address_reject_code = 550

[% END %]

[% END %]


[% IF pmg.mail.rejectunknown %]

unknown_client_reject_code = 550

[% END %]


[% IF pmg.mail.verifyreceivers %]

unverified_recipient_reject_code = [% pmg.mail.verifyreceivers %]

[% END %]


smtpd_client_connection_count_limit = [% pmg.mail.conn_count_limit %]

smtpd_client_connection_rate_limit = [% pmg.mail.conn_rate_limit %]

smtpd_client_message_rate_limit = [% pmg.mail.message_rate_limit %]


[% IF pmg.mail.tls %]

smtp_tls_security_level = may

smtp_tls_policy_maps = hash:/etc/pmg/tls_policy

smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt

smtpd_tls_security_level = may

smtpd_tls_cert_file = /etc/pmg/pmg-tls.pem

smtpd_tls_key_file = $smtpd_tls_cert_file

[% IF pmg.mail.tlslog %]

smtpd_tls_loglevel = 1

smtp_tls_loglevel = 1

[% END %]

[% IF pmg.mail.tlsheader %]

smtpd_tls_received_header = yes

[% END %]

[% END %]


smtp_tls_session_cache_database = btree:/var/lib/postfix/smtp_tls_session_cache

smtpd_tls_session_cache_database = btree:/var/lib/postfix/smtpd_tls_session_cache



default_destination_concurrency_limit = 40

lmtp_destination_concurrency_limit = 20

relay_destination_concurrency_limit = 20

smtp_destination_concurrency_limit = 20

virtual_destination_concurrency_limit = 20


recipient_delimiter = +
 
Feb 6, 2018
76
6
8
46
For clamav just use clamav-unofficial-sig.
Get it from github and not use the one bundled with debian as it really old.
Be sure to subscribe freely to the various feeds as documented in the Shell script.
 

heutger

Active Member
Apr 25, 2018
700
191
43
Fulda, Hessen, Germany
www.heutger.net
Did you the same setup as me? Could you provide help on dcc? How can I check, if it's really running. All spam mails I checked don't have any PYZOR or DCC "evidence".

I now worked on as suggested:

42 apt-get install unzip

as unzip is missing

43 cd /tmp
44 wget https://github.com/extremeshok/clamav-unofficial-sigs/archive/master.zip
45 unzip master.zip

to fetch clamav-unofficial-sigs from github

46 cp clamav-unofficial-sigs-master/clamav-unofficial-sigs.sh /usr/local/sbin/
47 chmod 755 /usr/local/sbin/clamav-unofficial-sigs.sh
48 mkdir /etc/clamav-unofficial-sigs
49 cp clamav-unofficial-sigs-master/config/* /etc/clamav-unofficial-sigs/
50 mkdir /var/log/clamav-unofficial-sigs

doing as described to prepare all files and folders

51 cd /etc/clamav-unofficial-sigs
52 cat /etc/*release*

to find out OS version

53 mv os.debian9.conf os.conf

to use the OS specific configuration

54 vi user.conf

adjusted content follows

55 /usr/local/sbin/clamav-unofficial-sigs.sh --install-cron
56 /usr/local/sbin/clamav-unofficial-sigs.sh --install-logrotate
57 /usr/local/sbin/clamav-unofficial-sigs.sh --install-man

to install as described

58 /usr/local/sbin/clamav-unofficial-sigs.sh

first run as described, just found issues here again, I reported bugs at github: securiteinfo.hdb does not download and MalwarePatrol different product code (I'm unsure why, but I got 32 instead of 8 which should have been assigned) ignored (although set in config), I additionally adjusted master.conf now, but need to wait 24 hours to recheck

59 cp /tmp/clamav-unofficial-sigs-master/systemd/* /etc/systemd/

as described for systemd

60 clamscan --debug 2>&1 /dev/null | grep "loaded"

as described to verify installation, looks good beside the issues above
 

heutger

Active Member
Apr 25, 2018
700
191
43
Fulda, Hessen, Germany
www.heutger.net
I just uncommented and adjusted in user.conf

malwarepatrol_receipt_code="xxxxx" => for sure, here is my receipt code set
malwarepatrol_product_code="32" => that's the product code my URL had and with this one, I'm able to download, I could not retest as --force also forced to retry the SecuriteInfo database, which does not download and hang the script
malwarepatrol_list="clamav_basic" # clamav_basic or clamav_ext
malwarepatrol_free="yes"

securiteinfo_authorisation_signature="xxxxx" => same here, I won't provide my credentials here

user_configuration_complete="yes" => not well documented, but need to uncomment, to activate user.conf
 
Feb 6, 2018
76
6
8
46
Hi

Here my dcc.service


[Unit]
Description=DCC (Distributed Checksum Clearinghouses) interface daemon
After=remote-fs.target systemd-journald-dev-log.socket

[Service]
Type=forking
PermissionsStartOnly=true
RuntimeDirectory=dcc
ExecStart=/var/dcc/libexec/dccifd
User=root
Group=root
Nice=1

#DCC writes pid file with "-" at the beginning which confuses systemd
#PIDFile=/run/dcc/dccifd.pid

[Install]
WantedBy=multi-user.target
 
Feb 6, 2018
76
6
8
46
FOR DCC:

1) edit /vardcc/dcc_conf
2) change to DCCIFD_ENABLE=on
3) Add to /etc/mail/spamassassin/custom.cf :
dcc_path /usr/local/bin/dccproc
dcc_home /var/dcc
dcc_dccifd_path /var/dcc/dccifd
dcc_body_max 999999
dcc_fuz1_max 999999
dcc_fuz2_max 999999
use_dcc 1
dcc_timeout 10

Then try to run spamassasin lint ad see if sa talks correctly with dccifd started from systemd (see my previous post)
 
Feb 6, 2018
76
6
8
46
I just uncommented and adjusted in user.conf

malwarepatrol_receipt_code="xxxxx" => for sure, here is my receipt code set
malwarepatrol_product_code="32" => that's the product code my URL had and with this one, I'm able to download, I could not retest as --force also forced to retry the SecuriteInfo database, which does not download and hang the script
malwarepatrol_list="clamav_basic" # clamav_basic or clamav_ext
malwarepatrol_free="yes"

securiteinfo_authorisation_signature="xxxxx" => same here, I won't provide my credentials here

user_configuration_complete="yes" => not well documented, but need to uncomment, to activate user.conf
I sugget to register to malwarepatrol and securiteinfo .
 

heutger

Active Member
Apr 25, 2018
700
191
43
Fulda, Hessen, Germany
www.heutger.net
Parallel I checked the quality of spam detection now with my adjustments and be very happy with. I'm just upset, that the last statistics are missing. I now decided for the first not to add any additional blacklists but try to run with the current setup. If spam detection will get worser, I will first just add barracudacentral as it's the most reliable database (however, if you google, you can find many complains about this list in conjunction with emailreg, looking a bit like UCEPROTECT seem to take money for unlisting with strange listing issues).

So next and for the first last two steps will be setting up an automated backup (using the backup script for settings backup plus backing up relevant paths, which include adjustments I did and uploading them to my backup location via lftp, if that's available for Debian also (as mentioned coming from CentOS systems, I always prefer)) and setting up a VPN connection to the server, that I can close SSH and GUI from internet (as system is hosted, I can't add a firewall in front).

If everything still works fine, I will consider a productive test in our company and will then purchase subscription therefor. I'm unsure, which one, depending on what support is really provided. E.g. if adjustment (checking) or performing is provided, maybe a greater support level would be fine.
 
Feb 6, 2018
76
6
8
46
Parallel I checked the quality of spam detection now with my adjustments and be very happy with. I'm just upset, that the last statistics are missing. I now decided for the first not to add any additional blacklists but try to run with the current setup. If spam detection will get worser, I will first just add barracudacentral as it's the most reliable database (however, if you google, you can find many complains about this list in conjunction with emailreg, looking a bit like UCEPROTECT seem to take money for unlisting with strange listing issues).

So next and for the first last two steps will be setting up an automated backup (using the backup script for settings backup plus backing up relevant paths, which include adjustments I did and uploading them to my backup location via lftp, if that's available for Debian also (as mentioned coming from CentOS systems, I always prefer)) and setting up a VPN connection to the server, that I can close SSH and GUI from internet (as system is hosted, I can't add a firewall in front).

If everything still works fine, I will consider a productive test in our company and will then purchase subscription therefor. I'm unsure, which one, depending on what support is really provided. E.g. if adjustment (checking) or performing is provided, maybe a greater support level would be fine.
I don't know if you made massive tweaks and mods the support will assist you.
As for rbl, i've preferred to use postscreen rbl and use the threeshold feature (see: http://rob0.nodns4.us/postscreen.html).
i've just using the rbl feature not any other checks (After-220 tests), just using pregreet and rbl.
The most notable feature is to use whitelist rbls to decrease points for blacklisted ip.
 
Feb 6, 2018
76
6
8
46
I did and got the codes, however securiteinfo first file got downloaded well, second it hangs, malwarepatrol doesn't work at all, as I got another product code, which I entered in user.conf, but got ignored
I've followed EXACTLY what docs of script said and it works
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!