Many thanks. I'm a bit new to systemd, where do I need to store this dcc.service file? Do I need to start the dcc.service then first? My recent dccifd symlink then is completely unneccessary?
Feature Requests
- Thread starter heutger
- Start date
Many thanks. I'm a bit new to systemd, where do I need to store this dcc.service file? Do I need to start the dcc.service then first? My recent dccifd symlink then is completely unneccessary?
ls -l /lib/systemd/system/dcc.service
-rw-r--r-- 1 root root 404 Mar 21 18:17 /lib/systemd/system/dcc.service
systemctl enable dcc
systemctl start dcc
I wold suggest to test FIRST the service outside systemd to see if it starts correctly
When did you set up? Maybe MalwarePatrol changed their URLs just recently. SecuriteInfo I believe in a temporary bug.
Sounds interesting, however, also needs adjustments, not available via GUI. I recently did not work with whitelists yet, although I don't believe in all the numbers, http://analyse.inps.de/?type=monthly&lang=de&service=&month=03&year=2018&sort=5 state also many false-negatives for the current only running whitelist with a big pool of IPs DNSWL.org
Hmm, I checked their subscription FAQ and you're right. So then I'm afraid, I would only go with the smallest subscription to have stable packages and stable package access. Maybe the other packages are meant for customers not aware of Linux, shells etc.
Thx, I was able to follow your explanations to change DCC to a service
--- .bash_history 2018-04-28 22:57:46.549979499 +0200
+++ .bash_history.bak 2018-04-28 22:01:20.952671659 +0200
@@ -14,10 +14,7 @@
./configure
make
make install
-vi /lib/systemd/system/dcc.service
-systemctl enable dcc
-systemctl start dcc
-vi /var/dcc/dcc_conf
+ln -s /var/dcc/libexec/dccifd /usr/local/bin/dccifd
vi /etc/mail/spamassassin/custom.cf
spamassassin -D --lint
echo "test" | spamassassin -D pyzor 2>&1 | less
--- .bash_history 2018-04-28 22:57:46.549979499 +0200
+++ .bash_history.bak 2018-04-28 22:01:20.952671659 +0200
@@ -14,10 +14,7 @@
./configure
make
make install
-vi /lib/systemd/system/dcc.service
-systemctl enable dcc
-systemctl start dcc
-vi /var/dcc/dcc_conf
+ln -s /var/dcc/libexec/dccifd /usr/local/bin/dccifd
vi /etc/mail/spamassassin/custom.cf
spamassassin -D --lint
echo "test" | spamassassin -D pyzor 2>&1 | less
I did two additional adjustments:
I removed singular.ttk.pte.hu from my blacklists as United Internet (1&1, GMX, web.de, ...) seems not be able to hold their servers under control and got listed. Although I whitelisted the DTAG/Deutsche Telekom, I won't do with United Internet as DTAG/Deutsche Telekom always get listed on Spamhaus or Spamcop, which from my point of opinion are industry standards, meanwhile singular.ttk.pte.hu is from a hungarian university for primary internal use, so it's better to remove this list from blocking and consider to add for tagging instead of whitelisting false-positives. I will think about adding bl.spameatingmonkey.net as many people like this list, but I will see.
I was in need to adjust /etc/clamav-unofficial-sigs/user.conf and set malwarepatrol_free="no" as otherwise my product code, which is 32 instead of 8, get ignored and the list is not working. I'm unsure, why I got another code than others, however with 8 it's not working and malwarepatrol_free="yes" seems to override malwarepatrol_product_code, although set to 32. Now everything seems to work fine, need to delete /var/lib/clamav-unofficial-sigs/configs/last-mbl-update.txt for testing.
Additional comment:
securiteinfo.hdb seems not to hang, it just take about 15 minutes, once it took 30 minutes as seen in the log file, to update the file. Strange, but that's the reason why it looked like the script hang. As the server, I'm testing Proxmox with, has 2.6 Tbit/s internet connectivity, internet connect can't be the reason. However, it's working now, that's the most important.
I removed singular.ttk.pte.hu from my blacklists as United Internet (1&1, GMX, web.de, ...) seems not be able to hold their servers under control and got listed. Although I whitelisted the DTAG/Deutsche Telekom, I won't do with United Internet as DTAG/Deutsche Telekom always get listed on Spamhaus or Spamcop, which from my point of opinion are industry standards, meanwhile singular.ttk.pte.hu is from a hungarian university for primary internal use, so it's better to remove this list from blocking and consider to add for tagging instead of whitelisting false-positives. I will think about adding bl.spameatingmonkey.net as many people like this list, but I will see.
I was in need to adjust /etc/clamav-unofficial-sigs/user.conf and set malwarepatrol_free="no" as otherwise my product code, which is 32 instead of 8, get ignored and the list is not working. I'm unsure, why I got another code than others, however with 8 it's not working and malwarepatrol_free="yes" seems to override malwarepatrol_product_code, although set to 32. Now everything seems to work fine, need to delete /var/lib/clamav-unofficial-sigs/configs/last-mbl-update.txt for testing.
Additional comment:
securiteinfo.hdb seems not to hang, it just take about 15 minutes, once it took 30 minutes as seen in the log file, to update the file. Strange, but that's the reason why it looked like the script hang. As the server, I'm testing Proxmox with, has 2.6 Tbit/s internet connectivity, internet connect can't be the reason. However, it's working now, that's the most important.
One more, I disabled Configuraion => Option => Use advanced statistic filters to be able to see statistics for incoming filtering only, otherwise would require outgoing mails also going through PMG to be able to see any statistics per user.
And last one for today:
As mentioned before, minimum Spam Level is 4 for me to prevent false-positives, running well, also I don't use Quarantine, I just Modify Spam Subject. I renamed this one as mentioned before. I also renamed the ones with Viruses and Dangerous Files incoming to Remove and replaced Block and Notify Admin with Removing attachment (Virus or Dangerous File).
As mentioned before, minimum Spam Level is 4 for me to prevent false-positives, running well, also I don't use Quarantine, I just Modify Spam Subject. I renamed this one as mentioned before. I also renamed the ones with Viruses and Dangerous Files incoming to Remove and replaced Block and Notify Admin with Removing attachment (Virus or Dangerous File).
Thx!! I never figure out this when experiencing the issue and nobody clarify this .
Work continued. As I changed DCC to a service and adjusted my "worklog", I will continue with number 64 (3 additional rows been added as stated above to change the symlink for dcc to real service setup, thanks to Davide).
I now set up OpenVPN access to the server only. So I used an OpenVPN setup script to be very pragmatic and just adjusted some values:
64 cd /tmp
65 wget https://github.com/Angristan/OpenVPN-install/archive/master.zip
66 unzip master.zip
67 cd OpenVPN-install-master
68 chmod +x openvpn-install.sh
69 ./openvpn-install.sh
I left the detected IP address (however recognized, that this IP address is used as remote address value in client config, which I changed later on to the hostname) left port and protocol as given, I chose resolvers DNS servers, as I also removed that line later on pushing DNS servers as I don't need a road warrior setup but just a server access setup, selected the fastest options for data channel, DH and RSA (as I just want to block out script kiddies etc. from accessing or trying to access SSH or GUI, but it's no rocket science done here), left the client name and let the script set up everything for me.
Welcome to the secure OpenVPN installer (github.com/Angristan/OpenVPN-install)
I need to ask you a few questions before starting the setup
You can leave the default options and just press enter if you are ok with them
I need to know the IPv4 address of the network interface you want OpenVPN listening to.
If your server is running behind a NAT, (e.g. LowEndSpirit, Scaleway) leave the IP address as it is. (local/private IP)
Otherwise, it should be your public IPv4 address.
IP address: xxx.xxx.xxx.xxx
What port do you want for OpenVPN?
Port: 1194
What protocol do you want for OpenVPN?
Unless UDP is blocked, you should not use TCP (unnecessarily slower)
Protocol [UDP/TCP]: UDP
What DNS do you want to use with the VPN?
1) Current system resolvers (from /etc/resolv.conf)
2) Cloudflare (Anycast: worldwide)
3) Quad9 (Anycast: worldwide)
4) FDN (France)
5) DNS.WATCH (Germany)
6) OpenDNS (Anycast: worldwide)
7) Google (Anycast: worldwide)
8) Yandex Basic (Russia)
9) AdGuard DNS (Russia)
DNS [1-8]: 1
See https://github.com/Angristan/OpenVPN-install#encryption to learn more about
the encryption in OpenVPN and the choices I made in this script.
Please note that all the choices proposed are secure (to a different degree)
and are still viable to date, unlike some default OpenVPN options
Choose which cipher you want to use for the data channel:
1) AES-128-CBC (fastest and sufficiently secure for everyone, recommended)
2) AES-192-CBC
3) AES-256-CBC
Alternatives to AES, use them only if you know what you're doing.
They are relatively slower but as secure as AES.
4) CAMELLIA-128-CBC
5) CAMELLIA-192-CBC
6) CAMELLIA-256-CBC
7) SEED-CBC
Cipher [1-7]: 1
Choose what size of Diffie-Hellman key you want to use:
1) 2048 bits (fastest)
2) 3072 bits (recommended, best compromise)
3) 4096 bits (most secure)
DH key size [1-3]: 1
Choose what size of RSA key you want to use:
1) 2048 bits (fastest)
2) 3072 bits (recommended, best compromise)
3) 4096 bits (most secure)
RSA key size [1-3]: 1
Finally, tell me a name for the client certificate and configuration
Please, use one word only, no special characters
Client name: client
Okay, that was all I needed. We are ready to setup your OpenVPN server now
Press any key to continue...
...
Finished!
Your client config is available at /root/client.ovpn
If you want to add more clients, you simply need to run this script another time!
70 vi /etc/openvpn/server.conf
I removed the both push lines
push "dhcp-option DNS 127.0.0.1"
push "redirect-gateway def1 bypass-dhcp"
because DHCP makes no sense and I don't need a VPN DNS for administrating my server and for sure I don't want all traffic going through the server
71 service openvpn restart
After changing the values, I restarted OpenVPN server
72 vi /root/client.ovpn
I changed the IP address to the hostname and removed the line
setenv opt block-outside-dns
for the same reason as above (this would prevent my local DNS server settings from working).
Then I downloaded the client.ovpn to my systems, added it to their config and tested the VPN connection.
73 ufw delete allow 8006
74 ufw allow from 10.8.0.2 to any port 8006
75 ufw delete allow ssh
76 ufw allow from 10.8.0.2 to any port ssh
I removed the old firewall rules allowing full internet access to port 8006 (first tested with GUI to prevent locking myself out from ssh before I have proofed concept) and ssh (after successful testing).
Now my last topic is to set up a backup script.
I now set up OpenVPN access to the server only. So I used an OpenVPN setup script to be very pragmatic and just adjusted some values:
64 cd /tmp
65 wget https://github.com/Angristan/OpenVPN-install/archive/master.zip
66 unzip master.zip
67 cd OpenVPN-install-master
68 chmod +x openvpn-install.sh
69 ./openvpn-install.sh
I left the detected IP address (however recognized, that this IP address is used as remote address value in client config, which I changed later on to the hostname) left port and protocol as given, I chose resolvers DNS servers, as I also removed that line later on pushing DNS servers as I don't need a road warrior setup but just a server access setup, selected the fastest options for data channel, DH and RSA (as I just want to block out script kiddies etc. from accessing or trying to access SSH or GUI, but it's no rocket science done here), left the client name and let the script set up everything for me.
Welcome to the secure OpenVPN installer (github.com/Angristan/OpenVPN-install)
I need to ask you a few questions before starting the setup
You can leave the default options and just press enter if you are ok with them
I need to know the IPv4 address of the network interface you want OpenVPN listening to.
If your server is running behind a NAT, (e.g. LowEndSpirit, Scaleway) leave the IP address as it is. (local/private IP)
Otherwise, it should be your public IPv4 address.
IP address: xxx.xxx.xxx.xxx
What port do you want for OpenVPN?
Port: 1194
What protocol do you want for OpenVPN?
Unless UDP is blocked, you should not use TCP (unnecessarily slower)
Protocol [UDP/TCP]: UDP
What DNS do you want to use with the VPN?
1) Current system resolvers (from /etc/resolv.conf)
2) Cloudflare (Anycast: worldwide)
3) Quad9 (Anycast: worldwide)
4) FDN (France)
5) DNS.WATCH (Germany)
6) OpenDNS (Anycast: worldwide)
7) Google (Anycast: worldwide)
8) Yandex Basic (Russia)
9) AdGuard DNS (Russia)
DNS [1-8]: 1
See https://github.com/Angristan/OpenVPN-install#encryption to learn more about
the encryption in OpenVPN and the choices I made in this script.
Please note that all the choices proposed are secure (to a different degree)
and are still viable to date, unlike some default OpenVPN options
Choose which cipher you want to use for the data channel:
1) AES-128-CBC (fastest and sufficiently secure for everyone, recommended)
2) AES-192-CBC
3) AES-256-CBC
Alternatives to AES, use them only if you know what you're doing.
They are relatively slower but as secure as AES.
4) CAMELLIA-128-CBC
5) CAMELLIA-192-CBC
6) CAMELLIA-256-CBC
7) SEED-CBC
Cipher [1-7]: 1
Choose what size of Diffie-Hellman key you want to use:
1) 2048 bits (fastest)
2) 3072 bits (recommended, best compromise)
3) 4096 bits (most secure)
DH key size [1-3]: 1
Choose what size of RSA key you want to use:
1) 2048 bits (fastest)
2) 3072 bits (recommended, best compromise)
3) 4096 bits (most secure)
RSA key size [1-3]: 1
Finally, tell me a name for the client certificate and configuration
Please, use one word only, no special characters
Client name: client
Okay, that was all I needed. We are ready to setup your OpenVPN server now
Press any key to continue...
...
Finished!
Your client config is available at /root/client.ovpn
If you want to add more clients, you simply need to run this script another time!
70 vi /etc/openvpn/server.conf
I removed the both push lines
push "dhcp-option DNS 127.0.0.1"
push "redirect-gateway def1 bypass-dhcp"
because DHCP makes no sense and I don't need a VPN DNS for administrating my server and for sure I don't want all traffic going through the server
71 service openvpn restart
After changing the values, I restarted OpenVPN server
72 vi /root/client.ovpn
I changed the IP address to the hostname and removed the line
setenv opt block-outside-dns
for the same reason as above (this would prevent my local DNS server settings from working).
Then I downloaded the client.ovpn to my systems, added it to their config and tested the VPN connection.
73 ufw delete allow 8006
74 ufw allow from 10.8.0.2 to any port 8006
75 ufw delete allow ssh
76 ufw allow from 10.8.0.2 to any port ssh
I removed the old firewall rules allowing full internet access to port 8006 (first tested with GUI to prevent locking myself out from ssh before I have proofed concept) and ssh (after successful testing).
Now my last topic is to set up a backup script.
I did it, for now, I'm finished with the setup, next step is to set another system up and test it for the company.
77 apt-get install lftp
78 mkdir /backup
79 mkdir /backup/node
80 mkdir /root/scripts
81 vi /root/scripts/backup.sh
#!/bin/bash
echo [`date +'%d.%m.%Y %H:%M'`] Starte Backup ...
echo [`date +'%d.%m.%Y %H:%M'`] Exportiere aktuelle Konfiguration [1/4] ...
rm /var/lib/pmg/backup/*
pmgbackup backup
# Zu sicherndes Verzeichnisse, z.B. /etc /root /var
sourcevrz='/etc /root /usr/local /var/lib/pmg/backup /var/log'
# Backupverzeichnis, z.B. /backup/node
backupvrz=/backup/node
# Variablendefinition
datum=$(date +'%Y%m%d')
dateiname=$backupvrz/backup$datum.tar
# -----------------------------------------------------
function f_delFiles()
# -----------------------------------------------------
# $1 Backupverzeichnis
{
loeschdatum=$(date --date='7 days ago' +'%Y%m%d')
rm $1/backup$loeschdatum.tar
}
echo [`date +'%d.%m.%Y %H:%M'`] Lösche Files in $backupvrz, die älter als 7 Tage sind [2/4] ...
f_delFiles $backupvrz
echo [`date +'%d.%m.%Y %H:%M'`] Sichere $sourcevrz nach $dateiname [3/4] ...
tar Pcf $dateiname $sourcevrz
echo [`date +'%d.%m.%Y %H:%M'`] Synchronisiere mit Onlinespeicher [4/4] ...
bash /root/scripts/upload.sh
echo [`date +'%d.%m.%Y %H:%M'`] Fertig!
82 vi /root/scripts/upload.sh
#!/bin/bash
HOST='xxxx'
USER='xxxx'
PASS='xxxx'
TARGETFOLDER='/xxxx'
SOURCEFOLDER='/backup'
lftp -e "
open $HOST
user $USER $PASS
lcd $SOURCEFOLDER
mirror --reverse --delete --use-cache --verbose $SOURCEFOLDER $TARGETFOLDER
bye
"
83 chmod +x /root/scripts/backup.sh
84 chmod +x /root/scripts/upload.sh
85 /root/scripts/backup.sh
Testing the script.
86 vi /etc/crontab
0 2 * * * root /root/scripts/backup.sh 2>&1 | /usr/bin/mail -r xxxx@xxxx -s "backup script output" xxxx@xxxx
77 apt-get install lftp
78 mkdir /backup
79 mkdir /backup/node
80 mkdir /root/scripts
81 vi /root/scripts/backup.sh
#!/bin/bash
echo [`date +'%d.%m.%Y %H:%M'`] Starte Backup ...
echo [`date +'%d.%m.%Y %H:%M'`] Exportiere aktuelle Konfiguration [1/4] ...
rm /var/lib/pmg/backup/*
pmgbackup backup
# Zu sicherndes Verzeichnisse, z.B. /etc /root /var
sourcevrz='/etc /root /usr/local /var/lib/pmg/backup /var/log'
# Backupverzeichnis, z.B. /backup/node
backupvrz=/backup/node
# Variablendefinition
datum=$(date +'%Y%m%d')
dateiname=$backupvrz/backup$datum.tar
# -----------------------------------------------------
function f_delFiles()
# -----------------------------------------------------
# $1 Backupverzeichnis
{
loeschdatum=$(date --date='7 days ago' +'%Y%m%d')
rm $1/backup$loeschdatum.tar
}
echo [`date +'%d.%m.%Y %H:%M'`] Lösche Files in $backupvrz, die älter als 7 Tage sind [2/4] ...
f_delFiles $backupvrz
echo [`date +'%d.%m.%Y %H:%M'`] Sichere $sourcevrz nach $dateiname [3/4] ...
tar Pcf $dateiname $sourcevrz
echo [`date +'%d.%m.%Y %H:%M'`] Synchronisiere mit Onlinespeicher [4/4] ...
bash /root/scripts/upload.sh
echo [`date +'%d.%m.%Y %H:%M'`] Fertig!
82 vi /root/scripts/upload.sh
#!/bin/bash
HOST='xxxx'
USER='xxxx'
PASS='xxxx'
TARGETFOLDER='/xxxx'
SOURCEFOLDER='/backup'
lftp -e "
open $HOST
user $USER $PASS
lcd $SOURCEFOLDER
mirror --reverse --delete --use-cache --verbose $SOURCEFOLDER $TARGETFOLDER
bye
"
83 chmod +x /root/scripts/backup.sh
84 chmod +x /root/scripts/upload.sh
85 /root/scripts/backup.sh
Testing the script.
86 vi /etc/crontab
0 2 * * * root /root/scripts/backup.sh 2>&1 | /usr/bin/mail -r xxxx@xxxx -s "backup script output" xxxx@xxxx
I'm not yet completely happy with spam detection (for commercial test), so I did some additional adjustments:
I increased required spam score to 6 because of false-positives
I changed for virus and dangerous attachments also to add a header VIRUS and DANGEROUS as usually that would also be somehow spam mails and to inform the receipient, that there are changes to the message
I added Barracudacentral with 1.4 points as blacklist in SpamAssassin custom.cf
I added additional rules
I added and enabled GeoIP filter (also for bayes)
I added fromreplyto plugin
# apt-get install re2c
# cd /etc/cron.hourly && wget sa.schaal-it.net/sa-update && chown root.root sa-update && chmod 755 sa-update
# vi sa-update
#!/bin/sh
# schaal @it
#
# Simple script to update SpamAssassin
SYSLOG_TAG=sa-update
compile=0
logger -d -t $SYSLOG_TAG "Start SA-Update"
sa-update --nogpg
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi
sa-update --nogpg --channel updates.spamassassin.org
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi
sa-update --nogpg --channel sa.zmi.at
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi
sa-update --nogpg --channel sa.schaal-it.net
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi
sa-update --nogpg --channel sought.rules.yerp.org
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi
sa-update --nogpg --channel spamassassin.heinlein-support.de
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi
if [ $compile -eq 1 ]; then
logger -d -t $SYSLOG_TAG "SA-Update found"
sa-compile
systemctl restart pmg-smtp-filter.service
else
logger -d -t $SYSLOG_TAG "No SA-Update found"
fi
# ./sa-update
# cd /tmp
# wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz
# wget http://geolite.maxmind.com/download/geoip/database/GeoIPv6.dat.gz
# gunzip GeoIP.dat.gz
# gunzip GeoIPv6.dat.gz
# mkdir /usr/share/GeoIP
# mv GeoIP.dat /usr/share/GeoIP/
# mv GeoIPv6.dat /usr/share/GeoIP/
# vi /etc/mail/spamassassin/custom.cf
loadplugin Mail::SpamAssassin:
lugin:yzor
use_pyzor 1
loadplugin Mail::SpamAssassin:lugin:CC
dcc_path /usr/local/bin/dccproc
dcc_home /var/dcc
dcc_dccifd_path /var/dcc/dccifd
dcc_body_max 999999
dcc_fuz1_max 999999
dcc_fuz2_max 999999
use_dcc 1
dcc_timeout 10
loadplugin Mail::SpamAssassin:lugin::RelayCountry
add_header all Relay-Country _RELAYCOUNTRY_
header RELAYCOUNTRY_BAD X-Relay-Countries =~ /(CN|RU|UA|RO|VN)/
describe RELAYCOUNTRY_BAD Relayed through spammy country at some point
score RELAYCOUNTRY_BAD 2.0
header RELAYCOUNTRY_GOOD X-Relay-Countries =~ /^(DE|AT|CH)/
describe RELAYCOUNTRY_GOOD First untrusted GW is DE, AT or CH
score RELAYCOUNTRY_GOOD -0.5
header RCVD_IN_BRBL eval:check_rbl('brbl-lastexternal', 'b.barracudacentral.org.', '127.0.0.2')
describe RCVD_IN_BRBL Received via a relay in Barracuda RBL
tflags RCVD_IN_BRBL net
score RCVD_IN_BRBL 1.4
# wget https://github.com/extremeshok/spamassassin-extremeshok_fromreplyto/archive/master.zip
# unzip master.zip
# mkdir /etc/mail/spamassassin/plugins/
# cp spamassassin-extremeshok_fromreplyto-master/plugins/* /etc/mail/spamassassin/plugins/
# cp spamassassin-extremeshok_fromreplyto-master/01_extremeshok_fromreplyto.cf /etc/mail/spamassassin/
# systemctl restart pmg-smtp-filter
Next step will be to add http://artinvoice.hu/spams/ for getting bayes performing better.
I increased required spam score to 6 because of false-positives
I changed for virus and dangerous attachments also to add a header VIRUS and DANGEROUS as usually that would also be somehow spam mails and to inform the receipient, that there are changes to the message
I added Barracudacentral with 1.4 points as blacklist in SpamAssassin custom.cf
I added additional rules
I added and enabled GeoIP filter (also for bayes)
I added fromreplyto plugin
# apt-get install re2c
# cd /etc/cron.hourly && wget sa.schaal-it.net/sa-update && chown root.root sa-update && chmod 755 sa-update
# vi sa-update
#!/bin/sh
# schaal @it
#
# Simple script to update SpamAssassin
SYSLOG_TAG=sa-update
compile=0
logger -d -t $SYSLOG_TAG "Start SA-Update"
sa-update --nogpg
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi
sa-update --nogpg --channel updates.spamassassin.org
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi
sa-update --nogpg --channel sa.zmi.at
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi
sa-update --nogpg --channel sa.schaal-it.net
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi
sa-update --nogpg --channel sought.rules.yerp.org
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi
sa-update --nogpg --channel spamassassin.heinlein-support.de
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi
if [ $compile -eq 1 ]; then
logger -d -t $SYSLOG_TAG "SA-Update found"
sa-compile
systemctl restart pmg-smtp-filter.service
else
logger -d -t $SYSLOG_TAG "No SA-Update found"
fi
# ./sa-update
# cd /tmp
# wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz
# wget http://geolite.maxmind.com/download/geoip/database/GeoIPv6.dat.gz
# gunzip GeoIP.dat.gz
# gunzip GeoIPv6.dat.gz
# mkdir /usr/share/GeoIP
# mv GeoIP.dat /usr/share/GeoIP/
# mv GeoIPv6.dat /usr/share/GeoIP/
# vi /etc/mail/spamassassin/custom.cf
loadplugin Mail::SpamAssassin:
lugin:yzoruse_pyzor 1
loadplugin Mail::SpamAssassin:
lugin:CCdcc_path /usr/local/bin/dccproc
dcc_home /var/dcc
dcc_dccifd_path /var/dcc/dccifd
dcc_body_max 999999
dcc_fuz1_max 999999
dcc_fuz2_max 999999
use_dcc 1
dcc_timeout 10
loadplugin Mail::SpamAssassin:
lugin::RelayCountryadd_header all Relay-Country _RELAYCOUNTRY_
header RELAYCOUNTRY_BAD X-Relay-Countries =~ /(CN|RU|UA|RO|VN)/
describe RELAYCOUNTRY_BAD Relayed through spammy country at some point
score RELAYCOUNTRY_BAD 2.0
header RELAYCOUNTRY_GOOD X-Relay-Countries =~ /^(DE|AT|CH)/
describe RELAYCOUNTRY_GOOD First untrusted GW is DE, AT or CH
score RELAYCOUNTRY_GOOD -0.5
header RCVD_IN_BRBL eval:check_rbl('brbl-lastexternal', 'b.barracudacentral.org.', '127.0.0.2')
describe RCVD_IN_BRBL Received via a relay in Barracuda RBL
tflags RCVD_IN_BRBL net
score RCVD_IN_BRBL 1.4
# wget https://github.com/extremeshok/spamassassin-extremeshok_fromreplyto/archive/master.zip
# unzip master.zip
# mkdir /etc/mail/spamassassin/plugins/
# cp spamassassin-extremeshok_fromreplyto-master/plugins/* /etc/mail/spamassassin/plugins/
# cp spamassassin-extremeshok_fromreplyto-master/01_extremeshok_fromreplyto.cf /etc/mail/spamassassin/
# systemctl restart pmg-smtp-filter
Next step will be to add http://artinvoice.hu/spams/ for getting bayes performing better.
One more thing: I adjusted my blacklists, I changed sbl-xbl.spamhaus.org to zen.spamhaus.org and moved manitu to the end because of too much false-positives in the past, I will check, how much hits occur to this list and may consider to remove it or I will change my setup to postscreen with weights and tresholds, then I may add barracuda at blocking level and meanwhile all other lists will then have a weight of * 2, manitu and barracuda will have * 1 and minimum treshold will be 2 (so both or any other need to have the server listed to blacklist it). Not yet such happy with this decision, so I will wait a while, I also contacted heise as well as manitu, that their blacklist is currently running into getting less usable, it recently had false-positive quotes like spamhaus or spamcop (or sth. in between), but currently it's more like GBUdb, WPBL or one of the others, which more often have/had false-positives.
Again just a very small update, I /dev/null'd sa-compile in the cron job sa-update to "disable" the mails telling me about recompilation after updates. Just found, that updates occure about three times the day.
Update done as announced:
# vi /etc/cron.daily/sa-learn
# chmod +x /etc/cron.daily/sa-learn
# /etc/cron.daily/sa-learn
#!/bin/bash
spamfile=spam--`date --date='1 day ago' '+%Y-%m-%d'`.gz
spamfile_unpacked=spam--`date --date='1 day ago' '+%Y-%m-%d'`
wget http://artinvoice.hu/spams/$spamfile
gunzip $spamfile
sa-learn --mbox --spam --progress $spamfile_unpacked
systemctl restart pmg-smtp-filter
rm -rf $spamfile $spamfile_unpacked
exit 0
So I need to adjust the date, as cron.daily is running in the morning, so I want to add the spams of the last day and not the few ones of the current day. Also the spamassassin restart did not fit, so I also need to adjust that one. Maybe I will need to adjust tomorrow some additional /dev/nulls to get the cronjob "quiet", but I first want to see, if it's really working, so I will first keep it as is.
# vi /etc/cron.daily/sa-learn
# chmod +x /etc/cron.daily/sa-learn
# /etc/cron.daily/sa-learn
#!/bin/bash
spamfile=spam--`date --date='1 day ago' '+%Y-%m-%d'`.gz
spamfile_unpacked=spam--`date --date='1 day ago' '+%Y-%m-%d'`
wget http://artinvoice.hu/spams/$spamfile
gunzip $spamfile
sa-learn --mbox --spam --progress $spamfile_unpacked
systemctl restart pmg-smtp-filter
rm -rf $spamfile $spamfile_unpacked
exit 0
So I need to adjust the date, as cron.daily is running in the morning, so I want to add the spams of the last day and not the few ones of the current day. Also the spamassassin restart did not fit, so I also need to adjust that one. Maybe I will need to adjust tomorrow some additional /dev/nulls to get the cronjob "quiet", but I first want to see, if it's really working, so I will first keep it as is.
I now also /dev/null wget and sa-learn, however, the download and import works and today I had the first time bayes_* tags in my messages. I need to say, I use one PMG for my private mails, so it's only my family and me, about 7 mailboxes to deliver mails to, so in a productive environment it would be faster to get spams and hams, but in my setup, I have many hams autolearned but nearly no spam and the filter seems need to have some ratio to start bayes-tagging. The commercial PMG test environment I just set up to send all the mails through of all domains, which are not the primary one, we usually use to talk to customers, so the mail flow is really really low. Also learning the spams now didn't help to get bayes-tagging starting.
I also checked occurence of barracuda in spamassassin vs. nix spam false-positives and will first not set the threshold thing. I will try to get in touch with heise to report about the increased false-positive rate.
I also checked occurence of barracuda in spamassassin vs. nix spam false-positives and will first not set the threshold thing. I will try to get in touch with heise to report about the increased false-positive rate.
Short update: /dev/null doesn't really work, I now use --quiet, although should not be available on some commands, it seems to work. However, I still get some response, so I test no 2>/dev/null for the errors (some KAM rules seem to have zero score). I will report, once then it's completely working.
I would be happy, if anyone can assist on how to test Pyzor and DCC. None of my mails yet got tagged with Pyzor or DCC, so I'm afraid, it doesn't work.
I also would be happy, if I could reject (hard reject) mails with a special spam score (e.g. >10).
I would be happy, if anyone can assist on how to test Pyzor and DCC. None of my mails yet got tagged with Pyzor or DCC, so I'm afraid, it doesn't work.
I also would be happy, if I could reject (hard reject) mails with a special spam score (e.g. >10).
Scripts are "quiet" now ;-)
Here how I changed them:
#!/bin/bash
spamfile=spam--`date --date='1 day ago' '+%Y-%m-%d'`.gz
spamfile_unpacked=spam--`date --date='1 day ago' '+%Y-%m-%d'`
wget --quiet http://artinvoice.hu/spams/$spamfile
gunzip $spamfile
sa-learn --mbox --spam --quiet $spamfile_unpacked
systemctl restart pmg-smtp-filter
rm -rf $spamfile $spamfile_unpacked
exit 0
#!/bin/sh
# schaal @it
#
# Simple script to update SpamAssassin
SYSLOG_TAG=sa-update
compile=0
logger -d -t $SYSLOG_TAG "Start SA-Update"
sa-update --nogpg
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi
sa-update --nogpg --channel updates.spamassassin.org
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi
sa-update --nogpg --channel sa.zmi.at
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi
sa-update --nogpg --channel sa.schaal-it.net
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi
sa-update --nogpg --channel sought.rules.yerp.org
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi
sa-update --nogpg --channel spamassassin.heinlein-support.de
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi
if [ $compile -eq 1 ]; then
logger -d -t $SYSLOG_TAG "SA-Update found"
sa-compile --quiet 2>/dev/null
systemctl restart pmg-smtp-filter.service
else
logger -d -t $SYSLOG_TAG "No SA-Update found"
fi
Here how I changed them:
#!/bin/bash
spamfile=spam--`date --date='1 day ago' '+%Y-%m-%d'`.gz
spamfile_unpacked=spam--`date --date='1 day ago' '+%Y-%m-%d'`
wget --quiet http://artinvoice.hu/spams/$spamfile
gunzip $spamfile
sa-learn --mbox --spam --quiet $spamfile_unpacked
systemctl restart pmg-smtp-filter
rm -rf $spamfile $spamfile_unpacked
exit 0
#!/bin/sh
# schaal @it
#
# Simple script to update SpamAssassin
SYSLOG_TAG=sa-update
compile=0
logger -d -t $SYSLOG_TAG "Start SA-Update"
sa-update --nogpg
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi
sa-update --nogpg --channel updates.spamassassin.org
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi
sa-update --nogpg --channel sa.zmi.at
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi
sa-update --nogpg --channel sa.schaal-it.net
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi
sa-update --nogpg --channel sought.rules.yerp.org
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi
sa-update --nogpg --channel spamassassin.heinlein-support.de
retval="$?"
if [ $retval -eq 0 ]; then compile=1; fi
if [ $compile -eq 1 ]; then
logger -d -t $SYSLOG_TAG "SA-Update found"
sa-compile --quiet 2>/dev/null
systemctl restart pmg-smtp-filter.service
else
logger -d -t $SYSLOG_TAG "No SA-Update found"
fi
I now tried for commercial use and now start to need to do some adjustments.
1. I learned, that sa-learn form "foreign" archives let start bayesian tagging fast, BUT not valid as "my" spam mails differ from "their" spam mails, so I get scores of 0%, 40%, 60% etc. which are far away from reality. So I performed
rm /etc/cron.daily/sa-learn
sa-learn --clear
and now start a new bayes database. Too sad :-(
2. I found a link here in the forum I followed to add mail subjects to the logs:
#cd /etc/pmg/templates
#vi main.cf.in
adding header_checks = regexp:/etc/postfix/header_checks
#vi /etc/postfix/header_checks
with content /^subject:/ INFO
#pmgconfig sync --restart 1
However, this does only work on valid mails, not on rejected ones (where I would like to see the headers). I checked for smtpd_delay_reject, but it's already enabled and just delay until RCPT TO and not until the message content. So I still search for a solution to add subjects to the mail log also on rejected mails.
3. I'm sad, that so many server operators are not able to set up their servers. I was required to disable reject unknown clients, reject unknown senders as well as SPF and to increase spam tagging score to 9 on the commercial test instance.
Also Manitu NiX Spam has now too much false positives, after getpocket, facebook, pinterest, I know saw Linkedin and Unitymedia as well. It's now really too much too accept that, so I will change now to postscreen with treshold. I'm also sure, that Pyzor and DCC don't work, so I will change the setup now. I will report on this adjustments lateron.
1. I learned, that sa-learn form "foreign" archives let start bayesian tagging fast, BUT not valid as "my" spam mails differ from "their" spam mails, so I get scores of 0%, 40%, 60% etc. which are far away from reality. So I performed
rm /etc/cron.daily/sa-learn
sa-learn --clear
and now start a new bayes database. Too sad :-(
2. I found a link here in the forum I followed to add mail subjects to the logs:
#cd /etc/pmg/templates
#vi main.cf.in
adding header_checks = regexp:/etc/postfix/header_checks
#vi /etc/postfix/header_checks
with content /^subject:/ INFO
#pmgconfig sync --restart 1
However, this does only work on valid mails, not on rejected ones (where I would like to see the headers). I checked for smtpd_delay_reject, but it's already enabled and just delay until RCPT TO and not until the message content. So I still search for a solution to add subjects to the mail log also on rejected mails.
3. I'm sad, that so many server operators are not able to set up their servers. I was required to disable reject unknown clients, reject unknown senders as well as SPF and to increase spam tagging score to 9 on the commercial test instance.
Also Manitu NiX Spam has now too much false positives, after getpocket, facebook, pinterest, I know saw Linkedin and Unitymedia as well. It's now really too much too accept that, so I will change now to postscreen with treshold. I'm also sure, that Pyzor and DCC don't work, so I will change the setup now. I will report on this adjustments lateron.