Tracking Center not in sync

I made my own solution with earlier ideas. Since I manage 4 big PMG servers, syncing and using cat with SSH on the fly is not an option. So I create a script to cache the syslog and merge them for all servers and changed the original: /usr/bin/pmg-log-tracker to place this cache file to the live (/var/log/syslog) location, load: /usr/bin/pmg-log-tracker-default and set the original syslog back (so this will not get corrupt).

Steps (logged in as root):

1. Back-up original: pmg-log-tracker
Code:
mv /usr/bin/pmg-log-tracker /usr/bin/pmg-log-tracker-default
2. Create new: /usr/bin/pmg-log-tracker file with this code:
Code:
mv /var/log/syslog /var/log/syslog-tmp
mv /tmp/syslog /var/log/syslog

/usr/bin/pmg-log-tracker-default "$@"

mv /var/log/syslog /tmp/syslog
mv /var/log/syslog-tmp /var/log/syslog
3. Give it execution rights.
Code:
chmod +x /usr/bin/pmg-log-tracker

Now create a script to generate: /tmp/syslog

1. Create file: /root/merge-syslog.sh with this code:
Code:
cat /var/log/syslog  | grep -E 'postfix' > /tmp/syslog
ssh root@mx2 -o ConnectTimeout=2 -o ConnectionAttempts=1 'cat /var/log/syslog | grep -E postfix' >> /tmp/syslog
ssh root@mx3 -o ConnectTimeout=2 -o ConnectionAttempts=1 'cat /var/log/syslog | grep -E postfix' >> /tmp/syslog
ssh root@mx4-o ConnectTimeout=2 -o ConnectionAttempts=1 'cat /var/log/syslog | grep -E postfix' >> /tmp/syslog
cat /tmp/syslog | sort -M > /tmp/syslog-2
uniq /tmp/syslog-2 /tmp/syslog
2. Give it execution rights.
Code:
chmod +x /root/merge-syslog.sh
3. Create cronjob, (runtime depends on time to execute /root/merge-syslog.sh)
Code:
*/10 * * * * /root/merge-syslog.sh

Warning, your data in the tracker-log can have a delay of 10 minutes with this example. With a small amount of mail, you can run it every 5 minutes or less.

Script is used at one node only, in my case mx1.
 
  • Like
Reactions: Skata and DerDanilo
Hmmm,

After reading the solutions posted here, I still do not like the idea of having to potentially break the system because it interferes with the way proxmox mail works right now. Is it too much to ask to implement a function in the web interface that points to a different log file? When using the command line, pmg-log-tracker can already do this. That would be a win-win as far as I'm concerned. All I would have to do is collect all needed combined logs on one node and let the system look at that. When that file does not exist, the default file (/var/log/syslog) could be parsed instead. Looks like a small change to me, but hey ... I'm just a user ;).

Best regards.
 
I made my own solution with earlier ideas. Since I manage 4 big PMG servers, syncing and using cat with SSH on the fly is not an option. So I create a script to cache the syslog and merge them for all servers and changed the original: /usr/bin/pmg-log-tracker to place this cache file to the live (/var/log/syslog) location, load: /usr/bin/pmg-log-tracker-default and set the original syslog back (so this will not get corrupt).

Steps (logged in as root):

1. Back-up original: pmg-log-tracker
Code:
mv /usr/bin/pmg-log-tracker /usr/bin/pmg-log-tracker-default
2. Create new: /usr/bin/pmg-log-tracker file with this code:
Code:
mv /var/log/syslog /var/log/syslog-tmp
mv /tmp/syslog /var/log/syslog

/usr/bin/pmg-log-tracker-default "$@"

mv /var/log/syslog /tmp/syslog
mv /var/log/syslog-tmp /var/log/syslog
3. Give it execution rights.
Code:
chmod +x /usr/bin/pmg-log-tracker

Now create a script to generate: /tmp/syslog

1. Create file: /root/merge-syslog.sh with this code:
Code:
cat /var/log/syslog  | grep -E 'postfix' > /tmp/syslog
ssh root@mx2 -o ConnectTimeout=2 -o ConnectionAttempts=1 'cat /var/log/syslog | grep -E postfix' >> /tmp/syslog
ssh root@mx3 -o ConnectTimeout=2 -o ConnectionAttempts=1 'cat /var/log/syslog | grep -E postfix' >> /tmp/syslog
ssh root@mx4-o ConnectTimeout=2 -o ConnectionAttempts=1 'cat /var/log/syslog | grep -E postfix' >> /tmp/syslog
cat /tmp/syslog | sort -M > /tmp/syslog-2
uniq /tmp/syslog-2 /tmp/syslog
2. Give it execution rights.
Code:
chmod +x /root/merge-syslog.sh
3. Create cronjob, (runtime depends on time to execute /root/merge-syslog.sh)
Code:
*/10 * * * * /root/merge-syslog.sh

Warning, your data in the tracker-log can have a delay of 10 minutes with this example. With a small amount of mail, you can run it every 5 minutes or less.

Script is used at one node only, in my case mx1.
Thank you, your solution made my life easier =) I have a cluster with 3 larger devices. More than 20,000 clean e-mails go through each device every day. Even with the amount, your script has no problems!
 
Hello

I found a simpler solution that runs without CronJob.

I have modified the file /usr/bin/pmg-tracker as follows:
/usr/bin/pmg-log-tracker-default "$@" | head -n -1
ssh root@mx2 /usr/bin/pmg-log-tracker-default "$@" | sed '/^#/ d'

Best regards
Martin Krüger
 
  • Like
Reactions: jdomenech
I found a simpler solution that runs without CronJob.
This is just excellent, well done !! So, in my case, on the master server mx02 I did this:

Bash:
mv /usr/bin/pmg-log-tracker /usr/bin/pmg-log-tracker-default

cat >/usr/bin/pmg-log-tracker <<'SCRIPT_END'
#!/bin/bash
/usr/bin/pmg-log-tracker-default "$@" | head -n -1
ssh root@mx01 /usr/bin/pmg-log-tracker-default "$@" | sed '/^#/ d'
ssh root@mx03 /usr/bin/pmg-log-tracker-default "$@" | sed '/^#/ d'
SCRIPT_END

chmod +x /usr/bin/pmg-log-tracker

Works perfectly.
 
  • Like
Reactions: jdomenech

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!