Tracking Center not in sync
- Thread starter Thomas Hukkelberg
- Start date
I made my own solution with earlier ideas. Since I manage 4 big PMG servers, syncing and using cat with SSH on the fly is not an option. So I create a script to cache the syslog and merge them for all servers and changed the original: /usr/bin/pmg-log-tracker to place this cache file to the live (/var/log/syslog) location, load: /usr/bin/pmg-log-tracker-default and set the original syslog back (so this will not get corrupt).
Steps (logged in as root):
1. Back-up original: pmg-log-tracker
2. Create new: /usr/bin/pmg-log-tracker file with this code:
3. Give it execution rights.
Now create a script to generate: /tmp/syslog
1. Create file: /root/merge-syslog.sh with this code:
2. Give it execution rights.
3. Create cronjob, (runtime depends on time to execute /root/merge-syslog.sh)
Warning, your data in the tracker-log can have a delay of 10 minutes with this example. With a small amount of mail, you can run it every 5 minutes or less.
Script is used at one node only, in my case mx1.
Steps (logged in as root):
1. Back-up original: pmg-log-tracker
Code:
mv /usr/bin/pmg-log-tracker /usr/bin/pmg-log-tracker-default
Code:
mv /var/log/syslog /var/log/syslog-tmp
mv /tmp/syslog /var/log/syslog
/usr/bin/pmg-log-tracker-default "$@"
mv /var/log/syslog /tmp/syslog
mv /var/log/syslog-tmp /var/log/syslog
Code:
chmod +x /usr/bin/pmg-log-trackerNow create a script to generate: /tmp/syslog
1. Create file: /root/merge-syslog.sh with this code:
Code:
cat /var/log/syslog | grep -E 'postfix' > /tmp/syslog
ssh root@mx2 -o ConnectTimeout=2 -o ConnectionAttempts=1 'cat /var/log/syslog | grep -E postfix' >> /tmp/syslog
ssh root@mx3 -o ConnectTimeout=2 -o ConnectionAttempts=1 'cat /var/log/syslog | grep -E postfix' >> /tmp/syslog
ssh root@mx4-o ConnectTimeout=2 -o ConnectionAttempts=1 'cat /var/log/syslog | grep -E postfix' >> /tmp/syslog
cat /tmp/syslog | sort -M > /tmp/syslog-2
uniq /tmp/syslog-2 /tmp/syslog
Code:
chmod +x /root/merge-syslog.sh
Code:
*/10 * * * * /root/merge-syslog.shWarning, your data in the tracker-log can have a delay of 10 minutes with this example. With a small amount of mail, you can run it every 5 minutes or less.
Script is used at one node only, in my case mx1.
Hmmm,
After reading the solutions posted here, I still do not like the idea of having to potentially break the system because it interferes with the way proxmox mail works right now. Is it too much to ask to implement a function in the web interface that points to a different log file? When using the command line, pmg-log-tracker can already do this. That would be a win-win as far as I'm concerned. All I would have to do is collect all needed combined logs on one node and let the system look at that. When that file does not exist, the default file (/var/log/syslog) could be parsed instead. Looks like a small change to me, but hey ... I'm just a user
.
Best regards.
After reading the solutions posted here, I still do not like the idea of having to potentially break the system because it interferes with the way proxmox mail works right now. Is it too much to ask to implement a function in the web interface that points to a different log file? When using the command line, pmg-log-tracker can already do this. That would be a win-win as far as I'm concerned. All I would have to do is collect all needed combined logs on one node and let the system look at that. When that file does not exist, the default file (/var/log/syslog) could be parsed instead. Looks like a small change to me, but hey ... I'm just a user
.Best regards.
Thank you, your solution made my life easier =) I have a cluster with 3 larger devices. More than 20,000 clean e-mails go through each device every day. Even with the amount, your script has no problems!
This is just excellent, well done !! So, in my case, on the master server mx02 I did this:
Bash:
mv /usr/bin/pmg-log-tracker /usr/bin/pmg-log-tracker-default
cat >/usr/bin/pmg-log-tracker <<'SCRIPT_END'
#!/bin/bash
/usr/bin/pmg-log-tracker-default "$@" | head -n -1
ssh root@mx01 /usr/bin/pmg-log-tracker-default "$@" | sed '/^#/ d'
ssh root@mx03 /usr/bin/pmg-log-tracker-default "$@" | sed '/^#/ d'
SCRIPT_END
chmod +x /usr/bin/pmg-log-trackerWorks perfectly.
Hello and thank you all for the inspiration on how to sync tracking-logs.
is this still needed in PMGv8? (I assume yes, as I just ran into a similiar issue on a test-cluster)
is there a sane way to publish all logs to all cluster members so anyone can search anywhere for mail tracking?
thx.
is this still needed in PMGv8? (I assume yes, as I just ran into a similiar issue on a test-cluster)
is there a sane way to publish all logs to all cluster members so anyone can search anywhere for mail tracking?
thx.
Hi,
This is what I do for sync log
Now, you can track all log from dashboard server1
Good luck
This is what I do for sync log
- I have 3 PMG with name server1, server2, server3
- server1 will become master and receive log from other members
- Edit file /etc/rsyslog.conf on server1 and uncomment these line
Code:
# provides UDP syslog reception
module(load="imudp")
input(type="imudp" port="514")- on server2, and server3, edit file /etc/rsyslog.conf and add this line in the bottom line
Code:
mail.* @ip-address-server1- restart rsyslog on all servers
Code:
systemctl restart rsyslogNow, you can track all log from dashboard server1
Good luck
Does this work on PMG v8? Mine does not 
Syncing is working fine, I can see all syslog and mail logs from node2 on master node1:
But the tracking centre does not see it, and nor does:
Seems PMG is reading from systemd-journal and not /var/log/syslog now.
Does anyone have any ideas?
Thanks,
C
Syncing is working fine, I can see all syslog and mail logs from node2 on master node1:
Code:
root@mailgate1:~# grep 24095067E3A9A40CC3A /var/log/syslog
2025-03-26T07:15:48+00:00 mailgate2 pmg-smtp-filter[12069]: 24095067E3A9A40CC3A: new mail message-id=<D9.CA.09278.899A3E76@i-07f2d4481de636190.mta3vrest.sd.prd.sparkpost>#012
2025-03-26T07:15:48+00:00 mailgate2 pmg-smtp-filter[12069]: 24095067E3A9A40CC3A: Subject: Build your rewards!#012
2025-03-26T07:15:51+00:00 mailgate2 pmg-smtp-filter[12069]: 24095067E3A9A40CC3A: SA score=0/5 time=2.923 bayes=undefined autolearn=disabled hits=DKIMWL_WL_HIGH(-0.443),DKIM_SIGNED(0.1),DKIM_VALID(-0.1),DKIM_VALID_AU(-0.1),DKIM_VALID_EF(-0.1),DMARC_PASS(-0.1),HTML_FONT_LOW_CONTRAST(0.001),HTML_MESSAGE(0.001),KAM_EU(0.5),POISEN_SPAM_PILL(0.1),POISEN_SPAM_PILL_1(0.1),POISEN_SPAM_PILL_2(0.1),SPF_HELO_NONE(0.001),SPF_PASS(-0.001)
2025-03-26T07:15:51+00:00 mailgate2 pmg-smtp-filter[12069]: 24095067E3A9A40CC3A: accept mail to <X@XXX.co.uk> (3059D240952) (rule: default-accept)
2025-03-26T07:15:51+00:00 mailgate2 pmg-smtp-filter[12069]: 24095067E3A9A40CC3A: processing time: 3.19 seconds (2.923, 0.157, 0)
2025-03-26T07:15:51+00:00 mailgate2 postfix/smtpd[14546]: proxy-accept: END-OF-MESSAGE: 250 2.5.0 OK (24095067E3A9A40CC3A); from=<msprvs1=20180EhecHOij=bounces-1898-534@em.screwfix.com> to=<X@XXX.co.uk> proto=ESMTP helo=<z2vlqbzr.em.screwfix.com>But the tracking centre does not see it, and nor does:
Code:
root@mailgate1:~# pmg-log-tracker --verbose | grep 24095067E3A9A40CC3A
root@mailgate1:~#Seems PMG is reading from systemd-journal and not /var/log/syslog now.
Does anyone have any ideas?
Thanks,
C
Last edited:
Just another year has passed without the Tracking Center being synchronized across the different cluster nodes. Are there any updates on this?
I am seriously questioning whether it still makes sense to pay for multiple licenses, since diagnostics is our primary responsibility at PMG. In its current state, this setup is extremely inconvenient.
Are there any plans from the developer team to keep the Tracking Center synchronized in a future version of PMG?
I am seriously questioning whether it still makes sense to pay for multiple licenses, since diagnostics is our primary responsibility at PMG. In its current state, this setup is extremely inconvenient.
Are there any plans from the developer team to keep the Tracking Center synchronized in a future version of PMG?
@duderino Great, thank you.
For PMG 9, I simply commented out the following lines and allowed TCP 514 in the firewall settings:
If it’s really that simple and has been available since PMG 7, I still don’t understand why this isn’t enabled out of the box by the developers.
Maybe the developers could put this back on the agenda so it becomes an officially supported feature and survives updates. It’s truly a very important feature for cluster environments.
Has anyone also tested whether this works in both directions? In other words, can the Tracking Center on the respective slaves also be kept in sync?
For PMG 9, I simply commented out the following lines and allowed TCP 514 in the firewall settings:
Code:
Master comment out:
# /etc/rsyslog.conf
module(load="imtcp")
input(type="imtcp" port="514")
Slave(s) add:
# /etc/rsyslog.d/pmg-master.conf
mail.* @@<master-ip>;RSYSLOG_ForwardFormat
On master and slave: systemctl restart rsyslogIf it’s really that simple and has been available since PMG 7, I still don’t understand why this isn’t enabled out of the box by the developers.
Maybe the developers could put this back on the agenda so it becomes an officially supported feature and survives updates. It’s truly a very important feature for cluster environments.
Has anyone also tested whether this works in both directions? In other words, can the Tracking Center on the respective slaves also be kept in sync?
Last edited: