[TUTORIAL] How to sync all Slave logs to the Master in real-time

[kez]

Tested Working on Mail Gateway 7.3-3

To get all cluster nodes to log their messages to the Master server > Tracking Center:

On Master:

Edit /etc/rsyslog.conf

# provides TCP syslog reception
module(load="imtcp")
input(type="imtcp" port="514")

Then: systemctl restart rsyslog

On Cluster memeber nodes:

Edit /etc/rsyslog.conf

###############
#### RULES ####
###############


#
# First some standard log files.  Log by facility.
#


*.*     @@<IPofMaster>

Then systemctl restart rsyslog

That's it! You now have all cluster member nodes logging to the Master.

PS - Remember to firewall/restrict port 514 on the Master server to only accept from the Slave member nodes.

 

[LeFred]

+1
And I use to specialize the master for admin & tracking. The slaves make SMTP works.
Good architecture when traffic growth ...

 

[vsupport]

Thanks..

 

[michabbs]

I tried to achieve this in current PMG 8. This looks strange to me:

Original log line on slave:
2023-07-10T20:22:09.310604+00:00 zack postfix/pickup[211940]: 4BAA1299E: uid=0 from=<root>

The same log line after transfer to master:
2023-07-10T20:22:09.310604+00:00 zack postfix[211940] 4BAA1299E: uid=0 from=<root>

Why they are not identical? What happened to /pickup part? How to fix it?

 

[vsupport]

Debian 12 Notes

https://www.debian.org/releases/sta...information.en.html#changes-to-system-logging

I do not know if this affects Proxmox as I have not yet made the jump to 8

 

[michabbs]

I am not able to get Tracking Center to work with shared logs on PMG 8. :-(

On master I added /etc/rsyslog.d/imtcp.conf:

# Provides TCP syslog reception
$ModLoad imtcp.so
$InputTCPServerRun 514
$AllowedSender TCP, ip.ip.ip.ip

On slave I added /etc/rsyslog.d/pmg-master.conf:

mail.*     @@slave.doman;RSYSLOG_SyslogProtocol23Format

Logs seems to be correcly sent from slave to master. I can see lines from both nodes in /var/log/mail.log on master:

2023-07-13T20:13:46.459954+00:00 master pmg-smtp-filter[276911]: end database maintenance (7 ms)
2023-07-13T20:14:50.615759+00:00 slave pmg-smtp-filter[303636] starting database maintenance

But in the Tracking Center on master there are only master-processed emails visible. :-(
Any ideas?

 

[LeFred]

I don't jump to v8 too... but it's important for me to know if this work in v8 !...

Did you try to simplify the config ? For example : without ";RSYSLOG_SyslogProtocol23Format" ...

 

[michabbs]

No luck :-(

 

[LeFred]

As vsupport notes, can you see if /var/log/syslog contains the lines too ?

 

[michabbs]

As vsupport notes, can you see if /var/log/syslog contains the lines too ?

Yes, everything is in /var/log/syslog too.

I am really curious why the log entries on both nodes are not identical (#4). The difference must come from some reason and this is where I would look for problems. But no idea what's the reason... :-(

 

[kienvt]

I see it seems to be an error from rsyslog. On PMG8 I edited the rsyslog configuration on the master node as follows and it solved the problem.

# provides TCP syslog reception
#module(load="imtcp")
#input(type="imtcp" port="514")
$ModLoad imtcp
$InputTCPServerRun 514

 

[michabbs]

Slave requires /etc/rsyslog.d/pmg-master.conf:

mail.*     @@master.address;RSYSLOG_ForwardFormat

...and voila! It works.