Debugging Filter

[Joern Bredereck]

Hi,

I'm using 5.2.1 and would like to know how to troubleshoot/debug the mail filter. The information provided in /var/log/mail.log is very limited. For example I'm having issues with the office-attachment filter. In some cases it's filtering attachments even though there should be a matching whitelist rule that should match first and make sure that the office attachments from specific addresses get delivered. In other cases, office files are being delivered which should have been removed. Is there any way to get more detailed debugging logs from the filtering engine?

Thanks in advance!

Jörn Bredereck

 

[tom]

How did you configure your "office-attachment" filter exactly?

Check the settings against the emails (check email headers) which are not caught by the rule.

 

[Joern Bredereck]

How did you configure your "office-attachment" filter exactly?

Check the settings against the emails (check email headers) which are not caught by the rule.

As for now the biggest problem is the not-working white-listing of certain senders. Today an office attachment got removed even though the sender was whitelisted.

How can I troubleshoot problems like these? Is the any way to increase the verbosity of the logs?

 

[tom]

I doubt that you can see this is the logs, as I assume the sender is not white-listed correctly or your rules are wrong, do what I wrote above.

 

[Joern Bredereck]

I doubt that you can see this is the logs, as I assume the sender is not white-listed correctly or your rules are wrong, do what I wrote above.

I think you misunderstood me. This thread isn't about my specific filtering problem but about the general logging capabilities of Proxmox Mail Gateway in order to troubleshoot and diagnose ANY kind of filtering problems. I only stated my filtering problem to give you an example WHY verbose logging is neccessary to troubleshoot those kinds of problems.

 

[tom]

If you create a rule to filter for xy@domain.com and emails from abc@domain.com are arriving, no logs will be created as this is not an error, just a wrong rule.
You have to check your rule system and the emails in question. Please check the from fields and and envelope sender. As long as you do not tell these details, no one can help.

You can query the syslog via GUI, als the Message Tracking Center is a good frontend for logs.

 

[Joern Bredereck]

If you create a rule to filter for xy@domain.com and emails from abc@domain.com are arriving, no logs will be created as this is not an error, just a wrong rule.

Obviously. But what if you create an "accept-rule" for xy@domain.com and attachments from xy@domain.com get removed anyway? The logical first step would be analyzing the logs to find out how the filter parsed and compared the FROM-Field of the email with the entries in the white-list. For example I found out by trying that the filter seems to parse only the ENVELOPE-FROM (Return-Path) and not the FROM-field of the email header (which can be a problem if the sender is using BATV [1]). It would have been helpful to see in the logs which value for "FROM" had been parsed by the filter. This would have saved me some try-and-error-troubleshooting.

If the Proxmox Mail Gateway doesn't support vebose logging like that, fine... that's all I wanted to know. But please don't suggest that such information wouldn't be useful while troubleshooting cases like these.


[1] https://en.wikipedia.org/wiki/Bounce_Address_Tag_Validation

 

[tom]

Obviously. But what if you create an "accept-rule" for xy@domain.com and attachments from xy@domain.com get removed anyway?

very unlikely to happen. in 99,99 % of such issues the rule setup is wrong. (statistics based on 14 years mail gateway support experience)

The logical first step would be analyzing the logs to find out how the filter parsed and compared the FROM-Field of the email with the entries in the white-list. For example I found out by trying that the filter seems to parse only the ENVELOPE-FROM (Return-Path) and not the FROM-field of the email header (which can be a problem if the sender is using BATV [1]). It would have been helpful to see in the logs which value for "FROM" had been parsed by the filter. This would have saved me some try-and-error-troubleshooting.

That's why I pointed you to the mail header - so you can see envelope-from and from fields which you can compare against your rule setup.

If the Proxmox Mail Gateway doesn't support vebose logging like that, fine... that's all I wanted to know. But please don't suggest that such information wouldn't be useful while troubleshooting cases like these.


[1] https://en.wikipedia.org/wiki/Bounce_Address_Tag_Validation

My tips are very useful and I do not really understand your complaints here and I am sorry that I wasted your time ...

 

[Joern Bredereck]

very unlikely to happen. in 99,99 % of such issues the rule setup is wrong. (statistics based on 14 years mail gateway support experience)

I didn't want to imply that it's a bug in the software. I'm actually pretty sure it's a faulty rule. But in order to find the fault in the rule it would be helpful to see how the filter parsed and compared the addresses.

That's why I pointed you to the mail header - so you can see envelope-from and from fields which you can compare against your rule setup.

Unfortunately we don't always have access to our customer's emails (including the headers) for privacy reasons. Most of the time all I have at my disposal to troubleshoot a customer's complaint about a not working rule are the logs. If Proxmox Mail Gateway doesn't provide the neccessary info in the logs then I can't help my customers.

My tips are very useful and I do not really understand your complaints here and I am sorry that I wasted your time ...

There's no reason the get personal here. I appreciate your help. Telling me that Proxmox Mail Gateway is NOT able to log the debugging info I need to do qualified support for my customers was helpful after all. This way we know whether Proxmox Mail Gateway is the right software for our business. We're in the process of evaluating PMG at this point and the lack of detailed logs would be a show stopper for us.

I understand you're part of the developer team and I can take your word for it, that detailed logs are nothing you're planing to implement in your software in the near future? If so then I will include your statement in my evaluation report. Please confirm that you're authorized to speak for the development team in that matter. Thank you.