How to get the real from address?

Discussion in 'Mail Gateway: Installation and configuration' started by heutger, May 27, 2018.

  1. heutger

    heutger Active Member

    Joined:
    Apr 25, 2018
    Messages:
    331
    Likes Received:
    78
    I have mails in the tracking center like being from 12345-abcd-abcd-1234-abcd-abcd-000000@eu-west-1.amazonses.com, but they aren't from this sender, they are from e.g. Samsung or other users of Amazon Web Services / Mail Services, same for other service providers like mcsv.net (Mailchimp), ... Some solutions (like my current rspamd test) are able to show the "real" sender, please can you provide on how to enable PMG to show them also? I would expect latest in the statistics (fetched from SpamAssassin as I believe) the real senders been shown?
     
    #1 heutger, May 27, 2018
    Last edited: May 28, 2018
  2. dietmar

    dietmar Proxmox Staff Member
    Staff Member

    Joined:
    Apr 28, 2005
    Messages:
    16,278
    Likes Received:
    279
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  3. heutger

    heutger Active Member

    Joined:
    Apr 25, 2018
    Messages:
    331
    Likes Received:
    78
    I am afraid, it is, but it doesn’t help to analyze the messages. So how to fetch the real from and/or can’t you use it in the SpamAssassin driven views, as there you know about them?
     
  4. dietmar

    dietmar Proxmox Staff Member
    Staff Member

    Joined:
    Apr 28, 2005
    Messages:
    16,278
    Likes Received:
    279
    The tracking center is not meant to be used for analyzing messages. You can use it to analyze SMTP problems.

    Sorry, but what do you think is the 'real' from?
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  5. heutger

    heutger Active Member

    Joined:
    Apr 25, 2018
    Messages:
    331
    Likes Received:
    78
    I use the tracking center to analyze if PMG is working correct and as expected. E.g. I check all the rejects and that way recognized too many false positives with nixspam and singulari blacklist. I recognized lame server admins at commercial test setup and disabled SPF, sender and client domain checking. I checked bayes_ occured in spamassassin symbols and that importing spam for sa-learn was a bad idea, I checked blocking of surbl and others and decided to install a local nameserver to prevent blocking, I checked dcc and pyzor started first after enabling modules in init.pre.in instead of using custom.cf as enabling seems to be too late there and I check nixspam, barracudacentral and wpbl quality after deprecating them to a weight 1 rbl in postscreen using a treshold of 2 as well as adding them as symbols to spamassassin with a score of 1.4. I also check autolearned ham and spam here. Somehow, if I would like to decide if PMG worked right and spam detection as well (however as already wrote, I would prefer to be able to jump to spamassassin scores etc. from here like been shown in the statistics, as SA score shown in postfix differ from the spam level been set in statistics (strangest thing is, that the spam levels are e.g. 12/5 (looks like 12 points of 5?)) and so I need to see the "real" from.

    OK, it's somehow vice versa, you show the real from, but I want to see, from whom the mail comes from, so what I mean with "real" from is what is later shown in the mail clients. This from seems to be written in quotation marks to superset the real sender address or superset in the body of the message because of solutions like Amazon SES, Mailchimp, but also others, would like to detect bounces meanwhile allowing recipients to respond to the mail directly (without the need of setting a different reply to or getting tagged like my recent fromto plugin from extremeshock did). It's obvious, that it's the reason to do so, if looking at senders like bounces+123456-abcd-address=domain.tld@sg.e.doodle.com which finally are shown as mailer@doodle.com sending a mail to address@domain.tld. However, meanwhile this senders are recognizable, 12345-12345-abcd-abcd-abcd-12345-000000@eu-west-1.amazonses.com isn't. On my rspamd test, I get both, ID which is the original address as recognized by PMG in both views (tracking center and statistics) also, but the "real" from is also shown there, helping me in deciding faster, if the mail was tagged, rejected or what else right or not.
     
  6. dietmar

    dietmar Proxmox Staff Member
    Staff Member

    Joined:
    Apr 28, 2005
    Messages:
    16,278
    Likes Received:
    279
    Are you aware of the fact that this information is faked most times? What your call 'real' is in effect wrong and misleading ...
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  7. heutger

    heutger Active Member

    Joined:
    Apr 25, 2018
    Messages:
    331
    Likes Received:
    78
    May be, so that's why it would be great to have both, "set" sender and "given" sender, so it's possible on legit changes of bounce checks in services like the big mailer services to see the sender, the mail will come from, meanwhile for faked ones still be able to see the incoming ID.
     
  8. heutger

    heutger Active Member

    Joined:
    Apr 25, 2018
    Messages:
    331
    Likes Received:
    78
    I have an open question on Stack Overflow, if it's possible to see From, To and Subject also on rejected mails (delaying reject seems not to delay enough) but for mails running into the filter, I found the solution for my question:

    /etc/postfix/header_checks:
    /^From:/ INFO
    /^To:/ INFO
    /^Subject:/ INFO

    and I see in the log expand in the tracking center the given from, to and subject beside the sender been announced by the delivering server. I just would now want to see this information in the tracking center and/or statistics as well. Anyone can help?
     
    luke5sky likes this.
  9. Spanguel

    Spanguel New Member

    Joined:
    Nov 4, 2013
    Messages:
    19
    Likes Received:
    1
    Hi.

    I created the file /etc/postfix/header_checks with the following content:
    /^subject:/ WARN
    /^to:/ WARN
    /^from:/ WARN
    /^Subject:/ WARN
    /^To:/ WARN
    /^From:/ WARN

    After that u need to tell that postfix. Postfix' config is build on startup so it will be overwritten.
    I decided to edit the template file to include that.

    So according to the doc here: https://<yourgateway>:8006/pmg-docs/pmg-admin-guide.html#_service_configuration_templates
    I copied the file /var/lib/pmg/templates/main.cf.in to /etc/pmg/templates/main.cf.in and added the following line at the end:
    header_checks = regexp:/etc/postfix/header_checks

    After restarting the postfix service the Tracking Center looks like the following:
    upload_2018-5-30_8-26-5.png

    Thats the first thing i was missing in the Tracking Center. I would like to have that Info in a cell.

    EDIT: i see you already have that setup but i leave it here for anyone else interessted in setting it up that way.

    I think the rejected Mail is rejected even before it tells us what and where it wants to send, so i dont think thats possible.
     
    #9 Spanguel, May 30, 2018
    Last edited: May 30, 2018
  10. heutger

    heutger Active Member

    Joined:
    Apr 25, 2018
    Messages:
    331
    Likes Received:
    78
    Hi, that’s what I did also, I just used the one uppercase rest lowercase variant. I’m not sure, why to add both variants, for just adding the subject as I did before all explanations only list one case variant.
     
  11. Spanguel

    Spanguel New Member

    Joined:
    Nov 4, 2013
    Messages:
    19
    Likes Received:
    1
    I haven't double checked the lower or upper case. I found that in a Forum and tested it. It worked, so i didn't pay any attention to the lower case. But I will change it to your template. And to INFO and not WARN. Thanks for that. :)
     
  12. heutger

    heutger Active Member

    Joined:
    Apr 25, 2018
    Messages:
    331
    Likes Received:
    78
    ;-) I just wondered, if it's required or not, but if it's required as some mails get lowercase, others get uppercase, my recent only one case subject addon wouldn't had worked and would require more combinations like FROM, TO and SUBJECT. I also read about warn and info, that info got introduced later to postfix, that's why this post (I also found, just changed the order and cleaned it up ;-)) get use of warn instead of info.
     
  13. heutger

    heutger Active Member

    Joined:
    Apr 25, 2018
    Messages:
    331
    Likes Received:
    78
  14. heutger

    heutger Active Member

    Joined:
    Apr 25, 2018
    Messages:
    331
    Likes Received:
    78
    Help wanted: The subject shown in the details with the adjustments above is sometimes encoded (UTF8, base64, ...) and I need to manually decode it. Is there an option, I was not able to found yet, to decode the subject for/in the subject line via header checks, so I'm not required any more to manually decode via decoding tools by copy and paste? Thanks!
     
  15. luke5sky

    luke5sky New Member

    Joined:
    Jul 30, 2018
    Messages:
    7
    Likes Received:
    0
    if you found an answear to the encoding problem i would be interested to know how you did it!

    Thank you
     
  16. heutger

    heutger Active Member

    Joined:
    Apr 25, 2018
    Messages:
    331
    Likes Received:
    78
    Sadly no
     
  17. heutger

    heutger Active Member

    Joined:
    Apr 25, 2018
    Messages:
    331
    Likes Received:
    78
    I found the encoding "type" now and also a converter, but still search for a suitable solution.

    A converter, which works for the encoded headers is http://dogmamix.com/MimeHeadersDecoder/ as it's no "real" UTF8.

    https://www.admin-magazin.de/News/Tipps/ADMIN-Tipp-UTF-8-im-E-Mail-Subject as well as https://www.linuxquestions.org/ques...slate-utf-8-in-email-subject-line-4175604007/ gives hints on how to decode, but won't be able to be done integrated in the postfix header_checks and also I can't check such header information via postfix header_checks for e.g. rejecting. I'm currently not such happy to run a script over the full log file to decode as this solutions would provide support on how to do, so I'm still struggling to find a better solution.
     
  18. luke5sky

    luke5sky New Member

    Joined:
    Jul 30, 2018
    Messages:
    7
    Likes Received:
    0
    i know the struggle…
    i am using the same converter and simply added it to the favorites of my browser.
    Not the best solution, but it works and it is not that much of an issue for us.
     
  19. heutger

    heutger Active Member

    Joined:
    Apr 25, 2018
    Messages:
    331
    Likes Received:
    78
    It's just for investigating spam mail filter, if it's working well. Currently I just check once the week for three issues:
    - spamass-milter rejects
    - sender address rejected
    - client host rejected

    All my lists work well and I had no complaints. In the past I also used and checked SPF but because of problems it's off now. Sender address and Client host rejects I had no useful ones anymore (in the past, I also saw valid hosts from Amazon being rejected by client host, although changed from unknown_client_hostname performing FCrDNS checks to just unknown_reverse_client_hostname which only does a PTR check), so most careful to be checked are only the milter rejects. Once the week I also train my private filter. So there are rare situations, I need to check, however, I still prefer automatism instead of always follow a favorite link in my browser and perform the conversations manually. So maybe someone else in this forum can help out.
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice