pmg-api 7.2-3 can turn on SMTPUTF8, breaking mail flow

T

trichardson

New Member
Jun 17, 2021
8
6
3
42
The modification to Utils.pm in pmg-api v7.2-3 will turn on SMTPUTF8 when it sees non-ASCII characters in the To/From addresses. This is problematic when emails are received with non-ASCII in the To/From fields and either 1) smptutf8_enable=no is configured in Postfix locally; or 2) the downstream SMTP server doesn't support SMTPUTF8. In case 1, SMTPUTF8 is enabled and postfix rejects the reinjected email. In case 2, the mail gets reinjected successfully with SMTPUTF8 enabled but then rejected by the downstream server which doesn't support it.

It seems like this would break quite a few installations out there that are similar to ours: lots of mail with non-ASCII in To/From and a downstream SMTP server that doesn't support SMTPUTF8 (Exchange 2010 :-X)

I disabled the line in Utils.pm that conditionally added the SMTPUTF8 option which fixed our problem, and that's fine because we don't want/can't have SMTPUTF8. A proper solution should probably check to see if SMTPUTF8 is enabled in the local Postfix before setting the option.

-Tom
 
  • Like
Reactions: LinuxCuba
trichardson said:
The modification to Utils.pm in pmg-api v7.2-3 will turn on SMTPUTF8 when it sees non-ASCII characters in the To/From addresses.
Click to expand...
yes - if the envelope addresses contain non-ascii characters then smtputf8 is needed ?

If you downstream server does not support smtputf8 you need to disable it in the PMG postfix config.

Did this work for you before 7.2-3 ? (As I was under the impression and had a few cases of similar setups that did have to disable smtputf8 in postfix)

Could you please provide some logs and sample mails (that did work before but not with 7.2-3) ?

Thanks!
 
Stoiko,

The timeline, from my point of view is this:
1. We have been successfully receiving mails with UTF8 in the envelope addresses for years, through PMG, into an Exchange 2010 server

2. An update applied on January 5 broke mail flow. That update bumped pmg-api to 7.2-3. Emails were being bounced by PMG Postfix because SMTPUTF8 was now being required, but not offered by Exchange 2010:
Jan 8 01:00:23 gtpmg postfix/smtp[34680]: 6BC25C0CCB: to=<REDACTED@availink.com>, relay=OUREXCHANGESERVER[10.209.144.32]:25, delay=0.06, delays=0.05/0/0/0, dsn=5.6.7, status=bounced (SMTPUTF8 is required, but was not offered by host 10.209.144.32[10.209.144.32])

3. Furthermore, it looks like the undeliverable reports sent back to the sender also failed because of lack of SMTPUTF8 on their side:
Jan 8 01:00:24 gtpmg postfix/smtp[34680]: 7D2BEC0CF2: to=<REDACTED@Sigurd.com.tw>, relay=mg.Sigurd.com.tw[59.120.59.92]:25, delay=1.1, delays=0/0/1.1/0, dsn=5.6.7, status=bounced (SMTPUTF8 is required, but was not offered by host mg.Sigurd.com.tw[59.120.59.92])

4. Having newly learned about SMTPUTF8, I added smtputf8_enable=no to PMG Postfix main.c with the understanding that that would make Postfix ignorant to SMTPUTF8, restoring historical behavior. However, after doing that, mail flow was broken in a new way. PMG Postfix was suddenly unhappy to see SMTPUTF8 requested by PMG when reinjecting the email:
Jan 11 09:08:16 gtpmg postfix/qmgr[91208]: 6230DC167B: from=<REDACTED@sigurd.com.tw>, size=119588, nrcpt=8 (queue active) Jan 11 09:08:16 gtpmg pmg-smtp-filter[103815]: 2023/01/11-09:08:16 CONNECT TCP Peer: "[127.0.0.1]:54472" Local: "[127.0.0.1]:10024" Jan 11 09:08:16 gtpmg pmg-smtp-filter[103815]: C156363BEC2D06365B: new mail message-id=<7FC68833625C0348A2E1A8C40E123C660160A162A8@MBX4.sigurd.com.tw>#012 Jan 11 09:08:17 gtpmg pmg-smtp-filter[103815]: C156363BEC2D06365B: SA score=0/5 time=1.346 bayes=0.00 autolearn=no autolearn_force=no hits=AWL(-0.048),BAYES_00(-1.9),DKIM_INVALI D(0.1),DKIM_SIGNED(0.1),HTML_FONT_FACE_BAD(0.001),HTML_MESSAGE(0.001),KAM_DMARC_STATUS(0.01),KAM_MANYTO(0.2),SPF_HELO_NONE(0.001),SPF_PASS(-0.001) Jan 11 09:08:17 gtpmg postfix/smtpd[111884]: connect from localhost.localdomain[127.0.0.1] Jan 11 09:08:17 gtpmg pmg-smtp-filter[103815]: smtp error - got: 555 5.5.4 Unsupported option: SMTPUTF8#012 Jan 11 09:08:17 gtpmg pmg-smtp-filter[103815]: smtp from: ERROR at /usr/share/perl5/PMG/Utils.pm line 279. Jan 11 09:08:17 gtpmg pmg-smtp-filter[103815]: C156363BEC2D06365B: reinject mail to <REDACTED@availink.com> (rule: Whitelist) failed Jan 11 09:08:17 gtpmg pmg-smtp-filter[103815]: C156363BEC2D06365B: reinject mail to <REDACTED@availink.com> (rule: Whitelist) failed Jan 11 09:08:17 gtpmg pmg-smtp-filter[103815]: C156363BEC2D06365B: reinject mail to <REDACTED@availink.com> (rule: Whitelist) failed Jan 11 09:08:17 gtpmg pmg-smtp-filter[103815]: C156363BEC2D06365B: reinject mail to <REDACTED@availink.com> (rule: Whitelist) failed Jan 11 09:08:17 gtpmg pmg-smtp-filter[103815]: C156363BEC2D06365B: reinject mail to <REDACTED@availink.com> (rule: Whitelist) failed Jan 11 09:08:17 gtpmg pmg-smtp-filter[103815]: C156363BEC2D06365B: reinject mail to <REDACTED@availink.com> (rule: Whitelist) failed Jan 11 09:08:17 gtpmg pmg-smtp-filter[103815]: C156363BEC2D06365B: reinject mail to <REDACTED@availink.com> (rule: Whitelist) failed Jan 11 09:08:17 gtpmg pmg-smtp-filter[103815]: C156363BEC2D06365B: reinject mail to <REDACTED@availink.com> (rule: Whitelist) failed Jan 11 09:08:17 gtpmg postfix/smtpd[111884]: lost connection after MAIL from localhost.localdomain[127.0.0.1] Jan 11 09:08:17 gtpmg postfix/smtpd[111884]: disconnect from localhost.localdomain[127.0.0.1] ehlo=1 xforward=1 mail=0/1 commands=2/3 Jan 11 09:08:17 gtpmg pmg-smtp-filter[103815]: C156363BEC2D06365B: processing time: 1.404 seconds (1.346, 0.019, 0) Jan 11 09:08:17 gtpmg postfix/lmtp[111846]: 6230DC167B: to=<REDACTED@availink.com>, relay=127.0.0.1[127.0.0.1]:10024, delay=11327, delays=11325/0/0.04/1.4, dsn=4.4.0, status=defe rred (host 127.0.0.1[127.0.0.1] said: 451 4.4.0 detected undelivered mail to <REDACTED@availink.com> (in reply to end of DATA command))

5. My hack to restore mail flow was to comment out the line that was enabling SMTPUTF8 when envelope addresses contain non-ASCII characters. /usr/share/perl5/PMG/Utils.pm line 267:
#HACK $mail_opts .= " SMTPUTF8" if mail_needs_smtputf8($entity, $sender, $targets);

My understanding about SMTPUTF8, learned solely through this experience and some reading about it, is that it's apparently not strictly necessary in order for mail to be exchanged with non-ASCII envelope addresses. We had been doing this for years without it. #3 above shows that the sender does not support SMTPUTF8 and was not requesting it, so even though modern Postfix had support for it enabled by default, it was not being requested upon forwarding to our Exchange server until the change in pmg-api 7.2-3 was made to assert it independently of whether support for it was enabled or not.

Unfortunately I don't have the emails on hand, but I did verify that the envelope addresses (to and from) contained UTF-8.
 
  • Like
Reactions: LinuxCuba and digaovaa
Hmm - this is quite odd - I did not expect that UTF8 characters would be accepted in general ...
and it's against most of the relevant RFCs (with smtp that of course does not mean that it would not work ... :/)

If you get another such mail It might help to have it here for some tests (along with the unchanged logs) - you can send them to me directly to not have them completely in public - if this is an option for you:
s.ivanov _at_ proxmox.com

trichardson said:
My understanding about SMTPUTF8, learned solely through this experience and some reading about it, is that it's apparently not strictly necessary in order for mail to be exchanged with non-ASCII envelope addresses.
Click to expand...
do you have any links for this? (I mean - your setup of the past years is proof that it somehow did work ... but more input would help us to get an informed decision how to handle this)

currently I'm considering a option in the pmg-config to ignore the match and just not use the extension...
 
  • Like
Reactions: LinuxCuba
Short update - after getting a screenshot of an affected mail via e-mail:

The issue could probably occur if the mail contains non-ascii characters in its headers.
While this is still technically not rfc-compliant (smtp-servers should use SMTPUTF8 flag if the headers contain non-ascii characters) I can very well imagine that this was working quite fine until the change introduced in pmg-api 7.2-3.

We'll look into it and are currently discussing how to adapt the code to not break smtputf8 compliance in general, while keeping setups working, that did so before the change.
 
  • Like
Reactions: LinuxCuba
trichardson said:
Turns out the X-DFrom and X-DSubject headers contain UTF-8. I believe this is in contradiction to the third paragraph of 3.2 of SMTPUTF8
Click to expand...
Yes - that was my suspicion - thanks for the sample mail - I'll try to get around to looking into the issue soon

and yes it's a violation of the RFC - but with mail - quite a few systems were in place and have not been touched long before that rfc came out (2012)
 
  • Like
Reactions: trichardson
have the same issue. pmg last update + ms exchange 2016.
it there any nondestructive workaround?

Jan 18 12:24:45 mx5 postfix/smtpd[49268]: connect from relay.sender.tld[xxx.xx.xxx.x] Jan 18 12:24:47 mx5 postfix/smtpd[49268]: NOQUEUE: client=relay.sender.tld[xxx.xx.xxx.x] Jan 18 12:24:47 mx5 pmg-smtp-filter[49207]: 2C179D63C7BADF640D4: new mail message-id=<Issue1674033867342@hq-exhc.corp.sender.tld>#012 Jan 18 12:24:48 mx5 pmg-smtp-filter[49207]: 2C179D63C7BADF640D4: SA score=0/5 time=0.831 bayes=0.00 autolearn=ham autolearn_force=no hits=AWL(0.564),BAYES_00(-1.9),DKIM_SIGNED(0.1),DKIM_VALID(-0.1),DKIM_VALID_AU(-0.1),DKIM_VALID_EF(-0.1),HTML_MESSAGE(0.001),SPF_HELO_NONE(0.001),SPF_PASS(-0.001) Jan 18 12:24:48 mx5 postfix/smtpd[49277]: connect from localhost.localdomain[127.0.0.1] Jan 18 12:24:48 mx5 postfix/smtpd[49277]: 449EF2C17C2: client=localhost.localdomain[127.0.0.1], orig_client=relay.sender.tld[xxx.xx.xxx.x] Jan 18 12:24:48 mx5 postfix/cleanup[49227]: 449EF2C17C2: message-id=<Issue1674033867342@hq-exhc.corp.sender.tld> Jan 18 12:24:48 mx5 postfix/qmgr[887]: 449EF2C17C2: from=<lab_yt_mail@sender.tld>, size=14611, nrcpt=1 (queue active) Jan 18 12:24:48 mx5 pmg-smtp-filter[49207]: 2C179D63C7BADF640D4: accept mail to <name.surname@my.tld> (449EF2C17C2) (rule: default-accept) Jan 18 12:24:48 mx5 postfix/smtpd[49277]: disconnect from localhost.localdomain[127.0.0.1] ehlo=1 xforward=1 mail=1 rcpt=1 data=1 commands=5 Jan 18 12:24:48 mx5 pmg-smtp-filter[49207]: 2C179D63C7BADF640D4: processing time: 0.921 seconds (0.831, 0.024, 0) Jan 18 12:24:48 mx5 postfix/smtpd[49268]: proxy-accept: END-OF-MESSAGE: 250 2.5.0 OK (2C179D63C7BADF640D4); from=<lab_yt_mail@sender.tld> to=<name.surname@my.tld> proto=ESMTP helo=<relay.sender.tld> Jan 18 12:24:48 mx5 postfix/smtpd[49268]: disconnect from relay.sender.tld[xxx.xx.xxx.x] ehlo=2 starttls=1 mail=1 rcpt=1 data=1 quit=1 commands=7 Jan 18 12:24:48 mx5 postfix/smtp[48573]: 449EF2C17C2: to=<name.surname@my.tld>, relay=exch_relay.my.tld[xx.xxx.x.x]:25, delay=0.58, delays=0.05/0/0.52/0, dsn=5.6.7, status=bounced (SMTPUTF8 is required, but was not offered by host exch_relay.my[xx.xxx.x.x]) Jan 18 12:24:48 mx5 postfix/qmgr[887]: 449EF2C17C2: removed

the fun is that sender mail system does not wants to receive bounce

Jan 18 12:24:48 mx5 postfix/cleanup[49227]: D31E62C17C4: message-id=<20230118092448.D31E62C17C4@mx5.my.tld> Jan 18 12:24:48 mx5 postfix/qmgr[887]: D31E62C17C4: from=<>, size=16681, nrcpt=1 (queue active) Jan 18 12:24:49 mx5 postfix/smtp[48544]: D31E62C17C4: to=<lab_yt_mail@sender.tld>, relay=relay.sender.tld[xx.xxx.x.xx]:25, delay=0.7, delays=0/0/0.7/0, dsn=5.6.7, status=bounced (SMTPUTF8 is required, but was not offered by host relay.srelay.sender.tld[xx.xxx.x.xx]) Jan 18 12:24:49 mx5 postfix/qmgr[887]: D31E62C17C4: removed
 
Last edited:
Not sure if you consider this "nondestructive", but this worked for me: I commented out line 267 of /usr/share/perl5/PMG/Utils.pm to disable the new check which was enabling SMTPUTF8.
Perl: 
#disabled this: $mail_opts .= " SMTPUTF8" if mail_needs_smtputf8($entity, $sender, $targets);
I also have smtputf8_enable = no in my /etc/postfix/main.cf
 
I was seeing the exact same thing. I assume you haven't yet disabled SMTPUTF8 in postfix. The first rejection you see is postfix being unhappy that your downstream Exch2016 server doesn't support SMTPUTF8. The second is because postfix tries to use SMTPUTF8 when sending the bounce message to the originating server (which also doesn't support SMTPUTF8). This is all caused by the one check added to Utils.pm that was enforcing the RFC: SMTPUTF8 *shall* be used when email headers contain UTF-8. But that only works in an ideal world :-(
 
  • Like
Reactions: LinuxCuba
Is there any news on this problem?
I too had to disable and SMTPUTF8 via templating engine and comment /usr/share/perl5/PMG/Utils.pm too get mail flow working properly again....
 
Stoiko, I reviewed your patches and largely agree with the approach. I might have added a configuration option to enable/disable SMTPUTF8 which could be used to retain PMG's original behavior w.r.t. locally-generated mails, but I also appreciate not wanting to complicate things. People with UTF-8 addresses should be supporting SMTPUTF8. "... I assume non-ascii local-parts to be a very fringe edge-case in environments where smtputf8 is not supported." => I agree
 
  • Like
Reactions: LinuxCuba
trichardson said:
Stoiko, I reviewed your patches and largely agree with the approach. I might have added a configuration option to enable/disable SMTPUTF8 which could be used to retain PMG's original behavior w.r.t. locally-generated mails, but I also appreciate not wanting to complicate things. People with UTF-8 addresses should be supporting SMTPUTF8. "... I assume non-ascii local-parts to be a very fringe edge-case in environments where smtputf8 is not supported." => I agree
Click to expand...
Thank you so much for taking the time to review them!

Adding an option to completely enable/disable it is also something we consider - but I'd like to avoid reading the config file for each mail that is being sent once more and if I'd rather do that with a minor release
 
After reviewing the update, I think as well that the proposed approaches “skipping the headers checking” incl. “For locally generated mail it detects if its needed by checking the envelope-addresses and the headers for non-ascii characters” makes real sense and stays close to the real world – the paradigm: either is the environment smtputf8 capable or not and to great extent it implies a switch behavior of completely enable/disable smtputf8 for locally generated mails.
 
  • Like
Reactions: LinuxCuba
Stoiko Ivanov said:
Hmm - this is quite odd - I did not expect that UTF8 characters would be accepted in general ...
and it's against most of the relevant RFCs (with smtp that of course does not mean that it would not work ... :/)

If you get another such mail It might help to have it here for some tests (along with the unchanged logs) - you can send them to me directly to not have them completely in public - if this is an option for you:
s.ivanov _at_ proxmox.com


do you have any links for this? (I mean - your setup of the past years is proof that it somehow did work ... but more input would help us to get an informed decision how to handle this)

currently I'm considering a option in the pmg-config to ignore the match and just not use the extension...
Click to expand...
Stoiko Ivanov The exact same thing happened to me, but instead of exchange, with zimbra, and I did exactly the same hack as my friend @trichardson, I've been using pmg for many years since its inception, and all my backend servers have been Zimbra that aren't enables SMTPUTF8 by default, and since I received the first mail bounce informed that the backend mail server did not support this feature, and researched the postfix documentation, I disabled it by default for everyone in PMG, using the postfix template, main.cf. Then in January 2023, the fateful problem occurred, and many customers complained. At first, they went with the reports, as could be seen in other threads in this forum, which resulted in the patch for the API.
https://forum.proxmox.com/threads/quranantine-emails-not-beeing-sent.119387
https://lists.proxmox.com/pipermail/pmg-devel/2022-December/002233.html

Then the problem was another as the same thing happened to him and @trichardson raises.
 
received an update to 7.2-4. what config should I edit to have it working?
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back