Mailproxy Transport and API

[WhoTookMyJiraa]

Hello,

i would like to ask for help with possibly a bug we encountered on PMG. We have a two node cluster of gateways and we are using it to filter the incoming mail. There is a list of aprox 100 domains we accept in "Mail Proxy - Relay domains" and destinations in "Mail Proxy - Transports".

While testing we did not encounter any problem with this setup and setting through GUI:

Relay domain: domain.tld
Host: imap.foo.bar
Protocol: smtp
Port: 25
Use MX: no

When adding the transport through API by calling:

curl --location 'https://pmg.foo.bar:8006/api2/json/config/transport' \
--header 'CSRFPreventionToken: REDACTED' \
--header 'Content-Type: application/json' \
--header 'Cookie: PMGAuthCookie=REDACTED \
--data '{
"domain":"domain.tld",
"comment" : "Added: 1.1.2023",
"host" : "imap.foo.bar",
"port" : 25,
"protocol" : "smtp",
"use_mx" : false
}'

the record gets added and can be seen in PMG gui on both gateways, but when someone sends an email to the domain.tld the mail is processed incorectly and ends on:

May 29 09:57:40 gw1 postfix/smtpd[2485533]: connect from mail-pj1-x1036.google.com[2607:f8b0:4864:20::1036] May 29 09:57:41 gw1 postfix/smtpd[2485533]: NOQUEUE: client=mail-pj1-x1036.google.com[2607:f8b0:4864:20::1036] May 29 09:57:41 gw1 pmg-smtp-filter[2486489]: 141C0464745AF5D658A: new mail message-id=<CAJNjfQ=jOSTLkQJWDumZF3twocJ-cWFRk9RNvRLExeFbDQWLDA@mail.gmail.com>#012 May 29 09:57:42 gw1 pmg-smtp-filter[2486489]: 141C0464745AF5D658A: SA score=0/5 time=0.889 bayes=0.00 autolearn=ham autolearn_force=no hits=AWL(0.452),BAYES_00(-1.9),DKIM_SIGNED(0.1),DKIM_VALID(-0.1),DKIM_VALID_AU(-0.1),DKIM_VALID_EF(-0.1),FREEMAIL_FROM(0.001),HTML_MESSAGE(0.001),RCVD_IN_DNSWL_NONE(-0.0001),SPF_HELO_NONE(0.001),SPF_PASS(-0.001),T_SCC_BODY_TEXT_LINE(-0.01) May 29 09:57:42 gw1 postfix/smtpd[2486304]: connect from localhost.localdomain[127.0.0.1] May 29 09:57:42 gw1 postfix/smtpd[2486304]: CA4171240BC: client=localhost.localdomain[127.0.0.1], orig_client=mail-pj1-x1036.google.com[2607:f8b0:4864:20::1036] May 29 09:57:42 gw1 postfix/cleanup[2486493]: CA4171240BC: message-id=<CAJNjfQ=jOSTLkQJWDumZF3twocJ-cWFRk9RNvRLExeFbDQWLDA@mail.gmail.com> May 29 09:57:42 gw1 postfix/qmgr[2320443]: CA4171240BC: from=<REDACTED@gmail.com>, size=4111, nrcpt=1 (queue active) May 29 09:57:42 gw1 postfix/smtpd[2486304]: disconnect from localhost.localdomain[127.0.0.1] ehlo=1 xforward=1 mail=1 rcpt=1 data=1 commands=5 May 29 09:57:42 gw1 pmg-smtp-filter[2486489]: 141C0464745AF5D658A: accept mail to <info@domain.tld> (CA4171240BC) (rule: default-accept) May 29 09:57:42 gw1 pmg-smtp-filter[2486489]: 141C0464745AF5D658A: processing time: 0.957 seconds (0.889, 0.039, 0) May 29 09:57:42 gw1 postfix/smtpd[2485533]: proxy-accept: END-OF-MESSAGE: 250 2.5.0 OK (141C0464745AF5D658A); from=<REDACTED@gmail.com> to=<info@domain.tld> proto=ESMTP helo=<mail-pj1-x1036.google.com> May 29 09:57:42 gw1 postfix/local[2486555]: CA4171240BC: to=<info@domain.tld>, relay=local, delay=0.05, delays=0.01/0.02/0/0.02, dsn=5.1.1, status=bounced (unknown user: "info") May 29 09:57:42 gw1 postfix/qmgr[2320443]: CA4171240BC: removed May 29 09:58:13 gw1 postfix/smtpd[2485533]: disconnect from mail-pj1-x1036.google.com[2607:f8b0:4864:20::1036] ehlo=2 starttls=1 mail=1 rcpt=1 bdat=1 quit=1 commands=7

The problem is that the GW tries to deliver the email localy instead of relaying it to the server imap.foo.bar.

This error persists for days. After adding any other record to the Mail Proxy - Transports setting or reloading postfix, the emails are delivered correctly to imap.foo.bar

gw1 postfix/smtp[2487449]: 81A4A1240BC: to=<info@domain.tld>, relay=imap.foo.bar[REDACTED]:25, delay=0.08, delays=0.01/0/0.04/0.03, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as 9396E20079)

Is there any bug or difference between adding the record in GUI or API?

Thank you for your help.

 

[Stoiko Ivanov]

relay=local,

I guess this is the root of the issue - check:
* your hostname
* /etc/resolv.conf (the first searchddomain entry is used to create the fqdn of PMG, which by default is used in the postfix config)
* any modifications you did to the postfix configuration

I assume that domain.tld ends up in the mydestination variable ... thus postfix tries to deliver mail for this domain locally

I hope this helps!

 

[WhoTookMyJiraa]

Well the problem is not that it does not deliver the emails localy, the default behaviour is set that way:


But the problem is that the visible config in Transports does not get applied in the backend.

hostname -f: gw1.mx.foo.bar (it is FQDN)
resolvconf:
search mx.foo.bar
nameserver validipv6ofNS1
nameserver validipv6ofNS2
nameserver validipv4ofNS1

no modifications to postfix

As i said we are using the pmg as gateways only, there are no local domains and users. I understand that we can set the Default domain for transport to imap.foo.bar but i would like to solve the bug.

 

[Stoiko Ivanov]

* how does /etc/pmg/transport look like - also check if /etc/pmg/transport.db has a newer mtime than /etc/pmg/transport
My guess is that you'd need to reload postfix for the changes to be picked up (and that your changes through the GUI just happened to coincide with the trivial-rewrite daemon picking up the changes faster - see https://www.postfix.org/trivial-rewrite.8.html

still the delivery to the local system remains odd .

 

[WhoTookMyJiraa]

Well when the bug first occured it lasted for 6 days and got fixed immediately after any manual change in the gui. I will check the mtimes tomorrow with my collegue. Still i cant see any way that the timing coincided with the reload.

We tried to check what does the "create" button does when pressed in the gui and it seems to call the same api endpoint as our curl. Any chance someone added a little "reload postfix" feature to the gui button?

 

[Stoiko Ivanov]

We tried to check what does the "create" button does when pressed in the gui and it seems to call the same api endpoint as our curl. Any chance someone added a little "reload postfix" feature to the gui button

not in general - I just tested it here - no reload happened

 

[WhoTookMyJiraa]

Hello, we have tried to add a transport by API and mtime of both transport and transpor.db changed and is the same.

cat /etc/pmg/transport | grep addeddomain.tld finds the domain but after doing cat on the transport.db file i also get random data but i can clearly see all the domains from transport config. WITHOUT the new domain. After adding and deleting "random transport" in GUI both transport and transport.db gets changed (mtime) but now i can find the addeddomain.tld

It looks like the transpor.db gets rebuild but from data before adding the data through api. Will do some more testing and will let you know.

 

[Stoiko Ivanov]

t looks like the transpor.db gets rebuild but from data before adding the data through api.

The api itself writes /etc/pmg/transport and then runs postmap on it ...
(and the GUI calls the same api endpoint you referenced...)