Selective smarthost (based on recipient, say outlook.com or live.com)

[AlexHK]

We've been using PMG for a couple of weeks but today I saw our IP address seems to be on a Microsoft blocklist - ignore this part, how to get unblocked is not the reason why I am posting.

Rather, I'd like to see how to get PMG to work with selective smarthosts or possibly some other suggestions for the future.

Before we had PMG between the mail server and the internet we had a useful option for sending mails, which came in handy a couple of times during the years:

Send all emails directly fist, and then to smart hosts if there are problems.

For Postfix, it does seem there are ways to set up selective smart hosts based on the recipients by adding check_recipient_mx_access in smtpd_*_restriction:

https://serverfault.com/questions/663418/relay-host-based-on-destination-mx-record/663435#663435

I tried a few different ways, but eventually gave up since non of them really worked - possibly due to my requirement of smtp auth for my smarthost which seems to work if I use relayhost on its own (see below) but not if in combination with the above solutions.

Another solution mentioned relay host depending to receiver address (untested):

https://tipstricks.itmatrix.eu/relaying-emails-dependent-on-either-sender-or-receiver/

Add in /etc/postfix/transport:
@example4.com relay:[relay.server.com]:25
someone@example5.com relay:[relay.server.com]:25
someoneelse@example6.com local
*                        smtp

What that work with PMG? Enabling SMTP authentication would also work?


Currently I am sending ALL mail via smarthost that requires auth via template in /etc/pmg/templates/main.cf.in, but I'd like to get a better solution in the future...

Current working config:

relayhost = [smtp.example.com]:587
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_sasl_security_options = noanonymous

Any ideas?

 

[utkonos]

This would be a useful feature. I have just transitioned a mail system to a PMG+365+(IMAP/POP/SMTP) and have encountered a speedbump with one external mail destination. That destination uses a mail security gateway which had a whitelist entry for the old mail system from ancient times before my time and from before anyone else with insitutional memory. Mail to that destination is being bounced. This is a slightly different use case from yours where you're on a blacklist and need to work to get off. But the feature solution in PMG I see as being the same.

PMG has a feature called "Transports" which is configured at Configuration > Mail Proxy > Transports. This feature operates on inbound email and can route email for a domain or user's address to a specific location. This is how I have setup the hybrid 365+traditional mail system. The 365 users are all individual Transports entries.

The feature that would solve your problem and mine would be "Outbound Transports" or the equivalent. It could be configured to operate on a specific sender email address (a user on a sending mail system on the "inside") or it can operate on a destination domain. Outbound mail traffic that meets the criteria would then be sent to the host, port, and transport protocol as configured.

For my use case, I would send mail from that particular user to the domain with the gateway to the old mail system's SMTP server for delivery while we get in touch with the other party and get their gateway updated. Our end user and their end users would then be able to continue with their work and not need to think about email problems.

For your use case, mail being sent to 365 could be put in an outbound Transport that delivers that email to a smarthost that you have setup outside PMG (or a second PMG on a different IP) or whatever solves your problem temporarily while you work out the blacklist issue. Again, your end users don't have to think about email problems while you work on a solution.

 

[AlexHK]

Thanks for your feedback. I tried to work with transports yesterday after posting the second example, but I didn't get it to work. It does indeed seem PMG only uses it for inbound mail.

I also noticed the manual describes /etc/pmg/transports but then in main.cf we have transport_maps = hash:/etc/pmg/transport (transport vs. transports), so I am not sure this is a typo or two different things.

It would be really great if PMG allows us to have different smarthosts based on receiver domain. Bonus points if the team could make smarthosts with SMTP authentication work from within the PMG web interface.

-- // --

PS: Just to be clear, our IP is not really on a typical blocklist and this shouldn't be a thread about getting unblocked. Google "block list (S3140)" or read this if you really must know.

 

[utkonos]

I experienced a problem with the connector from PMG to 365 where internal emails were being semi-greylisted. The email address lists are handled in the traditional half of the hybrid setup, so foo@example.com is sent to the IMAP/POP where it is split into separate envelopes for jack@example.com and jill@example.com which are then delivered locally, and romeo@example.com and juliet@example.com which are then sent back to PMG and then delivered to 365. Some lists have three or more 365 users on the list.

If there are three or more 365 users on the list, two of the 365 users will be delivered by 365 and one or however many more than two will be temporarily rejected (like greylist) and then queued on PMG. PMG then retries later, and the list is slowly delivered in pairs. No email was lost, but this made things very slow. I contacted Microsoft support and they added some kind of additional whitelist entry in the back end for the PMG IP address. From then on, everything was smooth sailing.

However, I never experienced getting blocked by MS. There was one single block on Yahoo and a single block on Google for test emails someone sent. I then taught people to send to a mailinator account we setup for testing. This allows test emails for whatever, but doesn't endanger sending irregular emails to a big provider and risking an adjustment to the score for our IP.

BTW: before working on the production domain for my customer, I setup a scale model with every piece: OpenSMTPD server, Dovecot+OpenSMTPD, 365 w/ connectors, and PMG on PVE and a test domain that the customer owned. I worked out all the issues before making the main transition there. One tool that I found helpful was:
https://www.mail-tester.com/

You can send three test emails to it for free per day. This and using mailinator for testing was crucial. The test system had a perfect score for sent emails on mail-tester.com. The only issues I then encountered were the strange 365 semi-greylisting and the old whitelists on external mail systems and the handful of mistaken test emails.

BTW: reading the problem listed at that answers.microsoft.com page:
"Please contact your Internet service provider since part of their network is on our block list"

Your IP may be on a bad neighborhood blacklist. I can think of two reasons for this off the top of my head:

1. Microsoft has their own bad neighborhood blacklist. In this case you can only talk to them about it to get it fixed.
2. Your IP is listed on either or both of UCEPROTECTL2 and UCEPROTECTL3.

This second case can be checked using https://mxtoolbox.com/ just plug in your IP and then navigate from the results to the blacklist check. You can also see if they have you listed directly using their own tool: https://www.uceprotect.net/en/rblcheck.php

If the IP range is listed in UCEPROTECTL2 or the ASN is listed in UCEPROTECTL3, they operate an extortion / mafia protection racket here:
https://www.whitelisted.org/

You plug the IP in that and they give you a price to have your IP whitelisted. The whole process feels smarmy, and please don't take anything I have said as a recommendation to do what they want or to pay them for anything. I'm just pointing out that this can be the source of a blacklist problem depending on whether the destination mail server uses UCEPROTECT in their filtration.

 

[AlexHK]

That was actually very helpful info, thanks a lot. Seems the network to which our new IP belongs to was indeed on UCEPROTECTL2. Moving the gateway to another completely unlisted IP now shows successful direct delivery (without smarthost) also to outlook.com.

I still wish there would be better options for directing outgoing mails via selective smarthosts. For instance with our mail server we could also have different smarthosts for different domains: domain1.com to smarthost1, domain2.com to smartahost2, etc (and all with proper SMTP authorization).

 

[carsten2]

This would be a useful feature. I have just transitioned a mail system to a PMG+365+(IMAP/POP/SMTP) and have encountered a speedbump with one external mail destination.

Are there any news from the proxmox team.
1) Outgoing smart host WITH authentication to the provider smart host.
2) Outgoging smarts hosts depending on destination domain (also with authentication)

 

[das1996]

Bump.

This would be a great feature.

As a self hosted residential user with a semi static ip (changes ~5 yrs), I've had good results sending to most destinations, however do run into some targets that outright refuse because th rdns lookup doesn't match pmg fqdn. It would be nice to have a round robin type set up where if it fails on direct send, then use **smarthost**.

 

[das1996]

I got nowhere trying to set the mapping with main.cf.in.

One post I found made reference to just setting up another transport. That works for specific domains when transport defined configuration/mail proxy/relaying/smarthost is used as primary. Similar to the OP, I want it to send directly first, to specified relays for defined sender/recipient domains and using another relay when either of the first fail or result in rejected/bounced.

 

[masterx]

bump, i have exactly the same issue. routing mails to different mx hosts depending on recipient domain (aka outbound transport in PMG language) is something which other products were able to do for decades.

i'd really like to see this in PMG.

 

[kavejo]

+1 Outbound conditional relay (with authentication) would be something I’d be interested too. Ideally based on sender or recipient domain.