Unable to whitelist servers for bad SPF

C

cglmicro

Member
Oct 12, 2020
101
11
23
51
Hi.

Some of my customers are forwarding emails from their old addresses (ex.: @gmail.com to their new domain addresses). My PMG's are blocking some incoming mail that are forward from an ISP (Videotron). I need to be able to tell PMG to never check RBL and SPF rules for emails coming from all of Videotron servers, they had some I found so far:
alt12.smtp-out.videotron.ca, alt22.smtp-out.videotron.ca, alt32.smtp-out.videotron.ca and alt42.smtp-out.videotron.ca.
I'm afraid that I'll miss some more, so I thought about using a REGULAR EXPRESSION to add *.smtp-out.videotron.ca, but it's not working.
Then I tried adding only the IP adresses that I know, but even this is still blocked !

Here is some details for 3 emails from the TRACKING CENTER of PMG:
Code: 
Nov 15 10:46:56 pmg10 postfix/smtpd[2389421]: connect from alt22.smtp-out.videotron.ca[70.80.0.73]
Nov 15 10:46:57 pmg10 postfix/smtpd[2389421]: NOQUEUE: reject: RCPT from alt22.smtp-out.videotron.ca[70.80.0.73]: 554 5.7.1 <nancygroulx@aubutgroulx.ca>: Recipient address rejected: Rejected by SPF: 70.80.0.73 is not a designated mailserver for d.aubry%40cglmicro.ca (context mfrom, on pmg10.legardeur.net); from=<d.aubry@cglmicro.ca> to=<nancygroulx@aubutgroulx.ca> proto=ESMTP helo=<alt22.smtp-out.videotron.ca>
Nov 15 10:47:57 pmg10 postfix/smtpd[2389421]: disconnect from alt22.smtp-out.videotron.ca[70.80.0.73] ehlo=2 starttls=1 mail=1 rcpt=0/1 rset=1 quit=1 commands=6/7

Nov 15 10:52:02 pmg10 postfix/smtpd[2405904]: connect from alt32.smtp-out.videotron.ca[24.53.0.21]
Nov 15 10:52:03 pmg10 postfix/smtpd[2405904]: NOQUEUE: reject: RCPT from alt32.smtp-out.videotron.ca[24.53.0.21]: 554 5.7.1 <nancygroulx@aubutgroulx.ca>: Recipient address rejected: Rejected by SPF: 24.53.0.21 is not a designated mailserver for d.aubry%40cglmicro.ca (context mfrom, on pmg10.legardeur.net); from=<d.aubry@cglmicro.ca> to=<nancygroulx@aubutgroulx.ca> proto=ESMTP helo=<alt32.smtp-out.videotron.ca>
Nov 15 10:53:03 pmg10 postfix/smtpd[2405904]: disconnect from alt32.smtp-out.videotron.ca[24.53.0.21] ehlo=2 starttls=1 mail=1 rcpt=0/1 rset=1 quit=1 commands=6/7

Nov 15 11:03:59 pmg11 postfix/smtpd[2913688]: connect from alt42.smtp-out.videotron.ca[23.233.128.29]
Nov 15 11:03:59 pmg11 postfix/smtpd[2913688]: NOQUEUE: reject: RCPT from alt42.smtp-out.videotron.ca[23.233.128.29]: 554 5.7.1 <nancygroulx@aubutgroulx.ca>: Recipient address rejected: Rejected by SPF: 23.233.128.29 is not a designated mailserver for d.aubry%40cglmicro.ca (context mfrom, on pmg11.legardeur.net); from=<d.aubry@cglmicro.ca> to=<nancygroulx@aubutgroulx.ca> proto=ESMTP helo=<alt42.smtp-out.videotron.ca>
Nov 15 11:04:59 pmg11 postfix/smtpd[2913688]: disconnect from alt42.smtp-out.videotron.ca[23.233.128.29] ehlo=2 starttls=1 mail=1 rcpt=0/1 rset=1 quit=1 commands=6/7

and now here is my setup:
1636993305472.png
1636993380646.png

What am I missing?

Thank you.
 
Hi Stoiko.

Thanks for your answer.

You are right, when I add the 4 IP address in MAIL PROXY > WHITELIST it work.
It also work if add the 4 domains alt12.smtp-out.videotron.ca to alt42.smtp-out.videotron.ca at the same place it work too.
This is only a temporary solution since I fear the day when I'll need to add let say the hundred of servers owned by MailChimp, Sendgrid, Microsoft, Google, or other. If they keep adding more server, it's impossible to keep track.

So I tried adding the regular expression *.smtp-out.videotron.ca and it was not passing through.

I also tried has suggested by you in this other thread https://forum.proxmox.com/threads/regular-expression-in-whitelist.68435/: this regular expression *\.smtp-out\.videotron\.ca and that one *\.smtp\-out\.videotron\.ca (but this last one returned this error in the tracing:
Code: 
Nov 15 16:04:41 pmg11 postfix/smtpd[3771362]: warning: regexp map /etc/postfix/senderaccess, line 1: Invalid preceding regular expression
Nov 15 16:04:41 pmg11 postfix/smtpd[3771362]: connect from alt32.smtp-out.videotron.ca[24.53.0.21]
Nov 15 16:04:42 pmg11 postfix/smtpd[3771362]: NOQUEUE: reject: RCPT from alt32.smtp-out.videotron.ca[24.53.0.21]: 554 5.7.1 <nancygroulx@aubutgroulx.ca>: Recipient address rejected: Rejected by SPF: 24.53.0.21 is not a designated mailserver for d.aubry%40cglmicro.ca (context mfrom, on pmg11.legardeur.net); from=<d.aubry@cglmicro.ca> to=<nancygroulx@aubutgroulx.ca> proto=ESMTP helo=<alt32.smtp-out.videotron.ca>

What am I missing?
 
Hi guys.

I'm still having issues with REGULAR EXPRESSION that isn't working for the MAIL PROXY > WHITELIST.

How to whitelist every servers of VIDEOTRON.CA for instance (i.e.: alt12.smtp-out.videotron.ca)? I tried also *.videotron.ca and *\.videotron\.ca but it's not working.

Help please :-(
 
Another FAILED attempt. I found out that the content I add in MAIL PROXY > WHITELIST if reflected in /etc/postfix/senderaccess.

By looking at the content of the file /etc/postfix/senderaccess, I deducted the REGEX I need to add in the GUI :
^.+@.+\.smtp-out\.videotron\.ca

to get the file to look like:
Code: 
root@pmg10:/etc/postfix# cat /etc/postfix/senderaccess
/^.+@.+\.smtp-out\.videotron\.ca$/ OK
/^.+@alt12\.smtp-out\.videotron\.caTEMPORARYDISABLED$/ OK
/^.+@alt14\.smtp-out\.videotron\.caTEMPORARYDISABLED$/ OK
/^.+@alt22\.smtp-out\.videotron\.caTEMPORARYDISABLED$/ OK
/^.+@alt32\.smtp-out\.videotron\.caTEMPORARYDISABLED$/ OK
/^.+@alt42\.smtp-out\.videotron\.caTEMPORARYDISABLED$/ OK
/^.+@mail4\.vertisoftpme\.com$/ OK
/^.+@mail5\.vertisoftpme\.com$/ OK
/^.+@mail6\.vertisoftpme\.com$/ OK
/^.+@mail7\.vertisoftpme\.com$/ OK

BUT IT STILL NOT WORK, ARRRRGGGGHHHHH !

Just to be clear: I don't want to whitelist something with an enveloppe FROM like xyz@alt12.videotron.ca, I just want to whitelist the server with a name pattern alt12.videotron.ca (or alt14, alt32, etc.). Is it the right place to do this?
 
You could consider disabling the SPF checks during the SMTP-dialogue (GUI->Configuration->Mail Proxy->Options)-
(might be easier ...)


cglmicro said:
warning: regexp map /etc/postfix/senderaccess, line 1: Invalid preceding regular expression
Click to expand...

is this line still in the logs - that might be the reason why all entries are ignored - please post your /etc/postfix/senderaccess
 
Hi Stoiko, glad to see you here :)

About disabling the SPF, I hope not having to do this because I really believe its role in spam fighting.

The "invalid preceding regular expression" is no longer appearing in my log, it was a one time thing during my tests.

Here again my /etc/postfix/senderaccess:
Code: 
root@pmg10:/etc/postfix# cat /etc/postfix/senderaccess
/^.+@.+\.smtp-out\.videotron\.ca$/ OK
/^.+@alt12\.smtp-out\.videotron\.caTEMPORARYDISABLED$/ OK
/^.+@alt14\.smtp-out\.videotron\.caTEMPORARYDISABLED$/ OK
/^.+@alt22\.smtp-out\.videotron\.caTEMPORARYDISABLED$/ OK
/^.+@alt32\.smtp-out\.videotron\.caTEMPORARYDISABLED$/ OK
/^.+@alt42\.smtp-out\.videotron\.caTEMPORARYDISABLED$/ OK
/^.+@mail4\.vertisoftpme\.com$/ OK
/^.+@mail5\.vertisoftpme\.com$/ OK
/^.+@mail6\.vertisoftpme\.com$/ OK
/^.+@mail7\.vertisoftpme\.com$/ OK

Thank you !!!!
 
cglmicro said:
This is only a temporary solution since I fear the day when I'll need to add let say the hundred of servers owned by MailChimp, Sendgrid, Microsoft, Google, or other. If they keep adding more server, it's impossible to keep track.
Click to expand...
if you plan on whitelisting most of the large mailproviders - I'd probably reconsider disabling SPF (it won't help too much anyways)

cglmicro said:
Nov 15 10:46:57 pmg10 postfix/smtpd[2389421]: NOQUEUE: reject: RCPT from alt22.smtp-out.videotron.ca[70.80.0.73]: 554 5.7.1 <nancygroulx@aubutgroulx.ca>: Recipient address rejected: Rejected by SPF: 70.80.0.73 is not a designated mailserver for d.aubry%40cglmicro.ca (context mfrom, on pmg10.legardeur.net); from=<d.aubry@cglmicro.ca> to=<nancygroulx@aubutgroulx.ca> proto=ESMTP helo=<alt22.smtp-out.videotron.ca>
Click to expand...
not quite sure I completely get your setup - I assume cglmicro.ca is your domain? - why is an ISP sending mails FROM your domain to your PMG?!

in any case to answer your question -> the entries in /etc/postfix/senderaccess are checked (see in /etc/postfix/main.cf, which itself is generated from the template (see the reference docs)) during the RCPT TO command - and are evaluated against the address/string postfix got from the MAIL FROM command (in this case <d.aubry@cglmicro.ca>) - this is not the hostname

If you really want to allow all mails coming from videotron servers (based on their hostname) - add another line in the main.cf template with a
check_client_access entry for a regex table

e.g.:
Code: 
smtpd_recipient_restrictions =
        permit_mynetworks
        reject_unauth_destination
        reject_non_fqdn_recipient
        check_client_access  regexp:/etc/postfix/YOUR_VIDEOTRON_REGEX_TABLE
        check_recipient_access  regexp:/etc/postfix/rcptaccess
[%- IF postfix.usepolicy %] check_sender_access  regexp:/etc/postfix/senderaccess[% END %]
[%- IF postfix.usepolicy %] check_client_access  cidr:/etc/postfix/clientaccess[% END %]
[%- IF postfix.usepolicy %] check_policy_service inet:127.0.0.1:10022[% END %]
[%- IF pmg.mail.verifyreceivers %] reject_unknown_recipient_domain[% END %]
[%- IF pmg.mail.verifyreceivers %] reject_unverified_recipient[% END %]

the postfix docs explain this quite well:
http://www.postfix.org/postconf.5.html#smtpd_client_restrictions
http://www.postfix.org/SMTPD_ACCESS_README.html

I hope this helps!

P.S. If you need a guaranteed answer on such questions - I'd recommend getting a subscription of level Basic or above - and opening a ticket in our enterprise support portal https://www.proxmox.com/en/proxmox-mail-gateway/pricing
We try our best to answer as many threads here as possible - but sometimes things are missed
 
if you plan on whitelisting most of the large mailproviders - I'd probably reconsider disabling SPF (it won't help too much anyways)
Click to expand...
I disagree. Anyone serious will plan things through if the use services like sendgid or MailChimp, and will add their servers to their SPF.

I have a suggestion, why don't you ask your customer to fix their SPF records?
Click to expand...
I agree with you, but in this case it's not applicable and you'll why in a few lines.

not quite sure I completely get your setup - I assume cglmicro.ca is your domain? - why is an ISP sending mails FROM your domain to your PMG?!
Click to expand...

Here is this actual case:
My customer @aubutgroulx.ca is filtered in my PMG. She also own an old email nancy_groulx@videotron.ca (Videotron in Quebec is a major ISP that provide email address ending with their domain name).She want to forward emails that she receive to her old address (@videotron.ca) to her new address (@aubutgroulx.ca).
If I send a test email from my email (@cglmicro.ca) to her old email (@videotron.ca), when Videotron transfert it to her new email (@aubutgroulx.ca), it's forwarding it has if @cglmicro.ca is sending the email, where in fact it's @videotron.ca that is sending it. I know it's not the best way to forward it, but it's the way Videotron is doing it. Therefor the SPF check fail since the email look like it's coming from @cglmicro.ca, but with a server (many server from Videotron) not authorized by @cglmicro.ca SPF.
I can't modify my @cglmicro.ca SPF (and every other SPF of every other person that sends mail to her old address).

...add another line in the main.cf template with a check_client_access entry for a regex table
Click to expand...
Will it survive all the updates and upgrades in the future, or will I need to readd the line from time to time?
Will try your solution for sure, thanks, thanks, and thanks !!

...I'd recommend getting a subscription of level Basic or above...
Click to expand...
I know, I hope some day I'll be able to monetize this service I give for free to my end users. For the moment it's too expensive. Why not offer something like "pay 100 $ for X support requests" and have these support request not expire? Will surely purchase some.

Thanks again and I'll be back with the results as soon has I'll be able to test it.
 
Call Videotron and ask them to forward correctly.
Click to expand...
Impossible. They manage millions of mailboxes, they won't change a thing.

Stoiko: I did the change in my /etc/postfix/main.cf:
Code: 
smtpd_recipient_restrictions =
        permit_mynetworks
        reject_unauth_destination
        reject_non_fqdn_recipient
        check_client_access     regexp:/etc/postfix/cglmicro_regex_table regexp:/etc/postfix/rcptaccess
        check_sender_access     regexp:/etc/postfix/senderaccess
        check_client_access     cidr:/etc/postfix/clientaccess
        check_policy_service    inet:127.0.0.1:10022
        reject_unknown_recipient_domain
        reject_unverified_recipient

and my file /etc/postfix/cglmicro_regex_table:
Code: 
/^.+\.smtp-out\.videotron\.ca$/ OK

It's still not working. Also, the change I'm making in my slave is lost and revert back to the default each time I restart postfix, but not on my master.
To your knowledge, is there a delay before it's applied on a node or should I "systemctl restart postfix" at every attempt?
Should it be added instead in smtpd_sender_restrictions rather than smtpd_recipient_restrictions ?
Should I leave the OK at the end?

Thanks again.
 
Last edited:
cglmicro said:
Stoiko: I did the change in my /etc/postfix/main.cf:
Click to expand...
as said - you need to edit the template for main.cf - not the file itself - see:
https://pmg.proxmox.com/pmg-docs/pmg-admin-guide.html#pmgconfig_template_engine

also make sure to restart postfix afterwards

I hopt this helps!
also I think you should add a line containing check_client_access so:
Code: 
smtpd_recipient_restrictions =
        permit_mynetworks
        reject_unauth_destination
        reject_non_fqdn_recipient
        check_client_access  regexp:/etc/postfix/cglmicro_regex_table
        check_recipient_access  regexp:/etc/postfix/rcptaccess
[%- IF postfix.usepolicy %] check_sender_access  regexp:/etc/postfix/senderaccess[% END %]
[%- IF postfix.usepolicy %] check_client_access  cidr:/etc/postfix/clientaccess[% END %]
[%- IF postfix.usepolicy %] check_policy_service inet:127.0.0.1:10022[% END %]
[%- IF pmg.mail.verifyreceivers %] reject_unknown_recipient_domain[% END %]
[%- IF pmg.mail.verifyreceivers %] reject_unverified_recipient[% END %]

Also for this to work in a cluster you should probably put the regex file in /etc/pmg/templates (that way it will get synchronized across your nodes)
 
YEAHHHH ! Your my hero.
Here is what worked for me.

First create this template folder if it doesn't exist :
Code: 
mkdir /etc/pmg/templates

Then copy the main template file in the newly created folder:
Code: 
cp /var/lib/pmg/templates/main.cf.in /etc/pmg/templates/

Then modify the file to add this line in the smtpd_recipient_restrictions section:
Code: 
        check_client_access     regexp:/etc/postfix/custom_regex_table.save

Create the file /etc/postfix/custom_regex_table.save that should contain something like :
Code: 
/^.+\.smtp-out\.videotron\.ca$/ OK
/^.+\.sendgrid\.net$/ OK
When the file was named simply /etc/postfix/custom_regex_table, postfix was deleting it every time the service restart.

CHMOD your regex file :
Code: 
chmod 0644 /etc/postfix/custom_regex_table.save
Without this permission, the log will tell you that the file won't exists.

Restart the sync and don't take chance restart also postfix:
Code: 
pmgconfig sync --restart 1
systemctl restart postfix

Your main.cf.in will sync to your slaves, but you will need to create (and update) your custom_regex_template.save on all your slave and "systemctl restart postif" on each nodes.

I'll do more tests to look in the logs and confirm that the changes are still there tomorrow.

Thanks again.
 
  • Like
Reactions: Stoiko Ivanov
Hi Stoiko.

I think there is something wrong, since some connexion are rejected:
Code: 
Dec 23 15:22:03 pmg10 postfix/postscreen[3350152]: NOQUEUE: reject: RCPT from [209.85.221.41]:36712: 550 5.7.1 Service unavailable; client [209.85.221.41] blocked using dnsbl.sorbs.net; from=<xyz@gmail.com>, to=<xyz@domain.tld>, proto=ESMTP, helo=<mail-wr1-f41.google.com>

Even if my GOOGLE.COM is whitelisted, see my files:
CSS: 
root@pmg10:/etc/postfix# cat /etc/pmg/templates/main.in.cf
...
smtpd_recipient_restrictions =
        permit_mynetworks
        reject_unauth_destination
        reject_non_fqdn_recipient
        check_client_access     regexp:/etc/postfix/cglmicro_regex_table.save
        check_recipient_access  regexp:/etc/postfix/rcptaccess
[%- IF postfix.usepolicy %] check_sender_access  regexp:/etc/postfix/senderaccess[% END %]
[%- IF postfix.usepolicy %] check_client_access  cidr:/etc/postfix/clientaccess[% END %]
[%- IF postfix.usepolicy %] check_policy_service inet:127.0.0.1:10022[% END %]
[%- IF pmg.mail.verifyreceivers %] reject_unknown_recipient_domain[% END %]
[%- IF pmg.mail.verifyreceivers %] reject_unverified_recipient[% END %]
...
    
    
root@pmg10:/etc/postfix# cat /etc/postfix/cglmicro_regex_table.save
/^.+\.smtp-out\.videotron\.ca$/ OK
/^.+\.srvr\.bell\.ca$/ OK
/^.+\.facebook\.com$/ OK
/^.+\.google\.com$/ OK
/^.+\.outlook\.com$/ OK
/^.+\.microsoft\.com$/ OK

What am I missing?
Thank you and merry Christmas to all!
 
The IP is consider spam via DNSBL. Either ask SORBS to unlist the IP or don't use SORBS as your DNSBL server.

Code: 
Dec 23 15:22:03 pmg10 postfix/postscreen[3350152]: NOQUEUE: reject: RCPT from [209.85.221.41]:36712: 550 5.7.1 Service unavailable; client [209.85.221.41] blocked using dnsbl.sorbs.net; from=<xyz@gmail.com>, to=<xyz@domain.tld>, proto=ESMTP, helo=<mail-wr1-f41.google.com>
 
  • Like
Reactions: Stoiko Ivanov
Hi hata_ph.

I know the IP is blacklisted, and this thread is exactly about that and finding a way to get around DNSBL for certain hosts.
For example, some FACEBOOK and GMAIL server IPs are blacklisted, some other not. I want PMG to accept every connexions from servers ending with certain FQDN. I don't want to get rid of SORBS for the other 99.99% connexions made to my servers.

Does it make more sense?
 
cglmicro said:
Hi hata_ph.

I know the IP is blacklisted, and this thread is exactly about that and finding a way to get around DNSBL for certain hosts.
For example, some FACEBOOK and GMAIL server IPs are blacklisted, some other not. I want PMG to accept every connexions from servers ending with certain FQDN. I don't want to get rid of SORBS for the other 99.99% connexions made to my servers.

Does it make more sense?
Click to expand...
It will not work . Either ask SORBS to unlist the spam IP or do not use SORBS as all.
Another option is disable DNSBL but then it will be another security issue.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back