Whitelist users to send and recieve instead of check against mailserver

[ChAoS]

Hello,

due to some circumstances we hat to change our mail gateways in our mid-size company. I decided to use PMG in cluster (2 servers) each connected to another internet line.

our old carrier did filtering (and couting users for license purposes) statically. I had manually to enter every mail address which should be able to send and receive mails.

I know PMG checks it against our mailserver and caches it within an own database.

The problem is, that not every mailaddress should be reachable from outside and also some users should only be able to mail internally.

I decided to export an email address list from our ad and I manually deleted all forbidden addresses.

There are now more than 800 entries.

Every user has at least 3 addresses (Max.mustermann@company, m.mustermann@company and mmustermann@company).

I imported this list via cmd pmgsh tool into the Who objects into a new created list "AllowedEmailAddresses" and created 4 rules with low priority.

Prio 30: In_AcceptAllowedAddresses - To field matches my list -> accept
Prio 29: Out_AcceptAllowedAddress - from field matches my list -> accept
prio 20: In_SpamNotAllowedAddresses - All mails (which not accepted before) put to spam
prio 20: Out_SpamNotAllowedAddresses - All mails (which not accepted before) put to spam

This works as excepted. My problem is now I have a "catch all" system and have to review the spam.
Also senders are not notified, when there is a typo in the Mail address.

Now my question is,
is there onother method to use this (already imported) list against the sender directly while SMTP connection?
I don't want to send NDR - as it is not more actual.

Thank you

Dirk

 

[Stoiko Ivanov]

hmm - I hope I understood your use-case correctly (if not - please explain it with different words, or example logs) - but (with a bit of change in the behavior) - this might work:
* enable before-queue filtering and change the Action of the NotAllowedAddresses to: 'Block'
* optionally add a BCC Action, which sends the mails to some archive-mailbox - in case you would need them

-> before-queue filtering and 'Block' results (in most cases (i.e. if all recipients of the mail are blocked) ) in a response-code of 5xx - which should notify the sender without sending out an NDR

(see https://pmg.proxmox.com/pmg-docs/pmg-admin-guide.html#_mail_proxy_configuration (section 4.7.5 ) on a bit more detail on before-queue filtering)

I hope this helps!

 

[ChAoS]

It seems to be THAT.

I will give it a try today evening or tomorrow.

When all works file we will buy 2 subscriptions. Due to the cancelled serice of our old carrier I had to react quickly.

Thank you, I will let you know if it works

Dirk

 

[ChAoS]

@Stoiko Ivanov

I reply myself in hope you'll read my reply

I tested before cue filtering and it blocks.

BUT

the rules are biting each other...

current config:

SPAM 3 -> Quarantaine
Allowed RCPT -> Accept
All other -> block

now, if someone sends mail to not existing users and SPAM goes over 3 -> mail is delivered and quarantained.

After much thinking yesterday evening (and drinking a beer) i got the conclusion it is not possible, even with sorting these rules.

I created new rules in hope it works but I have new trouble:

99 -> modify field -> DHRCPT: no
92 -> check if allowed RCPT, when yes: modify field -> DHRCPT: yes
so far it works, all mails get a tag in the header named DHRCPT with yes or no (I verified, it works)
but
rule
83 -> what object: DHRCPT: no -> block mail doesn't work.

therefore I created a what object and in it I created "match field" DHRCPT value: no (also tested .no and *no*)
but this rule does not work, it doesnt block mails, even they are DHRCPT yes or no.

When I remove the what object from that rule -> all mails are blocked, so the rule is in the right position and is active.

What is wrong?

Thank you

Dirk

 

[Stoiko Ivanov]

now, if someone sends mail to not existing users and SPAM goes over 3 -> mail is delivered and quarantained.

After much thinking yesterday evening (and drinking a beer) i got the conclusion it is not possible, even with sorting these rules.

Thanks for testing this and experimenting around...
Seems this is currently not possible with the rule-system by itself
(We are considering enhancing the rulesystem at some point to support things like 'not matching a Who Object' - but it is not planned for implementation soon (patches are always welcome of course))

However you should be able to achieve your desired behavior by adapting the postfix config using the templateing system:
https://pmg.proxmox.com/pmg-docs/pmg-admin-guide.html#pmgconfig_template_engine

I think (again without testing this explicitly) - adding another

 check_recipient_access  regexp:/etc/postfix/allowed_recipients

line to the smtpd_recipient_restrictions in the `main.cf.in` template (below the existing `check_recipient_access regexp:/etc/postfix/rcptaccess` line) , adding the allowed recipients to /etc/postfix/allowed_recipients (and running postmap on the file)

should achieve what you're aiming for.

Relevant postfix documenation on the topic:
http://www.postfix.org/SMTPD_ACCESS_README.html
http://www.postfix.org/postconf.5.html#smtpd_recipient_restrictions
http://www.postfix.org/access.5.html

depending on how often the list of allowed recipients change you can either manually maintain it - or use a cronjob to sync it from some source

therefore I created a what object and in it I created "match field" DHRCPT value: no (also tested .no and *no*)
but this rule does not work, it doesnt block mails, even they are DHRCPT yes or no.

The rule-system does not 'apply' the actions in order - meaning you cannot mark a mail with a header field and use that field as What Object in a rule with lower priority - so that's expected

I hope this helps!

 

[ChAoS]

Hello @Stoiko Ivanov ,

thanks for your competent and fast reply

hanks for testing this and experimenting around...

thats my nature, I can not accept the thigs go as they go, I want to know why it goes

Seems this is currently not possible with the rule-system by itself
(We are considering enhancing the rulesystem at some point to support things like 'not matching a Who Object' - but it is not planned for implementation soon (patches are always welcome of course))

yes, the IS NOT is really missing, there are only IS - rules in this system. So I found out that (not working) workaround

However you should be able to achieve your desired behavior by adapting the postfix config using the templateing system:
https://pmg.proxmox.com/pmg-docs/pmg-admin-guide.html#pmgconfig_template_engine

I think (again without testing this explicitly) - adding another

check_recipient_access regexp:/etc/postfix/allowed_recipients

That was what I done years before on my private postfix. I am not an expert, I followed many tutorials and setup my own mailserver on a VSP, running fine since years.

What about the syntax of the allowed_recipients?
Is it also like my private relay map?

name@domain.tld OK

must the OK also be inside?

does it make more sense to put the allowed_recipients file (and even it's postmap created db-file) to /etc/pmg/ as we are using a 2 node cluster and link it in the template?

The /etc/pmg/templates folder did not exist - is it correct?

Thank you again so much

Dirk

 

[Stoiko Ivanov]

yes, the IS NOT is really missing, there are only IS - rules in this system. So I found out that (not working) workaround

in that case that's true - however with the priorities you can create rulesets for many use-cases - and it's somewhat simpler) - but as said - extending this is on our minds ...

What about the syntax of the allowed_recipients?
Is it also like my private relay map?

from the access(5) manpage I think the following should work:

allowed1@domain.tld OK
allowed2@domain.tld OK
....
allowed_last@domain.tld OK
domain.tld REJECT

but please verify this in your test-setup.

does it make more sense to put the allowed_recipients file (and even it's postmap created db-file) to /etc/pmg/ as we are using a 2 node cluster and link it in the template?

hmm - that makes things a bit more complicated:
* not all files from /etc/pmg are synced across clusternodes
* however the files from master should end up on all nodes in /etc/pmg/master/
* /etc/pmg/templates should be kept in sync across nodes
* however if the file changes you need to run postmap on the file on all nodes (the .db files are excluded from syncing)

again - please test this - since it's not a common setup (and technically not supported - meaning I cannot promise that this will remain in sync for ever) - I'm writing this from memory ...

The /etc/pmg/templates folder did not exist - is it correct?

yes - you need to create it

I hope this helps!

 

[ChAoS]

Dear @Stoiko Ivanov

thank you again for your interest in helping me, perfect

I think (again without testing this explicitly) - adding another

check_recipient_access regexp:/etc/postfix/allowed_recipients

I had to change the line to

check_recipient_access hash:/etc/postfix/allowed_recipients

now it works as expected

Mails are directly blocked with the correct error, rcpt not exists, so sender has a clear answer

I know it is not supported and not comfortable, especially my collegues are not firm with linux, so I have to write a script and a good documentation about this

additionally I experimented around with the same file for senders restrictions. As mentioned in my first posts, here are some users (especially in storage) they only are allowed to mail internally. In the past it was regulated by our old carrier.

Now I can use my first method, a list in proxmox to allow senders and block not allowed but then I have to maintain 2 lists and keep them in sync.

So I tried to modify also the main.cf.in in templates to use the same list for checks but it not works.

smtpd_sender_restrictions =
        permit_mynetworks
        reject_non_fqdn_sender
        check_client_access     cidr:/etc/postfix/clientaccess
        check_sender_access     regexp:/etc/postfix/senderaccess
        check_sender_access     hash:/etc/postfix/allowed_recipients
        check_recipient_access  regexp:/etc/postfix/rcptaccess
[%- IF pmg.mail.rejectunknown %] reject_unknown_client_hostname[% END %]
[%- IF pmg.mail.rejectunknownsender %] reject_unknown_sender_domain[% END %]

smtpd_recipient_restrictions =

I get no errors in syslog and mails are not restcricted. So I think I am on the wrong section, cause we are talking about port 26 and not 25...

from the access(5) manpage I think the following should work:

allowed1@domain.tld OK
allowed2@domain.tld OK
....
allowed_last@domain.tld OK
domain.tld REJECT

That was 100% correct

hmm - that makes things a bit more complicated:
* not all files from /etc/pmg are synced across clusternodes
* however the files from master should end up on all nodes in /etc/pmg/master/
* /etc/pmg/templates should be kept in sync across nodes
* however if the file changes you need to run postmap on the file on all nodes (the .db files are excluded from syncing)

I tested a bit around and you are also 100% right, the text file is synchronisized between the hosts, the .db file not. So I SCPd it and that seems to be the easiest way to get the files to the 2nd host after a change, so if it will all go right the next days I am going to write a small sh script which postmaps the alias file, scp it to 2nd host and syncs / restarts the services.

The evening hours are the best to test, there is less mail traffic and I have an good overview, if some mails are blocked, which should be delivered.

THANK you again

Dirk

 

[Stoiko Ivanov]

additionally I experimented around with the same file for senders restrictions. As mentioned in my first posts, here are some users (especially in storage) they only are allowed to mail internally. In the past it was regulated by our old carrier.

Now I can use my first method, a list in proxmox to allow senders and block not allowed but then I have to maintain 2 lists and keep them in sync.

So I tried to modify also the main.cf.in in templates to use the same list for checks but it not works.

You need to adapt the settings for the smtpd listener on the internal port - this is in master.cf (thus you need to also override the master.cf.in template) - the internal port defaults to accepting mail from the trusted networks

but glad you're making good progress on your setup!

please keep us posted how it works out

 

[ChAoS]

You need to adapt the settings for the smtpd listener on the internal port - this is in master.cf (thus you need to also override the master.cf.in template) - the internal port defaults to accepting mail from the trusted networks

thank you thank you and again thank you

It was not really clear, google, try and error did the job...

copied the master.cf fom /etc/postfix to /etc/pmg/templates/master.cf.in

-o smtpd_sender_restrictions=check_sender_access,hash:/etc/postfix/allowed_recipients

important was the comma before hash:
now it seem to work, all mails from internally not listed in the allowed_recipients list are directly denied. For me it is a bit of outbreak protection, if something is compromitted, the risk is a bit smaller that someone uses our gateway with fake addresses

Hope you'll have a nice weekend

Dirk

 

[Stoiko Ivanov]

Thanks for reporting your success

Hope you'll have a nice weekend

I'm planning on it - wish you a nice weekend as well