SMTP Smuggling mitigation

[Stoiko Ivanov]

A new SMTP-protocol level spoofing attack was recently published:
https://sec-consult.com/blog/detail/smtp-smuggling-spoofing-e-mails-worldwide/

Proxmox Mailgateway uses postfix as it's mail transport agent, thus the fix for the vulnerability happens in postfix.
postfix upstream details: https://www.postfix.org/smtp-smuggling.html

pmg-api in version 8.0.11 and 7.3-11 respectively address the issue by not accepting bare newlines and rejecting pipelineing on the external port by setting:

smtpd_data_restrictions = reject_unauth_pipelining

smtpd_forbid_bare_newline = yes
smtpd_forbid_bare_newline_exclusions =
        $mynetworks,
        cidr:/etc/postfix/clientaccess

in the main.cf.in template (/var/lib/pmg/templates/main.cf.in)
and removing both restriction on the internal port in the master.cf.in template (/var/lib/pmg/templates/master.cf.in).

The long-term fix using smtpd_forbid_bare_newline = yes is needs the postfix packages in the bookworm-update or bullseye-update (see https://wiki.debian.org/StableUpdates) repositories respectively.
The stable-update repositories are enabled by default on installs from the PMG ISO and from the container-template.

If you have modified the postfix configuration templates make sure to adapt them to include those lines. The package installation procedure
will ask you about changes in the shipped config-files, since the configuration templates are handled with ucf (see https://packages.debian.org/bookworm/ucf)

After updating you can verify that the necessary configuration item has been set as follows:

# grep -E 'smtpd_data_restrictions|smtpd_forbid_bare_newline' /etc/postfix/main.cf /etc/postfix/master.cf

The expected output is:

/etc/postfix/main.cf:smtpd_data_restrictions = reject_unauth_pipelining
/etc/postfix/main.cf:smtpd_forbid_bare_newline = yes
/etc/postfix/main.cf:smtpd_forbid_bare_newline_exclusions =
/etc/postfix/master.cf:  -o smtpd_data_restrictions=
/etc/postfix/master.cf:  -o smtpd_forbid_bare_newline=no

The mitigation is limited to the external port in order to minimize issues with internal trusted hosts, which do not adhere to the SMTP protocol.
If your environment does not have such hosts removing the modification for the master.cf.in template (by copying it to /etc/pmg/templates will disable pipelining and accepting bare linefeeds also on the internal port.

Using smtpd_data_restrictions = reject_unauth_pipelining instead of the alternative smtpd_forbid_unauth_pipelining = yes was chosen to
have one consistent fix for both supported PMG releases (7.3 and 8.0).

The packages containing the fix are available on all repositories - the version forbiding bare newlines is available in the pmg-no-subscription repositories.

UPDATE: postfix upstream has addressed the issue by providing an option to forbid bare newlines ('\n' instead of '\r\n') as command separator.
Once this change has been backported to the versions available in Debian and been released, we will provide a further update enabling this option on the external port. The Mail Proxy Whitelist (https://pmg.proxmox.com/pmg-docs/pmg-admin-guide.html#pmgconfig_whitelist_overview) can optionally list exceptions for external hosts that do not adhere to the SMTP protocol.

UPDATE (02.01.2024): the packages which have an option to forbid bare newlines as command separator are available in the bookworm-updates and bullseye-updates Debian repositories - pmg-api 8.0.11 contains updated postfix configuration templates that turn this option on for the external port, and allow clients listed in the mail proxy whitelist to still send bare newlines. An updated version for PMG 7.3 will follow in the next days.

UPDATE (10.01.2024): the text was updated to reflect that PMG 7 and 8 now use the long-term fix, as the both versions are available in the respective pmg-no-subscription repositories

 

[Andrei9385]

Hello. Which module is responsible for pmg-api ?

Spoiler: Image

 

[dcsapak]

are you on the no-subscription repo ? (it's currently only available there) and did you refresh the package list?

pmg-api is the package, so it should be in the list in the 'Proxmox' group

 

[Andrei9385]

are you on the no-subscription repo ?

No, free version.

are you on the no-subscription repo ?

No, I'll do it at night.

 

[sb-jw]

No, free version.

So no-subscription.

There are only testing, no-subscription and enterprise repo. You definitely shouldn't use the testing repo for productive use.

 

[Andrei9385]

So no-subscription.

There are only testing, no-subscription and enterprise repo. You definitely shouldn't use the testing repo for productive use.

I'm sorry, I don't know what you're talking about. I can use the repository for free or not ? Do I have to buy support ?

 

[sb-jw]

There is no "free version" repo. There are only three repos for Proxmox products: testing, no-subscription and enterprise. Your equivalent would therefore be no-subscription repo, since you definitely shouldn't use the testing repo productively, whereas the no-subscription can definitely be used. However, if you place high demands on stable operation, need support or simply want to support the further development of Proxmox, then you can buy a subscription.

You can also find out more about the repos in the docs:
PVE: https://pve.proxmox.com/pve-docs/pve-admin-guide.html#sysadmin_package_repositories
PMG: https://pmg.proxmox.com/pmg-docs/pmg-admin-guide.html#pmg_package_repositories
PBS: https://pbs.proxmox.com/docs/installation.html#debian-package-repositories

 

[Robstarusa]

Thanks for the update & quick response on this vulnerabilit.

After running

grep smtpd\_data\_restrictions /etc/postfix/main.cf /etc/postfix/master.cf

I get

# grep smtpd\_data\_restrictions /etc/postfix/main.cf /etc/postfix/master.cf
/etc/postfix/master.cf:  -o smtpd_data_restrictions=

Any idea why my /etc/postfix/main.cf wasn't updated?

 

[Stoiko Ivanov]

grep smtpd\_data\_restrictions /etc/postfix/main.cf /etc/postfix/master.cf

sorry - I mispasted the grep command (the \ before _ do not need to be there)

but in your case - do you maybe have a modified main.cf.in template in /etc/pmg/templates? - if yes - you should have been asked what to do with the new shipped template

 

[Klug]

Hello.

Last version of Wietse's post on this also recommends "smtpd_discard_ehlo_keywords = chunking" (to disable BDAT, which inherently allows pipelining).

 

[kreops]

Hi all,
I have a cluster that I have just updated with pmg-api 7.3-10 and the postfix templates are updated correctly:

# grep smtpd_data_restrictions /var/lib/pmg/templates/master.cf.in /var/lib/pmg/templates/main.cf.in
/var/lib/pmg/templates/master.cf.in:  -o smtpd_data_restrictions=
/var/lib/pmg/templates/main.cf.in:smtpd_data_restrictions = reject_unauth_pipelining

But the postfix main.cf configuration did not change:

# grep smtpd_data_restrictions /etc/postfix/main.cf /etc/postfix/master.cf
/etc/postfix/master.cf:  -o smtpd_data_restrictions=

Any help would be appreciated.

Thank you

 

[sb-jw]

pmgconfig sync --restart 1

https://pmg.proxmox.com/pmg-docs/pmg-admin-guide.html#pmgconfig_template_engine

 

[kreops]

Thank you for your answer, but that didn't help.
Unfortunately status is still the ssame, only master.cf is updated.

# pmgconfig sync --restart 1
# grep smtpd_data_restrictions /etc/postfix/main.cf /etc/postfix/master.cf
/etc/postfix/master.cf:  -o smtpd_data_restrictions=

 

[sb-jw]

Have you checked whether it is still saved in the template? Have you checked it on both PMGs? Did you just try again in main.cf and then issue the sync command again?

 

[kreops]

yes, the templates are updated in both the nodes:

root@proxmox03:~# grep smtpd_data_restrictions /var/lib/pmg/templates/master.cf.in /var/lib/pmg/templates/main.cf.in
/var/lib/pmg/templates/master.cf.in:  -o smtpd_data_restrictions=
/var/lib/pmg/templates/main.cf.in:smtpd_data_restrictions = reject_unauth_pipelining
root@proxmox03:~# date
Thu 28 Dec 2023 09:23:42 PM CET

root@proxmox04:~# grep smtpd_data_restrictions /var/lib/pmg/templates/master.cf.in /var/lib/pmg/templates/main.cf.in
/var/lib/pmg/templates/master.cf.in:  -o smtpd_data_restrictions=
/var/lib/pmg/templates/main.cf.in:smtpd_data_restrictions = reject_unauth_pipelining
root@proxmox04:~# date
Thu 28 Dec 2023 09:23:45 PM CET

I just run the command

 pmgconfig sync --restart 1

again on both nodes simultaneously, with the result the result below:

root@proxmox03:~# date
Thu 28 Dec 2023 09:25:43 PM CET
root@proxmox03:~# pmgconfig sync --restart 1
root@proxmox03:~# grep smtpd_data_restrictions /var/lib/pmg/templates/master.cf.in /var/lib/pmg/templates/main.cf.in
/var/lib/pmg/templates/master.cf.in:  -o smtpd_data_restrictions=
/var/lib/pmg/templates/main.cf.in:smtpd_data_restrictions = reject_unauth_pipelining
root@proxmox03:~# grep smtpd_data_restrictions /etc/postfix/main.cf /etc/postfix/master.cf
/etc/postfix/master.cf:  -o smtpd_data_restrictions=
root@proxmox03:~# date
Thu 28 Dec 2023 09:26:09 PM CET

root@proxmox04:~# date
Thu 28 Dec 2023 09:25:43 PM CET
root@proxmox04:~# pmgconfig sync --restart 1
root@proxmox04:~# grep smtpd_data_restrictions /var/lib/pmg/templates/master.cf.in /var/lib/pmg/templates/main.cf.in
/var/lib/pmg/templates/master.cf.in:  -o smtpd_data_restrictions=
/var/lib/pmg/templates/main.cf.in:smtpd_data_restrictions = reject_unauth_pipelining
root@proxmox04:~# grep smtpd_data_restrictions /etc/postfix/main.cf /etc/postfix/master.cf
/etc/postfix/master.cf:  -o smtpd_data_restrictions=
root@proxmox04:~# date
Thu 28 Dec 2023 09:26:09 PM CET

 

[sb-jw]

So under PMG 8 I had no problems.

Maybe there is a bug in PMG 7.3 so that the template file is not recognized at all and it is therefore not used. Maybe you're planning on upgrading to PMG 8 anyway? Then it might have been taken care of.

 

[kreops]

I have another cluster with the same version where everything is. set correctly after the upgrade.
I'd like to have the same on both

 

[kreops]

I have another cluster with the same version where everything is. set correctly after the upgrade.
I'd like to have the same on both

I found the reason why /etc/postfix/main.cf was not updated: I already had a template for it in /etc/pmg/templates, therefore the new version included in the update is not taking into account!

You have to modify the template inside the directory /etc/pmg/templates, and run pmgconfig sync --restart 1, then the postfix configiguration is finally correct.

 

[Stoiko Ivanov]

So under PMG 8 I had no problems.

Maybe there is a bug in PMG 7.3 so that the template file is not recognized at all and it is therefore not used. Maybe you're planning on upgrading to PMG 8 anyway? Then it might have been taken care of.

The config-template handling has not changed since PMG 6.X (when ucf-handling was introduced)

 

[kreops]

Hello.

Last version of Wietse's post on this also recommends "smtpd_discard_ehlo_keywords = chunking" (to disable BDAT, which inherently allows pipelining).

Personally I have updated the template manually and added smtpd_discard_ehlo_keywords = chunking in /etc/pmg/templates/main.cf.in, maybe later Proxmox will add this into their own template.