How to log subjects of all incoming email?

koby · Sep 16, 2020

Hi ,
After that change I do manage to see the mail subject on the log ,
But it does not shown in the tracking center too
Any help please.

here is my change :

Code:

# PHASE 2 - parse mail
# on error: exit

        my $maxfiles = $pmg_cfg->get('clamav', 'archivemaxfiles')o;

        my $entity = $queue->parse_mail($maxfiles);

        $self->log (3, "$queue->{logid}: new mail message-id=%s", $queue->{msgid});

# Add by koby at 16/09/2020    

        $self->log (3, "$queue->{logid}: Subject: %s", PMG::Utils::decode_rfc1522(PVE::Tools::trim($entity->head->get('subject'))) || 'No Subject');
        my @fromarray = split('\s*,\s*', $entity->head->get('from') || $msginfo->{sender});
        $self->log (3, "$queue->{logid}: From: %s", PMG::Utils::decode_rfc1522(PVE::Tools::trim($fromarray[0])) );

# End

Any help please.

heutger · Sep 16, 2020

May you be able to contribute your code to be embedded in the official distribution?

koby · Sep 16, 2020

heutger said:
May you be able to contribute your code to be embedded in the official distribution?

koby · Sep 16, 2020

Hello heutger ,
This is not my code...
I took it from this forum.
It would be nice to have it in the product.
According to my problem , there is any help ?

heutger · Sep 16, 2020

Question is to @Miktash

TFrenz · Dec 3, 2020

In Version 6.3-5 it does not run.
Following error:
Dec 03 12:08:51 mta.ebf-frenz.de systemd[1]: Starting Proxmox SMTP Filter Daemon...
Dec 03 12:08:53 mta.ebf-frenz.de pmg-smtp-filter[710]: Global symbol "$entity" requires explicit package name (did you forget to declare "my $entity"?) at /usr/bin/pmg-smtp-filter line 838.
Dec 03 12:08:53 mta.ebf-frenz.de pmg-smtp-filter[710]: Global symbol "$entity" requires explicit package name (did you forget to declare "my $entity"?) at /usr/bin/pmg-smtp-filter line 839.
Dec 03 12:08:53 mta.ebf-frenz.de pmg-smtp-filter[710]: Execution of /usr/bin/pmg-smtp-filter aborted due to compilation errors.
Dec 03 12:08:53 mta.ebf-frenz.de systemd[1]: pmg-smtp-filter.service: Control process exited, code=exited, status=255/EXCEPTION
Dec 03 12:08:53 mta.ebf-frenz.de systemd[1]: pmg-smtp-filter.service: Failed with result 'exit-code'.
Dec 03 12:08:53 mta.ebf-frenz.de systemd[1]: Failed to start Proxmox SMTP Filter Daemon.

What can i do?

TFrenz · Dec 3, 2020

I have found my mistake.
i have inserted in Phase5 this is incorrect.
The phase2 is the correct place.

johannes21 · Jul 21, 2021

Thank you Miktash, it works.
But would it still be possible to display the subject as a column?

kez · May 11, 2023

Hi there,

This isn't working for me in v7.3-3

Code:

File: /usr/bin/pmg-smtp-filter

23 use PVE::SafeSyslog;
24 use PVE::ProcFSTools;
25 use PVE::INotify;
26 use PVE::Tools;

/// split ////

651 # PHASE 2 - parse mail
652 # on error: exit
653
654         my $maxfiles = $pmg_cfg->get('clamav', 'archivemaxfiles');
655
656         my $entity = $queue->parse_mail($maxfiles);
657
658         $self->log (3, "$queue->{logid}: new mail message-id=%s", $queue->{msgid});
659
660         # Custome code log subject and more..
661
662         $self->log (3, "$queue->{logid}: Subject: %s", PMG::Utils::decode_rfc1522(PVE::Tools::trim($entity->head->get('subject'))) || 'No Subject');
663         my @fromarray = split('\s*,\s*', $entity->head->get('from') || $msginfo->{sender});
664         $self->log (3, "$queue->{logid}: From: %s", PMG::Utils::decode_rfc1522(PVE::Tools::trim($fromarray[0])) );

Does anyone have any advice?

Thanks,
C

UPDATE

I changed it to this and it worked, i.e deleted the last 2 lines.

Code:

651 # PHASE 2 - parse mail
652 # on error: exit
653
654         my $maxfiles = $pmg_cfg->get('clamav', 'archivemaxfiles');
655
656         my $entity = $queue->parse_mail($maxfiles);
657
658         $self->log (3, "$queue->{logid}: new mail message-id=%s", $queue->{msgid});
659
660         # Custome code log subject and more..
661
662         $self->log (3, "$queue->{logid}: Subject: %s", PMG::Utils::decode_rfc1522(PVE::Tools::trim($entity->head->get('subject'))) || 'No Subject');

heutger · May 11, 2023

kez said:

Hi there,

This isn't working for me in v7.3-3

Code:

File: /usr/bin/pmg-smtp-filter

23 use PVE::SafeSyslog;
24 use PVE::ProcFSTools;
25 use PVE::INotify;
26 use PVE::Tools;

/// split ////

651 # PHASE 2 - parse mail
652 # on error: exit
653
654         my $maxfiles = $pmg_cfg->get('clamav', 'archivemaxfiles');
655
656         my $entity = $queue->parse_mail($maxfiles);
657
658         $self->log (3, "$queue->{logid}: new mail message-id=%s", $queue->{msgid});
659
660         # Custome code log subject and more..
661
662         $self->log (3, "$queue->{logid}: Subject: %s", PMG::Utils::decode_rfc1522(PVE::Tools::trim($entity->head->get('subject'))) || 'No Subject');
663         my @fromarray = split('\s*,\s*', $entity->head->get('from') || $msginfo->{sender});
664         $self->log (3, "$queue->{logid}: From: %s", PMG::Utils::decode_rfc1522(PVE::Tools::trim($fromarray[0])) );

Does anyone have any advice?

Thanks,
C

Please file a feature request at bugzilla.proxmox.com, so that this feature will be realized. In my advancing thread you can find a solution via header checks adding given from, subject as well. However, this is an adjustment and result in changing default behavior. Still looking for both since rejecting mails from score 3 and can’t verify the rejected mails from mailjet, sendgrid, etc. as they don’t show the from they will show in the mail as being handling mails for many customers. Technical address doesn’t help, subject would help as well.

kez · Jun 16, 2023

UPDATE -

This works fine for me:

Code:

650 # PHASE 2 - parse mail
651 # on error: exit
652
653         my $maxfiles = $pmg_cfg->get('clamav', 'archivemaxfiles');
654
655         my $entity = $queue->parse_mail($maxfiles);
656
657         $self->log (3, "$queue->{logid}: new mail message-id=%s", $queue->{msgid});
658
659         $self->log (3, "$queue->{logid}: Subject: %s", PMG::Utils::decode_rfc1522($entity->head->get ('subject', 0)) || 'No Subject');

Hope this helps!
C

josefb · Jun 23, 2023

Dear Promox,

please reconsider your concerns about subject logging.

Yes it can be problematic in relation to the GDPR, but in principle every admin with system access can read all forwarded mails (eg. a simple bcc can to the job or modification in the postfix-config or a tcpdump, ...).
I've worked with many spam-filter solutions in the past, but nearly all show the Subject and the body From in the central logging. Many of them also have the possibility to show the whole message or a least all headers. Without that the tracking of spam is really a pain.

If a user has concerns about the GDPR, then in my opinion this can only be resolved with an appropriate agreement anyway (-> Article 28 GDPR).

Just my 2 cents
bye Josef

jadsy2107 · Jun 26, 2023

kez said:

UPDATE -

This works fine for me:

Code:

650 # PHASE 2 - parse mail
651 # on error: exit
652
653         my $maxfiles = $pmg_cfg->get('clamav', 'archivemaxfiles');
654
655         my $entity = $queue->parse_mail($maxfiles);
656
657         $self->log (3, "$queue->{logid}: new mail message-id=%s", $queue->{msgid});
658
659         $self->log (3, "$queue->{logid}: Subject: %s", PMG::Utils::decode_rfc1522($entity->head->get ('subject', 0)) || 'No Subject');

Hope this helps!
C

This worked for me as well, Proxmox Mail Gateway 7.3-4
I'm assuming this will be overwritten in updates?

josefb · Jun 26, 2023

jadsy2107 said:
This worked for me as well, Proxmox Mail Gateway 7.3-4
I'm assuming this will be overwritten in updates?

It works for me also, but I'm pretty sure this will be overwritten in updates because the file is part of the pmg-api package.

josefb · Jun 26, 2023

just to give an idea how the tracking center looks in other spam-filter solutions. It would be really very helpful to see the Subject and the body From (Sender).

mach · Jun 30, 2023

I got it working on pmg 8 using this:

Diff:

--- pmg-smtp-filter.orig.8.0    2023-06-30 05:30:48.958121687 -0400
+++ pmg-smtp-filter    2023-07-06 03:00:52.579596405 -0400
@@ -23,6 +23,7 @@
 use PVE::SafeSyslog;
 use PVE::ProcFSTools;
 use PVE::INotify;
+use PVE::Tools;
 
 use Mail::SpamAssassin;
 use Mail::SpamAssassin::NetSet;
@@ -659,6 +660,9 @@
     my $entity = $queue->parse_mail($maxfiles);
 
     $self->log (3, "$queue->{logid}: new mail message-id=%s", $queue->{msgid});
+    $self->log (3, "$queue->{logid}: Subject: %s", PMG::Utils::decode_rfc1522($entity->head->get ('subject', 0)) || 'No Subject');
+    my @fromarray = split('\s*,\s*', PMG::Utils::decode_rfc1522($entity->head->get('from')) || $msginfo->{sender});
+    $self->log (3, "$queue->{logid}: From: %s", PMG::Utils::decode_rfc1522(PVE::Tools::trim($fromarray[0])) );
 
 # PHASE 3 - run external content analyzers
 # (SPAM analyzer is run on demand later)

tanvarian · Jan 24, 2024

mach said:

I got it working on pmg 8 using this:

Diff:

--- pmg-smtp-filter.orig.8.0    2023-06-30 05:30:48.958121687 -0400
+++ pmg-smtp-filter    2023-07-06 03:00:52.579596405 -0400
@@ -23,6 +23,7 @@
 use PVE::SafeSyslog;
 use PVE::ProcFSTools;
 use PVE::INotify;
+use PVE::Tools;
 
 use Mail::SpamAssassin;
 use Mail::SpamAssassin::NetSet;
@@ -659,6 +660,9 @@
     my $entity = $queue->parse_mail($maxfiles);
 
     $self->log (3, "$queue->{logid}: new mail message-id=%s", $queue->{msgid});
+    $self->log (3, "$queue->{logid}: Subject: %s", PMG::Utils::decode_rfc1522($entity->head->get ('subject', 0)) || 'No Subject');
+    my @fromarray = split('\s*,\s*', PMG::Utils::decode_rfc1522($entity->head->get('from')) || $msginfo->{sender});
+    $self->log (3, "$queue->{logid}: From: %s", PMG::Utils::decode_rfc1522(PVE::Tools::trim($fromarray[0])) );
 
 # PHASE 3 - run external content analyzers
 # (SPAM analyzer is run on demand later)

Hi,
its working fine as long as the subject does not contain any emojis. Is there any way to show subject containing emojis character?

heutger · Jan 24, 2024

tanvarian said:
Hi,
its working fine as long as the subject does not contain any emojis. Is there any way to show subject containing emojis character?

Upvote my Bugzillw feature request to add this option to the GUI. So that would work then for sure.

chcatz · Feb 6, 2024

... not nice but better than no subject in the log.(UTF-8 characters fix)

$self->log (3, "$queue->{logid}: Subject: %s", PMG::Utils::rfc1522_to_html($entity->head->get ('subject', 0)) || 'No Subject');

triggerhappy · Feb 29, 2024

Hello,

I'm trying to achieve the same but in the tracking center, i don't see any subject being logged. Also, is it possible to export the tracking center output to CSV ?

How to log subjects of all incoming email?

Renowned Member

Famous Member

Renowned Member

Renowned Member

Famous Member

Member

Member

New Member

Member

Famous Member

Member

Member

New Member

Member

Member

Member

New Member

Famous Member

Member

New Member