Policy Database Maintenance Issues

[Matthias Mayr]

Hello Everyone,

I receive emails regarding "Error: queue file write error" quite regular. In the Syslog it states:
Mar 29 11:07:36 my-hostname postfix/smtpd[219387]: warning: timeout talking to proxy 127.0.0.1:10024
Mar 29 11:07:36 my-hostname postfix/smtpd[219387]: proxy-reject: END-OF-MESSAGE: 451 4.3.0 Error: queue file write error; from [...]

While investigating this issue, I noticed that everytime I get those messages, there is also another message around:
Mar 29 11:07:22 my-hostname pmgpolicy[838]: starting policy database maintenance (greylist, rbl)
Mar 29 11:07:27 my-hostname pmgpolicy[838]: end policy database maintenance (5298 ms, 0 ms)

pmgpolicy start it's policy database maintenance and produces a high load (Load of 4 on a quadcore VM) and takes up to 8 seconds.

The load of the machine is around 200-500 Mails/d, running on a quadcore vm with 4G RAM on a PVE Ceph Cluster, backed by HDD (SSD would be possible if it is likely to resolve the issue).

How can I tweak the database maintenance? (When searching through the forum, most other users seems to have a duration of a few milliseconds not seconds on db maintenance)

or the otherway round

How can I handle this issue without tweaking db maintenance/what other reasons could be responsible for the smtp timeouts?

Thank you very much in advance

Regards
Matthias

 

[Stoiko Ivanov]

I receive emails regarding "Error: queue file write error" quite regular. In the Syslog it states:
Mar 29 11:07:36 my-hostname postfix/smtpd[219387]: warning: timeout talking to proxy 127.0.0.1:10024
Mar 29 11:07:36 my-hostname postfix/smtpd[219387]: proxy-reject: END-OF-MESSAGE: 451 4.3.0 Error: queue file write error; from [...]

hmm seems either pmg-smtp-filter crashes - or all of the workers are running quite long (which might point to a CPU bottleneck)

* do you have any custom rules/spamassassin configs in your setup?
* any other modifications ?

Could you please provide a larger sample of your journal (maybe we can spot an issue there)

the database maintanence of pmgpolicy should not really interfere with mail-processing - although you're right that 6 seconds seem a bit long….
* any interesting lines in the postresql log? (/var/log/postgresql/postgresql-13-main.log)

I hope this helps!

 

[Matthias Mayr]

Hello Stoiko Ivanov,

thank you for your reply. You were mainly right, it has nothing to do with the database maintenance. This was indeed another symptom.
I tried to find further patterns, and noticed that the issure is occurring when the backup runs.

I solved it by
- Moving PMG to ssd storage instead of HDD
- assigning more memory
- assigning more cpu cores

as I took all those three measures at once, I am not abled to tell, which option mainly is responsible for fixing the issue.

I am running PVE with PBS and never notices issues with performance during backup on any other machine.
Could you think of any issues why the machine is having severe performance issues during backup (Guest Agent is installed and active).

Thank you very much.

Regards
Matthias

 

[Stoiko Ivanov]

I am running PVE with PBS and never notices issues with performance during backup on any other machine.
Could you think of any issues why the machine is having severe performance issues during backup (Guest Agent is installed and active).

it very much depends on your setup:
* please share the VM/CT config of the PMG installation on PVE
* please share the Hardwarespecs PVE is running on
* how are your disks set-up?

 

[Matthias Mayr]

multinode ceph cluster with ceph pools consisting out of 1T Enterprise SSDs and 6T WD Gold SATA disks
all nodes are running 256G Memory and 24 Cores on current Epyc-Hardware.
All nodes connected with 10G
Ceph running on dedicated 10G
Corosync on 2x dedicated 1G

PMG is running as VM.
- 4G Memory (Currently upgraded to 8G), no ballooning
- 1 CPU, 4 Cores (Currently upgraded to 8 Cores)
- Running on ceph hdd pool (Currently SSD Pool) with 64G
- Virtio Network
- Virtio SCSI
- SeaBios
- Guest Agent
- OS Type Linux 5.x
- ACPI: yes
- KVM: yes