PMG CPU stuck at 100%

DerDanilo

Renowned Member
Jan 21, 2017
450
105
63
Since the upgrade to
Code:
pmg-api/7.1-1/7d5cde4b (running kernel: 5.13.19-2-pve)
the CPU seems to be stuck at 100%.

The following processes are burning the CPU cores like there is no tomorrow:

Code:
postgres: 13/main root Proxmox_ruledb [local] idle in transaction
pmgmirror

According to pmgcm the nodes are synced:
Code:
root@mx2:~# pmgcm status
NAME(CID)--------------IPADDRESS----ROLE-STATE---------UPTIME---LOAD----MEM---DISK
mx2(3)               xxx.xxx.xxx.xxx  node   A           00:18   1.87    58%    15%
mx1(1)               xxx.xxx.xxx.xxx  master A           00:44   0.83    55%    14%

This just started out of the blue. We have quiet a few addresses and domains in our rulesdb but the system was never slowed down like this before.
Also this doesn't seem to effect the master node at all.

How can we solve this?

Thanks!
 

DerDanilo

Renowned Member
Jan 21, 2017
450
105
63
I checked for high cpu usage with htop and wrote the findings above.
Anyways, the CPU usage is down for now - after I installed the 5.15.x Kernel on the PVE Nodes below. There seems to be some trouble with the 5.13.x and Ryzen 9 3900 CPUs.

I'll keep you posted.
 

Stoiko Ivanov

Proxmox Staff Member
Staff member
May 2, 2018
6,856
1,039
164
* Anything relevant in the syslog?
* Anything relevant in the postgresql log?

If in doubt maybe post both here (for the timeframe where the high cpu usage was observed) - maybe we can see something.

Should it happen again - opening a `psql Proxmox_ruledb` session and running `select * from pg_stat_activity` might give some more insight.
 

DerDanilo

Renowned Member
Jan 21, 2017
450
105
63
Interessting sections I've found in the high CPU timeframe so far. The postgres log seems to give a hint.
FYI: Prior to the CPU peak nothing was changed on the PMG vm or the underlaying PVE nodes.

Bash:
# Syslog

Dec 15 22:49:44 mx2 pmg-smtp-filter[1672]: Killing "1" children
Dec 15 22:49:44 mx2 pmg-smtp-filter[1689]: 2021/12/15-22:49:44 CONNECT TCP Peer: "[127.0.0.1]:42350" Local: "[127.0.0.1]:10024"
Dec 15 22:49:44 mx2 pmg-smtp-filter[11408]: WARNING: DBI::db=HASH(0x5592b40ea218)->disconnect invalidates 3 active statement handles (either destroy statement handles or call finish on them before disconnecting) at /usr/share/perl5/PMG/RuleDB.pm line 62.

Dec 15 23:00:05 mx2 systemd[1]: Reloading Proxmox Mail Gateway Policy Daemon.
Dec 15 23:00:05 mx2 systemd[1]: Reloaded Proxmox Mail Gateway Policy Daemon.
Dec 15 23:00:05 mx2 pmgpolicy[1660]: Received a SIG HUP
Dec 15 23:00:08 mx2 pmgpolicy[1660]: 2021/12/15-23:00:08 Server closing!
Dec 15 23:00:08 mx2 pmgpolicy[1660]: 2021/12/15-23:00:08 Re-exec server during HUP

Dec 15 23:00:11 mx2 pmgpolicy[1660]: WARNING: Pid_file created by this same process. Doing nothing.
Dec 15 23:00:11 mx2 pmgpolicy[1660]: WARNING: Pid_file created by this same process. Doing nothing.
Dec 15 23:00:11 mx2 pmgpolicy[1660]: 2021/12/15-23:00:11 main (type Net::Server::PreForkSimple) starting! pid(1660)
Dec 15 23:00:11 mx2 pmgpolicy[1660]: Binding open file descriptors
Dec 15 23:00:11 mx2 pmgpolicy[1660]: Binding to TCP port 10022 on host 127.0.0.1 with IPv4
Dec 15 23:00:11 mx2 pmgpolicy[1660]: Reassociating file descriptor 8 with TCP on [127.0.0.1]:10022, using IPv4
Dec 15 23:00:11 mx2 pmgpolicy[1660]: Group Not Defined.  Defaulting to EGID '0'
Dec 15 23:00:11 mx2 pmgpolicy[1660]: User Not Defined.  Defaulting to EUID '0'
Dec 15 23:00:11 mx2 pmgpolicy[1660]: Setting up serialization via flock
Dec 15 23:00:11 mx2 pmgpolicy[1660]: Policy daemon (re)started
Dec 15 23:00:11 mx2 pmgpolicy[1660]: Beginning prefork (5 processes)
Dec 15 23:00:11 mx2 pmgpolicy[1660]: Starting "5" children

Dec 15 23:00:19 mx2 systemd[1]: pmg-hourly.service: Succeeded.
Dec 15 23:00:19 mx2 systemd[1]: Finished Hourly Proxmox Mail Gateway activities.
Dec 15 23:00:19 mx2 systemd[1]: pmg-hourly.service: Consumed 16.142s CPU time.
Dec 15 23:00:21 mx2 postfix/postscreen[1694]: CONNECT from [10.0.1.12]:49026 to [10.0.1.102]:25

# postgresql log

2021-12-15 22:24:11.532 CET [754] LOG:  received fast shutdown request
2021-12-15 22:24:11.533 CET [754] LOG:  aborting any active transactions
2021-12-15 22:24:11.534 CET [3241866] root@Proxmox_ruledb FATAL:  terminating connection due to administrator command
2021-12-15 22:24:11.536 CET [3236541] root@Proxmox_ruledb FATAL:  terminating connection due to administrator command
2021-12-15 22:24:11.538 CET [3236518] root@Proxmox_ruledb FATAL:  terminating connection due to administrator command
2021-12-15 22:24:11.540 CET [3234461] root@Proxmox_ruledb FATAL:  terminating connection due to administrator command
2021-12-15 22:24:11.541 CET [3226453] root@Proxmox_ruledb FATAL:  terminating connection due to administrator command
2021-12-15 22:24:11.543 CET [3226245] root@Proxmox_ruledb FATAL:  terminating connection due to administrator command
2021-12-15 22:24:11.550 CET [754] LOG:  background worker "logical replication launcher" (PID 772) exited with exit code 1
2021-12-15 22:24:11.555 CET [3223436] root@Proxmox_ruledb FATAL:  terminating connection due to administrator command
2021-12-15 22:24:11.555 CET [3223436] root@Proxmox_ruledb LOG:  could not send data to client: Broken pipe
2021-12-15 22:24:11.592 CET [767] LOG:  shutting down
2021-12-15 22:24:11.623 CET [754] LOG:  database system is shut down
2021-12-15 22:31:40.863 CET [759] LOG:  starting PostgreSQL 13.5 (Debian 13.5-0+deb11u1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2021-12-15 22:31:40.864 CET [759] LOG:  listening on IPv6 address "::1", port 5432
2021-12-15 22:31:40.864 CET [759] LOG:  listening on IPv4 address "127.0.0.1", port 5432
2021-12-15 22:31:40.865 CET [759] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2021-12-15 22:31:40.869 CET [762] LOG:  database system was shut down at 2021-12-15 22:24:11 CET
2021-12-15 22:31:40.876 CET [759] LOG:  database system is ready to accept connections
2021-12-15 22:31:43.782 CET [937] root@Proxmox_ruledb ERROR:  constraint "localstat_time_key" of relation "localstat" does not exist
2021-12-15 22:31:43.782 CET [937] root@Proxmox_ruledb STATEMENT:  ALTER TABLE LocalStat DROP CONSTRAINT localstat_time_key
2021-12-15 23:55:22.749 CET [759] LOG:  received fast shutdown request
2021-12-15 23:55:22.756 CET [759] LOG:  aborting any active transactions
2021-12-15 23:55:22.763 CET [61239] root@Proxmox_ruledb FATAL:  terminating connection due to administrator command
2021-12-15 23:55:22.775 CET [759] LOG:  background worker "logical replication launcher" (PID 769) exited with exit code 1
2021-12-15 23:55:22.804 CET [763] LOG:  shutting down
2021-12-15 23:55:22.893 CET [759] LOG:  database system is shut down
2021-12-15 23:55:54.801 CET [758] LOG:  starting PostgreSQL 13.5 (Debian 13.5-0+deb11u1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2021-12-15 23:55:54.802 CET [758] LOG:  listening on IPv6 address "::1", port 5432
2021-12-15 23:55:54.802 CET [758] LOG:  listening on IPv4 address "127.0.0.1", port 5432
2021-12-15 23:55:54.804 CET [758] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2021-12-15 23:55:54.812 CET [765] LOG:  database system was shut down at 2021-12-15 23:55:22 CET
2021-12-15 23:55:54.825 CET [758] LOG:  database system is ready to accept connections
2021-12-15 23:55:59.325 CET [805] root@Proxmox_ruledb ERROR:  constraint "localstat_time_key" of relation "localstat" does not exist
2021-12-15 23:55:59.325 CET [805] root@Proxmox_ruledb STATEMENT:  ALTER TABLE LocalStat DROP CONSTRAINT localstat_time_key

2021-12-16 00:08:29.600 CET [765] LOG:  starting PostgreSQL 13.5 (Debian 13.5-0+deb11u1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2021-12-16 00:08:29.602 CET [765] LOG:  listening on IPv6 address "::1", port 5432
2021-12-16 00:08:29.602 CET [765] LOG:  listening on IPv4 address "127.0.0.1", port 5432
2021-12-16 00:08:29.606 CET [765] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2021-12-16 00:08:29.624 CET [768] LOG:  database system was interrupted; last known up at 2021-12-16 00:06:00 CET
2021-12-16 00:08:29.880 CET [768] LOG:  database system was not properly shut down; automatic recovery in progress
2021-12-16 00:08:29.889 CET [768] LOG:  redo starts at 64/93E3EEB8
2021-12-16 00:08:29.889 CET [768] LOG:  invalid record length at 64/93E3EFA0: wanted 24, got 0
2021-12-16 00:08:29.889 CET [768] LOG:  redo done at 64/93E3EF68
2021-12-16 00:08:29.927 CET [765] LOG:  database system is ready to accept connections
2021-12-16 00:08:36.857 CET [812] root@Proxmox_ruledb ERROR:  constraint "localstat_time_key" of relation "localstat" does not exist
2021-12-16 00:08:36.857 CET [812] root@Proxmox_ruledb STATEMENT:  ALTER TABLE LocalStat DROP CONSTRAINT localstat_time_key
2021-12-16 01:29:46.515 CET [769] LOG:  starting PostgreSQL 13.5 (Debian 13.5-0+deb11u1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2021-12-16 01:29:46.516 CET [769] LOG:  listening on IPv6 address "::1", port 5432
2021-12-16 01:29:46.516 CET [769] LOG:  listening on IPv4 address "127.0.0.1", port 5432
2021-12-16 01:29:46.516 CET [769] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2021-12-16 01:29:46.520 CET [773] LOG:  database system was interrupted; last known up at 2021-12-16 00:48:33 CET
2021-12-16 01:29:46.556 CET [773] LOG:  database system was not properly shut down; automatic recovery in progress
2021-12-16 01:29:46.558 CET [773] LOG:  redo starts at 64/93F7C690
2021-12-16 01:29:46.561 CET [773] LOG:  invalid record length at 64/93F9DB58: wanted 24, got 0
2021-12-16 01:29:46.562 CET [773] LOG:  redo done at 64/93F9DB20
2021-12-16 01:29:46.575 CET [769] LOG:  database system is ready to accept connections
2021-12-16 01:29:49.345 CET [811] root@Proxmox_ruledb ERROR:  constraint "localstat_time_key" of relation "localstat" does not exist
2021-12-16 01:29:49.345 CET [811] root@Proxmox_ruledb STATEMENT:  ALTER TABLE LocalStat DROP CONSTRAINT localstat_time_key
2021-12-16 01:47:55.923 CET [769] LOG:  received fast shutdown request
2021-12-16 01:47:56.009 CET [769] LOG:  aborting any active transactions
2021-12-16 01:47:56.014 CET [769] LOG:  background worker "logical replication launcher" (PID 783) exited with exit code 1
2021-12-16 01:47:56.014 CET [778] LOG:  shutting down
2021-12-16 01:47:56.048 CET [769] LOG:  database system is shut down
2021-12-16 01:50:15.188 CET [763] LOG:  starting PostgreSQL 13.5 (Debian 13.5-0+deb11u1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2021-12-16 01:50:15.188 CET [763] LOG:  listening on IPv6 address "::1", port 5432
2021-12-16 01:50:15.188 CET [763] LOG:  listening on IPv4 address "127.0.0.1", port 5432
2021-12-16 01:50:15.189 CET [763] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2021-12-16 01:50:15.193 CET [767] LOG:  database system was shut down at 2021-12-16 01:47:56 CET
2021-12-16 01:50:15.199 CET [763] LOG:  database system is ready to accept connections
2021-12-16 01:50:18.027 CET [805] root@Proxmox_ruledb ERROR:  constraint "localstat_time_key" of relation "localstat" does not exist
2021-12-16 01:50:18.027 CET [805] root@Proxmox_ruledb STATEMENT:  ALTER TABLE LocalStat DROP CONSTRAINT localstat_time_key
2021-12-16 02:32:56.195 CET [763] LOG:  received fast shutdown request
2021-12-16 02:32:56.197 CET [763] LOG:  aborting any active transactions
2021-12-16 02:32:56.198 CET [9045] root@Proxmox_ruledb FATAL:  terminating connection due to administrator command
2021-12-16 02:32:56.198 CET [763] LOG:  background worker "logical replication launcher" (PID 774) exited with exit code 1
2021-12-16 02:32:56.200 CET [11162] root@Proxmox_ruledb FATAL:  terminating connection due to administrator command
2021-12-16 02:32:56.201 CET [1605] root@Proxmox_ruledb FATAL:  terminating connection due to administrator command
2021-12-16 02:32:56.203 CET [8272] root@Proxmox_ruledb FATAL:  terminating connection due to administrator command
2021-12-16 02:32:56.205 CET [1619] root@Proxmox_ruledb FATAL:  terminating connection due to administrator command
2021-12-16 02:32:56.207 CET [29624] root@Proxmox_ruledb FATAL:  terminating connection due to administrator command
2021-12-16 02:32:56.209 CET [9808] root@Proxmox_ruledb FATAL:  terminating connection due to administrator command
2021-12-16 02:32:56.212 CET [11802] root@Proxmox_ruledb FATAL:  terminating connection due to administrator command
2021-12-16 02:32:56.217 CET [769] LOG:  shutting down
2021-12-16 02:32:56.253 CET [763] LOG:  database system is shut down
 

DerDanilo

Renowned Member
Jan 21, 2017
450
105
63
Bash:
2021-12-16 03:04:46.210 CET [769] LOG:  starting PostgreSQL 13.5 (Debian 13.5-0+deb11u1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2021-12-16 03:04:46.211 CET [769] LOG:  listening on IPv6 address "::1", port 5432
2021-12-16 03:04:46.211 CET [769] LOG:  listening on IPv4 address "127.0.0.1", port 5432
2021-12-16 03:04:46.212 CET [769] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2021-12-16 03:04:46.216 CET [771] LOG:  database system was shut down at 2021-12-16 02:32:56 CET
2021-12-16 03:04:46.223 CET [769] LOG:  database system is ready to accept connections
2021-12-16 03:04:49.065 CET [811] root@Proxmox_ruledb ERROR:  constraint "localstat_time_key" of relation "localstat" does not exist
2021-12-16 03:04:49.065 CET [811] root@Proxmox_ruledb STATEMENT:  ALTER TABLE LocalStat DROP CONSTRAINT localstat_time_key
root@mx2:~# more /var/log/postgresql/postgresql-13-main.log
2021-12-16 00:08:29.600 CET [765] LOG:  starting PostgreSQL 13.5 (Debian 13.5-0+deb11u1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2021-12-16 00:08:29.602 CET [765] LOG:  listening on IPv6 address "::1", port 5432
2021-12-16 00:08:29.602 CET [765] LOG:  listening on IPv4 address "127.0.0.1", port 5432
2021-12-16 00:08:29.606 CET [765] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2021-12-16 00:08:29.624 CET [768] LOG:  database system was interrupted; last known up at 2021-12-16 00:06:00 CET
2021-12-16 00:08:29.880 CET [768] LOG:  database system was not properly shut down; automatic recovery in progress
2021-12-16 00:08:29.889 CET [768] LOG:  redo starts at 64/93E3EEB8
2021-12-16 00:08:29.889 CET [768] LOG:  invalid record length at 64/93E3EFA0: wanted 24, got 0
2021-12-16 00:08:29.889 CET [768] LOG:  redo done at 64/93E3EF68
2021-12-16 00:08:29.927 CET [765] LOG:  database system is ready to accept connections
2021-12-16 00:08:36.857 CET [812] root@Proxmox_ruledb ERROR:  constraint "localstat_time_key" of relation "localstat" does not exist
2021-12-16 00:08:36.857 CET [812] root@Proxmox_ruledb STATEMENT:  ALTER TABLE LocalStat DROP CONSTRAINT localstat_time_key
2021-12-16 01:29:46.515 CET [769] LOG:  starting PostgreSQL 13.5 (Debian 13.5-0+deb11u1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2021-12-16 01:29:46.516 CET [769] LOG:  listening on IPv6 address "::1", port 5432
2021-12-16 01:29:46.516 CET [769] LOG:  listening on IPv4 address "127.0.0.1", port 5432
2021-12-16 01:29:46.516 CET [769] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2021-12-16 01:29:46.520 CET [773] LOG:  database system was interrupted; last known up at 2021-12-16 00:48:33 CET
2021-12-16 01:29:46.556 CET [773] LOG:  database system was not properly shut down; automatic recovery in progress
2021-12-16 01:29:46.558 CET [773] LOG:  redo starts at 64/93F7C690
2021-12-16 01:29:46.561 CET [773] LOG:  invalid record length at 64/93F9DB58: wanted 24, got 0
2021-12-16 01:29:46.562 CET [773] LOG:  redo done at 64/93F9DB20
2021-12-16 01:29:46.575 CET [769] LOG:  database system is ready to accept connections
2021-12-16 01:29:49.345 CET [811] root@Proxmox_ruledb ERROR:  constraint "localstat_time_key" of relation "localstat" does not exist
2021-12-16 01:29:49.345 CET [811] root@Proxmox_ruledb STATEMENT:  ALTER TABLE LocalStat DROP CONSTRAINT localstat_time_key
2021-12-16 01:47:55.923 CET [769] LOG:  received fast shutdown request
2021-12-16 01:47:56.009 CET [769] LOG:  aborting any active transactions
2021-12-16 01:47:56.014 CET [769] LOG:  background worker "logical replication launcher" (PID 783) exited with exit code 1
2021-12-16 01:47:56.014 CET [778] LOG:  shutting down
2021-12-16 01:47:56.048 CET [769] LOG:  database system is shut down
2021-12-16 01:50:15.188 CET [763] LOG:  starting PostgreSQL 13.5 (Debian 13.5-0+deb11u1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2021-12-16 01:50:15.188 CET [763] LOG:  listening on IPv6 address "::1", port 5432
2021-12-16 01:50:15.188 CET [763] LOG:  listening on IPv4 address "127.0.0.1", port 5432
2021-12-16 01:50:15.189 CET [763] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2021-12-16 01:50:15.193 CET [767] LOG:  database system was shut down at 2021-12-16 01:47:56 CET
2021-12-16 01:50:15.199 CET [763] LOG:  database system is ready to accept connections
2021-12-16 01:50:18.027 CET [805] root@Proxmox_ruledb ERROR:  constraint "localstat_time_key" of relation "localstat" does not exist
2021-12-16 01:50:18.027 CET [805] root@Proxmox_ruledb STATEMENT:  ALTER TABLE LocalStat DROP CONSTRAINT localstat_time_key
2021-12-16 02:32:56.195 CET [763] LOG:  received fast shutdown request
2021-12-16 02:32:56.197 CET [763] LOG:  aborting any active transactions
2021-12-16 02:32:56.198 CET [9045] root@Proxmox_ruledb FATAL:  terminating connection due to administrator command
2021-12-16 02:32:56.198 CET [763] LOG:  background worker "logical replication launcher" (PID 774) exited with exit code 1
2021-12-16 02:32:56.200 CET [11162] root@Proxmox_ruledb FATAL:  terminating connection due to administrator command
2021-12-16 02:32:56.201 CET [1605] root@Proxmox_ruledb FATAL:  terminating connection due to administrator command
2021-12-16 02:32:56.203 CET [8272] root@Proxmox_ruledb FATAL:  terminating connection due to administrator command
2021-12-16 02:32:56.205 CET [1619] root@Proxmox_ruledb FATAL:  terminating connection due to administrator command
2021-12-16 02:32:56.207 CET [29624] root@Proxmox_ruledb FATAL:  terminating connection due to administrator command
2021-12-16 02:32:56.209 CET [9808] root@Proxmox_ruledb FATAL:  terminating connection due to administrator command
2021-12-16 02:32:56.212 CET [11802] root@Proxmox_ruledb FATAL:  terminating connection due to administrator command
2021-12-16 02:32:56.217 CET [769] LOG:  shutting down
2021-12-16 02:32:56.253 CET [763] LOG:  database system is shut down
2021-12-16 03:04:46.210 CET [769] LOG:  starting PostgreSQL 13.5 (Debian 13.5-0+deb11u1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2021-12-16 03:04:46.211 CET [769] LOG:  listening on IPv6 address "::1", port 5432
2021-12-16 03:04:46.211 CET [769] LOG:  listening on IPv4 address "127.0.0.1", port 5432
2021-12-16 03:04:46.212 CET [769] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2021-12-16 03:04:46.216 CET [771] LOG:  database system was shut down at 2021-12-16 02:32:56 CET
2021-12-16 03:04:46.223 CET [769] LOG:  database system is ready to accept connections
2021-12-16 03:04:49.065 CET [811] root@Proxmox_ruledb ERROR:  constraint "localstat_time_key" of relation "localstat" does not exist
2021-12-16 03:04:49.065 CET [811] root@Proxmox_ruledb STATEMENT:  ALTER TABLE LocalStat DROP CONSTRAINT localstat_time_key
 

Stoiko Ivanov

Proxmox Staff Member
Staff member
May 2, 2018
6,856
1,039
164
Dec 15 22:49:44 mx2 pmg-smtp-filter[11408]: WARNING: DBI::db=HASH(0x5592b40ea218)->disconnect invalidates 3 active statement handles (either destroy statement handles or call finish on them before disconnecting) at /usr/share/perl5/PMG/RuleDB.pm line 62.
This line looks odd (in the sense that I don't recall having seen this (despite looking quite a lot at PMG logs).

The only thing in the postgres log - which does not look familiar and might be an indicator of something being off are the following lines:
2021-12-16 01:29:46.520 CET [773] LOG: database system was interrupted; last known up at 2021-12-16 00:48:33 CET
2021-12-16 01:29:46.556 CET [773] LOG: database system was not properly shut down; automatic recovery in progress
2021-12-16 01:29:46.558 CET [773] LOG: redo starts at 64/93F7C690
2021-12-16 01:29:46.561 CET [773] LOG: invalid record length at 64/93F9DB58: wanted 24, got 0
2021-12-16 01:29:46.562 CET [773] LOG: redo done at 64/93F9DB28

* You could try to create a backup of PMG in the GUI
* then import it to a new PMG instance - while checking the postgres logs - and reboot

If this works you could try restoring from backup on your live-system as well

I hope this helps!
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!