Statistics: Spam Scores shows negative values

ggzengel

Member
Jul 8, 2019
18
1
6
1618274150232.png

# pmgversion -v proxmox-mailgateway: 6.4-1 (API: 6.4-3/225ecb80, running kernel: 5.4.106-1-pve) pmg-api: 6.4-3 pmg-gui: 2.4-1 pve-kernel-5.4: 6.3-8 pve-kernel-helper: 6.3-8 pve-kernel-5.4.106-1-pve: 5.4.106-1 pve-kernel-5.4.103-1-pve: 5.4.103-1 pve-kernel-5.4.101-1-pve: 5.4.101-1 pve-kernel-5.4.78-2-pve: 5.4.78-2 pve-kernel-5.4.73-1-pve: 5.4.73-1 clamav-daemon: 0.102.4+dfsg-0+deb10u1 libarchive-perl: 3.3.3-1 libjs-extjs: 6.0.1-10 libjs-framework7: 4.4.7-1 libpve-apiclient-perl: 3.1-3 libpve-common-perl: 6.3-5 libpve-http-server-perl: 3.1-1 libxdgmime-perl: 0.01-5 lvm2: not correctly installed pmg-docs: 6.4-1 pmg-log-tracker: 2.1.9-1 postgresql-11: 11.11-0+deb10u1 proxmox-mini-journalreader: 1.1-1 proxmox-spamassassin: 3.4.5-1 proxmox-widget-toolkit: 2.4-9 pve-firmware: 3.2-2 pve-xtermjs: 4.7.0-3
 
Please post:
* a screenshot of the general Statistics page
* the output of: `pmgsh get /statistics/mail -starttime 1618178400 -endtime 1618264800`
* the output of: ` pmgsh get /statistics/spamscores -starttime 1618178400 -endtime 161826480`
(both outputs in CODE tags please)
 
1618306335279.png

Code:
# pmgsh get /statistics/mail -starttime 1618178400 -endtime 1618264800
200 OK
{
   "avptime" : 9.98109970092773,
   "bounces_in" : 2,
   "bounces_out" : 0,
   "bytes_in" : 1328831,
   "bytes_out" : 2568994,
   "count" : 295,
   "count_in" : 294,
   "count_out" : 1,
   "glcount" : 1,
   "junk_in" : 279,
   "junk_out" : 0,
   "pregreet_rejects" : 248,
   "rbl_rejects" : 27,
   "spamcount_in" : 3,
   "spamcount_out" : 0,
   "spfcount" : 0,
   "viruscount_in" : 0,
   "viruscount_out" : 0
}

Code:
# pmgsh get /statistics/spamscores -starttime 1618178400 -endtime 161826480
200 OK
[
   {
      "count" : 0,
      "level" : "0",
      "ratio" : 0
   },
   {
      "count" : 0,
      "level" : "1",
      "ratio" : 0
   },
   {
      "count" : 0,
      "level" : "2",
      "ratio" : 0
   },
   {
      "count" : 0,
      "level" : "3",
      "ratio" : 0
   },
   {
      "count" : 0,
      "level" : "4",
      "ratio" : 0
   },
   {
      "count" : 0,
      "level" : "5",
      "ratio" : 0
   },
   {
      "count" : 0,
      "level" : "6",
      "ratio" : 0
   },
   {
      "count" : 0,
      "level" : "7",
      "ratio" : 0
   },
   {
      "count" : 0,
      "level" : "8",
      "ratio" : 0
   },
   {
      "count" : 0,
      "level" : "9",
      "ratio" : 0
   },
   {
      "count" : 0,
      "level" : "10",
      "ratio" : 0
   }
]
 
Here the same day at UTC:

Code:
# cat /etc/timezone
Etc/UTC

# pmgsh get /statistics/mail -starttime 1618185600 -endtime 1618272000
200 OK
{
   "avptime" : 1.45419465135408,
   "bounces_in" : 7,
   "bounces_out" : 9,
   "bytes_in" : 44981912,
   "bytes_out" : 71711384,
   "count" : 573,
   "count_in" : 464,
   "count_out" : 109,
   "glcount" : 0,
   "junk_in" : 280,
   "junk_out" : 0,
   "pregreet_rejects" : 248,
   "rbl_rejects" : 27,
   "spamcount_in" : 5,
   "spamcount_out" : 0,
   "spfcount" : 0,
   "viruscount_in" : 0,
   "viruscount_out" : 0
}

# pmgsh get /statistics/spamscores -starttime 1618185600 -endtime 1618272000
200 OK
[
   {
      "count" : 155,
      "level" : "0",
      "ratio" : 0.82010582010582
   },
   {
      "count" : 11,
      "level" : "1",
      "ratio" : 0.0582010582010582
   },
   {
      "count" : 18,
      "level" : "2",
      "ratio" : 0.0952380952380952
   },
   {
      "count" : 0,
      "level" : "3",
      "ratio" : 0
   },
   {
      "count" : 1,
      "level" : "4",
      "ratio" : 0.00529100529100529
   },
   {
      "count" : 0,
      "level" : "5",
      "ratio" : 0
   },
   {
      "count" : 1,
      "level" : "6",
      "ratio" : 0.00529100529100529
   },
   {
      "count" : 1,
      "level" : "7",
      "ratio" : 0.00529100529100529
   },
   {
      "count" : 1,
      "level" : "8",
      "ratio" : 0.00529100529100529
   },
   {
      "count" : 0,
      "level" : "9",
      "ratio" : 0
   },
   {
      "count" : 1,
      "level" : "10",
      "ratio" : 0.00529100529100529
   }
]
 
The email tells an other story:

Code:
Proxmox Status Report - 2021-04-12 (lx-mailgateway)

System Status

Hostname    lx-mailgateway

Uptime    1 day 09:36

Version    pmg-api/6.4-3/225ecb80

Load    0.02

Memory    56.05%

Disk    17.03%

Incoming Mails (24 hours)

Incoming Mails    464

Junk Mails (60.34%)    280

Spam Mails (1.08%)    5

SPF rejects (0.00%)    0

Bounces (1.51%)    7

Virus Mails (0.00%)    0

Mail Traffic    42.898 MByte

Outgoing Mails (24 hours)

Outgoing Mails    109

Bounces (8.26%)    9

Mail Traffic    68.389 MByte

Spam Quarantine

Quarantine Size (MBytes)    12

Number of Mails    69

Average Size (Bytes)    181274

Average Spam Level    9
 
* the output of: ` pmgsh get /statistics/spamscores -starttime 1618178400 -endtime 161826480`
sorry - I mistyped - there is a 0 missing in the endtime:
`pmgsh get /statistics/spamscores -starttime 1618178400 -endtime 1618264800`
 
Code:
# pmgsh get /statistics/spamscores -starttime 1618178400 -endtime 1618264800
200 OK
[
   {
      "count" : -14,
      "level" : "0",
      "ratio" : -0.7
   },
   {
      "count" : 11,
      "level" : "1",
      "ratio" : 0.55
   },
   {
      "count" : 18,
      "level" : "2",
      "ratio" : 0.9
   },
   {
      "count" : 0,
      "level" : "3",
      "ratio" : 0
   },
   {
      "count" : 1,
      "level" : "4",
      "ratio" : 0.05
   },
   {
      "count" : 0,
      "level" : "5",
      "ratio" : 0
   },
   {
      "count" : 1,
      "level" : "6",
      "ratio" : 0.05
   },
   {
      "count" : 1,
      "level" : "7",
      "ratio" : 0.05
   },
   {
      "count" : 1,
      "level" : "8",
      "ratio" : 0.05
   },
   {
      "count" : 0,
      "level" : "9",
      "ratio" : 0
   },
   {
      "count" : 1,
      "level" : "10",
      "ratio" : 0.05
   }
]
 
On a hunch - is the statistics still off for April 12. if you check it now?

The numbers do add up in the UTC-case..
 
Nice drift ...

> Do not trust any statistics you did not fake yourself.

Code:
# date
Wed 14 Apr 2021 12:37:03 PM UTC

# pmgsh get /statistics/spamscores -starttime 1618178400 -endtime 1618264800
200 OK
[
   {
      "count" : -12,
      "level" : "0",
      "ratio" : -0.545454545454545
   },
   {
      "count" : 11,
      "level" : "1",
      "ratio" : 0.5
   },
   {
      "count" : 18,
      "level" : "2",
      "ratio" : 0.818181818181818
   },
   {
      "count" : 0,
      "level" : "3",
      "ratio" : 0
   },
   {
      "count" : 1,
      "level" : "4",
      "ratio" : 0.0454545454545455
   },
   {
      "count" : 0,
      "level" : "5",
      "ratio" : 0
   },
   {
      "count" : 1,
      "level" : "6",
      "ratio" : 0.0454545454545455
   },
   {
      "count" : 1,
      "level" : "7",
      "ratio" : 0.0454545454545455
   },
   {
      "count" : 1,
      "level" : "8",
      "ratio" : 0.0454545454545455
   },
   {
      "count" : 0,
      "level" : "9",
      "ratio" : 0
   },
   {
      "count" : 1,
      "level" : "10",
      "ratio" : 0.0454545454545455
   }
]
 
There is one more error:

On 11-04 there was 1 outgoing mail, but in global graphs it shows more.
On 12-04 there was more outgoing mail, but in global graphs it shows only one outgoing mail.

I think the value tables under global statistics is one day behind.

1618418548296.png
 
To avoid creating a new thread, we're seeing the same negative Score 0 reporting figure, which has a knock-on to the other score percentages.

Screenshot inline, which is from PMG 6.4-3. Did you ever get any further with potential "why"s? We've seen some emails scored negatively.

PMG-NegativeScore0-20210606.png

Hoping to hear back with many thanks!
 
To avoid creating a new thread, we're seeing the same negative Score 0 reporting figure, which has a knock-on to the other score percentages.

Following on from this, since the above screenshot we've deployed inbound filtering to PMG as well, which has reduced/improved the problem.

PMG-NegScore-Improved20210607.png

It's earlier in the day relative to when we took the screenshot yesterday, though the percentages are far more sane.

Very curious if anyone/Proxmox has any clues here as to what's causing the behaviour behind-the-scenes?

Thank you!
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!