Statistics: Spam Scores shows negative values

ggzengel · Apr 13, 2021

# pmgversion -v
proxmox-mailgateway: 6.4-1 (API: 6.4-3/225ecb80, running kernel: 5.4.106-1-pve)
pmg-api: 6.4-3
pmg-gui: 2.4-1
pve-kernel-5.4: 6.3-8
pve-kernel-helper: 6.3-8
pve-kernel-5.4.106-1-pve: 5.4.106-1
pve-kernel-5.4.103-1-pve: 5.4.103-1
pve-kernel-5.4.101-1-pve: 5.4.101-1
pve-kernel-5.4.78-2-pve: 5.4.78-2
pve-kernel-5.4.73-1-pve: 5.4.73-1
clamav-daemon: 0.102.4+dfsg-0+deb10u1
libarchive-perl: 3.3.3-1
libjs-extjs: 6.0.1-10
libjs-framework7: 4.4.7-1
libpve-apiclient-perl: 3.1-3
libpve-common-perl: 6.3-5
libpve-http-server-perl: 3.1-1
libxdgmime-perl: 0.01-5
lvm2: not correctly installed
pmg-docs: 6.4-1
pmg-log-tracker: 2.1.9-1
postgresql-11: 11.11-0+deb10u1
proxmox-mini-journalreader: 1.1-1
proxmox-spamassassin: 3.4.5-1
proxmox-widget-toolkit: 2.4-9
pve-firmware: 3.2-2
pve-xtermjs: 4.7.0-3

Stoiko Ivanov · Apr 13, 2021

Please post:
* a screenshot of the general Statistics page
* the output of: `pmgsh get /statistics/mail -starttime 1618178400 -endtime 1618264800`
* the output of: ` pmgsh get /statistics/spamscores -starttime 1618178400 -endtime 161826480`
(both outputs in CODE tags please)

ggzengel · Apr 13, 2021

Code:

# pmgsh get /statistics/mail -starttime 1618178400 -endtime 1618264800
200 OK
{
   "avptime" : 9.98109970092773,
   "bounces_in" : 2,
   "bounces_out" : 0,
   "bytes_in" : 1328831,
   "bytes_out" : 2568994,
   "count" : 295,
   "count_in" : 294,
   "count_out" : 1,
   "glcount" : 1,
   "junk_in" : 279,
   "junk_out" : 0,
   "pregreet_rejects" : 248,
   "rbl_rejects" : 27,
   "spamcount_in" : 3,
   "spamcount_out" : 0,
   "spfcount" : 0,
   "viruscount_in" : 0,
   "viruscount_out" : 0
}

Code:

# pmgsh get /statistics/spamscores -starttime 1618178400 -endtime 161826480
200 OK
[
   {
      "count" : 0,
      "level" : "0",
      "ratio" : 0
   },
   {
      "count" : 0,
      "level" : "1",
      "ratio" : 0
   },
   {
      "count" : 0,
      "level" : "2",
      "ratio" : 0
   },
   {
      "count" : 0,
      "level" : "3",
      "ratio" : 0
   },
   {
      "count" : 0,
      "level" : "4",
      "ratio" : 0
   },
   {
      "count" : 0,
      "level" : "5",
      "ratio" : 0
   },
   {
      "count" : 0,
      "level" : "6",
      "ratio" : 0
   },
   {
      "count" : 0,
      "level" : "7",
      "ratio" : 0
   },
   {
      "count" : 0,
      "level" : "8",
      "ratio" : 0
   },
   {
      "count" : 0,
      "level" : "9",
      "ratio" : 0
   },
   {
      "count" : 0,
      "level" : "10",
      "ratio" : 0
   }
]

ggzengel · Apr 13, 2021

Here the same day at UTC:

Code:

# cat /etc/timezone
Etc/UTC

# pmgsh get /statistics/mail -starttime 1618185600 -endtime 1618272000
200 OK
{
   "avptime" : 1.45419465135408,
   "bounces_in" : 7,
   "bounces_out" : 9,
   "bytes_in" : 44981912,
   "bytes_out" : 71711384,
   "count" : 573,
   "count_in" : 464,
   "count_out" : 109,
   "glcount" : 0,
   "junk_in" : 280,
   "junk_out" : 0,
   "pregreet_rejects" : 248,
   "rbl_rejects" : 27,
   "spamcount_in" : 5,
   "spamcount_out" : 0,
   "spfcount" : 0,
   "viruscount_in" : 0,
   "viruscount_out" : 0
}

# pmgsh get /statistics/spamscores -starttime 1618185600 -endtime 1618272000
200 OK
[
   {
      "count" : 155,
      "level" : "0",
      "ratio" : 0.82010582010582
   },
   {
      "count" : 11,
      "level" : "1",
      "ratio" : 0.0582010582010582
   },
   {
      "count" : 18,
      "level" : "2",
      "ratio" : 0.0952380952380952
   },
   {
      "count" : 0,
      "level" : "3",
      "ratio" : 0
   },
   {
      "count" : 1,
      "level" : "4",
      "ratio" : 0.00529100529100529
   },
   {
      "count" : 0,
      "level" : "5",
      "ratio" : 0
   },
   {
      "count" : 1,
      "level" : "6",
      "ratio" : 0.00529100529100529
   },
   {
      "count" : 1,
      "level" : "7",
      "ratio" : 0.00529100529100529
   },
   {
      "count" : 1,
      "level" : "8",
      "ratio" : 0.00529100529100529
   },
   {
      "count" : 0,
      "level" : "9",
      "ratio" : 0
   },
   {
      "count" : 1,
      "level" : "10",
      "ratio" : 0.00529100529100529
   }
]

ggzengel · Apr 13, 2021

The email tells an other story:

Code:

Proxmox Status Report - 2021-04-12 (lx-mailgateway)

System Status

Hostname    lx-mailgateway

Uptime    1 day 09:36

Version    pmg-api/6.4-3/225ecb80

Load    0.02

Memory    56.05%

Disk    17.03%

Incoming Mails (24 hours)

Incoming Mails    464

Junk Mails (60.34%)    280

Spam Mails (1.08%)    5

SPF rejects (0.00%)    0

Bounces (1.51%)    7

Virus Mails (0.00%)    0

Mail Traffic    42.898 MByte

Outgoing Mails (24 hours)

Outgoing Mails    109

Bounces (8.26%)    9

Mail Traffic    68.389 MByte

Spam Quarantine

Quarantine Size (MBytes)    12

Number of Mails    69

Average Size (Bytes)    181274

Average Spam Level    9

Stoiko Ivanov · Apr 13, 2021

Stoiko Ivanov said:
* the output of: ` pmgsh get /statistics/spamscores -starttime 1618178400 -endtime 161826480`

sorry - I mistyped - there is a 0 missing in the endtime:
`pmgsh get /statistics/spamscores -starttime 1618178400 -endtime 1618264800`

ggzengel · Apr 13, 2021

Code:

# pmgsh get /statistics/spamscores -starttime 1618178400 -endtime 1618264800
200 OK
[
   {
      "count" : -14,
      "level" : "0",
      "ratio" : -0.7
   },
   {
      "count" : 11,
      "level" : "1",
      "ratio" : 0.55
   },
   {
      "count" : 18,
      "level" : "2",
      "ratio" : 0.9
   },
   {
      "count" : 0,
      "level" : "3",
      "ratio" : 0
   },
   {
      "count" : 1,
      "level" : "4",
      "ratio" : 0.05
   },
   {
      "count" : 0,
      "level" : "5",
      "ratio" : 0
   },
   {
      "count" : 1,
      "level" : "6",
      "ratio" : 0.05
   },
   {
      "count" : 1,
      "level" : "7",
      "ratio" : 0.05
   },
   {
      "count" : 1,
      "level" : "8",
      "ratio" : 0.05
   },
   {
      "count" : 0,
      "level" : "9",
      "ratio" : 0
   },
   {
      "count" : 1,
      "level" : "10",
      "ratio" : 0.05
   }
]

Stoiko Ivanov · Apr 14, 2021

On a hunch - is the statistics still off for April 12. if you check it now?

The numbers do add up in the UTC-case..

ggzengel · Apr 14, 2021

Nice drift ...

> Do not trust any statistics you did not fake yourself.

Code:

# date
Wed 14 Apr 2021 12:37:03 PM UTC

# pmgsh get /statistics/spamscores -starttime 1618178400 -endtime 1618264800
200 OK
[
   {
      "count" : -12,
      "level" : "0",
      "ratio" : -0.545454545454545
   },
   {
      "count" : 11,
      "level" : "1",
      "ratio" : 0.5
   },
   {
      "count" : 18,
      "level" : "2",
      "ratio" : 0.818181818181818
   },
   {
      "count" : 0,
      "level" : "3",
      "ratio" : 0
   },
   {
      "count" : 1,
      "level" : "4",
      "ratio" : 0.0454545454545455
   },
   {
      "count" : 0,
      "level" : "5",
      "ratio" : 0
   },
   {
      "count" : 1,
      "level" : "6",
      "ratio" : 0.0454545454545455
   },
   {
      "count" : 1,
      "level" : "7",
      "ratio" : 0.0454545454545455
   },
   {
      "count" : 1,
      "level" : "8",
      "ratio" : 0.0454545454545455
   },
   {
      "count" : 0,
      "level" : "9",
      "ratio" : 0
   },
   {
      "count" : 1,
      "level" : "10",
      "ratio" : 0.0454545454545455
   }
]

ggzengel · Apr 14, 2021

There is one more error:

On 11-04 there was 1 outgoing mail, but in global graphs it shows more.
On 12-04 there was more outgoing mail, but in global graphs it shows only one outgoing mail.

I think the value tables under global statistics is one day behind.

ggzengel · Apr 14, 2021

ggzengel · Apr 21, 2021

Did this help?

linux · Jun 6, 2021

To avoid creating a new thread, we're seeing the same negative Score 0 reporting figure, which has a knock-on to the other score percentages.

Screenshot inline, which is from PMG 6.4-3. Did you ever get any further with potential "why"s? We've seen some emails scored negatively.

Hoping to hear back with many thanks!

linux · Jun 7, 2021

linux said:
To avoid creating a new thread, we're seeing the same negative Score 0 reporting figure, which has a knock-on to the other score percentages.

Following on from this, since the above screenshot we've deployed inbound filtering to PMG as well, which has reduced/improved the problem.

It's earlier in the day relative to when we took the screenshot yesterday, though the percentages are far more sane.

Very curious if anyone/Proxmox has any clues here as to what's causing the behaviour behind-the-scenes?

Thank you!

Search

Search

Statistics: Spam Scores shows negative values

ggzengel

Member

Stoiko Ivanov

Proxmox Staff Member

ggzengel

Member

ggzengel

Member

ggzengel

Member

Stoiko Ivanov

Proxmox Staff Member

ggzengel

Member

Stoiko Ivanov

Proxmox Staff Member

ggzengel

Member

ggzengel

Member

ggzengel

Member

ggzengel

Member

linux

Member

linux

Member