Is there a built-in tool for removing duplicate entries? Is it possible to add a check for duplicates when creating a new record?
Faced with large white and black lists, I discovered that there were no checks for duplicate entries in the database, so I had to write a “crutch” to correct this problem. I apologize in advance for the code, I am not a programmer, I wrote this program for my own purposes
Faced with large white and black lists, I discovered that there were no checks for duplicate entries in the database, so I had to write a “crutch” to correct this problem. I apologize in advance for the code, I am not a programmer, I wrote this program for my own purposes
Program (CLI) for searching and removing duplicate email/domain records in the Blacklist/Whitelist tables of the Proxmox Mail Gateway program
This program is written "on the knee", the author, as usual, does not bear any responsibility for its work, use it at your own peril and risk
There are 4 files in the archive:
pmgdupe.exe - the program itself
pmgdupe.ini - text settings file
pmgdupe.pb - source code for the PureBasic 5.73 LTS compiler
readme.txt - this file
First of all, go to the PMG mail server console via SSH and issue the command:
In response we get the following output:
We are interested in "Global blacklist", look at its identifier - in this case 27. Now we upload values with this identifier to the source.json file:
The resulting source.json is our table with duplicates. I find duplicates most often in the domain and email objects, so I made a program specifically for them; if you want other values, rewrite them yourself.
We copy the table to a Windows machine (for example via ssh), check the settings in pmgdupe.ini, it contains three values - the source file (Source), the output file of the script (Destination) and the number of the array of objects (27), before using it you need to write your parameters there .
We run pmgdupe, get a .sh script as output, which we transfer via SSH to PMG and set the “executable” attribute to it, after which we run it. If all is well, duplicate entries in the table will be deleted and only one entry will be left.
This program is written "on the knee", the author, as usual, does not bear any responsibility for its work, use it at your own peril and risk
There are 4 files in the archive:
pmgdupe.exe - the program itself
pmgdupe.ini - text settings file
pmgdupe.pb - source code for the PureBasic 5.73 LTS compiler
readme.txt - this file
First of all, go to the PMG mail server console via SSH and issue the command:
pmgsh get /config/ruledb/whoIn response we get the following output:
[ { "id": 27, "info" : "Global blacklist", "name" : "Blacklist" }, { "id" : 28, "info" : "Global whitelist", "name" : "Whitelist" }]We are interested in "Global blacklist", look at its identifier - in this case 27. Now we upload values with this identifier to the source.json file:
pmgsh get /config/ruledb/who/27/objects >source.jsonThe resulting source.json is our table with duplicates. I find duplicates most often in the domain and email objects, so I made a program specifically for them; if you want other values, rewrite them yourself.
We copy the table to a Windows machine (for example via ssh), check the settings in pmgdupe.ini, it contains three values - the source file (Source), the output file of the script (Destination) and the number of the array of objects (27), before using it you need to write your parameters there .
We run pmgdupe, get a .sh script as output, which we transfer via SSH to PMG and set the “executable” attribute to it, after which we run it. If all is well, duplicate entries in the table will be deleted and only one entry will be left.