[SOLVED] Importing big blacklists and whitelists from another spam gateway

[Fathi]

Hi,
I am evaluating pmg on a production environment and would like to import our current white and blacklists from the scrollout F1 mail gateway. I have these white/black emails and domains in plain text, so is it possible to import them from cli ? from text file or with a loop command, or by any other batch mode, ...
TIA

 

[mira]

Take a look at pmgsh (https://pmg.proxmox.com/pmg-docs/pmg-admin-guide.html#chapter_pmgsh) together with the API Viewer (https://pmg.proxmox.com/pmg-docs/api-viewer/index.html). You can use it to automate the process of adding your white- and blacklists.

 

[Fathi]

Take a look at pmgsh (https://pmg.proxmox.com/pmg-docs/pmg-admin-guide.html#chapter_pmgsh) together with the API Viewer (https://pmg.proxmox.com/pmg-docs/api-viewer/index.html). You can use it to automate the process of adding your white- and blacklists.

Hi,
Thank you for your reply. I tried it, it works only if I specify a pmail with root role parameter. What I need is to add entries to global white/blacklists not to per user white/blacklist. Could you, give me more hints please.

 

[hata_ph]

https://forum.proxmox.com/threads/import-csv-to-blacklist.48300/#post-228240

 

[mira]

The rule white- and blacklists are in config/ruledb/who. To add for example an email address to the whitelist, get the ID of the whitelist with pmgsh get config/ruledb/who. Once you have the ID you can use pmgsh create config/ruledb/who/<whitelist-ID>/email --email <email address> to add it to the whitelist.

 

[Fathi]

Thank you,
I finely managed to import my white and black lists (emails + domain). But, i made a mistake running the batches twice, and found that emails and domains have been imported twice. While this has no impact on accuracy, it could have an impact on performance. Is there a way to remove these duplicates ? And as many of ower spammers use <random_big_number@qq.com> and the likes and other probably unique inexistent email addresses, is there a way to expire email addresses that have not been seen since xxx days, so to have the processing time of black lists at reasonable levels ? A similar way could be done, by cron, to expurge blacklisted domains that doesn't exist any more.

 

[mira]

You could try greylisting. This will return a temporary error (4xx) the first time a new tuple (IP, sender, receiver) appears, which leads to the sending mail server to retry it later. Most spam mails won't be resent, while legitimate mails should.
So the second time a mail is sent, it will pass through just fine and the tuple will be cached for 30 days.

You can also whitelist certain senders to make sure they always get through on the first try.

 

[aamirkabli]

Thank you,
I finely managed to import my white and black lists (emails + domain). But, i made a mistake running the batches twice, and found that emails and domains have been imported twice. While this has no impact on accuracy, it could have an impact on performance. Is there a way to remove these duplicates ? And as many of ower spammers use <random_big_number@qq.com> and the likes and other probably unique inexistent email addresses, is there a way to expire email addresses that have not been seen since xxx days, so to have the processing time of black lists at reasonable levels ? A similar way could be done, by cron, to expurge blacklisted domains that doesn't exist any more.

Hi Fathi,
Can you please let me know how you managed to import your white and black lists (emails + domain).

 

[Fathi]

Hi Fathi,
Can you please let me know how you managed to import your white and black lists (emails + domain).

Hi @aamirkabli ,
I followed @mira instruction up on this thread https://forum.proxmox.com/threads/i...s-from-another-spam-gateway.74839/post-333558