Auto-Whitelist

[BryanPearson]

Does checking the "Use auto-whitelists" in the Spam Detector Configuration simply activate the Spamassassin AWL, or does it add all outbound emails' recipient addresses to the Global Whitelist?

If we could automatically add the outbound email addresses to our global whitelist, it would really help us, as we are getting too many false positives.


I am using a trial version now, and have about 14 days left.

 

[martin]

Does checking the "Use auto-whitelists" in the Spam Detector Configuration simply activate the Spamassassin AWL

yes.

, or does it add all outbound emails' recipient addresses to the Global Whitelist?

If we could automatically add the outbound email addresses to our global whitelist, it would really help us, as we are getting too many false positives.

no, that is not possible. I suggest to adapt the settings and rules to your requirements to get best results.

First, I suggest to activate the following:

- greylisting
- SPF
- RBL
- receiver veriftication

with these settings you will block a lot of emails already on smtp level, before the spam rules begins to analyse.

Second, increase the spam level (default is 5) to 8. (see what object spam).

If this does not give you the wanted results, we have to analyze the false positives by looking into the mail headers (if any) and we can add some custom rules.

 

[BryanPearson]

Greylisting, SPF not viable for customer email

When I talk to my users about using greylisting, they uniformly reject it as counter-productive. My users send and receive email constantly in conjunction with phone calls. If the email were routinely delayed in either direction by even ten minutes, it would be almost unanimously viewed as a big mistake.

SPF is not very useful to us as an indicator of spamminess, as our customer base has a very wide variety of email arrangements, and very few of them use servers with SPF records.

I have now enabled RBL, and allowed the traffic through the firewall. It is too soon to see how effective that will be.

Receiver verification for us is tricky. I absolutely want the initial SMTP session to be terminated if the recipient is not present. This is very good. But the mail gateway NEEDS to be able to independently handle the list of valid addresses. There should be a REJECT action on the mail gateway, that rejects SMTP sessions containing traffic for addresses not in a local list. This is especially important when the mail server is inaccessible for some reason, and when there may be more than one mail gateway in the chain of custody for security or redundancy purposes.

The BLOCK action simply loses the mail, right? A REJECT option that accepts only those matching the inbound list, and rejects everything else is better for our customers. If my email address is ernie.klarn@mymail.com, and someone attempts to send mail to ernesto.klarn@mymail.com or erni.klarn@mymail.com, he will get a non-delivery report from HIS server if the session with my gateway is rejected. This is very useful to the customer.

When the PROXMOX gateway is not the first one in the chain of mail-handling servers, it is too late to reject the session, and when it is not the last one in the chain, it can't consult the email server to verify the receivers.

Automatically adding outbound mail addresses to a whitelist is a BIG deal to us. Our customers may email someone here, then call when their email is ignored (quarantined as spam). When they call, they give someone their email address. Our employee can add them to their personal whitelist, which won't help when they are emailing a department or someone else at the company, or forward the email address to me to add to the global whitelist, which is a pain for me, and must be repeated when the customer's email address changes.

This shouldn't be about transferring the effort we used to spend on dealing with spam to time spent dealing with the anti-spam infrastructure. Automatically adding any outbound email address to the global whitelist is a very hands-off approach that saves everyone the most time.

BTW, thanks for the ability to add custom.cf to the Spamassassin ruleset. I added several rules (after a crash course in regexp) that have helped immensely with both spam detection and reducing false positives. It is a big deal to me, and certainly something any competitor will have to offer to be considered. But without localized receiver verification and real auto-whitelisting, Proxmox will at best be only part of our anti-spam solution.

 

[tom]

hi again,

When I talk to my users about using greylisting, they uniformly reject it as counter-productive. My users send and receive email constantly in conjunction with phone calls. If the email were routinely delayed in either direction by even ten minutes, it would be almost unanimously viewed as a big mistake.

an email sender is maximum greylisted one time, then he will be added automatically to the known sender list. BTW, if the sender has a well configured email system, he will NEVER get greylisted (SPF record).

greylisting is extremly effective against unwanted emails, if you do not use it you miss this.

SPF is not very useful to us as an indicator of spamminess, as our customer base has a very wide variety of email arrangements, and very few of them use servers with SPF records.

if you enable SPF and your customer has NO spf configured, nothing happens here. the trick is, if your customer has a valid SPF (and more and more do this) then your proxmox can see this and can reject spammer due to the spf record. so I cannot see any problem activating spf on your proxmox.

I have now enabled RBL, and allowed the traffic through the firewall. It is too soon to see how effective that will be.

yes. you can query the syslog for this, eg. for spamhaus rejects. go to the console and type for example:

> grep spamhaus.org/query /var/log/syslog

Receiver verification for us is tricky. I absolutely want the initial SMTP session to be terminated if the recipient is not present. This is very good. But the mail gateway NEEDS to be able to independently handle the list of valid addresses. There should be a REJECT action on the mail gateway, that rejects SMTP sessions containing traffic for addresses not in a local list. This is especially important when the mail server is inaccessible for some reason, and when there may be more than one mail gateway in the chain of custody for security or redundancy purposes.

just enable receiver verification (yes, 450). giving an temp. failure. so if you system is not up, you will get the emails later. this is a save setup. if you really need a local list, use the LDAP connection. this sync. all email addresses from an ms active dir to you proxmox, locally cached.

The BLOCK action simply loses the mail, right?

yes.

A REJECT option that accepts only those matching the inbound list, and rejects everything else is better for our customers. If my email address is ernie.klarn@mymail.com, and someone attempts to send mail to ernesto.klarn@mymail.com or erni.klarn@mymail.com, he will get a non-delivery report from HIS server if the session with my gateway is rejected. This is very useful to the customer.

yes, that is no problem here.

When the PROXMOX gateway is not the first one in the chain of mail-handling servers, it is too late to reject the session, and when it is not the last one in the chain, it can't consult the email server to verify the receivers.

if you want to reject on smtp level, this server has to be on the first place. this point is, that the emails are not transmitted in this case. if you accepts on the first place everything, you loose this functionality. if proxmox is on the second place, you can something with the ruleset and the LDAP connection. but note, the emails are already in your network (bandwith).

Automatically adding outbound mail addresses to a whitelist is a BIG deal to us. Our customers may email someone here, then call when their email is ignored (quarantined as spam). When they call, they give someone their email address. Our employee can add them to their personal whitelist, which won't help when they are emailing a department or someone else at the company, or forward the email address to me to add to the global whitelist, which is a pain for me, and must be repeated when the customer's email address changes.

This shouldn't be about transferring the effort we used to spend on dealing with spam to time spent dealing with the anti-spam infrastructure. Automatically adding any outbound email address to the global whitelist is a very hands-off approach that saves everyone the most time.

I am quite sure that this is a not good idea for most situations, building up such whitelist have several sideeffect which are not wanted. we can think of enabling such an option in the next release. (2.1 in almost finished, I talk here about 2.2 or later).

BTW, thanks for the ability to add custom.cf to the Spamassassin ruleset. I added several rules (after a crash course in regexp) that have helped immensely with both spam detection and reducing false positives. It is a big deal to me, and certainly something any competitor will have to offer to be considered. But without localized receiver verification and real auto-whitelisting, Proxmox will at best be only part of our anti-spam solution.

see above, LDAP is doing localized receiver verification.

my opinion: I do not think that its a good idea to have 2 spamfilters (or just for redundancy, build a HA cluster with Proxmox) most effective against spam are smtp level checks and therefore this spamserver has to be on the first place. we have sites where more than one million email want to come, but more than 96 % are already rejected on smtp level - before the messages are transfered, no NDR, just worryfree (just think of botnets, etc.).

 

[BryanPearson]

I am quite sure that this is a not good idea for most situations, building up such whitelist have several sideeffect which are not wanted. we can think of enabling such an option in the next release. (2.1 in almost finished, I talk here about 2.2 or later).

I can't imagine what the side effects could be, unless you postulate that:
1) Our customers sometimes spam us and sometimes they don't.
2) We return email to spammers.

Microsoft does this in their Outlook Junk-Email filtering. You can check an option to "Automatically add people I email to the safe-senders list." and for individuals, this makes sense, wouldn't you agree? I mean, if spam comes in from an email address that is not on the safe senders list (whitelist) I sure could take it off the list in a hurry, right?

But with company departmental email, Joe may be handling email contact for customer Bill, and one day Joe is gone. Mary is now handling Joe's mail, but she had no way of getting Joe's whitelist. Since Bill's email is now ignored as spam (he suffers from poor IT support and a crappy email client) he becomes someone else's customer. Joe wouldn't even have to leave - he could just refer Bill to Mary, and poof! - he is off our radar.

Bill shouldn't have to prove he is a customer over and over each time he emails someone else in the company. An automatically managed global whitelist completely fixes that from Bill's perspective. And happy customers mean happy companies.
Yes, this list would end up being rather large. But I suspect for most companies the list would be IMMENSELY valuable in reducing false positives. And isn't the risk of false positives the reason people aren't as aggressive as they could be in choosing a spam score threshold? And how could this list be more difficult to manage than the greylist?

As it is, it looks as though we will be purchasing the product. We haven't yet picked the version we want. But we would pay $1000 just to add that feature.

Thanks for your time. I appreciate your responses. They have been to the point and very helpful.

 

[tom]

I am quite sure that this is a not good idea for most situations, building up such whitelist have several sideeffect which are not wanted. we can think of enabling such an option in the next release. (2.1 in almost finished, I talk here about 2.2 or later).

I can't imagine what the side effects could be, unless you postulate that:
1) Our customers sometimes spam us and sometimes they don't.
2) We return email to spammers.

Microsoft does this in their Outlook Junk-Email filtering. You can check an option to "Automatically add people I email to the safe-senders list." and for individuals, this makes sense, wouldn't you agree? I mean, if spam comes in from an email address that is not on the safe senders list (whitelist) I sure could take it off the list in a hurry, right?

But with company departmental email, Joe may be handling email contact for customer Bill, and one day Joe is gone. Mary is now handling Joe's mail, but she had no way of getting Joe's whitelist. Since Bill's email is now ignored as spam (he suffers from poor IT support and a crappy email client) he becomes someone else's customer. Joe wouldn't even have to leave - he could just refer Bill to Mary, and poof! - he is off our radar.

Bill shouldn't have to prove he is a customer over and over each time he emails someone else in the company. An automatically managed global whitelist completely fixes that from Bill's perspective. And happy customers mean happy companies.
Yes, this list would end up being rather large. But I suspect for most companies the list would be IMMENSELY valuable in reducing false positives. And isn't the risk of false positives the reason people aren't as aggressive as they could be in choosing a spam score threshold? And how could this list be more difficult to manage than the greylist?

As it is, it looks as though we will be purchasing the product. We haven't yet picked the version we want. But we would pay $1000 just to add that feature.

hi bryan,

I suggest that you should discuss this feature request directly with Martin (CEO, martin@proxmox.com), with him you can negotiate the details.

Thanks for your time. I appreciate your responses. They have been to the point and very helpful.

thanks also for your postings, we need this feedback to understand customer needs.

 

[BryanPearson]

Auto-Whitelist useful after all?

Last night I had a bit of a brainstorm.

If I create a signature line that everyone in the company uses, like:

Wheaties® are the BEST cereal ever!

And then give it a negative spam score in the custom.cf of -100 (clearly ham) then any replies to our outgoing messages would automatically get a very high ham rating, which would tend to persist, thanks to the Auto-Whitelist feature of Spamassassin averaging the spam score of email from the customer's address. Even if subsequent emails were to be missing the signature line, the averaging effect should still keep them from being filtered.

So while it isn't quite what I was hoping for, it is still very manageable. All our employees would have to do is send an email to the customer asking them to simply reply, to ensure further email from that customer is not mistaken for spam.

Do I have this right? Will this plan work the way I have described?

BTW, we are readily beating the Outlook filtering now, and the Proxmox isn't even the Internet-facing server. When we move it, I expect another jump in effectiveness. We plan to order today.