[gentoo-user] Greylisting vs. reject_rbl_client

[gentoo-user] Greylisting vs. reject_rbl_client

[Grant]

I've followed the steps outlined here to eliminate spam up to the
section on "SPF and greylisting" on the second page:

http://www.freesoftwaremagazine.com/articles/focus_spam_postfix/

The author is really into greylisting:

"If you take nothing else from this article, let it be that
greylisting is a Good Thing and your customers will love you for using
it"

but the feedback I got on it from this list was not as positive.  Of
the stuff I've implemented, these lines seem to have been the most
effective:

reject_rbl_client relays.ordb.org,
reject_rbl_client list.dsbl.org,
reject_rbl_client sbl-xbl.spamhaus.org,

with the spamhaus.org line doing most of the work according to the logs.

Do you think the reject_rbl_client stuff is safer than greylisting?

- Grant
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Greylisting vs. reject_rbl_client

[kashani]

Grant wrote:
> Do you think the reject_rbl_client stuff is safer than greylisting?
> 
> - Grant

1. Blacklists have the HIGHEST false positive rate of any anti-spam 
technique other than sending all mail to /dev/null. 34%
http://www.paulgraham.com/falsepositives.html

2. Blacklists block the least amount of spam. 24%
So it's wrong more often than right.

3. All Blacklists are run by jackasses. Yes, even the ones you like.
http://www.internetnews.com/xSP/article.php/8_1143551
http://www.peacefire.org/anti-spam/group-statement.5-17-2001.html
http://www.networkworld.com/research/2001/0910feat.html

and far too much personal experience*

	In my experience over the past two to three years greylisting and 
simple header checks have blocked 99% of spam before it gets to the 
queue and generated less admin overhead with false positives and other 
nonsense. I'd call its accuracy a solid 99.9% since I've only had to 
whitelist three sets of servers over the years, YMMV. It might not be 
99.9 for everyone, but it will be far better than blacklisting. There 
are some quirks with greylisting, but overall it's been very effective 
without much downside.

I can't say enough bad things about blacklisting.

kashani

* The first ISP I worked for actually hosted public.com which has 
probably been the most hijacked domain ever. It's a fun Monday morning 
when some moron decided to block your entire ISP without actually 
looking at the headers. It gets slightly less fun the fifth and sixth 
time it happens. Homicide is considered when they assume they are 
automatically right, are as rude as possible to you, and then stall for 
a day before they grudgingly remove you.
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Greylisting vs. reject_rbl_client

[Grant]

> > Do you think the reject_rbl_client stuff is safer than greylisting?
> >
> > - Grant
>
> 1. Blacklists have the HIGHEST false positive rate of any anti-spam
> technique other than sending all mail to /dev/null. 34%
> http://www.paulgraham.com/falsepositives.html
>
> 2. Blacklists block the least amount of spam. 24%
> So it's wrong more often than right.
>
> 3. All Blacklists are run by jackasses. Yes, even the ones you like.
> http://www.internetnews.com/xSP/article.php/8_1143551
> http://www.peacefire.org/anti-spam/group-statement.5-17-2001.html
> http://www.networkworld.com/research/2001/0910feat.html
>
> and far too much personal experience*
>
>         In my experience over the past two to three years greylisting and
> simple header checks have blocked 99% of spam before it gets to the
> queue and generated less admin overhead with false positives and other
> nonsense. I'd call its accuracy a solid 99.9% since I've only had to
> whitelist three sets of servers over the years, YMMV. It might not be
> 99.9 for everyone, but it will be far better than blacklisting. There
> are some quirks with greylisting, but overall it's been very effective
> without much downside.
>
> I can't say enough bad things about blacklisting.
>
> kashani
>
> * The first ISP I worked for actually hosted public.com which has
> probably been the most hijacked domain ever. It's a fun Monday morning
> when some moron decided to block your entire ISP without actually
> looking at the headers. It gets slightly less fun the fifth and sixth
> time it happens. Homicide is considered when they assume they are
> automatically right, are as rude as possible to you, and then stall for
> a day before they grudgingly remove you.

I've removed the blacklisting and thank you very much for your
response.  I guess I'm back to greylisting and/or content filtering.

- Grant
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Greylisting vs. reject_rbl_client

[Grant]

> > Do you think the reject_rbl_client stuff is safer than greylisting?
> >
> > - Grant
>
> 1. Blacklists have the HIGHEST false positive rate of any anti-spam
> technique other than sending all mail to /dev/null. 34%
> http://www.paulgraham.com/falsepositives.html
>
> 2. Blacklists block the least amount of spam. 24%
> So it's wrong more often than right.
>
> 3. All Blacklists are run by jackasses. Yes, even the ones you like.
> http://www.internetnews.com/xSP/article.php/8_1143551
> http://www.peacefire.org/anti-spam/group-statement.5-17-2001.html
> http://www.networkworld.com/research/2001/0910feat.html
>
> and far too much personal experience*
>
>         In my experience over the past two to three years greylisting and
> simple header checks have blocked 99% of spam before it gets to the
> queue and generated less admin overhead with false positives and other
> nonsense. I'd call its accuracy a solid 99.9% since I've only had to
> whitelist three sets of servers over the years, YMMV. It might not be
> 99.9 for everyone, but it will be far better than blacklisting. There
> are some quirks with greylisting, but overall it's been very effective
> without much downside.
>
> I can't say enough bad things about blacklisting.
>
> kashani
>
> * The first ISP I worked for actually hosted public.com which has
> probably been the most hijacked domain ever. It's a fun Monday morning
> when some moron decided to block your entire ISP without actually
> looking at the headers. It gets slightly less fun the fifth and sixth
> time it happens. Homicide is considered when they assume they are
> automatically right, are as rude as possible to you, and then stall for
> a day before they grudgingly remove you.

Do you think this postfix anti-spam configuration is OK:

smtpd_delay_reject = yes
smtpd_helo_required = yes
smtpd_helo_restrictions =
        permit_mynetworks,
        reject_non_fqdn_hostname,
        reject_invalid_hostname,
        permit
smtpd_sender_restrictions =
        permit_mynetworks,
        reject_non_fqdn_sender,
        reject_unknown_sender_domain,
        permit
smtpd_recipient_restrictions =
        permit_mynetworks,
        reject_non_fqdn_recipient,
        reject_unknown_recipient_domain,
        reject_unauth_destination,
        permit

Would it be OK to remove the following aliases since I never use them:

# Well-known aliases -- these should be filled in!
root:           grant
operator:       grant

# Standard RFC2142 aliases
abuse:              grant
ftp:                grant
hostmaster:         grant
news:               grant
noc:                grant
security:           grant
usenet:             grant
uucp:               grant
webmaster:          grant
www:                grant
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Greylisting vs. reject_rbl_client

[Nick Rout]

On Mon, 21 Aug 2006 20:28:16 -0700
Grant <emailgrant@gmail.com> wrote:

> Would it be OK to remove the following aliases since I never use them:
> 
> # Well-known aliases -- these should be filled in!
> root:           grant
> operator:       grant
> 
> # Standard RFC2142 aliases
> abuse:              grant
> ftp:                grant
> hostmaster:         grant
> news:               grant
> noc:                grant
> security:           grant
> usenet:             grant
> uucp:               grant
> webmaster:          grant
> www:                grant

Don't remove the root one, some packages send emails to their owner, and if they are (for example) privileged daemons the message will go to root. It needs to be pointed elsewhere.

The others, I am not sure. You probably should have an abuse address and a postmaster address, other than that I am not sure what the custom is or what the rfc's say about what email accounts are expected in a domain. Don't forget that your web server (if you have one) may deliver pages that say to contact a certain mail address in the event of problems, obviously that address should be available.
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Greylisting vs. reject_rbl_client

[Grant]

> > Would it be OK to remove the following aliases since I never use them:
> >
> > # Well-known aliases -- these should be filled in!
> > root:           grant
> > operator:       grant
> >
> > # Standard RFC2142 aliases
> > abuse:              grant
> > ftp:                grant
> > hostmaster:         grant
> > news:               grant
> > noc:                grant
> > security:           grant
> > usenet:             grant
> > uucp:               grant
> > webmaster:          grant
> > www:                grant
>
> Don't remove the root one, some packages send emails to their owner, and if they are (for example) privileged daemons the message will go to root. It needs to be pointed elsewhere.
>
> The others, I am not sure. You probably should have an abuse address and a postmaster address, other than that I am not sure what the custom is or what the rfc's say about what email accounts are expected in a domain. Don't forget that your web server (if you have one) may deliver pages that say to contact a certain mail address in the event of problems, obviously that address should be available.

Good call.  I'll keep root and abuse and you reminded me to add one
for the user my shopping cart program runs as.

- Grant
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Greylisting vs. reject_rbl_client

[kashani]

Grant wrote:
> Do you think this postfix anti-spam configuration is OK:
> 
> smtpd_delay_reject = yes
> smtpd_helo_required = yes
> smtpd_helo_restrictions =
>        permit_mynetworks,
>        reject_non_fqdn_hostname,
>        reject_invalid_hostname,
>        permit

I'd be careful with non_fqdn_hostname

> smtpd_sender_restrictions =
>        permit_mynetworks,
>        reject_non_fqdn_sender,
>        reject_unknown_sender_domain,
>        permit
> smtpd_recipient_restrictions =
>        permit_mynetworks,
>        reject_non_fqdn_recipient,
>        reject_unknown_recipient_domain,
>        reject_unauth_destination,
>        permit

That's pretty much what I run and you might want to look at 
smtpd_data_restrictions as well.

> Would it be OK to remove the following aliases since I never use them:

It's good form to keep them on your server and compile with the relvent 
RFC which specifies these.

kashani
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Greylisting vs. reject_rbl_client

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 191 bytes --]

On Thu, 24 Aug 2006 17:42:07 -0700, kashani wrote:

> I'd be careful with non_fqdn_hostname

Why?


-- 
Neil Bothwick

"I'm Not Sure If I'm Homosexual", Said Tom, Half In Earnest.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-user] Greylisting vs. reject_rbl_client

[Grant]

> > Do you think this postfix anti-spam configuration is OK:
> >
> > smtpd_delay_reject = yes
> > smtpd_helo_required = yes
> > smtpd_helo_restrictions =
> >        permit_mynetworks,
> >        reject_non_fqdn_hostname,
> >        reject_invalid_hostname,
> >        permit
>
> I'd be careful with non_fqdn_hostname

What's wrong with that?  Here's how the postfix docs describe it:

reject_non_fqdn_helo_hostname (with Postfix < 2.3: reject_non_fqdn_hostname)
Reject the request when the HELO or EHLO hostname is not in
fully-qualified domain form, as required by the RFC.

> > smtpd_sender_restrictions =
> >        permit_mynetworks,
> >        reject_non_fqdn_sender,
> >        reject_unknown_sender_domain,
> >        permit
> > smtpd_recipient_restrictions =
> >        permit_mynetworks,
> >        reject_non_fqdn_recipient,
> >        reject_unknown_recipient_domain,
> >        reject_unauth_destination,
> >        permit
>
> That's pretty much what I run and you might want to look at
> smtpd_data_restrictions as well.

What do you use with smtpd_data_restrictions?  I was considering
reject_unauth_pipelining but the docs have me confused with the "Note"
below:

reject_unauth_pipelining
Reject the request when the client sends SMTP commands ahead of time
where it is not allowed, or when the client sends SMTP commands ahead
of time without knowing that Postfix actually supports ESMTP command
pipelining. This stops mail from bulk mail software that improperly
uses ESMTP command pipelining in order to speed up deliveries.
Note: reject_unauth_pipelining is not useful outside
smtpd_data_restrictions when 1) the client uses ESMTP (EHLO instead of
HELO) and 2) with "smtpd_delay_reject = yes" (the default). The use of
reject_unauth_pipelining in the other restriction contexts is
therefore not recommended.

> > Would it be OK to remove the following aliases since I never use them:
>
> It's good form to keep them on your server and compile with the relvent
> RFC which specifies these.

Those aliases must be bringing in some spam though.

- Grant
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Greylisting vs. reject_rbl_client

[kashani]

Grant wrote:
>> I'd be careful with non_fqdn_hostname
> 
> What's wrong with that?  Here's how the postfix docs describe it:
> 
> reject_non_fqdn_helo_hostname (with Postfix < 2.3: 
> reject_non_fqdn_hostname)
> Reject the request when the HELO or EHLO hostname is not in
> fully-qualified domain form, as required by the RFC.

	Nothing is wrong with it, but that tends to be the one that bounces the 
most mail erroneously at least for me. In a perfect world there would be 
no problem with it, but in reality we have MS 2003 boxes reporting 
themselves as 2003WS-01 without a FQDN when they attempt to relay.

>> > smtpd_sender_restrictions =
>> >        permit_mynetworks,
>> >        reject_non_fqdn_sender,
>> >        reject_unknown_sender_domain,
>> >        permit
>> > smtpd_recipient_restrictions =
>> >        permit_mynetworks,
>> >        reject_non_fqdn_recipient,
>> >        reject_unknown_recipient_domain,
>> >        reject_unauth_destination,
>> >        permit
>>
>> That's pretty much what I run and you might want to look at
>> smtpd_data_restrictions as well.
> 
> What do you use with smtpd_data_restrictions?  I was considering
> reject_unauth_pipelining but the docs have me confused with the "Note"
> below:
> 
> reject_unauth_pipelining
> Reject the request when the client sends SMTP commands ahead of time
> where it is not allowed, or when the client sends SMTP commands ahead
> of time without knowing that Postfix actually supports ESMTP command
> pipelining. This stops mail from bulk mail software that improperly
> uses ESMTP command pipelining in order to speed up deliveries.
> Note: reject_unauth_pipelining is not useful outside
> smtpd_data_restrictions when 1) the client uses ESMTP (EHLO instead of
> HELO) and 2) with "smtpd_delay_reject = yes" (the default). The use of
> reject_unauth_pipelining in the other restriction contexts is
> therefore not recommended.

er hmmm, I'm still using Postfix 2.2 which doesn't have all the neat 2.3 
stuff yet. In 2.2 you'd put pipelining under smtpd recipient 
restrictions, but it appears that would cause some issues in 2.3 though 
just setting it under data restrictions would work fine if I'm reading 
it right.

kashani
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Greylisting vs. reject_rbl_client

[Preston Hagar]

[-- Attachment #1: Type: text/plain, Size: 4264 bytes --]

I think the thing you have to keep in mind is how strict you want to be.  I
am in control of mail servers for three different "organizations", my
personal email, my consulting side business, and the real estate company I
work full time for.  Each one has a varying degree of how acceptable false
positives are.  For my personal email, I use the following settings:

smtpd_delay_reject = yes
smtpd_helo_required = yes
smtpd_helo_restrictions =
    permit_mynetworks,
    check_helo_access
         hash:/etc/postfix/helo_access,
    reject_non_fqdn_hostname,
    reject_invalid_hostname,
    permit

smtpd_sender_restrictions =
   permit_sasl_authenticated,
   permit_mynetworks,
   reject_non_fqdn_sender,
   reject_unknown_sender_domain,
   permit

smtpd_recipient_restrictions =
   reject_unauth_pipelining,
   reject_non_fqdn_recipient,
   reject_unknown_recipient_domain,
   permit_mynetworks,
   permit_sasl_authenticated,
   reject_unauth_destination,
   reject_rbl_client relays.ordb.org,
   reject_rbl_client list.dsbl.org,
   reject_rbl_client sbl-xbl.spamhaus.org,
   reject_rbl_client bl.spamcop.net,
   reject_rbl_client dnsbl.njabl.org,
   check_policy_service inet:127.0.0.1:60000
   permit


The check_policy_service inet:127.0.0.1:60000 is for postgrey, a greylisting
service.  Personally, I hate spam so much, I would rather block my own
mother's emails than get spam.  The checks above with postgrey have brought
me from about 15-20 spam a day to 0 since I have had them in effect.  On the
otherhand, for the real estate company I work for, some of these options are
not acceptable.  Postgrey is not acceptable since many of our users expect
email to be "instant"  I know there is no technical guarantee of this, but
they get upset no matter what if their emails take too long.  Using postgrey
on my own personal server, I make it delay for 5 minutes.  Most mail servers
will keep trying and I will get the message within 10-15 minutes max.  I
have had a few times, though, that the mail took 4 or 5 hours before it was
tried again.  A solution I use for this for my consulting business is to
whitelist all of my clients domains.  With the real estate company, however,
there are too many different users working with too many different and new
companies to try and whitelist them all.

I actually do keep the rbls for all installations.  I completely understand
and agree with the previous posters frustration with rbls.  I have had the
same trouble myself.  I have had our company be on spamhaus and spamcop and
different times in the last couple of years.  It usually is a headache, but
in general, they are more good than harm.  At the real estate company,  I
have about 200 users that I can assure you would let me hear about it very
quickly if they were not getting their messages.  So far, I have not had any
complaints using the rbls above.

Anyway, as I said before, you just have to find the proper balance for your
situation.  If you or your company cannot afford to lose even one email, it
is probably best to put in very limited checks and use client-side filtering
with Junk mailboxes.  If speed of delivery is important, greylisting is not
a good option.  If you just want to stop the spam and it isn't the end of
the world if you lose a possibly legitimate mail, then try out my settings
above.

Hope this helps,

Preston

On 8/21/06, Grant <emailgrant@gmail.com> wrote:
>
> I've followed the steps outlined here to eliminate spam up to the
> section on "SPF and greylisting" on the second page:
>
> http://www.freesoftwaremagazine.com/articles/focus_spam_postfix/
>
> The author is really into greylisting:
>
> "If you take nothing else from this article, let it be that
> greylisting is a Good Thing and your customers will love you for using
> it"
>
> but the feedback I got on it from this list was not as positive.  Of
> the stuff I've implemented, these lines seem to have been the most
> effective:
>
> reject_rbl_client relays.ordb.org,
> reject_rbl_client list.dsbl.org,
> reject_rbl_client sbl-xbl.spamhaus.org,
>
> with the spamhaus.org line doing most of the work according to the logs.
>
> Do you think the reject_rbl_client stuff is safer than greylisting?
>
> - Grant
> --
> gentoo-user@gentoo.org mailing list
>
>

[-- Attachment #2: Type: text/html, Size: 5736 bytes --]

Re: [gentoo-user] Greylisting vs. reject_rbl_client

[kashani]

Preston Hagar wrote:
> Using postgrey on 
> my own personal server, I make it delay for 5 minutes.  Most mail 
> servers will keep trying and I will get the message within 10-15 minutes 
> max.  I have had a few times, though, that the mail took 4 or 5 hours 
> before it was tried again. 

	hotmail will retry every minute for 3 minutes. If you make the delay 30 
secs you will see mail come in much quicker from a number of places 
without accepting more spam. It's been my experience that mail is either 
retried or it is not and the actual time of the delay doesn't matter. 
Also setting the whitelist time to 63 days will keep infrequent emailers 
from getting greylisted as often.

You might also checkout sqlgrey which has some nice twists like 
whitelisting a domain after x successful deliveries in y time. The 
ebuild is in bugzilla.

http://bugs.gentoo.org/show_bug.cgi?id=71535

kashani
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Greylisting vs. reject_rbl_client

[Preston Hagar]

[-- Attachment #1: Type: text/plain, Size: 1491 bytes --]

On 8/25/06, kashani <kashani-list@badapple.net> wrote:
>
> Preston Hagar wrote:
> > Using postgrey on
> > my own personal server, I make it delay for 5 minutes.  Most mail
> > servers will keep trying and I will get the message within 10-15 minutes
> > max.  I have had a few times, though, that the mail took 4 or 5 hours
> > before it was tried again.
>
>         hotmail will retry every minute for 3 minutes. If you make the
> delay 30
> secs you will see mail come in much quicker from a number of places
> without accepting more spam. It's been my experience that mail is either
> retried or it is not and the actual time of the delay doesn't matter.
> Also setting the whitelist time to 63 days will keep infrequent emailers
> from getting greylisted as often.
>
> You might also checkout sqlgrey which has some nice twists like
> whitelisting a domain after x successful deliveries in y time. The
> ebuild is in bugzilla.
>
> http://bugs.gentoo.org/show_bug.cgi?id=71535
>
> kashani
> --
> gentoo-user@gentoo.org mailing list


postgrey does the  whitelist a domain after so many sucessful tries.  I do
use that as well.  Don't get me wrong, most of my mail is delivered right
after the 5 minute limit.  I just wanted to warn the OP that on a few
occasions, mail can take longer.  According to RFC specs, it can take up to
4 hours.  With the real estate company I work for, even one message taking 4
hours is unacceptable.

Thanks for your input.  I will check out sqlgrey.

Preston

[-- Attachment #2: Type: text/html, Size: 2051 bytes --]