[gentoo-user] Greylisting idea

[gentoo-user] Greylisting idea

[Grant]

Greylisting seems to be the most effective way of eliminating unwanted
email.  The problem is that it also has the potential to eliminate a
legitimate email.  Couldn't a feature be added to greylisting software
that dispatches an email to the sender of any email that is
temporarily rejected and doesn't retry within a certain amount of
time?  The email could say something like, "Your message of {date} was
rejected as possible spam.  Please call us at {phone_number}."

- Grant
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Greylisting idea

[Richard Fish]

On 8/28/06, Grant <emailgrant@gmail.com> wrote:
> Greylisting seems to be the most effective way of eliminating unwanted
> email.  The problem is that it also has the potential to eliminate a
> legitimate email.  Couldn't a feature be added to greylisting software
> that dispatches an email to the sender of any email that is
> temporarily rejected and doesn't retry within a certain amount of
> time?  The email could say something like, "Your message of {date} was
> rejected as possible spam.  Please call us at {phone_number}."

att.biz accounts do something like this.  It isn't a phone call, but
"go to $website and enter $code to unblock your mail to $recepient."

-Richard
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Greylisting idea

[Toby Cubitt]

On Mon, Aug 28, 2006 at 05:38:37PM -0700, Richard Fish wrote:
> On 8/28/06, Grant <emailgrant@gmail.com> wrote:
> >Greylisting seems to be the most effective way of eliminating unwanted
> >email.  The problem is that it also has the potential to eliminate a
> >legitimate email.  Couldn't a feature be added to greylisting software
> >that dispatches an email to the sender of any email that is
> >temporarily rejected and doesn't retry within a certain amount of
> >time?  The email could say something like, "Your message of {date} was
> >rejected as possible spam.  Please call us at {phone_number}."
> 
> att.biz accounts do something like this.  It isn't a phone call, but
> "go to $website and enter $code to unblock your mail to $recepient."

TMDA (in portage) could be set up to do something like this, I
believe.

Toby
-- 
PhD Student
Quantum Information Theory group
Max Planck Institute for Quantum Optics
Garching, Germany

email: toby@dr-qubit.org
web: www.dr-qubit.org
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Greylisting idea

[Grant]

> > >Greylisting seems to be the most effective way of eliminating unwanted
> > >email.  The problem is that it also has the potential to eliminate a
> > >legitimate email.  Couldn't a feature be added to greylisting software
> > >that dispatches an email to the sender of any email that is
> > >temporarily rejected and doesn't retry within a certain amount of
> > >time?  The email could say something like, "Your message of {date} was
> > >rejected as possible spam.  Please call us at {phone_number}."
> >
> > att.biz accounts do something like this.  It isn't a phone call, but
> > "go to $website and enter $code to unblock your mail to $recepient."
>
> TMDA (in portage) could be set up to do something like this, I
> believe.

I'm going to go ahead and try greylisting.  It sounds like regular
postfix checks can reject legitimate email just like greylisting can
end up doing, but postfix checks are cutting spam in half and it
sounds like greylisting will do a lot better.

I'm going to replace the following postix config:

smtpd_delay_reject = yes
smtpd_helo_required = yes
smtpd_helo_restrictions =
        permit_mynetworks,
        reject_non_fqdn_hostname,
        reject_invalid_hostname,
        permit
smtpd_sender_restrictions =
        permit_mynetworks,
        reject_non_fqdn_sender,
        reject_unknown_sender_domain,
        permit
smtpd_recipient_restrictions =
        permit_mynetworks,
        reject_non_fqdn_recipient,
        reject_unknown_recipient_domain,
        reject_unauth_destination,
        permit
smtpd_data_restrictions =
        reject_unauth_pipelining,
        permit

with this:

smtpd_recipient_restrictions =
        permit_mynetworks,
        check_policy_service inet:127.0.0.1:10030
        reject_unauth_destination,
        permit

How does that look?

- Grant
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Greylisting idea

[Nick Rout]

On Mon, 28 Aug 2006 17:30:48 -0700
Grant <emailgrant@gmail.com> wrote:

> Greylisting seems to be the most effective way of eliminating unwanted
> email.  The problem is that it also has the potential to eliminate a
> legitimate email.  Couldn't a feature be added to greylisting software
> that dispatches an email to the sender of any email that is
> temporarily rejected and doesn't retry within a certain amount of
> time?  The email could say something like, "Your message of {date} was
> rejected as possible spam.  Please call us at {phone_number}."
> 
> - Grant

No, thats just silly. That would send a reply to every email that is a spam. Most spam has non-existent (or forged) "from" address, which means the mail will just bounce or go to someone whose address has been forged.

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Greylisting idea

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 517 bytes --]

On Tue, 29 Aug 2006 17:04:40 +1200, Nick Rout wrote:

> No, thats just silly. That would send a reply to every email that is a
> spam. Most spam has non-existent (or forged) "from" address, which
> means the mail will just bounce or go to someone whose address has been
> forged.

And the rest will confirm to the spammer that they have a valid address,
along with instructions for having their spam accepted.


-- 
Neil Bothwick

I am Barry Norman of the Borg - you will be assimilated - and why not?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-user] Greylisting idea

[Grant]

> > No, thats just silly. That would send a reply to every email that is a
> > spam. Most spam has non-existent (or forged) "from" address, which
> > means the mail will just bounce or go to someone whose address has been
> > forged.
>
> And the rest will confirm to the spammer that they have a valid address,
> along with instructions for having their spam accepted.

The bounced messages would be easy to avoid and it's not necessary to
include instructions about how to get the spam accepted.  Forged
addresses could be a problem though.  Is there any way to verify that
the sender address is legitimate without sending an email there?

- Grant
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Greylisting idea

[kashani]

Grant wrote:
> Greylisting seems to be the most effective way of eliminating unwanted
> email.  The problem is that it also has the potential to eliminate a
> legitimate email.  Couldn't a feature be added to greylisting software
> that dispatches an email to the sender of any email that is
> temporarily rejected and doesn't retry within a certain amount of
> time?  The email could say something like, "Your message of {date} was
> rejected as possible spam.  Please call us at {phone_number}."
> 
> - Grant

	I'd just configure a retry time of something large if you're worried 
about it. IIRC the default is one day and you could raise it to two days 
on a slow system without worrying that the db is getting too large.

On the other hand it might be interesting to return an email like this:

"Hi I'm the greylisting policy server. It took your mail system 36 hours 
to retry. Is your email admin insane? Almost every email server in 
existence has reasonable defaults that your idiot decided to mess with. 
I highly recommend someone take a walk down to his cube and give them a 
good smack in the back of the head."

kashani
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Greylisting idea

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 492 bytes --]

On Tue, 29 Aug 2006 07:55:45 -0700, Grant wrote:

> Forged
> addresses could be a problem though.  Is there any way to verify that
> the sender address is legitimate without sending an email there?

The real problem is that the forged addresses are often real. I get
enough spam without adding to it by getting one of these for each spam
that is sent out with my address in the From: header.


-- 
Neil Bothwick

Did you know that eskimos have 17 different words for linguist?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-user] Greylisting idea

[Grant]

> > Greylisting seems to be the most effective way of eliminating unwanted
> > email.  The problem is that it also has the potential to eliminate a
> > legitimate email.  Couldn't a feature be added to greylisting software
> > that dispatches an email to the sender of any email that is
> > temporarily rejected and doesn't retry within a certain amount of
> > time?  The email could say something like, "Your message of {date} was
> > rejected as possible spam.  Please call us at {phone_number}."
> >
> > - Grant
>
>         I'd just configure a retry time of something large if you're worried
> about it. IIRC the default is one day and you could raise it to two days
> on a slow system without worrying that the db is getting too large.

How exactly are legitimate messages lost through greylisting?  I've
come up with these:

1. legitimate messages that don't retry (someone mentioned Amazon newsletters)

2. legitimate messages that take longer than the maximum specified
retry period to retry (has anyone run into a mail server that takes
longer than a day to retry?)

3. legitimate messages that retry from a different server each time
they retry (someone mentioned that they have seen this)

- Grant
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Greylisting idea

[kashani]

Grant wrote:
> How exactly are legitimate messages lost through greylisting?  I've
> come up with these:
> 
> 1. legitimate messages that don't retry (someone mentioned Amazon 
> newsletters)

The postgrey whitelist included in the build covers some of the major 
ones. I'd question these being legitimate emails and I'd question this 
being a legitimate way to run your mail system, but this is where you'd 
likely see mail lost.

> 2. legitimate messages that take longer than the maximum specified
> retry period to retry (has anyone run into a mail server that takes
> longer than a day to retry?)

No. Most I've seen is 12 hours at a small DSL provider in LA. The 
fastest is Hotmail at 30 seconds.

> 3. legitimate messages that retry from a different server each time
> they retry (someone mentioned that they have seen this)

I've seen Dreamhost do this and I still can't fathom the idea behind it. 
unless webserver outgoing connections are originating from a NAT DHCP 
pool or something weird. However setting the IP check to be the first 24 
bits, aka match on the class C, makes this go away in every case I'm 
aware of.

In cases 2 and 3 the original mail sender would get their email returned 
  after the standard four day timeout whereas the mail goes completely 
into the ether in case 1.

kashani
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Greylisting idea

[Grant]

> > How exactly are legitimate messages lost through greylisting?  I've
> > come up with these:
> >
> > 1. legitimate messages that don't retry (someone mentioned Amazon
> > newsletters)
>
> The postgrey whitelist included in the build covers some of the major
> ones. I'd question these being legitimate emails and I'd question this
> being a legitimate way to run your mail system, but this is where you'd
> likely see mail lost.

Nice.  I didn't know postgrey had a default whitelist.

> > 2. legitimate messages that take longer than the maximum specified
> > retry period to retry (has anyone run into a mail server that takes
> > longer than a day to retry?)
>
> No. Most I've seen is 12 hours at a small DSL provider in LA. The
> fastest is Hotmail at 30 seconds.

Good news.

> > 3. legitimate messages that retry from a different server each time
> > they retry (someone mentioned that they have seen this)
>
> I've seen Dreamhost do this and I still can't fathom the idea behind it.
> unless webserver outgoing connections are originating from a NAT DHCP
> pool or something weird. However setting the IP check to be the first 24
> bits, aka match on the class C, makes this go away in every case I'm
> aware of.

Any drawbacks to that?  Is this what you mean:

# --lookup-by-subnet     strip the last 8 bits from IP addresses (default)

or this:

# --lookup-by-host     do not strip the last 8 bits from IP addresses

or something else?

> In cases 2 and 3 the original mail sender would get their email returned
>   after the standard four day timeout whereas the mail goes completely
> into the ether in case 1.

Why wouldn't the email be returned to the sender in case 1?

By the way, I've been greylisting for about 24 hours and spam has been
reduced by about 99.5%.

- Grant
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Greylisting idea

[kashani]

Grant wrote:
> Any drawbacks to that?  Is this what you mean:
> 
> # --lookup-by-subnet     strip the last 8 bits from IP addresses (default)

Yep this one and no drawbacks I can think of.

> Why wouldn't the email be returned to the sender in case 1?

	Because number 1 is entirely composed of newsletters, automated 
responses, etc. Someone at Amazon sat down one day and realized that 
their mail queues were full of crap email. So rather than have a twenty 
server farm to send email, they took a short cut. The email is generated 
on the fly and piped directly to the socket for immediately delivery. 
I'm totally guessing here, but that's roughly how I'd do it. If the mail 
fails, the entire thing is dropped on the floor and maybe a db gets 
updated to reflect that it was never sent. You can send a ton of email 
this way because you never take the I/O hit of running the email through 
outbound MTA queues on the originating side. Because it never enters a 
"real" MTA it never gets retried when it fails.

kashani
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Greylisting idea

[Grant]

> > Any drawbacks to that?  Is this what you mean:
> >
> > # --lookup-by-subnet     strip the last 8 bits from IP addresses (default)
>
> Yep this one and no drawbacks I can think of.

Cool, it's the default anyway.

> > Why wouldn't the email be returned to the sender in case 1?
>
>         Because number 1 is entirely composed of newsletters, automated
> responses, etc. Someone at Amazon sat down one day and realized that
> their mail queues were full of crap email. So rather than have a twenty
> server farm to send email, they took a short cut. The email is generated
> on the fly and piped directly to the socket for immediately delivery.
> I'm totally guessing here, but that's roughly how I'd do it. If the mail
> fails, the entire thing is dropped on the floor and maybe a db gets
> updated to reflect that it was never sent. You can send a ton of email
> this way because you never take the I/O hit of running the email through
> outbound MTA queues on the originating side. Because it never enters a
> "real" MTA it never gets retried when it fails.

That makes sense.  I was thinking the messages in case 1 were sent
from a normal mail server that was configured to not retry, but it
sounds like you're talking about mail that goes through a totally
different sending process.

- Grant
-- 
gentoo-user@gentoo.org mailing list