From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) by finch.gentoo.org (Postfix) with ESMTP id BB691138CCF for ; Tue, 12 May 2015 07:18:34 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id ED0E7E0851; Tue, 12 May 2015 07:18:28 +0000 (UTC) Received: from mail.syndicat.com (mail.syndicat.com [62.146.89.62]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id DAD5DE0843 for ; Tue, 12 May 2015 07:18:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=syndicat.com; s=x; h=Content-Type:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:To:From; bh=XfpMhvjgiNYT56fmTykL51adCxGW1U4GMyL31IdZWkE=; b=rEOlcsTZn6Dn76Ogvc3rUx2YjSn+Dblxw0n4mdqPvoG3EhJxgDccCkgI1MqXm2LvbvvQ/XWb3UIMBRqUJ8+4EJdi6S5RslwjCjucJTG6fPkrgf8bwrfu6rByJtuOMRWrCwQ8eOA4kLgRz9rLemcmX3OxGOVnP/K551plkgPGqog=; Received: from localhost.syndicat.com ([127.0.0.1] helo=localhost) by mail.syndicat.com with esmtp (Syndicat.com PostHamster 4.84) (envelope-from ) id 1Ys4SP-0000dW-Gc for gentoo-dev@lists.gentoo.org; Tue, 12 May 2015 09:18:25 +0200 X-Virus-Scanned: amavisd-new at syndicat.com Received: from mail.syndicat.com ([127.0.0.1]) by localhost (mail.syndicat.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2fJ-8v9gdCKc for ; Tue, 12 May 2015 09:18:25 +0200 (CEST) Received: from p50875a67.dip0.t-ipconnect.de ([80.135.90.103] helo=gongo.localnet) by mail.syndicat.com with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Syndicat.com PostHamster 4.84) (envelope-from ) id 1Ys4SP-0001eY-7U for gentoo-dev@lists.gentoo.org; Tue, 12 May 2015 09:18:25 +0200 From: Niels Dettenbach To: gentoo-dev@lists.gentoo.org Subject: Re: [gentoo-dev] Anti-spam changes: proposal to drop spammy mail Date: Tue, 12 May 2015 09:18:15 +0200 Message-ID: <2025350.DvjFDXzWaF@gongo> Organization: Syndicat IT&Internet User-Agent: KMail/4.14.6 (Linux/4.0.1-niels; KDE/4.14.7; x86_64; ; ) In-Reply-To: References: <1626925.WQM6IekEy6@gongo> Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-dev@lists.gentoo.org Reply-to: gentoo-dev@lists.gentoo.org MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart6913539.Ak0EWNXmf7"; micalg="pgp-sha512"; protocol="application/pgp-signature" X-Archives-Salt: 7cef33f9-e570-43b5-91bd-26a52ff54e8f X-Archives-Hash: f8c40a9c5275e66d236c76af751868e6 --nextPart6913539.Ak0EWNXmf7 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="us-ascii" Am Montag, 11. Mai 2015, 20:36:18 schrieb Robin H. Johnson: > There are people that still accept mail that violates standards? yes, and there are mail sites and/or mail clients sending standard violating= emails. But the more truth is that there are many points within standards which= are interpreted differently from different peoples / groups (or even m= ailer software developers) and there is no real clear / hard "border" w= hat is a violence (and "could be dropped") and what is not - at least i= f you did not want to loose ham traffic for your users. The email oecosystem does not dpend from a single RFC today - more and = more basic parts of existing internet mail and it's features are define= d in further RFCs or are conclusive from each other. Two very typical examples: 1.) The sender domain has no MX nor abuse contact (i.e. RFC 2142) Many pro level mass mailers do not have an "working" abuse contact, but= there are still many smaller sites out which doesnt have too (because = of limited DNS access or lack of knowledge). Dropping mail from such si= tes will lead you to loosing mails (even if it "just" hits one in thous= and ham mails). 2.) BCC Header Most Mailers today are filtering out BCC recipient headers at some poin= t while this is not defined in the RFCs and still discussed hardly how = far the deletion of BCC headers are breaking standards, resulting in po= ssible lost of emails. See i.e. Phillip Hazels (EXIMs) statements in th= e net. > My above statement is for mail that we ACCEPTED. If it violates > standards, it's already denied at SMTP time. hmmm, you mean some more to very basic points of the standards. =20 > smtpd_restriction_classes =3D restrictive,permissive > restrictive =3D > reject_invalid_hostname > reject_non_fqdn_hostname > reject_non_fqdn_recipient > reject_non_fqdn_sender > reject_unknown_sender_domain > reject_unknown_recipient_domain > check_sender_mx_access cidr:/etc/postfix/bogus_mx_records > check_sender_access pcre:/etc/postfix/sender_access_control.pcre > check_sender_access pcre:/etc/postfix/sender_access_control-alias= es.pcre > check_helo_access pcre:/etc/postfix/helo_checks > reject_unverified_sender > check_client_access cidr:/etc/postfix/filter.cidr > permit > permissive =3D > permit If it helps you a bit further, i can explain the basics of our setup, d= eveloped over nearly 20 years now, handling just a few hundredthousands= smtp sessions per day and having NO spam folder or similiar (which wou= ld not save any time of the email user at the end) - but easily could b= e ad[a|o]pted per i.e. SIEVE to lead out less hard / more "unclear" spa= m into folders (i.e. instead of that mail where we make greylisting usu= ally). Because sendmail and postfix was to ressource inefficient for us someti= mes in the early stages, we decided to go to EXIM (Phillip Hazel) - an = own build optimized for our needs - including even some own mods today.= We avoided running SA from Amavis because of inefficiency. Until today our incoming path goes: - EXIM with EXIM SA at SMTP time Means we use Spamassassin directly at SMTP time, which allows us to dyn= amically "react" or further actively investigate a incoming smtp sessio= n if required. SA is only invoked for non authenticated mail over netwo= rk btw.. Before exim contacts spamassassin at this stage, we run a bunch of chec= ks in EXIM similiar to yours above, but some more (see down) which drop= the connection or write data for further processing into the headers. = If the connection is still alive, we run a hand full of RDNSBL checks, = which "could reject" the session and then a hand full which just writes= warnings into headers plus data for further processing steps. If the sessions still "lives", EXIM contacts Spamassassin over socket a= nd=20 Here we have 3 "routes": =09- low spam -> Mails is going trough DIRECTLY more then SA 2.3/3.0 - possible spam -> Greylisting (3 times TEMP Rejec= t) more then SA 5.2 - spam -> REJECT more then SA 33.0 - blackhole =2D> This kind of REJECT hits around 5 - 10% of spam connections, all oth= er spam is usually catched before without the full email / mail body re= cieved. Greylisting is "remembering" each contact<->contact handle and "quasi w= hitelists" the sender email after greylisting once to avoid further del= ays in the future - this helps very well for mail sites and/or clients = which uses mail systems with bad reputation while "working OK". SA EXIM is able to do teergrubing as well, but we did not use it in mos= t situations - except partly in dictionary attacks. At this point, parts of the mail traffic is going to an AMAVIS-NG for v= irus filtering only (user decide for it byself here) - no SA or RDNSBL = again / at this place. EXIM SA is no longer maintained officially, so we maintain it byself in= to actual EXIM source trees (would be nice to get it into Gentoos EXIM = ebuild - i.e. by a USE flag - would help here if someone is interested = =2D and if someone has a newer, at least same efficient solution it would= be nioce to know). =20 Overview of EXIM checks of incoming SMTP sessions (parts of this are im= plemented in your postfix rules too): =2D-- snip --- =3D HELO/EHLO required by SMTP RFC See http://www.syndicat.com/faq/ema= il/no_helo/ =3D Forged IP detected in HELO (it's mine) - $sender_helo_name See htt= p://www.syndicat.com/faq/email/forged_ip/ =3D Forged IP detected in HELO: $sender_helo_name =3D Forged IP detected in HELO - $sender_helo_name !=3D $sender_host_ad= dress See http://www.syndicat.com/faq/email/forged_ip/ =3D Forged hostname detected in HELO - you are not $sender_helo_name Se= e http://www.syndicat.com/faq/email/forged_ip/ =3D HELO is our IP =3D $sender_helo_name is a silly HELO. =3D RFC 1918 IP address in HELO ( See http://www.syndicat.com/faq/email= /rfc1918-helo/ ) =3D $sender_address_domain is a silly domain. (i.e. localhost) =3D HELO should be hostname but is $sender_helo_name . ( See http://www= .syndicat.com/faq/email/helo_nohostname/ ) =3D HELO should be Fully Qualified Domain Name Host.Domain.Tld ( See RF= C821 or http://www.syndicat.com/faq/email/helo_nofqdn/ ) =3D Forged hostname detected in HELO - $sender_helo_name is one of our = domains =3D Only one recipient accepted for NULL sender =3D (DROP) too many unknown users (${eval:$rcpt_fail_count+1} failed re= cipients) =3D Dictionary attack (${eval:$rcpt_fail_count+1} failed recipients). =3D> Teergrube: dictionary attack (${eval:$rcpt_fail_count+1} failed re= cipients) =3D unknown user =3D X-Broken-Reverse-DNS: no DNS for IP address $sender_host_address =3D acl_mail: (WARN-ONLY) Cannot reverse DNS $sender_host_address =3D X-Broken-Reverse-DNS: no DNS for IP address $sender_host_address =3D Content Policy Restriction: Mails to undisclosed recipients are not= permitted. =3D No contact MX - rfc-ignorant host $sender_host_name $sender_host_ad= dress . ( See http://www.syndicat.com/faq/email/rfc_ignorant/ ) =3D (WARN-ONLY, no reliable check possible) No MX abuse contact - rfc-i= gnorant host $sender_host_name $sender_host_address . ( See http://www.= syndicat.com/faq/email/rfc_ignorant/ ) =2D-- snap --- then RDNSBL: deny =3D sbl-xbl.spamhaus.org : cbl.abuseat.org : zen.spamhaus.org : b.= barracudacentral.org : psbl.surriel.com : ix.dnsbl.manitu.net warn =3D dnsbl-3.uceprotect.net : ubl.unsubscore.com : dnsbl-1.uceprote= ct.net : dnsbl.sorbs.net hth a bit. cheerioh, Niels. =2D-=20 --- Niels Dettenbach Syndicat IT & Internet http://www.syndicat.com PGP: https://syndicat.com/pub_key.asc --- =20 --nextPart6913539.Ak0EWNXmf7 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part. Content-Transfer-Encoding: 7Bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAABCgAGBQJVUak3AAoJEA27WohFkipEBDIP/AwEygWoTCaz/lu0e6Bm97Ew HbPQSVsFNk26vy/z3QUx9YTCjOQ+oBP5VmspzPOZUoxc2JMcJ7tkKK9TFWypwhXx 3klnas/eWi/01T7qNNaoTPKtu/OOkekSginf8uVi75C9Rreco//YjdxGkMnv6TUO xxNHFD7FFjXkRt+EvsfMik9qPXxxTMgMAu9bhhtzUkBklaGPlvA8FcRVOfmMk37X CzN2MQIvnQF7ClQH27bW0I8s7gOHSWjncUOrLggyslJO49pD2aL87X+5xlfCXptZ uBuAtc/2/mvhfxQCFpdZ2iPKcPZh5pQdB69qs+XrBbiKKtXVLlDhATN4ovn6IhVS 9MJgUtVJrxdZKK8TOG8C9P5QVZ3NlZPR2GHjhQfYNZeLkDuSaYiJWWr8gaXw31e4 1kVREFNzXQ6U3IUBzj9XG+dJ1Y39C+cj0mLYrYVbkzKiNuarYKV8RQ5qfdxAHowv ZcQgPDanAHtAKv7fVI5MIBA34gbN/WdXLEbiV7dZoF9UlM9kSqzzStRuuIV3hetu sbW7fQFSnJbOMgjtREU46DB9BwM7xr19KkhQkY7ErBgDGrCtl/ZFI074Dy5xlNhH jHTajQUIXczvjCLRJ9a44JjuXbjiyJBYNvxaXwBFqW6XBDLqi5TI2i/S7Kt22nfO zyYtuFnDB/avBTDTplns =J+PC -----END PGP SIGNATURE----- --nextPart6913539.Ak0EWNXmf7--