From mboxrd@z Thu Jan 1 00:00:00 1970
Received: from pigeon.gentoo.org ([208.92.234.80] helo=lists.gentoo.org)
by finch.gentoo.org with esmtp (Exim 4.60)
(envelope-from <gentoo-dev+bounces-49880-garchives=archives.gentoo.org@lists.gentoo.org>)
id 1S0BRg-0001gf-LI
for garchives@archives.gentoo.org; Wed, 22 Feb 2012 12:37:20 +0000
Received: from pigeon.gentoo.org (localhost [127.0.0.1])
by pigeon.gentoo.org (Postfix) with SMTP id 8D53CE1809;
Wed, 22 Feb 2012 12:37:11 +0000 (UTC)
Received: from mail-bk0-f53.google.com (mail-bk0-f53.google.com [209.85.214.53])
by pigeon.gentoo.org (Postfix) with ESMTP id E8175E103C
for <gentoo-dev@lists.gentoo.org>; Wed, 22 Feb 2012 12:36:42 +0000 (UTC)
Received: by bkcit16 with SMTP id it16so7017502bkc.40
for <gentoo-dev@lists.gentoo.org>; Wed, 22 Feb 2012 04:36:42 -0800 (PST)
Received-SPF: pass (google.com: domain of corentin.chary@gmail.com designates 10.204.174.13 as permitted sender) client-ip=10.204.174.13;
Authentication-Results: mr.google.com; spf=pass (google.com: domain of corentin.chary@gmail.com designates 10.204.174.13 as permitted sender) smtp.mail=corentin.chary@gmail.com; dkim=pass header.i=corentin.chary@gmail.com
Received: from mr.google.com ([10.204.174.13])
by 10.204.174.13 with SMTP id r13mr15527393bkz.122.1329914202023 (num_hops = 1);
Wed, 22 Feb 2012 04:36:42 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=gamma;
h=mime-version:in-reply-to:references:date:message-id:subject:from:to
:content-type:content-transfer-encoding;
bh=kDNo+A0YkxTfVaEjB4jDhJUD+dr45J9hxud668yDTUI=;
b=eku0Jq+pb99OvOn5FfyPcCjpymh8x7bAwU/x7WXFVlIHeZAYWpn3uT1NQQgjuKE0E8
9yEnwdpPhFwQMW5G7TBdNTKiRP2B7x9edL4w+bhcLCSEUF0ZSUcjlDFgcH9FSzaibpm/
QpGXGkXfl8+iigK/0Y1inYHFxI8YQUc8NcFRU=
Precedence: bulk
List-Post: <mailto:gentoo-dev@lists.gentoo.org>
List-Help: <mailto:gentoo-dev+help@lists.gentoo.org>
List-Unsubscribe: <mailto:gentoo-dev+unsubscribe@lists.gentoo.org>
List-Subscribe: <mailto:gentoo-dev+subscribe@lists.gentoo.org>
List-Id: Gentoo Linux mail <gentoo-dev.gentoo.org>
X-BeenThere: gentoo-dev@lists.gentoo.org
Reply-to: gentoo-dev@lists.gentoo.org
MIME-Version: 1.0
Received: by 10.204.174.13 with SMTP id r13mr12510370bkz.122.1329914201087;
Wed, 22 Feb 2012 04:36:41 -0800 (PST)
Received: by 10.204.22.17 with HTTP; Wed, 22 Feb 2012 04:36:41 -0800 (PST)
In-Reply-To: <1329913223.32539.5.camel@daedalus.lan>
References: <CAHR064hRyxGuRD3VbQjNb5_5NOVJqpbJ7Ckad22gv9_oU2EaFQ@mail.gmail.com>
<CAHR064jUAtfg+or8u9gDYS4EUTtQnHUGWyHZ4v2vVZM3Khm=ig@mail.gmail.com>
<1329913223.32539.5.camel@daedalus.lan>
Date: Wed, 22 Feb 2012 13:36:41 +0100
Message-ID: <CAHR064jNGOmRpwAU6ZnexKnknCPR2jN1nYj0qrn9ud=AtJTdhg@mail.gmail.com>
Subject: Re: [gentoo-dev] Re: Gentoo Janitor scripts
From: Corentin Chary <corentin.chary@gmail.com>
To: gentoo-dev@lists.gentoo.org
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-Archives-Salt: c7422ba6-1a34-466a-86f5-400d102feb7d
X-Archives-Hash: 9a27b619b28451192a47746891a479be
On Wed, Feb 22, 2012 at 1:20 PM, Mart Raudsepp <leio@gentoo.org> wrote:
> On K, 2012-02-22 at 09:48 +0100, Corentin Chary wrote:
>> I did a quick script to count most used prefixes in SRC_URI yesterday
>> (https://github.com/iksaif/portage-janitor/blob/master/mirrors.py)
>>
>> Here is the (filtered) result:
>>
>> $ eix --only-names | python mirrors.py --count
>> 960 =C2=A0 =C2=A0 http://dev.gentoo.org
>> 372 =C2=A0 =C2=A0 http://xorg.freedesktop.org
>> 372 =C2=A0 =C2=A0 http://xorg.freedesktop.org/releases
>> 372 =C2=A0 =C2=A0 http://xorg.freedesktop.org/releases/individual
>> 306 =C2=A0 =C2=A0 http://pear.php.net
>> 306 =C2=A0 =C2=A0 http://pear.php.net/get
>> 256 =C2=A0 =C2=A0 http://oss.tresys.com
>> 255 =C2=A0 =C2=A0 http://oss.tresys.com/files
>> 255 =C2=A0 =C2=A0 http://oss.tresys.com/files/refpolicy
>> 225 =C2=A0 =C2=A0 http://hackage.haskell.org/packages
>> 225 =C2=A0 =C2=A0 http://hackage.haskell.org/packages/archive
>> 225 =C2=A0 =C2=A0 http://hackage.haskell.org
>> 206 =C2=A0 =C2=A0 http://ftp.xemacs.org
>> 201 =C2=A0 =C2=A0 https://github.com
>> 196 =C2=A0 =C2=A0 http://ftp.xemacs.org/pub
>> 196 =C2=A0 =C2=A0 http://ftp.xemacs.org/pub/xemacs
>> 193 =C2=A0 =C2=A0 http://ftp.xemacs.org/pub/xemacs/packages
>> 181 =C2=A0 =C2=A0 http://gstreamer.freedesktop.org
>> 181 =C2=A0 =C2=A0 http://gstreamer.freedesktop.org/src
>> 175 =C2=A0 =C2=A0 http://launchpad.net
>> 175 =C2=A0 =C2=A0 http://linuxgazette.net
>> 143 =C2=A0 =C2=A0 http://github.com
>> 130 =C2=A0 =C2=A0 http://pear.horde.org
>> 130 =C2=A0 =C2=A0 http://pear.horde.org/get
>> 101 =C2=A0 =C2=A0 http://savannah.nongnu.org/download
>> 101 =C2=A0 =C2=A0 http://savannah.nongnu.org
>> 100 =C2=A0 =C2=A0 http://get.qt.nokia.com
>> 97 =C2=A0 =C2=A0 =C2=A0ftp://sources.redhat.com/pub
>> 97 =C2=A0 =C2=A0 =C2=A0ftp://sources.redhat.com
>> 96 =C2=A0 =C2=A0 =C2=A0http://get.qt.nokia.com/qt
>> 95 =C2=A0 =C2=A0 =C2=A0http://get.qt.nokia.com/qt/source
>> 90 =C2=A0 =C2=A0 =C2=A0http://download.gna.org
>> 75 =C2=A0 =C2=A0 =C2=A0http://pecl.php.net
>> 75 =C2=A0 =C2=A0 =C2=A0http://pecl.php.net/get
>> 72 =C2=A0 =C2=A0 =C2=A0http://components.ez.no/get
>> 72 =C2=A0 =C2=A0 =C2=A0http://components.ez.no
>> 69 =C2=A0 =C2=A0 =C2=A0https://fedorahosted.org
>> 67 =C2=A0 =C2=A0 =C2=A0http://www.phrack.org/archives
>> 67 =C2=A0 =C2=A0 =C2=A0http://www.phrack.org/archives/tgz
>> 67 =C2=A0 =C2=A0 =C2=A0http://www.phrack.org
>>
>>
>> From that output we can easilly find out new entries to
>> thirdpartymirrors, for example:
>> gentoo-dev =C2=A0 =C2=A0http://dev.gentoo.org
>> xorg =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 http://xorg.freedesktop.o=
rg
>> gna =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0http://download.gna.=
org
>> pecl =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 http://pecl.php.net
>> pear =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 http://pear.php.net
>> github =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0https://github.com http://githu=
b.com
>> xemacs =C2=A0 =C2=A0 =C2=A0 http://ftp.xemacs.org/pub/ ftp://ftp.sa.xema=
cs.org/pub/
>> launchpad =C2=A0 =C2=A0http://launchpad.net
>> redhat =C2=A0 =C2=A0 =C2=A0 =C2=A0 ftp://sources.redhat.com/pub/ (and pr=
obably others !)
>> etc...
>>
>> The good part is that once you've modified thirdpartymirrors with new
>> mirrors, running mirrors.py --all will generate a big patch for all
>> your ebuilds to use those new mirrors !
>
> If you want this, then you should better figure out actual upstream
> mirroring systems and their list of mirrors they would want us to use.
> Until such, this seems to be just for shortening SRC_URI addresses when
> an upstream tarball domain name or path repeats, and that's definitely
> not what thirdpartymirrors is for.
Yes, of course, that was just a quick example, not something definitive.
But lets some examples:
- http://xorg.freedesktop.org, it's easy to find mirror for that one,
http://ftp.x.org/pub/ for example
- github: packages seems to use http and https, this script can help
to standardize the url used
- "960 =C2=A0 =C2=A0 http://dev.gentoo.org": that's a lot of package hoste=
d
here, is that really right ?
And still, thirdpartymirrors have some entries with only one "mirror"
and I believe factorizing SRC_URIs is a good thing (if something
changes, you just patch thirdpartymirros, not hundreds of ebuilds).
--=20
Corentin Chary
http://xf.iksaif.net