From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) by finch.gentoo.org (Postfix) with ESMTP id 10DE8138010 for ; Sun, 16 Sep 2012 13:15:57 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 9B15921C073; Sun, 16 Sep 2012 13:15:47 +0000 (UTC) Received: from mail-pb0-f53.google.com (mail-pb0-f53.google.com [209.85.160.53]) by pigeon.gentoo.org (Postfix) with ESMTP id 1B8A821C071 for ; Sun, 16 Sep 2012 13:14:51 +0000 (UTC) Received: by pbbro2 with SMTP id ro2so8554964pbb.40 for ; Sun, 16 Sep 2012 06:14:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=G6oN1fsKVYmZEwXyrRLfoE40+fIEt17RlPIJ/qKt5FU=; b=oSAQYtolAqpk002618A40h11/X9mrm8P7mVmwUOM+niDuD+s27lV2WfXx731ZJeDbd VFueW86Bkxj572L+iyL99RuRUgpveprPz8wu0UhTB5QoEBOK3626J2XPeYGzof7U7wZO ejlFwUtyK0NdSt0X480GHMC12iJTSaPk9qUQAYDnUjOUDmF5tAZig94nkZAk50t4Ynn0 x2YV/eXE+0UF/GBZAEcnr8M/qoZPpgym2bQbrQmtZTNrwcKnjnGhcIRkSkQlIcW1vHQ/ VIP+yVIA0GZTIWH7DMMR81+sENCbHvHqoiqmCQm/yZNnl+HDMZlulNZefO09/zZyZlqG 1plQ== Received: by 10.66.77.7 with SMTP id o7mr14228086paw.37.1347801291411; Sun, 16 Sep 2012 06:14:51 -0700 (PDT) Received: from smtp.gmail.com:587 (74-95-192-101-SFBA.hfc.comcastbusiness.net. [74.95.192.101]) by mx.google.com with ESMTPS id it5sm5025177pbc.10.2012.09.16.06.14.49 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 16 Sep 2012 06:14:50 -0700 (PDT) Received: by smtp.gmail.com:587 (sSMTP sendmail emulation); Sun, 16 Sep 2012 06:15:01 -0700 Date: Sun, 16 Sep 2012 06:15:01 -0700 From: Brian Harring To: Ben de Groot Cc: gentoo-dev@lists.gentoo.org Subject: Re: [gentoo-dev] example conversion of gentoo-x86 current deps to unified dependencies Message-ID: <20120916131501.GA23030@localhost> References: <20120907221051.4a7a6bde@pomiocik.lan> <504A5599.7060506@gentoo.org> <20120911021617.GE8036@localhost> <20120913221732.GE28593@localhost> <20120915203318.GH28593@localhost> <20120916000336.37f516f6@pomiocik.lan> <20120916012026.GI28593@localhost> Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-dev@lists.gentoo.org Reply-to: gentoo-dev@lists.gentoo.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-Archives-Salt: 00d29b16-292a-4abf-9e15-4ad5759e479b X-Archives-Hash: cbfb5b4de1efc574b02f9e9fb11dfab1 On Sun, Sep 16, 2012 at 03:39:22PM +0800, Ben de Groot wrote: > On 16 September 2012 09:20, Brian Harring wrote: > > Dumps are at > > http://dev.gentoo.org/~ferringb/unified-dependencies-example/ . > > > > Herds, if you want to see what your pkgs would look like, look at > > http://dev.gentoo.org/~ferringb/unified-dependencies-example/herds/ . > > > > If you'd like to see an *example effect* it has on what gets displayed > > to the user (aka, after all major use conditionals are stripped), look > > at > > http://dev.gentoo.org/~ferringb/unified-dependencies-example/user-visible.txt > > ; warning, that's a 55MB file. The syntax in use there isn't great, > > but as said, it's an example. > > > > ... > > > > Additionally, the form used here makes *no assumption about default > > context*; in any final solution we use, a default context would be > > wise- say build,run. Again, an example of what I mean. The dumps were regenerated; a default context of 'build,run' was added. Basically in the absense of an explicit dep: targetting, dep:build,run is assumed. Essentially it makes the deps cleaner to read for the common case, while also reducing the footprint at the cache, and vdb level; see http://dev.gentoo.org/~ferringb/unified-dependencies-example/vdb-effect.txt > Thanks. I have given it a quick overview for the qt herd. I still > don't see what using DEPENDENCIES adds to what we do now with separate > *DEPEND variables. I see no convincing reason to change what we do. Ok, so here's some stats: in the tree, we have 31360 ebuilds, and 194 eclasses; grand total of 31554 sources of metadata content (I say metadata since vapier has eblits in use which are just phase functions- I'm not scanning those intentionally). Doing some simple scans of the tree, here's some stats; note these stats are duplicated in the glep (they're nice selling points, thus might as well): 1) 746 hits in the tree for COMMON_DEPEND; that's 2%, and the usages I'm aware of have been for literally, what it sounds like- depends that are both DEPEND and RDEPEND. 2) scanning for assignments of RDEPEND=.*|$\{?DEPEND\}? gets a hit on 5343 unique sources. Searching for the inverse gets a hit on 10008 unique sources. Meaning that right there, ~48.6% of the tree is duplicating deps between the two forms. This puts us to 16083 unique sources in the tree that would benefit in some form (~51%). 3) What's interesting about that 51% is the eapi groupings; in EAPI4, the autosetting of RDEPEND="${RDEPEND:-${DEPEND}}" was discontinued. Roughly 50% of the initial 51% match is EAPI4; the rest are eapi0-3. 4) Again, keep in mind that the grep's in use above are single line matches- I'm definitely missing some ebuilds, and for complex dependencies that are appended/set by the eclass, likely missing a lot of that too. So... basically, people are already doing this manually with their own intermediate vars. Just a rough but mildly entertaining stat there's basically 8.38MB worth of normalized, literal fricking dependency; using a crappy algo (rather than a human doing the deps who can do it better), 37.4% (3.1MB) of that is removed via going to dependencies. It goes without saying that this would be a helluva lot less torturous on a proper PM implementation that parses the tree once, and renders- let alone repoman being able to avoid repeat scans of things it already has examined. Mind you, portage doesn't do that, but this would be good incentive to do a proper tree. :) > As I've said before on IRC, we need a good costs/benefits overview. > Right now I only see costs (migrating ebuilds and eclasses) and no > benefits. Offhand... I wouldn't be pushing for this if I didn't think this would be a boon over all- both for devs, and PMs. I think the actual 'cost' probably got lost in the noise of the various flames; what I'm proposing is basically zero cost for devs, it shoves the work to the PM, leaving the option for devs to do a more fine grained form if they can. I'm starting a seperate thread w/ a glep for this; I think you should take a look at the exact details of the specification including how DEPEND/RDEPEND/PDEPEND are handled in parallel to DEPENDENCIES (short version: if existent, they're automatically folded into DEPENDENCIES in my proposal); the end result is basically zero pain transition, while enabling us to start getting gains. Cheers- ~brian