Re: [gentoo-dev] example conversion of gentoo-x86 current deps to unified dependencies

[Brian Harring <ferringb@gmail.com>]

On Sun, Sep 16, 2012 at 03:39:22PM +0800, Ben de Groot wrote:
> On 16 September 2012 09:20, Brian Harring <ferringb@gmail.com> wrote:
> > Dumps are at
> > http://dev.gentoo.org/~ferringb/unified-dependencies-example/ .
> >
> > Herds, if you want to see what your pkgs would look like, look at
> > http://dev.gentoo.org/~ferringb/unified-dependencies-example/herds/ .
> >
> > If you'd like to see an *example effect* it has on what gets displayed
> > to the user (aka, after all major use conditionals are stripped), look
> > at
> > http://dev.gentoo.org/~ferringb/unified-dependencies-example/user-visible.txt
> > ; warning, that's a 55MB file.  The syntax in use there isn't great,
> > but as said, it's an example.
> >
> > ...
> >
> > Additionally, the form used here makes *no assumption about default
> > context*; in any final solution we use, a default context would be
> > wise- say build,run.  Again, an example of what I mean.

The dumps were regenerated; a default context of 'build,run' was 
added.  Basically in the absense of an explicit dep: targetting, 
dep:build,run is assumed.

Essentially it makes the deps cleaner to read for the common case, 
while also reducing the footprint at the cache, and vdb level; see
http://dev.gentoo.org/~ferringb/unified-dependencies-example/vdb-effect.txt 


> Thanks. I have given it a quick overview for the qt herd. I still
> don't see what using DEPENDENCIES adds to what we do now with separate
> *DEPEND variables. I see no convincing reason to change what we do.

Ok, so here's some stats: in the tree, we have 31360 ebuilds, and 194 
eclasses; grand total of 31554 sources of metadata content (I say 
metadata since vapier has eblits in use which are just phase 
functions- I'm not scanning those intentionally).

Doing some simple scans of the tree, here's some stats; note these 
stats are duplicated in the glep (they're nice selling points, thus 
might as well):

1) 746 hits in the tree for COMMON_DEPEND; that's 2%, and the usages 
I'm aware of have been for literally, what it sounds like- depends 
that are both DEPEND and RDEPEND.

2) scanning for assignments of RDEPEND=.*|$\{?DEPEND\}? gets a hit on 
5343 unique sources.  Searching for the inverse gets a hit on 10008 
unique sources.  Meaning that right there, ~48.6% of the tree is 
duplicating deps between the two forms.  This puts us to 16083 unique 
sources in the tree that would benefit in some form (~51%).

3) What's interesting about that 51% is the eapi groupings; in EAPI4,
the autosetting of RDEPEND="${RDEPEND:-${DEPEND}}" was discontinued.  
Roughly 50% of the initial 51% match is EAPI4; the rest are eapi0-3.

4) Again, keep in mind that the grep's in use above are single line 
matches- I'm definitely missing some ebuilds, and for complex 
dependencies that are appended/set by the eclass, likely missing a lot 
of that too.

So... basically, people are already doing this manually with their own 
intermediate vars.

Just a rough but mildly entertaining stat there's basically 8.38MB 
worth of normalized, literal fricking dependency; using a crappy algo 
(rather than a human doing the deps who can do it better), 37.4% 
(3.1MB) of that is removed via going to dependencies.  It goes 
without saying that this would be a helluva lot less torturous on a 
proper PM implementation that parses the tree once, and renders- let 
alone repoman being able to avoid repeat scans of things it already 
has examined.

Mind you, portage doesn't do that, but this would be good incentive to 
do a proper tree. :)


> As I've said before on IRC, we need a good costs/benefits overview.
> Right now I only see costs (migrating ebuilds and eclasses) and no
> benefits.

Offhand... I wouldn't be pushing for this if I didn't think this would 
be a boon over all- both for devs, and PMs.

I think the actual 'cost' probably got lost in the noise of the 
various flames; what I'm proposing is basically zero cost for devs, it 
shoves the work to the PM, leaving the option for devs to do a more 
fine grained form if they can.

I'm starting a seperate thread w/ a glep for this; I think you should 
take a look at the exact details of the specification including how
DEPEND/RDEPEND/PDEPEND are handled in parallel to DEPENDENCIES (short 
version: if existent, they're automatically folded into DEPENDENCIES 
in my proposal); the end result is basically zero pain transition, 
while enabling us to start getting gains.

Cheers-
~brian