Re: [gentoo-pms] GLEP: gentoo sync based unified deps proposal

[Brian Harring <ferringb@gmail.com>]

On Sun, Sep 16, 2012 at 03:39:49PM +0100, Ciaran McCreesh wrote:
> On Sun, 16 Sep 2012 06:52:11 -0700
> Brian Harring <ferringb@gmail.com> wrote:
> > The live version of the doc is available at 
> > http://dev.gentoo.org/~ferringb/unified-dependencies/extensible_dependencies.html 
> 
> I think you're being a bit glib with your dismissal of the labels
> parsing scheme. You've got the following problems, that labels don't
> have:
> 
> You dismiss the dep:build? ( dep:run? ( ) ) problem as "don't do that".

x? ( !x? ( dep ) ) already exists syntactically.  dep:build? ( 
dep:run? ( blah ) ) is no different, so yes, I dismiss it- frankly 
with prejudice since you keep bringing that example up but ignoring 
that it's not a new issue (plus ignoring that it never occurs in the 
wild because it's a self-solving issue).


> Labels doesn't have this problem: it doesn't try to reuse an existing
> syntax precisely because the existing syntax is extremely awkward for
> this kind of thing.

Labels have a human comprehension problem, and require a fair amount 
more work for the various parsers.

You may not agree on that view, but there seems to be some consensus 
on that (as much as one ever gets in gentoo at least).


> You say there's an implict "build,run". This can't be an implicit
> "dep:build,run? ( )" using that syntax, however, since then you
> wouldn't be able to put a dep:post? inside it by the above. 

You're misunderstanding; you're thinking about it as to how exheres 
labels are done which is basically
build+run: ${DEPENDENCIES}

One thing to keep in mind here; I don't mind making the parser do some 
extra work if it's easier on devs to deal with (including 
aesthetically; like it or not, dependencies is needed, and screaming 
"labels are the one true religion" isn't going to get us to where we 
need to go- pragmatic compromise will).

Either way, in the absense of an explicit dep context, dep:build,run? 
is assumed.  The parser/visitation implementation is admittedly more 
complex, but it's not in the realm of 'hard'.  This is assuming a 
non-shit implementation mind you.

Either way, my focus here is on the human, as I've stated.


> So again
> you've got a special case to deal with simply because of the syntax --
> your syntax doesn't correspond directly to the meaning of dependencies.

"doesn't correspond directly to the meaning of dependencies" is a 
vague assertion; back that one up.  I'm assuming you're complaining 
that conditionals are being used (the "but they're not USE flags!" bit 
which is a subjective view of the conditional namespace).

If that's not what you're referencing, then frankly I'll just go with 
"explain to me how this same critique doesn't apply to labels".


> There's also the issue of what negations do at the top level...

Yeah, I did skimp on that one; technically speaking, negations aren't 
required if they prove too much of a pain in the ass.  Negation at the 
top level could be interpretted two ways:

1) negating against all possible dep types; thus a !dep:build? would 
be dep:post,run? .  Too slick in my view, but who knows, othes may 
think it straight forward.

2) Treat it as a negation of the implicit dep:build,run; meaning 
!dep:build? would be dep:run?.

Unsure of which is preferably at this juncture.

> You discourage grouping of dependencies by the use flag that selects
> them (see xscreensaver in your examples). Logically, "xscreensaver
> related things" belong together.

I do not discourage grouping.  The example is an automated 
conversion, as explicitly mentioned in the doc.

A secondary condensing of those deps I'll add to make clear it's 
supported.


> Your negation example also tries to
> bring related dependencies together. But it's a mess. What if one of
> the dependencies was a build dependency, and one a run dependency?
> You'd end up with something silly like this:
> 
>     dep:build,run? (
>         !dep:run? ( dev-util/diffball )
>         !dep:build? ( dev-util/bsdiff )
>     )

Ciaran, we use a fucking bash format.  We lost the ability to block 
silly idiocy long ago via that choice.

As said, 'x? ( !x? ( dep ) )' already exists, despite being dumb.  The 
beauty of it however is that the syntax rules themselves make it such 
that it doesn't ever actually come up in real world usage- the dep 
wouldn't be used fundamentally.

My intention is a syntax/format that is natural to the dev, and 
doesn't force them to do silly shit.  If they choose to do silly shit, 
that's on their head; trying to explicitly ban all possible dumb uses 
just makes the rules worse via increased complexity.

Also, just to be clear, this critique applies to exheres labels just 
the same. That example, rewritten to drop the negations is thus:

dep:build,run? ( 
  dep:build? ( dev-util/diffball )
  dep:run? ( dev-util/bsdiff )
)

Yep.  That's pretty dumb.  But no less retarded than someone doing the 
following in labels:

build+run:
  build: dev-util/diffball
  run: dev-util/bsdiff

Which I'll note isn't banned in your usage, despite the idiocy of such 
a construct.



> Your syntax also prevents the following:
> 
>     DEPENDENCIES="foo? ( $(make_foo_deps blah) )"

Err, no it doesn't.  I think you're reading too literally into the 
example mplayer translation I put in the doc- again, that was just a 
quicky, automated form, you can push dep:blah down beneath 
conditionals as necessary/desired.

If you see something claiming otherwise, or implying otherwise in the 
glep, please tell me exactly where so I can fix the wording.

> 
> and would encourage something like this instead:
> 
>     DEPENDENCIES="
>         dep:build? ( foo? ( $(make_foo_build_deps blah) ) )
>         dep:run? ( foo? ( $(make_foo_run_deps blah) ) )
>         dep:build,run? ( foo? ( $(make_foo_build_and_run_deps blah) ) )
> 
> which makes it much harder for the foo.eclass authors to switch what
> kinds of dependencies they use. For example, if foo.eclass starts
> needing an install dependency, your syntax requires every foo user to
> be updated, whereas labels does not. To get around that, you'd have to
> allow deeply embedded dep: blocks.

This critique is based on the same misunderstanding, just to be clear.


> You're also still speaking in terms of "rendering DEPEND" etc.

Babysteps.  First switch the internals to DEPENDENCIES, render from 
that to *DEPEND; in doing so, it's minimally disruptive for the PM, 
and for devs.

> That's
> not really what we want, though. As you've observed, a lot of atoms are
> in both DEPEND and RDEPEND. Conceptually, it makes much more sense to
> consider such an atom to be a single dependency that is both a build
> and a runtime dependency than it does to consider it as two unrelated
> dependencies. This is especially the case now that we have := slot
> dependencies.

Agreed, but we're not going to get to where we want without going 
through transition phases; at the PM level, minimally these 
enhancements:

1) first, collapse dependencies down, than render the *DEPEND views,
  thus enabling easy and quick initial integration; effectively
  no impact on the api/functionality of the PM at this phase.
2) Start converting the internals of the PM over such that it 
  operates, directly from that parsed tree, rathrer having to
  operate on a flatten/reduced/rendered version of it for each 
  context.
3) Once that's in place, start teaching the resolver how to make 
  better decisions via the better dependency groupping.

That's the rough roadmap for PM's were this to be added, exempting 
integration of any new dependency types.

For devs, they're not going to migrated at the flip of a switch.  Thus 
the same general transition approach;

1) switch the focus to DEPENDENCIES via collapsing 
  depend/rdepend/pdepend into it.  This allows existing code to work 
  in EAPI5 (or whichever this lands in).
2) New dependency forms should be added to DEPENDENCIES only; this is
  a bit of a carrot to get folks to migrate.
3) As ebuilds/eclasses get further along, and <EAPI5 support gets
  increasingly dropped, devs are encouraged (and hopefully willing
  since the syntax should make their lives easier) to migrate to the
  new form.
4) Some point down the line when we've neared an appropriate critical 
  mass, we discontinue the no longer necessary *DEPEND appending into
  DEPENDENCIES; basically deprecate and/then ban *DEPEND in the EAPI 
  of that day.  *DEPEND fades into the dust, same as how implicit 
  RDEPEND=${RDEPEND-${DEPEND}} was phased out.


> Ultimately, it comes down to the observation that the flag? ( ) syntax
> is strongly nested and hierarchical, but dependency roles aren't.

There is a bit of truth in that views on flag? ( ) vs the random-ass 
context labeling (which is hierarchical- keep in mind your stack 
pushing/popping confusion).

That said, just because it *is*, doesn't mean it has to get in the 
way.  Devs are already quite familiar w/ the hierarchical/nesting 
behaviour of flag? ( ); I seriously doubt they are going to have 
problems making dep:* work.

Basically, yes, there are some limitations; do they matter?  Not 
really, no.

Is it a perfect scheme?  While a neat reuse of existing syntax, no, 
it's not perfect.

But it's realistic, and *doable*.  That counts for a fucking helluva 
lot more than chasing the perfect solution.

I expect you'll inject some rant, but frankly REQUIRED_USE is a good 
example of what I'm talking about; the synax sucks, it's a bit 
painful, but it filled a needed gap and got the job done- 10% of EAPI4 
ebuild are using it after all (please skip the required-use rant also, 
whatever the flaws, one cannot argue w/ the usage).


> Labels can give all the advantages of your proposal (including the
> backwards compatibility, if that's desired), but without the need to
> shoehorn the idea into an unsuitable syntax.

If you want your proposal to go anywhere, you're going to need a 
better transition plan then "and.... then devs convert their 
ebuilds/eclasses".  I'd suggested it prior, but no traction there.

Either way, you push your syntax, I'll push mine; I'm fine w/ devs 
choosing the one that best fits their flow.

~harring