[gentoo-pms] GLEP: gentoo sync based unified deps proposal

[gentoo-pms] GLEP: gentoo sync based unified deps proposal

[Brian Harring]

Folks-

Keeping it short and quick, a basic glep has been written for what I'm 
proposing for DEPENDENCIES enhancement.

The live version of the doc is available at 
http://dev.gentoo.org/~ferringb/unified-dependencies/extensible_dependencies.html 

Wording fixes will occur, but the core concept shouldn't change (and 
if it does, per PEP standards, the changes will be noted/tracked).

Please have a read through; it should be clear/concise as to why PM 
authors are pushing for a unified dependencies to be done, what we can 
get from it now, what it enables moving forward, and the direct cost 
to devs should this idea be implemented.

Just to be absolutely clear, what I'm proposing is basically zero cost 
for devs- they can (and should when it's in their benefit) switch to 
the new syntax when it makes their job easier; it can be used in 
parallel to existing DEPEND/RDEPEND/PDEPEND both to ease 
transition/compatibility.

In addition, a prototype portage patch has been cut for dependencies 
support (needs work/full validation), and a matching pkgcore one has 
been cut.

I realize it's a complex subject; if you have questions, please feel 
to ask.

Thanks, and pardon the ancillary/daft noise that has gone with this 
subject.

~harring

Re: [gentoo-pms] GLEP: gentoo sync based unified deps proposal

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 2990 bytes --]

On Sun, 16 Sep 2012 06:52:11 -0700
Brian Harring <ferringb@gmail.com> wrote:
> The live version of the doc is available at 
> http://dev.gentoo.org/~ferringb/unified-dependencies/extensible_dependencies.html 

I think you're being a bit glib with your dismissal of the labels
parsing scheme. You've got the following problems, that labels don't
have:

You dismiss the dep:build? ( dep:run? ( ) ) problem as "don't do that".
Labels doesn't have this problem: it doesn't try to reuse an existing
syntax precisely because the existing syntax is extremely awkward for
this kind of thing.

You say there's an implict "build,run". This can't be an implicit
"dep:build,run? ( )" using that syntax, however, since then you
wouldn't be able to put a dep:post? inside it by the above. So again
you've got a special case to deal with simply because of the syntax --
your syntax doesn't correspond directly to the meaning of dependencies.

There's also the issue of what negations do at the top level...

You discourage grouping of dependencies by the use flag that selects
them (see xscreensaver in your examples). Logically, "xscreensaver
related things" belong together. Your negation example also tries to
bring related dependencies together. But it's a mess. What if one of
the dependencies was a build dependency, and one a run dependency?
You'd end up with something silly like this:

    dep:build,run? (
        !dep:run? ( dev-util/diffball )
        !dep:build? ( dev-util/bsdiff )
    )

Your syntax also prevents the following:

    DEPENDENCIES="foo? ( $(make_foo_deps blah) )"

and would encourage something like this instead:

    DEPENDENCIES="
        dep:build? ( foo? ( $(make_foo_build_deps blah) ) )
        dep:run? ( foo? ( $(make_foo_run_deps blah) ) )
        dep:build,run? ( foo? ( $(make_foo_build_and_run_deps blah) ) )

which makes it much harder for the foo.eclass authors to switch what
kinds of dependencies they use. For example, if foo.eclass starts
needing an install dependency, your syntax requires every foo user to
be updated, whereas labels does not. To get around that, you'd have to
allow deeply embedded dep: blocks.

You're also still speaking in terms of "rendering DEPEND" etc. That's
not really what we want, though. As you've observed, a lot of atoms are
in both DEPEND and RDEPEND. Conceptually, it makes much more sense to
consider such an atom to be a single dependency that is both a build
and a runtime dependency than it does to consider it as two unrelated
dependencies. This is especially the case now that we have := slot
dependencies.

Ultimately, it comes down to the observation that the flag? ( ) syntax
is strongly nested and hierarchical, but dependency roles aren't.
Labels can give all the advantages of your proposal (including the
backwards compatibility, if that's desired), but without the need to
shoehorn the idea into an unsuitable syntax.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-pms] GLEP: gentoo sync based unified deps proposal

[Brian Harring]

On Sun, Sep 16, 2012 at 03:39:49PM +0100, Ciaran McCreesh wrote:
> On Sun, 16 Sep 2012 06:52:11 -0700
> Brian Harring <ferringb@gmail.com> wrote:
> > The live version of the doc is available at 
> > http://dev.gentoo.org/~ferringb/unified-dependencies/extensible_dependencies.html 
> 
> I think you're being a bit glib with your dismissal of the labels
> parsing scheme. You've got the following problems, that labels don't
> have:
> 
> You dismiss the dep:build? ( dep:run? ( ) ) problem as "don't do that".

x? ( !x? ( dep ) ) already exists syntactically.  dep:build? ( 
dep:run? ( blah ) ) is no different, so yes, I dismiss it- frankly 
with prejudice since you keep bringing that example up but ignoring 
that it's not a new issue (plus ignoring that it never occurs in the 
wild because it's a self-solving issue).


> Labels doesn't have this problem: it doesn't try to reuse an existing
> syntax precisely because the existing syntax is extremely awkward for
> this kind of thing.

Labels have a human comprehension problem, and require a fair amount 
more work for the various parsers.

You may not agree on that view, but there seems to be some consensus 
on that (as much as one ever gets in gentoo at least).


> You say there's an implict "build,run". This can't be an implicit
> "dep:build,run? ( )" using that syntax, however, since then you
> wouldn't be able to put a dep:post? inside it by the above. 

You're misunderstanding; you're thinking about it as to how exheres 
labels are done which is basically
build+run: ${DEPENDENCIES}

One thing to keep in mind here; I don't mind making the parser do some 
extra work if it's easier on devs to deal with (including 
aesthetically; like it or not, dependencies is needed, and screaming 
"labels are the one true religion" isn't going to get us to where we 
need to go- pragmatic compromise will).

Either way, in the absense of an explicit dep context, dep:build,run? 
is assumed.  The parser/visitation implementation is admittedly more 
complex, but it's not in the realm of 'hard'.  This is assuming a 
non-shit implementation mind you.

Either way, my focus here is on the human, as I've stated.


> So again
> you've got a special case to deal with simply because of the syntax --
> your syntax doesn't correspond directly to the meaning of dependencies.

"doesn't correspond directly to the meaning of dependencies" is a 
vague assertion; back that one up.  I'm assuming you're complaining 
that conditionals are being used (the "but they're not USE flags!" bit 
which is a subjective view of the conditional namespace).

If that's not what you're referencing, then frankly I'll just go with 
"explain to me how this same critique doesn't apply to labels".


> There's also the issue of what negations do at the top level...

Yeah, I did skimp on that one; technically speaking, negations aren't 
required if they prove too much of a pain in the ass.  Negation at the 
top level could be interpretted two ways:

1) negating against all possible dep types; thus a !dep:build? would 
be dep:post,run? .  Too slick in my view, but who knows, othes may 
think it straight forward.

2) Treat it as a negation of the implicit dep:build,run; meaning 
!dep:build? would be dep:run?.

Unsure of which is preferably at this juncture.

> You discourage grouping of dependencies by the use flag that selects
> them (see xscreensaver in your examples). Logically, "xscreensaver
> related things" belong together.

I do not discourage grouping.  The example is an automated 
conversion, as explicitly mentioned in the doc.

A secondary condensing of those deps I'll add to make clear it's 
supported.


> Your negation example also tries to
> bring related dependencies together. But it's a mess. What if one of
> the dependencies was a build dependency, and one a run dependency?
> You'd end up with something silly like this:
> 
>     dep:build,run? (
>         !dep:run? ( dev-util/diffball )
>         !dep:build? ( dev-util/bsdiff )
>     )

Ciaran, we use a fucking bash format.  We lost the ability to block 
silly idiocy long ago via that choice.

As said, 'x? ( !x? ( dep ) )' already exists, despite being dumb.  The 
beauty of it however is that the syntax rules themselves make it such 
that it doesn't ever actually come up in real world usage- the dep 
wouldn't be used fundamentally.

My intention is a syntax/format that is natural to the dev, and 
doesn't force them to do silly shit.  If they choose to do silly shit, 
that's on their head; trying to explicitly ban all possible dumb uses 
just makes the rules worse via increased complexity.

Also, just to be clear, this critique applies to exheres labels just 
the same. That example, rewritten to drop the negations is thus:

dep:build,run? ( 
  dep:build? ( dev-util/diffball )
  dep:run? ( dev-util/bsdiff )
)

Yep.  That's pretty dumb.  But no less retarded than someone doing the 
following in labels:

build+run:
  build: dev-util/diffball
  run: dev-util/bsdiff

Which I'll note isn't banned in your usage, despite the idiocy of such 
a construct.



> Your syntax also prevents the following:
> 
>     DEPENDENCIES="foo? ( $(make_foo_deps blah) )"

Err, no it doesn't.  I think you're reading too literally into the 
example mplayer translation I put in the doc- again, that was just a 
quicky, automated form, you can push dep:blah down beneath 
conditionals as necessary/desired.

If you see something claiming otherwise, or implying otherwise in the 
glep, please tell me exactly where so I can fix the wording.

> 
> and would encourage something like this instead:
> 
>     DEPENDENCIES="
>         dep:build? ( foo? ( $(make_foo_build_deps blah) ) )
>         dep:run? ( foo? ( $(make_foo_run_deps blah) ) )
>         dep:build,run? ( foo? ( $(make_foo_build_and_run_deps blah) ) )
> 
> which makes it much harder for the foo.eclass authors to switch what
> kinds of dependencies they use. For example, if foo.eclass starts
> needing an install dependency, your syntax requires every foo user to
> be updated, whereas labels does not. To get around that, you'd have to
> allow deeply embedded dep: blocks.

This critique is based on the same misunderstanding, just to be clear.


> You're also still speaking in terms of "rendering DEPEND" etc.

Babysteps.  First switch the internals to DEPENDENCIES, render from 
that to *DEPEND; in doing so, it's minimally disruptive for the PM, 
and for devs.

> That's
> not really what we want, though. As you've observed, a lot of atoms are
> in both DEPEND and RDEPEND. Conceptually, it makes much more sense to
> consider such an atom to be a single dependency that is both a build
> and a runtime dependency than it does to consider it as two unrelated
> dependencies. This is especially the case now that we have := slot
> dependencies.

Agreed, but we're not going to get to where we want without going 
through transition phases; at the PM level, minimally these 
enhancements:

1) first, collapse dependencies down, than render the *DEPEND views,
  thus enabling easy and quick initial integration; effectively
  no impact on the api/functionality of the PM at this phase.
2) Start converting the internals of the PM over such that it 
  operates, directly from that parsed tree, rathrer having to
  operate on a flatten/reduced/rendered version of it for each 
  context.
3) Once that's in place, start teaching the resolver how to make 
  better decisions via the better dependency groupping.

That's the rough roadmap for PM's were this to be added, exempting 
integration of any new dependency types.

For devs, they're not going to migrated at the flip of a switch.  Thus 
the same general transition approach;

1) switch the focus to DEPENDENCIES via collapsing 
  depend/rdepend/pdepend into it.  This allows existing code to work 
  in EAPI5 (or whichever this lands in).
2) New dependency forms should be added to DEPENDENCIES only; this is
  a bit of a carrot to get folks to migrate.
3) As ebuilds/eclasses get further along, and <EAPI5 support gets
  increasingly dropped, devs are encouraged (and hopefully willing
  since the syntax should make their lives easier) to migrate to the
  new form.
4) Some point down the line when we've neared an appropriate critical 
  mass, we discontinue the no longer necessary *DEPEND appending into
  DEPENDENCIES; basically deprecate and/then ban *DEPEND in the EAPI 
  of that day.  *DEPEND fades into the dust, same as how implicit 
  RDEPEND=${RDEPEND-${DEPEND}} was phased out.


> Ultimately, it comes down to the observation that the flag? ( ) syntax
> is strongly nested and hierarchical, but dependency roles aren't.

There is a bit of truth in that views on flag? ( ) vs the random-ass 
context labeling (which is hierarchical- keep in mind your stack 
pushing/popping confusion).

That said, just because it *is*, doesn't mean it has to get in the 
way.  Devs are already quite familiar w/ the hierarchical/nesting 
behaviour of flag? ( ); I seriously doubt they are going to have 
problems making dep:* work.

Basically, yes, there are some limitations; do they matter?  Not 
really, no.

Is it a perfect scheme?  While a neat reuse of existing syntax, no, 
it's not perfect.

But it's realistic, and *doable*.  That counts for a fucking helluva 
lot more than chasing the perfect solution.

I expect you'll inject some rant, but frankly REQUIRED_USE is a good 
example of what I'm talking about; the synax sucks, it's a bit 
painful, but it filled a needed gap and got the job done- 10% of EAPI4 
ebuild are using it after all (please skip the required-use rant also, 
whatever the flaws, one cannot argue w/ the usage).


> Labels can give all the advantages of your proposal (including the
> backwards compatibility, if that's desired), but without the need to
> shoehorn the idea into an unsuitable syntax.

If you want your proposal to go anywhere, you're going to need a 
better transition plan then "and.... then devs convert their 
ebuilds/eclasses".  I'd suggested it prior, but no traction there.

Either way, you push your syntax, I'll push mine; I'm fine w/ devs 
choosing the one that best fits their flow.

~harring

Re: [gentoo-pms] GLEP: gentoo sync based unified deps proposal

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 3732 bytes --]

On Sun, 16 Sep 2012 09:05:28 -0700
Brian Harring <ferringb@gmail.com> wrote:
> > Labels doesn't have this problem: it doesn't try to reuse an
> > existing syntax precisely because the existing syntax is extremely
> > awkward for this kind of thing.
> 
> Labels have a human comprehension problem, and require a fair amount 
> more work for the various parsers.
> 
> You may not agree on that view, but there seems to be some consensus 
> on that (as much as one ever gets in gentoo at least).

I've never heard that view coming from anyone who has actually used
labels. It's only come from people who haven't tried using it, and who
have a history of disagreeing with anything that says 'Exherbo' on it.
You're taking about consensus among people who have never tried it
because they don't like it; consensus among people who have tried it is
that the labels syntax is good.

> My intention is a syntax/format that is natural to the dev, and 
> doesn't force them to do silly shit.

Labels already solve that. We know because we've got extensive
experience with them. Adoption of labels has been demonstrated to be
easy, both for former Gentoo developers and for people who haven't
previously written ebuilds.

We *know* that labels are easy to learn and easy to use. We also know
that they admit an efficient implementation, that they compose nicely,
that they allow dependencies to be specified accurately and that they
scale to larger numbers of dependency classes.

> > Your syntax also prevents the following:
> > 
> >     DEPENDENCIES="foo? ( $(make_foo_deps blah) )"
> 
> Err, no it doesn't.  I think you're reading too literally into the 
> example mplayer translation I put in the doc- again, that was just a 
> quicky, automated form, you can push dep:blah down beneath 
> conditionals as necessary/desired.
> 
> If you see something claiming otherwise, or implying otherwise in the 
> glep, please tell me exactly where so I can fix the wording.

The point is that nesting prevents composition. Labels are context
insensitive, which allows groups of dependencies to be added anywhere,
whereas dep: blocks can only be added if the surrounding groups are
specified in a particular way.

> 1) first, collapse dependencies down, than render the *DEPEND views,
>   thus enabling easy and quick initial integration; effectively
>   no impact on the api/functionality of the PM at this phase.

Specification in terms of rendering has a huge problem, though.
Remembering the crazy rules Gentoo has for || ( flag? ( ) ), what does
this do?

    || ( dep:build? ( a ) dep:run? ( b ) )

> > Ultimately, it comes down to the observation that the flag? ( )
> > syntax is strongly nested and hierarchical, but dependency roles
> > aren't.
> 
> There is a bit of truth in that views on flag? ( ) vs the random-ass 
> context labeling (which is hierarchical- keep in mind your stack 
> pushing/popping confusion).

There's not any stack confusion in practice. Labels only have slightly
complicated rules to allow every side case to be covered. You're taking
the "don't do that" approach to nesting weirdness; labels go the
"specify it precisely" route instead.

> > Labels can give all the advantages of your proposal (including the
> > backwards compatibility, if that's desired), but without the need to
> > shoehorn the idea into an unsuitable syntax.
> 
> If you want your proposal to go anywhere, you're going to need a 
> better transition plan then "and.... then devs convert their 
> ebuilds/eclasses".  I'd suggested it prior, but no traction there.

Your "rewrite *DEPEND" approach can just as easily be used with labels.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[gentoo-pms] Re: [gentoo-dev] GLEP: gentoo sync based unified deps proposal

[Brian Harring]

On Sun, Sep 16, 2012 at 07:32:39PM +0300, Alex Alexander wrote:
>    On Sep 16, 2012 4:55 PM, "Brian Harring" <[1]ferringb@gmail.com> wrote:
>    >
>    > Folks-
>    >
>    > Keeping it short and quick, a basic glep has been written for what
>    I'm
>    > proposing for DEPENDENCIES enhancement.
>    >
>    > The live version of the doc is available at
>    >
>    [2]http://dev.gentoo.org/~ferringb/unified-dependencies/extensible_depe
>    ndencies.html
> 
>    Am I the only one who thinks that this dep:{build,...} thing looks
>    really ugly and is hard to read?
> 
>    IMO simply removing the "dep" part would greatly improve things:

That 'dep' part isn't great, but it's added for a reason; to unify 
with USE_EXPAND/use group intended syntax.  There's a reference in 
there to 
http://www.gossamer-threads.com/lists/gentoo/dev/260069#260069 which 
I'll formalize soon enough.


>    DEPENDENCIES="
>    :build,run? ( ... )
>    :run? ( ... )
>    "

For your suggestion, consider it if we *do* fxi USE expand- via using 
the same <namespace>:<setting> form.

Using app-admin/mcollective ad an example, it's deps are thus:

DEPEND="ruby_targets_ruby18? ( dev-lang/ruby:1.8 )
        ruby_targets_ree18? ( dev-lang/ruby-enterprise:1.8 )"
RDEPEND="dev-ruby/stomp
        ruby_targets_ruby18? ( dev-lang/ruby:1.8 )
        ruby_targets_ree18? ( dev-lang/ruby-enterprise:1.8 )"

Which, if USE_EXPAND targets were groupped, would go from this
  ruby_targets_ruby18? ( dev-lang/ruby:1.8 )
  ruby_targets_ree18? ( dev-lang/ruby-enterprise:1.8 )
  dep:run? ( dev-ruby/stomp )"

to this:
  ruby:targets_ruby18? ( dev-lang/ruby:1.8 )
  ruby:targets_ree18? ( dev-lang/ruby-enterprise:1.8 )
  :run? ( dev-ruby/stomp )



>    s/:/@/ would also be interesting

Just a note; the character choosen was *intentionally* one that isn't 
a valid use character.  @ is a valid character due to linguas.  See 
the thread I referenced (ciaran's response, then my response).

Short version; to use @, we need use subgroups; thus
linguas@ca@valencia .


>    DEPENDENCIES="
>    @build,run? ( ... )
>    @run? ( ... )
>    "

DEPENDENCIES="
  ruby@targets_ruby18? ( dev-lang/ruby:1.8 )
  ruby@targets_ree18? ( dev-lang/ruby-enterprise:1.8 )
  @run? ( dev-ruby/stomp )"

Using equivalent syntax for mcollective.

I'm not a huge fan of dep:, and I'm a bit wary of a bare 
@{run,test,whatever} since it carries with it an implicit "this is 
targetting the dep namespace".

That said, I'm not opposed to it- just as I said, I'm a bit wary at 
first glance.

Comments?
~harring

[gentoo-pms] Re: [gentoo-dev] GLEP: gentoo sync based unified deps proposal

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 602 bytes --]

> test depends: to specifically mark those dependencies that are only
> needed for when the pkg is being tested; effectively ephemeral
> build/run time depends that go away once testing is completed.

Does that mean that USE=test is going away somehow?

Also, could you please stop spreading FUD with your examples? A quick
glance shows that what you have expanded there, a fairly reasonable
Gentoo dev will solve using:

RDEPEND="..."
DEPEND="${RDEPEND}
    ..."

So if you really want to show some advantages, please compare it with
*real* code.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

[gentoo-pms] Re: [gentoo-dev] GLEP: gentoo sync based unified deps proposal

[Brian Harring]

On Tue, Sep 18, 2012 at 10:25:51AM +0200, Micha?? G??rny wrote:
> > test depends: to specifically mark those dependencies that are only
> > needed for when the pkg is being tested; effectively ephemeral
> > build/run time depends that go away once testing is completed.
> 
> Does that mean that USE=test is going away somehow?

If you think it through, a test use flag still is needed in the cases 
where the rdep itself would change if test was enabled; such a source 
is fairy rare, but not always just someone being moronic- certain 
cases to do testing, the tests need to reach in fairly deeply and 
recompilation for compile vs test isn't exposed.


> Also, could you please stop spreading FUD with your examples?

It's not FUD; it's rendered deps, and a demonstration of how they 
collapse down naturally on their own regardless of how you generate 
them.

Quite frankly, it's a fairly effective demonstration in my views, but 
so it goes.

> A quick
> glance shows that what you have expanded there, a fairly reasonable
> Gentoo dev will solve using:
>
> RDEPEND="[common depends]"
> DEPEND="${RDEPEND}
>     [build only depends]"

from diffball (under current EAPIs)

"""
RDEPEND=">=sys-libs/zlib-1.1.4
        >=app-arch/bzip2-1.0.2
        app-arch/xz-utils"
DEPEND="${RDEPEND}
        virtual/pkgconfig"
"""

becomes the following under the proposal:

"""
DEPENDENCIES=">=sys-libs/zlib-1.1.4
        >=app-arch/bzip2-1.0.2
        app-arch/xz-utils"
        dep:build? ( virtual/pkgconfig )"
"""

Suspect I may add that to the doc; it's a good example of the ground 
level simple gains for devs inherent in the proposal- thanks for 
helping improve it.


> So if you really want to show some advantages, please compare it with
> *real* code.

I think I'll take the risk, and assume people capable of discussing 
DEPENDENCIES and vaguely knowledgable in the ebuild format will be 
able to understand how their ebuilds will change; thus I'll skip that 
request of yours.


A productive suggestion for you; you should go looking through the 
tree finding cases where DEPENENCIES is a regression in form at the 
shell level, or rendered deps level.

Should you manage to find something that's not contrived or 
intentionally cracktastic, I expect people would be interested.

~harring

[gentoo-pms] Re: [gentoo-dev] GLEP: gentoo sync based unified deps proposal

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1722 bytes --]

On Tue, 18 Sep 2012 02:24:26 -0700
Brian Harring <ferringb@gmail.com> wrote:

> On Tue, Sep 18, 2012 at 10:25:51AM +0200, Micha?? G??rny wrote:
> > > test depends: to specifically mark those dependencies that are
> > > only needed for when the pkg is being tested; effectively
> > > ephemeral build/run time depends that go away once testing is
> > > completed.
> > 
> > Does that mean that USE=test is going away somehow?
> 
> If you think it through, a test use flag still is needed in the cases 
> where the rdep itself would change if test was enabled; such a source 
> is fairy rare, but not always just someone being moronic- certain 
> cases to do testing, the tests need to reach in fairly deeply and 
> recompilation for compile vs test isn't exposed.

Yes, and sometimes we're doing 'use test'. I simply don't see how
adding a separate group of dependencies just for 'test' phase is going
to help us. They fit just fine into build-time dependencies right now.

> > A quick
> > glance shows that what you have expanded there, a fairly reasonable
> > Gentoo dev will solve using:
> >
> > RDEPEND="[common depends]"
> > DEPEND="${RDEPEND}
> >     [build only depends]"
> 
> from diffball (under current EAPIs)
> 
> """
> RDEPEND=">=sys-libs/zlib-1.1.4
>         >=app-arch/bzip2-1.0.2
>         app-arch/xz-utils"
> DEPEND="${RDEPEND}
>         virtual/pkgconfig"
> """
> 
> becomes the following under the proposal:
> 
> """
> DEPENDENCIES=">=sys-libs/zlib-1.1.4
>         >=app-arch/bzip2-1.0.2
>         app-arch/xz-utils"
>         dep:build? ( virtual/pkgconfig )"
> """

Err, shouldn't the first three deps be namespaced?

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

[gentoo-pms] Re: [gentoo-dev] GLEP: gentoo sync based unified deps proposas

[Brian Harring]

On Tue, Sep 18, 2012 at 11:47:42AM +0200, Micha?? G??rny wrote:
> On Tue, 18 Sep 2012 02:24:26 -0700
> Brian Harring <ferringb@gmail.com> wrote:
> 
> > On Tue, Sep 18, 2012 at 10:25:51AM +0200, Micha?? G??rny wrote:
> > > > test depends: to specifically mark those dependencies that are
> > > > only needed for when the pkg is being tested; effectively
> > > > ephemeral build/run time depends that go away once testing is
> > > > completed.
> > > 
> > > Does that mean that USE=test is going away somehow?
> > 
> > If you think it through, a test use flag still is needed in the cases 
> > where the rdep itself would change if test was enabled; such a source 
> > is fairy rare, but not always just someone being moronic- certain 
> > cases to do testing, the tests need to reach in fairly deeply and 
> > recompilation for compile vs test isn't exposed.
> 
> Yes, and sometimes we're doing 'use test'. I simply don't see how
> adding a separate group of dependencies just for 'test' phase is going
> to help us.
> They fit just fine into build-time dependencies right now.

I'm going to assume you  typo'd "build-time" into "run-time"; on the 
offchance you've never written actual test code, to test the code you 
have to *run* the results.

Simple example, portage doesn't need eselect nor logrotate, nor afaik 
selinux or paxutils, till runtime since it doesn't test those 
pathways.

A non-crap resolver can exploit that gap when it comes to 
parallelization.

Just heading off an email from you, no, you cannot just stick it into 
RDEPEND then.

If you did so, the test deps would be locked into the required runtime 
graph for as long as the pkg was installed.

If in doubt of how that matters; trace the usage of gtest, nose, etc.  
Nose is a good example additionally since a properly setup setup.py, 
the pkg doesn't need nose for build- just strictly for test.



> > > A quick
> > > glance shows that what you have expanded there, a fairly reasonable
> > > Gentoo dev will solve using:
> > >
> > > RDEPEND="[common depends]"
> > > DEPEND="${RDEPEND}
> > >     [build only depends]"
> > 
> > from diffball (under current EAPIs)
> > 
> > """
> > RDEPEND=">=sys-libs/zlib-1.1.4
> >         >=app-arch/bzip2-1.0.2
> >         app-arch/xz-utils"
> > DEPEND="${RDEPEND}
> >         virtual/pkgconfig"
> > """
> > 
> > becomes the following under the proposal:
> > 
> > """
> > DEPENDENCIES=">=sys-libs/zlib-1.1.4
> >         >=app-arch/bzip2-1.0.2
> >         app-arch/xz-utils"
> >         dep:build? ( virtual/pkgconfig )"
> > """
> 
> Err, shouldn't the first three deps be namespaced?

No.

Please read the glep, specifically the section "basic rules".


Also, you come up with a valid criticism, valid point, etc, something 
*worthwhile*, I'll respond.  If it doesn't meet that criteria, assume 
I won't respond (feel free to bitch to the council during whatever 
vote occurs for this GLEP that I ignored your noise; it's a risk I'll 
willingly take).

~harring

Re: [gentoo-pms] GLEP: gentoo sync based unified deps proposal

[Brian Harring]

Pardon the delay; got busy with work, plus to actually address your
claims re: labels (or refute, as I intend to do)... data was 
necessary.

So I went and got the data. :)

Analysis was done roughly 09/17 or so; just looping back and 
commenting now however.


On Sun, Sep 16, 2012 at 05:59:21PM +0100, Ciaran McCreesh wrote:
> > > Your syntax also prevents the following:
> > > 
> > >     DEPENDENCIES="foo? ( $(make_foo_deps blah) )"
> > 
> > Err, no it doesn't.  I think you're reading too literally into the 
> > example mplayer translation I put in the doc- again, that was just a 
> > quicky, automated form, you can push dep:blah down beneath 
> > conditionals as necessary/desired.
> > 
> > If you see something claiming otherwise, or implying otherwise in the 
> > glep, please tell me exactly where so I can fix the wording.
> 
> The point is that nesting prevents composition. Labels are context
> insensitive, which allows groups of dependencies to be added anywhere,
> whereas dep: blocks can only be added if the surrounding groups are
> specified in a particular way.

Fun fact; peoples usage of labels in exherbo is thus:

build+run:
  set of deps
run:
  set of deps/conditionals/etc

You get the idea.  Technically, each block is usually wrapped in 
()... which frankly is a sign that the context switch third party 
code can introduce is problematic.  Basically, y'all are 
already using labels in exactly the fashion I propose, just 
with different synax.

I couldn't find a *single* instance of the following in usage in any git.exherbo repo:

build+run:
  dep1
  x? ( 
    dep2
    test:
      dep3
  )

Ie, a nested override.

Exherbo deps themselves basically dispute the claim that nesting 
somehow blocks people from doing composition; fact is, y'all can do 
nesting/context switching of a label w/in a block, but y'all don't at 
all.  This strongly makes me think you're either blowing smoke up 
folks asses or that you're chasing perfection ignoring the realities 
of how this is used on the ground.

Now, if there are actually examples of it in use in y'alls trees- 
(ones that aren't added after I send this email mind you ;) ) please 
point them out.  Offhand, there's actually 4 pkgs that do 
stacking/nesting of deps, although it's accidental and doesn't 
actually do it for gain.  They are:

dev-haskell/language-c[=0.3.2]::haskell
dev-haskell/language-javascript[=0.5.2]::haskell
kde/kdemultimedia[~scm]::kde
kde/kdemultimedia[=4.8.5]::kde

basically
DEPENDENCIES="
  ( build: some deps
    # dev forgot to close the block
    $(some dep_generator that forces a label immediately, per the norm)
  )"

While the norm is
DEPENDENCIES="
  ( build: some deps )
  ( $(some dep_generator that forces a label immediately) )"

Interesting sidenote btw; every usage I've found is directly 
translatable to my proposal, w/out any loss of expression in use.

Now, it's possible I fucked up.  I strongly doubt it however.  Please 
provide examples either way- else y'alls own dependencies disprove 
your claims about nesting being evil.

Either way, the data for that is at 
http://dev.gentoo.org/~ferringb/unified-dependencies/labels/translated-to-use-deps.txt


> > 1) first, collapse dependencies down, than render the *DEPEND views,
> >   thus enabling easy and quick initial integration; effectively
> >   no impact on the api/functionality of the PM at this phase.
> 
> Specification in terms of rendering has a huge problem, though.
> Remembering the crazy rules Gentoo has for || ( flag? ( ) ), what does
> this do?
> 
>     || ( dep:build? ( a ) dep:run? ( b ) )

Honestly, I was waiting for you to bring this up :)

You're conflating two different things here;
1) someone being a dumb ass and writing what's effectively a || ( 
atom) block, just doing so in a manner w/out any reason to do so.

2) Your ongoing jihad against || (), specifically the occasionally 
valid complaint that build/rdepend different means the resolver can 
get stuck in certain pathways when slots are involved, abi, etc.

Either way, in my proposal, I'm not going to single that out and try 
blocking it.  The rendered version of it is still stable, albeit if 
it's build/run it's unlikely to be desired if there is ABI involved 
(for non ABI, specifically self-bootstrapping codebases, I suspect 
someone could come up with a valid construct- sed has something 
similar if memory serves).

Worth noting, the following idiocy is valid:

x? ( dev-util/diffball )
!dev-util/diffball

Which is stupid, but syntactically correct.  Nor is this a new issue, 
thus I don't particularly agree with your approach of trying to sink 
the proposal via an orthogonal problem.

This is why we have QA tools.
 

> > > Ultimately, it comes down to the observation that the flag? ( )
> > > syntax is strongly nested and hierarchical, but dependency roles
> > > aren't.
> > 
> > There is a bit of truth in that views on flag? ( ) vs the random-ass 
> > context labeling (which is hierarchical- keep in mind your stack 
> > pushing/popping confusion).
> 
> There's not any stack confusion in practice. Labels only have slightly
> complicated rules to allow every side case to be covered. You're taking
> the "don't do that" approach to nesting weirdness; labels go the
> "specify it precisely" route instead.

In practice, all usages explicitly go out of their way to protect 
themselves from the invoking context, and they use labels *exactly the 
same way as dep:build,run? ( blah )* would be used.

Basically the only real world benefit I've seen of labels vs what I've 
proposed is a *slight* collapsing of the tree- although that is more 
than offset by the fact all dep generating functions introduce () 
barriers to protect the invoking scope from their label adjustments.

Clarifying, this is better in labels:
"""
build+run:
  blah
run:
  monkeys
"""
equivalent in my proposal would be
"""
blah
dep:run? ( monkeys )
"""

The difference there isn't exactly world shattering, so it's not a 
strong arguing point for labels.


> > > Labels can give all the advantages of your proposal (including the
> > > backwards compatibility, if that's desired), but without the need to
> > > shoehorn the idea into an unsuitable syntax.
> > 
> > If you want your proposal to go anywhere, you're going to need a 
> > better transition plan then "and.... then devs convert their 
> > ebuilds/eclasses".  I'd suggested it prior, but no traction there.
> 
> Your "rewrite *DEPEND" approach can just as easily be used with labels.

Just the same as the real world usage of labels could just as easily 
be done via dep:blah.

Not sure it's worth continuing this discussion point frankly; there is 
equivalence for all real world usage.  Not really in dispute (you'll 
try I'm sure, but without actual backing examples this time around I'm 
not particularly interested in listening).

If what you've got to say is "you can do this in labels, and I think 
you should labels"; ok, noted, end of discussion.

Other points I'm interested in; the mentioning of "what does
!build:test? ( blah ) mean while in implicit build,run" for example 
was useful and a valid critique (one I've not yet addressed in the 
doc); stuff like that you've got a better chance killing my proposal 
then continuing with the "yes but labels are better than sliced bread.  
you just don't understand" angle of discussion ;)

Either way, via 
http://dev.gentoo.org/~ferringb/unified-dependencies/labels/translated-to-use-deps.txt 
, I think it's pretty clear labels in real world usage aren't bringing 
anything to the tabel that we wouldn't have via my proposal; that 
leaves labels as just a different syntax (perhaps aesthetically more 
pleasing at first glance, but the label stacking bit via exheres 
analysis is proven to be something people explicitly go out of their 
way to protect against; meaning the aesthetics have a mental 
model cost).

~harring

[gentoo-pms] Re: [gentoo-dev] GLEP: gentoo sync based unified deps proposal

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 535 bytes --]

On Sun, 16 Sep 2012 06:52:11 -0700
Brian Harring <ferringb@gmail.com> wrote:

> Keeping it short and quick, a basic glep has been written for what I'm 
> proposing for DEPENDENCIES enhancement.
> 
> The live version of the doc is available at 
> http://dev.gentoo.org/~ferringb/unified-dependencies/extensible_dependencies.html 

One more question -- are we going to keep 'foo,bar?' syntax as
a special case applying only to dependency atoms or are we going to
extend it to USE flags?

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

[gentoo-pms] Re: [gentoo-dev] GLEP: gentoo sync based unified deps proposal

[Brian Harring]

On Wed, Sep 26, 2012 at 08:58:54AM +0200, Micha?? G??rny wrote:
> On Sun, 16 Sep 2012 06:52:11 -0700
> Brian Harring <ferringb@gmail.com> wrote:
> 
> > Keeping it short and quick, a basic glep has been written for what I'm 
> > proposing for DEPENDENCIES enhancement.
> > 
> > The live version of the doc is available at 
> > http://dev.gentoo.org/~ferringb/unified-dependencies/extensible_dependencies.html 
> 
> One more question -- are we going to keep 'foo,bar?' syntax as
> a special case applying only to dependency atoms or are we going to
> extend it to USE flags?

It's fairly dep specific; that's effectivelly foo|bar when you think 
about it; it multiplies out to dep:foo? ( that block ) dep:bar? ( that 
block ) in a dumb PM (smarter one just leaves the tree collapsed and 
filters as it goes).

Phrased another way, I'm not sure we really need shorthand for the 
following:

x? ( blah )
y? ( blah )
z? ( blah )

into
x,y,z? ( blah )

It's a rare case; I could only foresee that potentially being of use 
for arch flags; ie, amd64,x86? ( blah ); which I'd write as 
"arch:amd64,x86? ( blah )" personally since I don't like the notion of 
introducing ',' into raw, non use group flags.

That said, I don't hugely care; people think it's useful, then have at 
it.
~harring

[gentoo-pms] Re: [gentoo-dev] GLEP: gentoo sync based unified deps proposal

[Brian Harring]

On Wed, Sep 26, 2012 at 08:58:54AM +0200, Micha?? G??rny wrote:
> On Sun, 16 Sep 2012 06:52:11 -0700
> Brian Harring <ferringb@gmail.com> wrote:
> 
> > Keeping it short and quick, a basic glep has been written for what I'm 
> > proposing for DEPENDENCIES enhancement.
> > 
> > The live version of the doc is available at 
> > http://dev.gentoo.org/~ferringb/unified-dependencies/extensible_dependencies.html 
> 
> One more question -- are we going to keep 'foo,bar?' syntax as
> a special case applying only to dependency atoms or are we going to
> extend it to USE flags?

Note that's dep:foo,bar; not a bare "allow any use flags to be OR'd 
together".  In light of the fact it *is* just an expansion hack, the 
usage is semi limited although there are scenarios for it; arches, 
namely (if amd64 or x86, use this, if mips, that, etc).

I have no preference either way; extending it outside of dep isn't 
necessary if people hate it, although as said, there are some 
potential uses for it.

That said, if we were to start using it, the ',' as an 'or' operator 
mildly sucks; dep:build|run also sucks (hard to read), and 
dep:build+run, to me at least, implies 'and'.  And yep, bikeshedding 
potential there.

~harrin

Re: [gentoo-pms] GLEP: gentoo sync based unified deps proposal

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 2585 bytes --]

On Tue, 25 Sep 2012 15:46:14 -0700
Brian Harring <ferringb@gmail.com> wrote:
> Fun fact; peoples usage of labels in exherbo is thus:
> 
> build+run:
>   set of deps
> run:
>   set of deps/conditionals/etc

That's largely because there are a lot of former Gentoo developers
there who all said "oh, yeah, I forgot we could do it the other way"
when this was pointed out...

> > Specification in terms of rendering has a huge problem, though.
> > Remembering the crazy rules Gentoo has for || ( flag? ( ) ), what
> > does this do?
> > 
> >     || ( dep:build? ( a ) dep:run? ( b ) )
> 
> Honestly, I was waiting for you to bring this up :)
> 
> You're conflating two different things here;
> 1) someone being a dumb ass and writing what's effectively a || ( 
> atom) block, just doing so in a manner w/out any reason to do so.
> 
> 2) Your ongoing jihad against || (), specifically the occasionally 
> valid complaint that build/rdepend different means the resolver can 
> get stuck in certain pathways when slots are involved, abi, etc.
> 
> Either way, in my proposal, I'm not going to single that out and try 
> blocking it.  The rendered version of it is still stable, albeit if 
> it's build/run it's unlikely to be desired if there is ABI involved 
> (for non ABI, specifically self-bootstrapping codebases, I suspect 
> someone could come up with a valid construct- sed has something 
> similar if memory serves).

The rendered version ends up as ( a b ), in effect... It doesn't end up
as || ( a (at build time) b (at runtime) ).

> Which is stupid, but syntactically correct.  Nor is this a new issue, 
> thus I don't particularly agree with your approach of trying to sink 
> the proposal via an orthogonal problem.

No, you're introducing a new kind of weirdness for || ( ) here.

> Either way, via 
> http://dev.gentoo.org/~ferringb/unified-dependencies/labels/translated-to-use-deps.txt 
> , I think it's pretty clear labels in real world usage aren't
> bringing anything to the tabel that we wouldn't have via my proposal;
> that leaves labels as just a different syntax (perhaps aesthetically
> more pleasing at first glance, but the label stacking bit via exheres 
> analysis is proven to be something people explicitly go out of their 
> way to protect against; meaning the aesthetics have a mental 
> model cost).

It's not "go out of their way to protect against". It's "there's an
easy way of making sure everything is composable". Your
misappropriation of use flags doesn't have that.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-pms] GLEP: gentoo sync based unified deps proposal

[Brian Harring]

On Sat, Sep 29, 2012 at 05:05:09PM +0100, Ciaran McCreesh wrote:
> On Tue, 25 Sep 2012 15:46:14 -0700
> Brian Harring <ferringb@gmail.com> wrote:
> > Fun fact; peoples usage of labels in exherbo is thus:
> > 
> > build+run:
> >   set of deps
> > run:
> >   set of deps/conditionals/etc
> 
> That's largely because there are a lot of former Gentoo developers
> there who all said "oh, yeah, I forgot we could do it the other way"
> when this was pointed out...

I analyzed *all* exheres on git.exherbo.

To be crystal clear, these include your packages.

You yourself didn't use nested labels.  So either the author of labels 
'forgot' he could use it, or just didn't find the nesting actually 
useful.

Considering I've not found any examples where nesting /would/ be 
useful, I'm inclined to agree w/ y'alls usage- that nesting doesn't 
matter.

So... real world usage removes one of the core arguments of labels, 
leaving it just as "it's a new syntax/aesthetically more pleasing" in 
comparison to dep:build? ( blah ) form.

Not expecting you'll agree with that statement based on the facts of 
y'alls own repo... so if you're going to retort, bust out actual 
examples from eithe trees, where nesting would be preferable to the 
form people use now please; else just drop it (-your- own usage of 
labels disproves your claim; thus why I want actual examples now if 
that point will be debated any further).


> > > Specification in terms of rendering has a huge problem, though.
> > > Remembering the crazy rules Gentoo has for || ( flag? ( ) ), what
> > > does this do?
> > > 
> > >     || ( dep:build? ( a ) dep:run? ( b ) )
> > 
> > Honestly, I was waiting for you to bring this up :)
> > 
> > You're conflating two different things here;
> > 1) someone being a dumb ass and writing what's effectively a || ( 
> > atom) block, just doing so in a manner w/out any reason to do so.
> > 
> > 2) Your ongoing jihad against || (), specifically the occasionally 
> > valid complaint that build/rdepend different means the resolver can 
> > get stuck in certain pathways when slots are involved, abi, etc.
> > 
> > Either way, in my proposal, I'm not going to single that out and try 
> > blocking it.  The rendered version of it is still stable, albeit if 
> > it's build/run it's unlikely to be desired if there is ABI involved 
> > (for non ABI, specifically self-bootstrapping codebases, I suspect 
> > someone could come up with a valid construct- sed has something 
> > similar if memory serves).
> 
> The rendered version ends up as ( a b ), in effect... It doesn't end up
> as || ( a (at build time) b (at runtime) ).

Er, I assume you left out some chars there.  The rendered version 
there isn't ( a b ); in old form it is:
DEPEND=|| ( a )
RDEPEND=|| ( b )

This is why I label it a stupid use of syntax, but not ultimately 
harmful.


> > Which is stupid, but syntactically correct.  Nor is this a new issue, 
> > thus I don't particularly agree with your approach of trying to sink 
> > the proposal via an orthogonal problem.
> 
> No, you're introducing a new kind of weirdness for || ( ) here.

Na uh, you're the smelly face!

As I said, and via correcting your misrendering, this isn't 
introducing anything truly new here; people can/have done '|| ( a )'; 
it's a stupid construct, and for paludis it means it /does/ treat that 
as an OR block (for hte rest, we do the more obvious tree collapses 
during parsing, folding "a ( b )" down into "a b", same for "a || ( b 
)" into "a b" since they're the same thing).


> > Either way, via 
> > http://dev.gentoo.org/~ferringb/unified-dependencies/labels/translated-to-use-deps.txt 
> > , I think it's pretty clear labels in real world usage aren't
> > bringing anything to the tabel that we wouldn't have via my proposal;
> > that leaves labels as just a different syntax (perhaps aesthetically
> > more pleasing at first glance, but the label stacking bit via exheres 
> > analysis is proven to be something people explicitly go out of their 
> > way to protect against; meaning the aesthetics have a mental 
> > model cost).
> 
> It's not "go out of their way to protect against". It's "there's an
> easy way of making sure everything is composable". Your
> misappropriation of use flags doesn't have that.

Again, you're pulling a "na uh, you're the smelly face" counter 
argument.

Bluntly, you want something that is academically possible, but 
pragmatically/realistically unneeded- meaning the gains are basically 
not there, but the costs most definitely are.

Now, for exherbo were y'all lack actual versioned format (exheres-0 
has changed heavily since it's inception), and chucked everything and 
did it from scratch (right?  or do I need to do a copyright analysis 
in addition?)... the situation differs.  You can invent whatever 
syntax you want, since you're starting from scratch, changing the 
mental mode for parsing is fine.

We however are *not* starting from scratch.  This shifts what we'll 
accept for costs/gains ratio; frankly, the fact y'all don't make use 
of those claimed 'gains' makes me think y'all tried something and it 
turned out to be non-useful; it occurs in formats (ebuild format is 
littered w/ shit like that unfortunately).

Either way, this is gentoo, we're talking about the ebuild format; 
just the same as everyone shredded mgorny's ass for proposing we 
mangle atom syntax w/out gain, and proposal you push for the deptree 
changing, has to have significant gains for changing the existing 
form; aesthetics at a cost aren't enough.

~harring

Re: [gentoo-pms] GLEP: gentoo sync based unified deps proposal

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 3196 bytes --]

On Sun, 30 Sep 2012 13:14:53 -0700
Brian Harring <ferringb@gmail.com> wrote:
> > That's largely because there are a lot of former Gentoo developers
> > there who all said "oh, yeah, I forgot we could do it the other way"
> > when this was pointed out...
> 
> I analyzed *all* exheres on git.exherbo.
> 
> To be crystal clear, these include your packages.
> 
> You yourself didn't use nested labels.  So either the author of
> labels 'forgot' he could use it, or just didn't find the nesting
> actually useful.

That's a rather disingenuous claim, considering how none of the
packages I maintain have any non-trivial dependencies... You could
equally well say that every single package I maintain makes use of
nested labels in every single place where they're relevant.

> So... real world usage removes one of the core arguments of labels, 
> leaving it just as "it's a new syntax/aesthetically more pleasing" in 
> comparison to dep:build? ( blah ) form.
> 
> Not expecting you'll agree with that statement based on the facts of 
> y'alls own repo... so if you're going to retort, bust out actual 
> examples from eithe trees, where nesting would be preferable to the 
> form people use now please; else just drop it (-your- own usage of 
> labels disproves your claim; thus why I want actual examples now if 
> that point will be debated any further).

It's not just that it's more aesthetically pleasing. There are two
arguments favouring labels over use abuse.

The first is that it doesn't have perverse behaviour associated with it
like your misappropriation of use conditionals does. If you put labels
deep in a tree, it's well defined. If you put your conditionals
anywhere except the top level, crazy stuff happens.

The second is that it starts the conceptual shift from "cat/pkg is a
build dep, and cat/pkg is a run dep" to "cat/pkg is a dep that is
required for build and run".

> Bluntly, you want something that is academically possible, but 
> pragmatically/realistically unneeded- meaning the gains are basically 
> not there, but the costs most definitely are.

No, I want something where things that are different look different.
Dependency types are nothing like use flags, so they shouldn't look the
same.

> Either way, this is gentoo, we're talking about the ebuild format; 
> just the same as everyone shredded mgorny's ass for proposing we 
> mangle atom syntax w/out gain, and proposal you push for the deptree 
> changing, has to have significant gains for changing the existing 
> form; aesthetics at a cost aren't enough.

The gain is that you have a syntax designed for what's being
represented, not an existing syntax abused to sort of (but not
quite, because it's still defined in terms of rendering down) do new
things.

Really, all it takes to see that your syntax is bad is one tiny little
example:

    dep:build? ( dep:run? ( cat/pkg ) )

There shouldn't be any need to add in a repoman check to catch that
kind of thing. The problem is entirely caused by bad syntax design.
Implementing labels is not difficult, and the extra cost is worth it to
get a good design.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-pms] GLEP: gentoo sync based unified deps proposal

[Brian Harring]

On Sun, Sep 30, 2012 at 09:30:18PM +0100, Ciaran McCreesh wrote:
> On Sun, 30 Sep 2012 13:14:53 -0700
> Brian Harring <ferringb@gmail.com> wrote:
> > > That's largely because there are a lot of former Gentoo developers
> > > there who all said "oh, yeah, I forgot we could do it the other way"
> > > when this was pointed out...
> > 
> > I analyzed *all* exheres on git.exherbo.
> > 
> > To be crystal clear, these include your packages.
> > 
> > You yourself didn't use nested labels.  So either the author of
> > labels 'forgot' he could use it, or just didn't find the nesting
> > actually useful.
> 
> That's a rather disingenuous claim, considering how none of the
> packages I maintain have any non-trivial dependencies... You could
> equally well say that every single package I maintain makes use of
> nested labels in every single place where they're relevant.

Admittedly, that was a punch nearing the belt or a bit below; that 
said it's not disenguous.

Reality is, our current form can handle deps generally fine- what you 
label as trivial is the vast majority- I argue effectively all.

And I intentionally gave you a way to disprove that; find real world 
dependency examples to disprove it.


> > So... real world usage removes one of the core arguments of labels, 
> > leaving it just as "it's a new syntax/aesthetically more pleasing" in 
> > comparison to dep:build? ( blah ) form.
> > 
> > Not expecting you'll agree with that statement based on the facts of 
> > y'alls own repo... so if you're going to retort, bust out actual 
> > examples from eithe trees, where nesting would be preferable to the 
> > form people use now please; else just drop it (-your- own usage of 
> > labels disproves your claim; thus why I want actual examples now if 
> > that point will be debated any further).
> 
> It's not just that it's more aesthetically pleasing. There are two
> arguments favouring labels over use abuse.
> 
> The first is that it doesn't have perverse behaviour associated with it
> like your misappropriation of use conditionals does. If you put labels
> deep in a tree, it's well defined. If you put your conditionals
> anywhere except the top level, crazy stuff happens.

This statement of yours however is fairly disenguous; label behaviour 
when nested suffers the same mental parsing oddity (wait, we're in 
build context, and this is test?  Wtf happens there?).  With 'use 
abuse' however, the situation is clear:

dep:build,run? ( x? ( dep:test? ( blah ) foon ) monkeys )

Means that 'blah' target doesn't show up.  Which is the *same* as what 
happens if someone did thus in our existing syntax:

x? ( !x? ( blah ) )

Worth noting, you didn't ban that from exherbo; you left that to sort 
itself out, same as I'm doing for dep:blah flags.  Were I punching 
below the belt, the word 'hypocritical' would likely be involved.


> The second is that it starts the conceptual shift from "cat/pkg is a
> build dep, and cat/pkg is a run dep" to "cat/pkg is a dep that is
> required for build and run".

Fairly weak argument at best; you're claiming that via labels, 
"contextually they know it's these deps" in comparison to via 
dep:build "contextually they know it's exposed only in build".

Same difference.


> > Bluntly, you want something that is academically possible, but 
> > pragmatically/realistically unneeded- meaning the gains are basically 
> > not there, but the costs most definitely are.
> 
> No, I want something where things that are different look different.
> Dependency types are nothing like use flags, so they shouldn't look the
> same.

I'll buy that argument, and to some degree- agree.

I just view that as academic wankery without real world gain.

So like I said, academically possible, but 
pragmatically/unrealistically unneded.

No amount of arguing is going to dissuade me on that view either, 
although real world tree examples *might*- aka, stop talking, go 
analyzing.


> > Either way, this is gentoo, we're talking about the ebuild format; 
> > just the same as everyone shredded mgorny's ass for proposing we 
> > mangle atom syntax w/out gain, and proposal you push for the deptree 
> > changing, has to have significant gains for changing the existing 
> > form; aesthetics at a cost aren't enough.
> 
> The gain is that you have a syntax designed for what's being
> represented, not an existing syntax abused to sort of (but not
> quite, because it's still defined in terms of rendering down) do new
> things.
> 
> Really, all it takes to see that your syntax is bad is one tiny little
> example:
> 
>     dep:build? ( dep:run? ( cat/pkg ) )

x? ( !x? ( cat/pkg ) )

Outlaw that in paludis/exherbo, and *perhaps* I'd listen to you.

You're going into broken record mode; the fact people can do stupid 
shit if they're willingly trying to abuse the syntax isn't much of an 
argument- especially considering the PM can handle it either way, 
rendering it out to a noop.


> There shouldn't be any need to add in a repoman check to catch that
> kind of thing. The problem is entirely caused by bad syntax design.
> Implementing labels is not difficult, and the extra cost is worth it to
> get a good design.

As I've said, there isn't a need for repoman checks.  It's *suggested* 
since as I've stated, the underlying idiocy should be spotted in our 
existing deps.

That said, repoman isn't necessary; such a construct solves itself via 
the deps dropping out; and before you try arguing that, your argument 
effectively would be based on "if someone specifies the deps wrong..." 
which means they're already up shit creek.

Either way, pulling another "done with this thread" bit to wrap this 
up; you don't like the proposal, got it.

In my proposal, I am addressing labels; will fold in your claims, but 
those claims basically are shit- however, if you *did* find a 
conflicting nested example that wasn't contrived, preferablly 
multiple, I'd like those examples so I can include them into the 
proposal (give labels a fair hand, basically).

I don't think you're going to find any, let alone one; some 
artificially structured ones perhaps, but I'm not interested in those- 
I'm looking for real world deps where conflicting nested is the best 
form.

Go find 'em; either way, moving on from the current discussion form 
(also known as "broken record mode").

cheers-
~harring

Re: [gentoo-pms] GLEP: gentoo sync based unified deps proposal

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 3480 bytes --]

On Sun, 30 Sep 2012 14:42:14 -0700
Brian Harring <ferringb@gmail.com> wrote:
> Reality is, our current form can handle deps generally fine- what you 
> label as trivial is the vast majority- I argue effectively all.

We could do away with half of the current feature set if we were only
interested in making things nice for the "vast majority" of cases...

> This statement of yours however is fairly disenguous; label behaviour 
> when nested suffers the same mental parsing oddity (wait, we're in 
> build context, and this is test?  Wtf happens there?).

No, label behaviour is local, and scope independent.

> Means that 'blah' target doesn't show up.  Which is the *same* as
> what happens if someone did thus in our existing syntax:
> 
> x? ( !x? ( blah ) )
> 
> Worth noting, you didn't ban that from exherbo; you left that to sort 
> itself out, same as I'm doing for dep:blah flags.  Were I punching 
> below the belt, the word 'hypocritical' would likely be involved.

That's "not fixing an existing screw-up", which is not the same at all
as "adding a new screw-up".

> > The second is that it starts the conceptual shift from "cat/pkg is a
> > build dep, and cat/pkg is a run dep" to "cat/pkg is a dep that is
> > required for build and run".
> 
> Fairly weak argument at best; you're claiming that via labels, 
> "contextually they know it's these deps" in comparison to via 
> dep:build "contextually they know it's exposed only in build".
> 
> Same difference.

It's rather a big deal now that we have := dependencies.

> > No, I want something where things that are different look different.
> > Dependency types are nothing like use flags, so they shouldn't look
> > the same.
> 
> I'll buy that argument, and to some degree- agree.
> 
> I just view that as academic wankery without real world gain.
> 
> So like I said, academically possible, but 
> pragmatically/unrealistically unneded.

Labels are almost as easy to implement as your "filtering" model, and
they come with the benefit of a better syntax for developers. Even if
labels are noticably more work to implement, which I'm not convinced is
the case, it's worth paying that price a couple of times in package
manglers rather than having developers pay the price of worse syntax in
thousands of ebuilds. Filtering is a whole new mechanism anyway.

But here's the thing: when you sell something as "pragmatic", what
you're really saying is "it's wrong, I know it's wrong, and I'm going
to pretend that wrong is a good thing". Getting it wrong should be
something you do only after you're sure you can't afford get it right;
it shouldn't be something you're proud of.

> In my proposal, I am addressing labels; will fold in your claims, but 
> those claims basically are shit- however, if you *did* find a 
> conflicting nested example that wasn't contrived, preferablly 
> multiple, I'd like those examples so I can include them into the 
> proposal (give labels a fair hand, basically).

You already have an example in your proposal, in the form of mplayer's
X? ( ) dependencies.

But that's missing the point. Even if you pretend complicated
dependencies don't exist, labels are still by far the better proposal.
Your argument boils down to "it's more pragmatic to do a quick and dirty
implementation in Portage and thus force the technical debt onto every
single ebuild than it is to do it cleanly".

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-pms] GLEP: gentoo sync based unified deps proposal

[Brian Harring]

On Sun, Sep 30, 2012 at 10:53:40PM +0100, Ciaran McCreesh wrote:
> But here's the thing: when you sell something as "pragmatic", what
> you're really saying is "it's wrong, I know it's wrong, and I'm going
> to pretend that wrong is a good thing". Getting it wrong should be
> something you do only after you're sure you can't afford get it right;
> it shouldn't be something you're proud of.

No, when I say pragmatic, what I'm actually saying is that people who 
can't focus on cost/gain, by large, haven't had real jobs (else they 
would've had that perfectionism/decreasing gains ground out of them 
sooner or later), and are spending their time whacking off chasing a 
mythical 'perfect' solution.

Academic wankery, is the short version.  You're good at technical, but 
you frequently do the academic wanking crap which leads to things 
dead-ending... plus wasted time because to you, 'pragmatic' is a dirty 
word (compromise?  Heaven forbid!).


> > In my proposal, I am addressing labels; will fold in your claims, but 
> > those claims basically are shit- however, if you *did* find a 
> > conflicting nested example that wasn't contrived, preferablly 
> > multiple, I'd like those examples so I can include them into the 
> > proposal (give labels a fair hand, basically).
> 
> You already have an example in your proposal, in the form of mplayer's
> X? ( ) dependencies.

I said nested conflicting labels.  Meaning 
"build: x? ( dar run: blah )"

which isn't the case for any of mplayer deps.

Actual examples from the tree where a conflicting nested label is 
preferable.  That isn't one of 'em, and you're unwillingness/inability 
to point out real world examples is just digging a deeper ditch for 
your argument.


> But that's missing the point. Even if you pretend complicated
> dependencies don't exist, labels are still by far the better proposal.
> Your argument boils down to "it's more pragmatic to do a quick and dirty
> implementation in Portage and thus force the technical debt onto every
> single ebuild than it is to do it cleanly".

My argument boils down to thus:

We are not exherbo- we do not have the luxury of chucking out syntax 
and pulling NIH renaming of things for shits and giggles.  Especially 
if the new syntax is directly translatable into a tweak of our 
existing syntax (a tweak that we should do anyways- recall I built 
this off of fixing USE_EXPAND).

Your argument boils down to "it's not labels, ignore that it's 
aesthetic knob polishing (you can do the same w/ our existent 
syntax, thus the analogy of waxing it I truly mean), use labels 
because I'll berate you incessently till you accede".

Beauty of open source, you want it, go do it.

If in, what, 4 years?  3?  You've not been able to get off your ass, 
write a proposal, hell, do a portage patch (you've *never* done 
portage patches of any worth, bluntly- I know this since in the past I 
used to fix shit you requested), you lose your voice in the matter.

Considering your points boil down to aesthetic academic wanking at 
this point... put up, or shut up, really is that simple.

As said, you come up w/ real world examples, I'll include them; else 
persist and I'll just fold the academic wankery description of labels 
into the glep if you'd truly like me to (or you piss me off enough I 
do so to be a dick).

~harring

Re: [gentoo-pms] GLEP: gentoo sync based unified deps proposal

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 3620 bytes --]

On Sun, 30 Sep 2012 16:56:56 -0700
Brian Harring <ferringb@gmail.com> wrote:
> On Sun, Sep 30, 2012 at 10:53:40PM +0100, Ciaran McCreesh wrote:
> > But here's the thing: when you sell something as "pragmatic", what
> > you're really saying is "it's wrong, I know it's wrong, and I'm
> > going to pretend that wrong is a good thing". Getting it wrong
> > should be something you do only after you're sure you can't afford
> > get it right; it shouldn't be something you're proud of.
> 
> No, when I say pragmatic, what I'm actually saying is that people who 
> can't focus on cost/gain, by large, haven't had real jobs (else they 
> would've had that perfectionism/decreasing gains ground out of them 
> sooner or later), and are spending their time whacking off chasing a 
> mythical 'perfect' solution.

I don't know whether you're aware of this, but a small number (cost per
ebuild) multiplied by a big number (lots of ebuilds) can be larger than
a medium sized number (cost of implementing a good solution). I realise
this is a sophisticated technique I'm using here, but I assure you
multiplication has been used in some industries for a few years now.

> Academic wankery, is the short version.  You're good at technical,
> but you frequently do the academic wanking crap which leads to things 
> dead-ending... plus wasted time because to you, 'pragmatic' is a
> dirty word (compromise?  Heaven forbid!).

Or looking at it another way: you're so eager to deliver a "compromise"
and a "pragmatic" solution (at the expense of ebuild writers
everywhere) that you immediately rule out a "good" solution just so
you can push the virtues of doing it wrong. Doing it wrong should be a
last resort, not something you look to do at any given opportunity.

> > > In my proposal, I am addressing labels; will fold in your claims,
> > > but those claims basically are shit- however, if you *did* find a 
> > > conflicting nested example that wasn't contrived, preferablly 
> > > multiple, I'd like those examples so I can include them into the 
> > > proposal (give labels a fair hand, basically).
> > 
> > You already have an example in your proposal, in the form of
> > mplayer's X? ( ) dependencies.
> 
> I said nested conflicting labels.  Meaning 
> "build: x? ( dar run: blah )"
> 
> which isn't the case for any of mplayer deps.

x? ( build: a run: b ) *is* nested "conflicting".

You're still failing to understand the point of labels parsing rules,
though: the point is to make uses like the above well defined and
consistent.

> We are not exherbo- we do not have the luxury of chucking out syntax 
> and pulling NIH renaming of things for shits and giggles.  Especially 
> if the new syntax is directly translatable into a tweak of our 
> existing syntax (a tweak that we should do anyways- recall I built 
> this off of fixing USE_EXPAND).

This isn't chucking out syntax. It's augmenting existing syntax. What
you're doing is trying to shove something new onto an existing syntax
which doesn't fit it.

You should know this from REQUIRED_USE: that's a perfect example of
going too far to reuse existing syntax.

> Your argument boils down to "it's not labels, ignore that it's 
> aesthetic knob polishing (you can do the same w/ our existent 
> syntax, thus the analogy of waxing it I truly mean), use labels 
> because I'll berate you incessently till you accede".

It's about giving ebuild developers a good format to work with. That
sort of thing matters. There are a lot of ebuilds and not many package
manglers.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-pms] GLEP: gentoo sync based unified deps proposal

[Brian Harring]

On Mon, Oct 01, 2012 at 08:13:49AM +0100, Ciaran McCreesh wrote:
> x? ( build: a run: b ) *is* nested "conflicting".
> 
> You're still failing to understand the point of labels parsing rules,
> though: the point is to make uses like the above well defined and
> consistent.

I understand them just fine; you're just either very fucking daft, 
which I have a hard time believing, or lieing through your teeth 
(which fits a decade of behaviour including multiple suspensions for 
exactly that behaviour).

Implicit labels context is build+run.  Meaning the following
> x? ( build: a run: b ) *is* nested "conflicting".

is actually

build+run x? ( build: a run: b )

Which isn't a nested conflict- subset, not conflict.

You argue labels are required so people can do nested conflicts; 
meaning the following extreme example:

run x? ( build: a test: b )

And as I nicely pointed out, /not a single fucking exheres/ does that.  
you've yet to pull out an example contradicting that analysis in 
addition.


So... with that in mind- I'm doing two things; 1) can't force you 
back under a bridge, instead I'll do the killfile equivalent for a few 
weeks, 2) my original proposal if you kept being a tool seems 
appropriate:

"""
As said, you come up w/ real world examples, I'll include them; else
persist and I'll just fold the academic wankery description of labels
into the glep if you'd truly like me to (or you piss me off enough I
do so to be a dick).
"""

What I truly love about that solution there is that it's both 
accurate, and if I play my cards right, I may be able to get a glep 
passed calling your proposal academic wankery; minimally, it'll be fun 
from my standpoint to try, so at least something came out of the last 
few emails from you.

hugs and kisses-
~harring

Re: [gentoo-pms] GLEP: gentoo sync based unified deps proposal

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 2436 bytes --]

On Mon, 1 Oct 2012 02:01:32 -0700
Brian Harring <ferringb@gmail.com> wrote:
> On Mon, Oct 01, 2012 at 08:13:49AM +0100, Ciaran McCreesh wrote:
> > x? ( build: a run: b ) *is* nested "conflicting".
> > 
> > You're still failing to understand the point of labels parsing
> > rules, though: the point is to make uses like the above well
> > defined and consistent.
> 
> I understand them just fine; you're just either very fucking daft, 
> which I have a hard time believing, or lieing through your teeth 
> (which fits a decade of behaviour including multiple suspensions for 
> exactly that behaviour).
> 
> Implicit labels context is build+run.  Meaning the following
> > x? ( build: a run: b ) *is* nested "conflicting".
> 
> is actually
> 
> build+run x? ( build: a run: b )
> 
> Which isn't a nested conflict- subset, not conflict.

As I said right at the start, you're special-casing the top level to
something that can't normally be expressed using the syntax.

> You argue labels are required so people can do nested conflicts; 
> meaning the following extreme example:
> 
> run x? ( build: a test: b )
> 
> And as I nicely pointed out, /not a single fucking exheres/ does
> that. you've yet to pull out an example contradicting that analysis
> in addition.

No, I argue that having well-defined parsing rules means it doesn't
matter if someone does do that. Meaning, no special case for the top
level.

Your rules require a handler to say "have I seen any dep: blocks
further up the tree than my current position? If yes, handle this dep:
block one way; otherwise, handle it a different way". With labels, all
you do is initialise the label stack with build+run, and then no
special case handling is required.

That's what you should be putting in the GLEP. Not examples, but a big
fat warning that your syntax requires a very strange special case rule
to handle your default build+run behaviour.

> What I truly love about that solution there is that it's both 
> accurate, and if I play my cards right, I may be able to get a glep 
> passed calling your proposal academic wankery; minimally, it'll be
> fun from my standpoint to try, so at least something came out of the
> last few emails from you.

Oh come on, we all know that unnecessarily screwing up the syntax won't
make DEPENDENCIES be sufficiently un-exherbo-looking to get it passed...

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]