[gentoo-dev] rfc: calling all eclass phase functions by default

[gentoo-dev] rfc: calling all eclass phase functions by default

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 2202 bytes --]

All,

there is an ongoing discussion about how we handle eclass phase
functions by default [1].

Currently, EXPORT_FUNCTIONS is called in eclasses, and because of the
way this works, the phase function that is exported last in the chain of
inherited eclasses is the one that is called for a given phase,
overriding the PMS default phase function.

The initial proposal is to change this behaviour so that the PMS default
phase functions call all matching phase functions from inherited
eclasses in sequence.

For example:

- your ebuild inherits foo and bar and each of them have src_unpack
  functions. With this new behaviour, the default src_unpack would run
  foo_src_unpack, bar_src_unpack, then perform its own actions.

I strongly oppose this change, because I feel it will make our
entire tree very unpredictable at best. I realize this might eliminate
boilerplating from our tree. Weighing that against the possible
ramifications in this big of a change in automagic behaviour, I think
the cost is much higher than the gain.

I am also not very comfortable with our current state, because it has
a lot of uncertainty in terms of how the eclass phase functions are
called.

My counter proposal to this is that we stop calling eclass phase
functions automatically, and to minimize the amount of boilerplating
  we would have to do, we use a variable, such as ECLASS_PHASES  which
  would be defined at the ebuild level and contain a list of the eclass
  phase functions we want to run automatically.

Going back to my previous example, say your ebuild does the following:

-- code begins here --

inherit foo bar

# Foo and bar both have src_unpack and src_install functions.
# we want foo's src_unpack and bar's src_install:

ECLASS_PHASES="foo_src_unpack
	bar_src_install"

-- code ends here ---

If ECLASS_PHASES is undefined, I think the default should be to not run
the eclass phase functions.

Yes, this means there is some boilerplating. However, there are some
strong advantages:

- this is no longer dependent on order of inheritance.
- The ebuild author knows exactly which eclass phase functions will
  be run.

Thoughts?

William

[1] https://bugs.gentoo.org/show_bugs.cgi?id=516014

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] rfc: calling all eclass phase functions by default

[Kent Fredric]

[-- Attachment #1: Type: text/plain, Size: 3815 bytes --]

On 17 August 2014 09:54, William Hubbs <williamh@gentoo.org> wrote:

> I strongly oppose this change, because I feel it will make our
> entire tree very unpredictable at best. I realize this might eliminate
> boilerplating from our tree. Weighing that against the possible
> ramifications in this big of a change in automagic behaviour, I think
> the cost is much higher than the gain.
>
>

I agree on that part, for what its worth.

The current system we have in place is an analogue of "role based" or
"trait based" inheritance, where you have very straight forward composition
behavior with regards to the final ebuild.

One of the parent classes "wins", and if you don't like this, you have to
adapt it manually.

This in practice usually proves more useful than diamond style multiple
inheritance, that while seeming like a nice idea in concept, in practice
proves more complicated than its worth.

The only downside I see is we don't have a clear straight forward collision
strategy, which is essential for most role based systems:

For instance,

  dog.eclass : function bark
  tree.eclass: function bark

Proves not to be resolvable automatically in a sensible way.

Collison systems I've seen usually do one of two things:

- In the event of a collision, demand the consumer resolve the problem by
redefining the function the collision occurs on in terms of its composite
parts. ( which is basically what we already do )
- Declare syntax to "exclude" a potential collision from either composite
part.

Our only real difference at present is unlike these systems, we assume we
can simply guess which one wins and just choose it automatically, where
collision systems tend to force you to deal with the situation if any
collision occurs.



> I am also not very comfortable with our current state, because it has
> a lot of uncertainty in terms of how the eclass phase functions are
> called.
>
> My counter proposal to this is that we stop calling eclass phase
> functions automatically, and to minimize the amount of boilerplating
>   we would have to do, we use a variable, such as ECLASS_PHASES  which
>   would be defined at the ebuild level and contain a list of the eclass
>   phase functions we want to run automatically.
>
> Going back to my previous example, say your ebuild does the following:
>
> -- code begins here --
>
> inherit foo bar
>
> # Foo and bar both have src_unpack and src_install functions.
> # we want foo's src_unpack and bar's src_install:
>
> ECLASS_PHASES="foo_src_unpack
>         bar_src_install"
>
> -- code ends here ---
>
> If ECLASS_PHASES is undefined, I think the default should be to not run
> the eclass phase functions.
>
> Yes, this means there is some boilerplating. However, there are some
> strong advantages:
>
> - this is no longer dependent on order of inheritance.
> - The ebuild author knows exactly which eclass phase functions will
>   be run.
>
>
This proposal, seems reasonable, given my comments. I anticipate however
its biggest downside would be
the requirement to state *all* the functions you want, which would lead to
maintenance headaches
due to people forgetting to declare they wanted some function or other.

So if you could sculpt it to be broader by default and have less scope for
developer error, that'd be an improvement.

--- code start --
ECLASS_EXCLUDE="foo_src_unpack bar_src_unpack"
inherit foo bar baz


--- code end ---

here, src_unpack would be baz_src_unpack *regardless* of composition order
because "foo" and "bar" were barred from being used, and baz took
precedence as a result.

If baz provides no src_unpack, then the ebuild in question is simply left
without one.

You'll still need to declare src_unpack explicitly if you need use
conditional behaviour, however.
-- 
Kent

*KENTNL* - https://metacpan.org/author/KENTNL

[-- Attachment #2: Type: text/html, Size: 5277 bytes --]

Re: [gentoo-dev] rfc: calling all eclass phase functions by default

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 4657 bytes --]

Dnia 2014-08-16, o godz. 16:54:28
William Hubbs <williamh@gentoo.org> napisał(a):

> The initial proposal is to change this behaviour so that the PMS default
> phase functions call all matching phase functions from inherited
> eclasses in sequence.
> 
> For example:
> 
> - your ebuild inherits foo and bar and each of them have src_unpack
>   functions. With this new behaviour, the default src_unpack would run
>   foo_src_unpack, bar_src_unpack, then perform its own actions.

Why doesn't it call default_src_unpack too?

> I strongly oppose this change, because I feel it will make our
> entire tree very unpredictable at best. I realize this might eliminate
> boilerplating from our tree. Weighing that against the possible
> ramifications in this big of a change in automagic behaviour, I think
> the cost is much higher than the gain.

'Unpredictable' is very lightly said. 'Complete mayhem' seems much more
appropriate.

Right now, finding proper phase functions may be a bit hard due to
indirect inherits. Imagine all the fun of debugging phase functions
when every single indirect inherits adds to the stack. I don't want
even to discuss the issue of ordering them all.

For example, a quick grep suggests that 72 eclasses declare
src_compile(). Now imagine how many eclasses end up inherited in more
complex ebuilds. Of course, the final number would be smaller since
many eclasses will simply have to drop phase functions to avoid forcing
every single ebuild to redefine phases to avoid colliding behavior.

However, I exaggerate a bit here. The issue won't hit too many people
because the learning curve would look more like a cliff which you will
be climbing up upside down using only your tongue, during a storm.

> I am also not very comfortable with our current state, because it has
> a lot of uncertainty in terms of how the eclass phase functions are
> called.

That is not really a problem of PMS behavior but a problem with
the eclasses. If eclasses were done at least semi-reasonably, you
would be able to easily guess which phase functions are exported by
which eclass.

> My counter proposal to this is that we stop calling eclass phase
> functions automatically, and to minimize the amount of boilerplating
>   we would have to do, we use a variable, such as ECLASS_PHASES  which
>   would be defined at the ebuild level and contain a list of the eclass
>   phase functions we want to run automatically.
> 
> Going back to my previous example, say your ebuild does the following:
> 
> -- code begins here --
> 
> inherit foo bar
> 
> # Foo and bar both have src_unpack and src_install functions.
> # we want foo's src_unpack and bar's src_install:
> 
> ECLASS_PHASES="foo_src_unpack
> 	bar_src_install"
> 
> -- code ends here ---

And how is this exactly different than:

  src_unpack() { foo_src_unpack; }
  src_install() { bar_src_install; }

forced by your previous idea? As I see it, it's just a redundant way of
doing the same thing. A few bytes shorter maybe, at the cost of having
to control and consider another variable.

What happens if ebuild specifies both ECLASS_PHASES="foo_src_unpack"
and explicit src_unpack()?

> If ECLASS_PHASES is undefined, I think the default should be to not run
> the eclass phase functions.

Do we want to run default EAPI phases? Do we want to have an extra
variable to control these?

> Yes, this means there is some boilerplating. However, there are some
> strong advantages:
> 
> - this is no longer dependent on order of inheritance.
> - The ebuild author knows exactly which eclass phase functions will
>   be run.
> 
> Thoughts?

The current behavior compared to the two extremes suggested here sounds
like a good compromise. It's not perfect but it works very well for
most of the ebuilds, keeping only a few overly verbose and a few
unpredictable.

The first suggested idea is the kind of extreme trying to solve
the overly verbose ebuilds. While it could work in the end, it will
certainly make getting all ebuilds and eclasses right a pain. Moreover,
it is full of pitfalls and hidden issues.

The second idea is the complete opposite extreme. It kills all
automation, forcing verbose references in every ebuild. Imagine all
those simple Perl modules that have to list all the phase functions...
and imagine all the extra pitfalls of missing phase function calls.

That said, I suggest you focus your efforts on solving the real issue
-- that is, fixing the eclasses -- rather than trying to shoehorn
an ugly workaround into PMS.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] rfc: calling all eclass phase functions by default

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 1865 bytes --]

On Sun, Aug 17, 2014 at 10:32:14AM +1200, Kent Fredric wrote:
> On 17 August 2014 09:54, William Hubbs <williamh@gentoo.org> wrote:
> > My counter proposal to this is that we stop calling eclass phase
> > functions automatically, and to minimize the amount of boilerplating
> >   we would have to do, we use a variable, such as ECLASS_PHASES  which
> >   would be defined at the ebuild level and contain a list of the eclass
> >   phase functions we want to run automatically.
>
> This proposal, seems reasonable, given my comments. I anticipate however
> its biggest downside would be
> the requirement to state *all* the functions you want, which would lead to
> maintenance headaches
> due to people forgetting to declare they wanted some function or other.
> 
> So if you could sculpt it to be broader by default and have less scope for
> developer error, that'd be an improvement.
> 
> --- code start --
> ECLASS_EXCLUDE="foo_src_unpack bar_src_unpack"
> inherit foo bar baz
> 
> 
> --- code end ---
> 
> here, src_unpack would be baz_src_unpack *regardless* of composition order
> because "foo" and "bar" were barred from being used, and baz took
> precedence as a result.
> 
> If baz provides no src_unpack, then the ebuild in question is simply left
> without one.

My concern about reverse logic like excludes is this:

-- code start --

# foo and bas provide src_unpack, but you don't want the PMS default
# src_unpack to run them, and bar does not provide src_unpack:
# You want the rest of the PMS default src_unpack actions to run, so you
# don't write src_unpack.

ECLASS_EXCLUDE="foo_src_unpack bas_src_unpack"
inherit foo bar bas

-- code stop --

This works fine until the eclass maintainer for bar.eclass decides to
add bar_src_unpack.

As soon as that happens, your ebuild is broken.

William


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] rfc: calling all eclass phase functions by default

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 3556 bytes --]

On Sun, Aug 17, 2014 at 12:54:17AM +0200, Michał Górny wrote:
> Dnia 2014-08-16, o godz. 16:54:28
> William Hubbs <williamh@gentoo.org> napisał(a):
> 
> > The initial proposal is to change this behaviour so that the PMS default
> > phase functions call all matching phase functions from inherited
> > eclasses in sequence.
> > 
> > For example:
> > 
> > - your ebuild inherits foo and bar and each of them have src_unpack
> >   functions. With this new behaviour, the default src_unpack would run
> >   foo_src_unpack, bar_src_unpack, then perform its own actions.
> 
> Why doesn't it call default_src_unpack too?

In the original proposal, the loop that ran foo_src_unpack,
bar_src_unpack, etc would be part of default_src_unpack.

> > I strongly oppose this change, because I feel it will make our
> > entire tree very unpredictable at best. I realize this might eliminate
> > boilerplating from our tree. Weighing that against the possible
> > ramifications in this big of a change in automagic behaviour, I think
> > the cost is much higher than the gain.
> 
> 'Unpredictable' is very lightly said. 'Complete mayhem' seems much more
> appropriate.
> 
> Right now, finding proper phase functions may be a bit hard due to
> indirect inherits. Imagine all the fun of debugging phase functions
> when every single indirect inherits adds to the stack. I don't want
> even to discuss the issue of ordering them all.

Agreed. It definitely would be a mess.

> > I am also not very comfortable with our current state, because it has
> > a lot of uncertainty in terms of how the eclass phase functions are
> > called.
> 
> That is not really a problem of PMS behavior but a problem with
> the eclasses. If eclasses were done at least semi-reasonably, you
> would be able to easily guess which phase functions are exported by
> which eclass.

I'm all ears as they say; feel free to start a new thread on redesigning
our eclasses. I have no idea how they could be done differently
to make this easier.

> 
> > My counter proposal to this is that we stop calling eclass phase
> > functions automatically, and to minimize the amount of boilerplating
> >   we would have to do, we use a variable, such as ECLASS_PHASES  which
> >   would be defined at the ebuild level and contain a list of the eclass
> >   phase functions we want to run automatically.
> > 
> > Going back to my previous example, say your ebuild does the following:
> > 
> > -- code begins here --
> > 
> > inherit foo bar
> > 
> > # Foo and bar both have src_unpack and src_install functions.
> > # we want foo's src_unpack and bar's src_install:
> > 
> > ECLASS_PHASES="foo_src_unpack
> > 	bar_src_install"
> > 
> > -- code ends here ---
> 
> And how is this exactly different than:
> 
>   src_unpack() { foo_src_unpack; }
>   src_install() { bar_src_install; }
> 
> forced by your previous idea? As I see it, it's just a redundant way of
> doing the same thing. A few bytes shorter maybe, at the cost of having
> to control and consider another variable.
 
 I'm not arguing that either.

> What happens if ebuild specifies both ECLASS_PHASES="foo_src_unpack"
> and explicit src_unpack()?

The src_unpack() in the ebuild wins.

> That said, I suggest you focus your efforts on solving the real issue
> -- that is, fixing the eclasses -- rather than trying to shoehorn
> an ugly workaround into PMS.

Like I said above, I'm all ears; start a new thread about how the eclasses
can be fixed.

William


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[gentoo-dev] Re: rfc: calling all eclass phase functions by default

[Duncan]

William Hubbs posted on Sat, 16 Aug 2014 18:01:08 -0500 as excerpted:

> My concern about reverse logic like excludes is this:
> 
> -- code start --
> 
> # foo and bas provide src_unpack, but you don't want the PMS default
> # src_unpack to run them, and bar does not provide src_unpack:
> # You want the rest of the PMS default src_unpack actions to run, so you
> # don't write src_unpack.
> 
> ECLASS_EXCLUDE="foo_src_unpack bas_src_unpack"
> inherit foo bar bas
> 
> -- code stop --
> 
> This works fine until the eclass maintainer for bar.eclass decides to
> add bar_src_unpack.
> 
> As soon as that happens, your ebuild is broken.

Of course that can be foreseen since the eclass is directly inherited and 
an ebuild writer could do...

ECLASS_EXCLUDE="foo_src_unpack bas_src_unpack bar_src_unpack"

... even if there isn't one, since an ebuild writer can be presumed to 
know what his inherit line is.

But what about Nth generation inherits?  Suppose the foo eclass 
maintainer decides to inherit raz, and there's a raz_src_unpack, what 
then?

So with reverse-logic excludes, to be sure he gets what he intended in 
the default_src_unpack wanted scenario above, an ebuild writer will have 
to create a src_unpack that simply calls default_src_unpack.  Of course 
that's the way it is today too, but it's much more of a trap with reverse-
logic as now you pretty-much have to do your src_unpack simply calling 
default_src_unpack from the get-go, while with reverse-logic, it might be 
fine from the get-go, and only break in some rather hidden and non-
intuitive Nth-level inheritance case much later.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] rfc: calling all eclass phase functions by default

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 938 bytes --]

>>>>> On Sat, 16 Aug 2014, William Hubbs wrote:

> The initial proposal is to change this behaviour so that the PMS
> default phase functions call all matching phase functions from
> inherited eclasses in sequence.

> I strongly oppose this change, because I feel it will make our
> entire tree very unpredictable at best. [...]

+1

> My counter proposal to this is that we stop calling eclass phase
> functions automatically, and to minimize the amount of boilerplating
> we would have to do, we use a variable, such as ECLASS_PHASES which
> would be defined at the ebuild level and contain a list of the
> eclass phase functions we want to run automatically.

I'm strongly opposed against this proposal, too. Our current system
essentially works. Also, already now you can suppress automatic
calling of phase functions at the eclass level (by not exporting them)
and at the ebuild level (by defining explicit phase functions).

Ulrich

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] rfc: calling all eclass phase functions by default

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 998 bytes --]

Dnia 2014-08-17, o godz. 10:32:14
Kent Fredric <kentfredric@gmail.com> napisał(a):

> So if you could sculpt it to be broader by default and have less scope for
> developer error, that'd be an improvement.
> 
> --- code start --
> ECLASS_EXCLUDE="foo_src_unpack bar_src_unpack"
> inherit foo bar baz
> 
> 
> --- code end ---
> 
> here, src_unpack would be baz_src_unpack *regardless* of composition order
> because "foo" and "bar" were barred from being used, and baz took
> precedence as a result.

Wow, so you suggest replacing a solution where you have to re-declare
all the phases with one in which you have to opt-out of all phases of
all eclasses...

This thread spreads more great ideas every minute. Soon enough, we will
ban eclasses and require every ebuild to write everything inline just
to be sure. Preferably using kernel calls from assembly to avoid as much
middleware as possible, and make ebuilds as verbose as possible.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] rfc: calling all eclass phase functions by default

["Paweł Hajdan, Jr."]

[-- Attachment #1: Type: text/plain, Size: 3262 bytes --]

On 8/17/14, 12:32 AM, Kent Fredric wrote:
> Collison systems I've seen usually do one of two things:
> 
> - In the event of a collision, demand the consumer resolve the problem by
> redefining the function the collision occurs on in terms of its composite
> parts. ( which is basically what we already do )
> - Declare syntax to "exclude" a potential collision from either composite
> part.
> 
> Our only real difference at present is unlike these systems, we assume we
> can simply guess which one wins and just choose it automatically, where
> collision systems tend to force you to deal with the situation if any
> collision occurs.

This makes sense to me. Can we consider starting with just a repoman
warning for the collision cases?

The warning would make the problem more visible to ebuild writers. Then
we already have a solution that works, i.e. explicitly defining the
phase function in the ebuild, possibly calling the eclass functions.

My understanding is people not being aware of the problem is the main
issue here, not the ability to address it.

Paweł

>> I am also not very comfortable with our current state, because it has
>> a lot of uncertainty in terms of how the eclass phase functions are
>> called.
>>
>> My counter proposal to this is that we stop calling eclass phase
>> functions automatically, and to minimize the amount of boilerplating
>>   we would have to do, we use a variable, such as ECLASS_PHASES  which
>>   would be defined at the ebuild level and contain a list of the eclass
>>   phase functions we want to run automatically.
>>
>> Going back to my previous example, say your ebuild does the following:
>>
>> -- code begins here --
>>
>> inherit foo bar
>>
>> # Foo and bar both have src_unpack and src_install functions.
>> # we want foo's src_unpack and bar's src_install:
>>
>> ECLASS_PHASES="foo_src_unpack
>>         bar_src_install"
>>
>> -- code ends here ---
>>
>> If ECLASS_PHASES is undefined, I think the default should be to not run
>> the eclass phase functions.
>>
>> Yes, this means there is some boilerplating. However, there are some
>> strong advantages:
>>
>> - this is no longer dependent on order of inheritance.
>> - The ebuild author knows exactly which eclass phase functions will
>>   be run.
>>
>>
> This proposal, seems reasonable, given my comments. I anticipate however
> its biggest downside would be
> the requirement to state *all* the functions you want, which would lead to
> maintenance headaches
> due to people forgetting to declare they wanted some function or other.
> 
> So if you could sculpt it to be broader by default and have less scope for
> developer error, that'd be an improvement.
> 
> --- code start --
> ECLASS_EXCLUDE="foo_src_unpack bar_src_unpack"
> inherit foo bar baz
> 
> 
> --- code end ---
> 
> here, src_unpack would be baz_src_unpack *regardless* of composition order
> because "foo" and "bar" were barred from being used, and baz took
> precedence as a result.
> 
> If baz provides no src_unpack, then the ebuild in question is simply left
> without one.
> 
> You'll still need to declare src_unpack explicitly if you need use
> conditional behaviour, however.
> 



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 841 bytes --]

Re: [gentoo-dev] rfc: calling all eclass phase functions by default

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1986 bytes --]

Dnia 2014-08-17, o godz. 09:06:04
"Paweł Hajdan, Jr." <phajdan.jr@gentoo.org> napisał(a):

> On 8/17/14, 12:32 AM, Kent Fredric wrote:
> > Collison systems I've seen usually do one of two things:
> > 
> > - In the event of a collision, demand the consumer resolve the problem by
> > redefining the function the collision occurs on in terms of its composite
> > parts. ( which is basically what we already do )
> > - Declare syntax to "exclude" a potential collision from either composite
> > part.
> > 
> > Our only real difference at present is unlike these systems, we assume we
> > can simply guess which one wins and just choose it automatically, where
> > collision systems tend to force you to deal with the situation if any
> > collision occurs.
> 
> This makes sense to me. Can we consider starting with just a repoman
> warning for the collision cases?

Not really. First, you have to run a policy that prohibits solving them
through inherit order through QA. And saying for myself, I doubt I
would vote for a proposal which forces me to write more ebuild code.

Once QA agrees on a policy, we can add a matching repoman check.
Otherwise, it's full of false positives and a topic of bikeshed,
and then it is reverted.

> The warning would make the problem more visible to ebuild writers. Then
> we already have a solution that works, i.e. explicitly defining the
> phase function in the ebuild, possibly calling the eclass functions.
> 
> My understanding is people not being aware of the problem is the main
> issue here, not the ability to address it.

What we could do is printing the phase function names when starting
them, e.g.:

  >>> [foo_src_compile] Compiling sources in ...

As for another idea, we could warn if an eclass overrides phase
function via implicit inherit without redefining it. As I see it, this
is the biggest issue here, and a solution that's relatively easy to
accept.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] rfc: calling all eclass phase functions by default

["Paweł Hajdan, Jr."]

[-- Attachment #1: Type: text/plain, Size: 905 bytes --]

On 8/17/14, 9:18 AM, Michał Górny wrote:
> Dnia 2014-08-17, o godz. 09:06:04
> "Paweł Hajdan, Jr." <phajdan.jr@gentoo.org> napisał(a):
>> The warning would make the problem more visible to ebuild writers. Then
>> we already have a solution that works, i.e. explicitly defining the
>> phase function in the ebuild, possibly calling the eclass functions.
>>
>> My understanding is people not being aware of the problem is the main
>> issue here, not the ability to address it.
> 
> What we could do is printing the phase function names when starting
> them, e.g.:
> 
>   >>> [foo_src_compile] Compiling sources in ...
> 
> As for another idea, we could warn if an eclass overrides phase
> function via implicit inherit without redefining it. As I see it, this
> is the biggest issue here, and a solution that's relatively easy to
> accept.

Both of these sound good to me.

Paweł



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 841 bytes --]

Re: [gentoo-dev] rfc: calling all eclass phase functions by default

[Kent Fredric]

[-- Attachment #1: Type: text/plain, Size: 1978 bytes --]

On 17 August 2014 19:03, Michał Górny <mgorny@gentoo.org> wrote:

>
> > So if you could sculpt it to be broader by default and have less scope
> for
> > developer error, that'd be an improvement.
> >
> > --- code start --
> > ECLASS_EXCLUDE="foo_src_unpack bar_src_unpack"
> > inherit foo bar baz
> >
> >
> > --- code end ---
> >
> > here, src_unpack would be baz_src_unpack *regardless* of composition
> order
> > because "foo" and "bar" were barred from being used, and baz took
> > precedence as a result.
>
> Wow, so you suggest replacing a solution where you have to re-declare
> all the phases with one in which you have to opt-out of all phases of
> all eclasses...
>
> This thread spreads more great ideas every minute. Soon enough, we will
> ban eclasses and require every ebuild to write everything inline just
> to be sure. Preferably using kernel calls from assembly to avoid as much
> middleware as possible, and make ebuilds as verbose as possible.



Indeed, I think I shall recall my suggestion, because the number of times
you *ever* want to express that in eclasses is rare and none.

Either you want to expressly say "I want these phases to run in this order,
regardless of the import ordering", or "I want some sensible default to
take place based on import ordering"

Saying "I want this subset of possible phases to not happen, but we might
anticipate that somebody creates a function later that breaks our class and
that be ok"

Who ever does that. Seriously.

You would literally be enumerating  ( @INHERITED - THEONEIWANT  ) every
time when you could simply declare the one you want by doing " function(){
THEONEIWANT }".

In short, "Just override the function if you need to" is about as simple
and straight forward and not mental gymnastics as you're going to get.

Otherwise we're just paving a superhighway with good-intention-tiles.

-- 
Kent

*KENTNL* - https://metacpan.org/author/KENTNL

[-- Attachment #2: Type: text/html, Size: 2924 bytes --]

Re: [gentoo-dev] rfc: calling all eclass phase functions by default

[Rich Freeman]

On Sun, Aug 17, 2014 at 2:54 AM, Ulrich Mueller <ulm@gentoo.org> wrote:
>>>>>> On Sat, 16 Aug 2014, William Hubbs wrote:
>
>> My counter proposal to this is that we stop calling eclass phase
>> functions automatically, and to minimize the amount of boilerplating
>> we would have to do, we use a variable, such as ECLASS_PHASES which
>> would be defined at the ebuild level and contain a list of the
>> eclass phase functions we want to run automatically.
>
> I'm strongly opposed against this proposal, too. Our current system
> essentially works. Also, already now you can suppress automatic
> calling of phase functions at the eclass level (by not exporting them)
> and at the ebuild level (by defining explicit phase functions).
>

Honestly, I'm not sure that the status quo a great solution.  I think
we're much better off if eclasses are designed with the need to call
them explicitly kept in mind.

Rather than one big phase override that does 5 things, they should
have 5 separate calls to do each of those things in turn, and perhaps
one convenience function that calls all 5 in sequence.

Then if you want to use two eclasses that each expect to do 5 things
in the same phase you have a hope of being able to call them in a way
that makes sense, and not have things break anytime either of the
eclasses change.

Sure, you CAN work around many of these issues today, but the current
practice encourages eclass developers to build their eclasses with the
assumption that they are the only eclass around, and to avoid
educating their consumers about the need to call functions, little
documentation, etc.  The thinking is that there is no need for the foo
eclass maintainers to warn developers about the need to trigger their
new phase function, because it will happen automatically.  The problem
is that some percentage of the time, this won't actually happen.  It
is just infrequent enough that we tolerate it.

Thinking about it a bit more, the one case where the status quo might
be helpful is for eclasses that aren't general-consumption.  For
example, there are a million kde packages in the tree, and if it makes
sense to have each of those just set a few variables in the package
and inherit phase functions from a master kde eclass that makes
maintenance far simpler.  This tends not to cause problems because all
the ebuilds that use the eclass are maintained by the same project and
are closely related (and I don't think it is a good idea to use this
behavior for a kde eclass that anything that links to libkde is
intended to use - just for packages that are part of kde-meta and so
on).

Unfortunately, with our current design we can't exclude inheritance
for some eclasses and not others.  Possible options to get around this
include having different types of eclasses controlled by some setting
in the eclass file, something other than eclasses like meta-ebuilds
which can be inherited by other ebuilds perhaps with some kind of
namespace isolation (like private classes in most programming
languages), and two different inherit statements that behave
differently (which is basically the same as two kinds of eclasses, but
it gives control to the ebuild maintainers instead of the eclass
maintainers so that when things change there is a different kind of
pain).

Ok, I realize I'm thinking out loud here so I'll just leave it at
that.  I'm open to arguments - I just don't think the current state is
as clean as it could be so we really should consider any reasonable
proposals for change.

Rich

Re: [gentoo-dev] rfc: calling all eclass phase functions by default

[Sergey Popov]

[-- Attachment #1: Type: text/plain, Size: 2719 bytes --]

17.08.2014 01:54, William Hubbs пишет:
> All,
> 
> there is an ongoing discussion about how we handle eclass phase
> functions by default [1].
> 
> Currently, EXPORT_FUNCTIONS is called in eclasses, and because of the
> way this works, the phase function that is exported last in the chain of
> inherited eclasses is the one that is called for a given phase,
> overriding the PMS default phase function.
> 
> The initial proposal is to change this behaviour so that the PMS default
> phase functions call all matching phase functions from inherited
> eclasses in sequence.
> 
> For example:
> 
> - your ebuild inherits foo and bar and each of them have src_unpack
>   functions. With this new behaviour, the default src_unpack would run
>   foo_src_unpack, bar_src_unpack, then perform its own actions.
> 
> I strongly oppose this change, because I feel it will make our
> entire tree very unpredictable at best. I realize this might eliminate
> boilerplating from our tree. Weighing that against the possible
> ramifications in this big of a change in automagic behaviour, I think
> the cost is much higher than the gain.
> 
> I am also not very comfortable with our current state, because it has
> a lot of uncertainty in terms of how the eclass phase functions are
> called.
> 
> My counter proposal to this is that we stop calling eclass phase
> functions automatically, and to minimize the amount of boilerplating
>   we would have to do, we use a variable, such as ECLASS_PHASES  which
>   would be defined at the ebuild level and contain a list of the eclass
>   phase functions we want to run automatically.
> 
> Going back to my previous example, say your ebuild does the following:
> 
> -- code begins here --
> 
> inherit foo bar
> 
> # Foo and bar both have src_unpack and src_install functions.
> # we want foo's src_unpack and bar's src_install:
> 
> ECLASS_PHASES="foo_src_unpack
> 	bar_src_install"
> 
> -- code ends here ---
> 
> If ECLASS_PHASES is undefined, I think the default should be to not run
> the eclass phase functions.
> 
> Yes, this means there is some boilerplating. However, there are some
> strong advantages:
> 
> - this is no longer dependent on order of inheritance.
> - The ebuild author knows exactly which eclass phase functions will
>   be run.
> 
> Thoughts?
> 
> William
> 
> [1] https://bugs.gentoo.org/show_bugs.cgi?id=516014
> 

You have my strong opposition on such change as well. It will turn
ebuilds into unreadable and undpredictable mess, please do not do that

-- 
Best regards, Sergey Popov
Gentoo developer
Gentoo Desktop Effects project lead
Gentoo Proxy maintainers project lead


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 538 bytes --]

Re: [gentoo-dev] rfc: calling all eclass phase functions by default

[Rich Freeman]

On Mon, Aug 18, 2014 at 4:54 AM, Sergey Popov <pinkbyte@gentoo.org> wrote:
> 17.08.2014 01:54, William Hubbs пишет:
>>
>> # Foo and bar both have src_unpack and src_install functions.
>> # we want foo's src_unpack and bar's src_install:
>>
>> ECLASS_PHASES="foo_src_unpack
>>       bar_src_install"
>
> You have my strong opposition on such change as well. It will turn
> ebuilds into unreadable and undpredictable mess, please do not do that
>

I'm not sure I follow your complaint.  He is talking about adding one
line to an ebuild.  I'm not sure how that is unreadable, and the
algorithm you quoted looks fairly predictable to me as well.

Certainly it is less convenient than not having to do anything to pull
in eclass-defined phase functions, and it requires ebuilds to be
updated when eclasses are updated to add new phase functions.  That
could be problematic for cases like KDE/X11/etc where you have a large
collection of short ebuilds with all the logic in an eclass.

I just want to make sure I'm understanding your concern in case there
is a new issue being raised.

Rich

Re: [gentoo-dev] rfc: calling all eclass phase functions by default

[hasufell]

Sergey Popov:
> 
> You have my strong opposition on such change as well. It will turn
> ebuilds into unreadable and undpredictable mess, please do not do that
> 

They are already fairly unreadable and unpredictable.

Re: [gentoo-dev] rfc: calling all eclass phase functions by default

[Sergey Popov]

[-- Attachment #1: Type: text/plain, Size: 923 bytes --]

18.08.2014 16:04, hasufell пишет:
>> You have my strong opposition on such change as well. It will turn
>> ebuilds into unreadable and undpredictable mess, please do not do that
>>
> 
> They are already fairly unreadable and unpredictable.
> 

For you - maybe. But not for me.

I am NOT talking about hacks like putting additional *.as files through
echo(hello Boost ebuild) or doing something crazy with subshells.

But most of the eclass and ebuilds are readable quite simple if you read
devmanual, PMS and have a brain.

Of course, there are sometimes non-trivial stuff that is hard to read.

But majority of ebuilds and eclasses are fine to understand and predict.

So, without examples from you, this discussion will lead to nowhere, so,
please let's stop it.

-- 
Best regards, Sergey Popov
Gentoo developer
Gentoo Desktop Effects project lead
Gentoo Proxy maintainers project lead


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 538 bytes --]

Re: [gentoo-dev] rfc: calling all eclass phase functions by default

[Sergey Popov]

[-- Attachment #1: Type: text/plain, Size: 1494 bytes --]

18.08.2014 14:44, Rich Freeman пишет:
> On Mon, Aug 18, 2014 at 4:54 AM, Sergey Popov <pinkbyte@gentoo.org> wrote:
>> 17.08.2014 01:54, William Hubbs пишет:
>>>
>>> # Foo and bar both have src_unpack and src_install functions.
>>> # we want foo's src_unpack and bar's src_install:
>>>
>>> ECLASS_PHASES="foo_src_unpack
>>>       bar_src_install"
>>
>> You have my strong opposition on such change as well. It will turn
>> ebuilds into unreadable and undpredictable mess, please do not do that
>>
> 
> I'm not sure I follow your complaint.  He is talking about adding one
> line to an ebuild.  I'm not sure how that is unreadable, and the
> algorithm you quoted looks fairly predictable to me as well.
> 
> Certainly it is less convenient than not having to do anything to pull
> in eclass-defined phase functions, and it requires ebuilds to be
> updated when eclasses are updated to add new phase functions.  That
> could be problematic for cases like KDE/X11/etc where you have a large
> collection of short ebuilds with all the logic in an eclass.
> 
> I just want to make sure I'm understanding your concern in case there
> is a new issue being raised.

What's bad with overriding needed functions and saying which exported
functions(from eclasses) to execute and in which order?

Is this approach flaw? In which ways?

-- 
Best regards, Sergey Popov
Gentoo developer
Gentoo Desktop Effects project lead
Gentoo Proxy maintainers project lead


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 538 bytes --]

Re: [gentoo-dev] rfc: calling all eclass phase functions by default

[hasufell]

Sergey Popov:
> 18.08.2014 16:04, hasufell пишет:
>>> You have my strong opposition on such change as well. It will turn
>>> ebuilds into unreadable and undpredictable mess, please do not do that
>>>
>>
>> They are already fairly unreadable and unpredictable.
>>
> 
> For you - maybe. But not for me.
> 
> I am NOT talking about hacks like putting additional *.as files through
> echo(hello Boost ebuild) or doing something crazy with subshells.
> 
> But most of the eclass and ebuilds are readable quite simple if you read
> devmanual, PMS and have a brain.
> 
> Of course, there are sometimes non-trivial stuff that is hard to read.
> 
> But majority of ebuilds and eclasses are fine to understand and predict.
> 
> So, without examples from you, this discussion will lead to nowhere, so,
> please let's stop it.
> 

From my time as a sunrise dev I strongly disagree. People have problems
with understanding the mess, including actual programmers. They have
enough technical understanding, but not the time or motivation to go
through all those funny pitfalls which are NOT properly documented in
devmanual.

The most popular example is what we are talking about right now:
indirect inheritance for example via games.eclass which inherits
base.eclass but does not export src_unpack so stuff like unpacker.eclass
and git-2.eclass will likely just do nothing if you inherit them before
games.eclass (which is required by games herd policy)... uhm. I doubt
you would have guessed this one if you saw the plain ebuild. I know the
pitfall, so I see it just from looking at the inherit line. But it is
far from being obvious.

Re: [gentoo-dev] rfc: calling all eclass phase functions by default

[hasufell]

hasufell:
> Sergey Popov:
>> 18.08.2014 16:04, hasufell пишет:
>>>> You have my strong opposition on such change as well. It will turn
>>>> ebuilds into unreadable and undpredictable mess, please do not do that
>>>>
>>>
>>> They are already fairly unreadable and unpredictable.
>>>
>>
>> For you - maybe. But not for me.
>>
>> I am NOT talking about hacks like putting additional *.as files through
>> echo(hello Boost ebuild) or doing something crazy with subshells.
>>
>> But most of the eclass and ebuilds are readable quite simple if you read
>> devmanual, PMS and have a brain.
>>
>> Of course, there are sometimes non-trivial stuff that is hard to read.
>>
>> But majority of ebuilds and eclasses are fine to understand and predict.
>>
>> So, without examples from you, this discussion will lead to nowhere, so,
>> please let's stop it.
>>
> 
> From my time as a sunrise dev I strongly disagree. People have problems
> with understanding the mess, including actual programmers. They have
> enough technical understanding, but not the time or motivation to go
> through all those funny pitfalls which are NOT properly documented in
> devmanual.
> 
> The most popular example is what we are talking about right now:
> indirect inheritance for example via games.eclass which inherits
> base.eclass but does not export src_unpack so stuff like unpacker.eclass
> and git-2.eclass will likely just do nothing if you inherit them before
> games.eclass (which is required by games herd policy)... uhm. I doubt
> you would have guessed this one if you saw the plain ebuild. I know the
> pitfall, so I see it just from looking at the inherit line. But it is
> far from being obvious.
> 

Even more interesting... you can work around this by inheriting
base.eclass explicitly before e.g. unpacker.eclass, something like

inherit base unpacker games

=> unpacker_src_unpack() is carried out by default (and the ebuild
breaks if someone thinks the base.eclass is useless and removes it)

inherit unpacker games

=> unpacker_src_unpack is not carried out by default although
games.eclass does not directly overwrite it

If you think any of this is sensible...

Re: [gentoo-dev] rfc: calling all eclass phase functions by default

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2178 bytes --]

Dnia 2014-08-18, o godz. 12:41:11
hasufell <hasufell@gentoo.org> napisał(a):

> hasufell:
> > Sergey Popov:
> >> 18.08.2014 16:04, hasufell пишет:
> >>>> You have my strong opposition on such change as well. It will turn
> >>>> ebuilds into unreadable and undpredictable mess, please do not do that
> >>>>
> >>>
> >>> They are already fairly unreadable and unpredictable.
> >>>
> >>
> >> For you - maybe. But not for me.
> >>
> >> I am NOT talking about hacks like putting additional *.as files through
> >> echo(hello Boost ebuild) or doing something crazy with subshells.
> >>
> >> But most of the eclass and ebuilds are readable quite simple if you read
> >> devmanual, PMS and have a brain.
> >>
> >> Of course, there are sometimes non-trivial stuff that is hard to read.
> >>
> >> But majority of ebuilds and eclasses are fine to understand and predict.
> >>
> >> So, without examples from you, this discussion will lead to nowhere, so,
> >> please let's stop it.
> >>
> > 
> > From my time as a sunrise dev I strongly disagree. People have problems
> > with understanding the mess, including actual programmers. They have
> > enough technical understanding, but not the time or motivation to go
> > through all those funny pitfalls which are NOT properly documented in
> > devmanual.
> > 
> > The most popular example is what we are talking about right now:
> > indirect inheritance for example via games.eclass which inherits
> > base.eclass but does not export src_unpack so stuff like unpacker.eclass
> > and git-2.eclass will likely just do nothing if you inherit them before
> > games.eclass (which is required by games herd policy)... uhm. I doubt
> > you would have guessed this one if you saw the plain ebuild. I know the
> > pitfall, so I see it just from looking at the inherit line. But it is
> > far from being obvious.
> > 
> 
> Even more interesting... you can work around this by inheriting
> base.eclass explicitly before e.g. unpacker.eclass, something like
> 
> inherit base unpacker games

This is a bug with vapier's approach at spanking and will be fixed.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] rfc: calling all eclass phase functions by default

[hasufell]

hasufell:
> 
> Even more interesting... you can work around this by inheriting
> base.eclass explicitly before e.g. unpacker.eclass, something like
> 
> inherit base unpacker games
> 
> => unpacker_src_unpack() is carried out by default (and the ebuild
> breaks if someone thinks the base.eclass is useless and removes it)
> 
> inherit unpacker games
> 
> => unpacker_src_unpack is not carried out by default although
> games.eclass does not directly overwrite it
> 
> If you think any of this is sensible...
> 

Almost forgot, of course this does not work if you expect
unpacker_src_unpacker() to run:
inherit unpacker games base

as well as
inherit unpacker base games

however
inherit games unpacker base

will work.

And now... guess why the games herd made it a policy to always inherit
games.eclass last. Because of the unpredictability of eclasses and that
they may randomly add exported phase functions. It's a bit paranoid, but
understandable, since we don't have any real rules here.

So in the end 3 eclasses all tell you "inherit me last! really!". Good
luck with figuring out how to make a gnome game with python and multilib
support work together. I can predict the days such a review would take
in #gentoo-sunrise. Not less than 3.

Re: [gentoo-dev] rfc: calling all eclass phase functions by default

[Chris Reffett]

On 8/18/2014 8:56 AM, hasufell wrote:
> hasufell:
>>
>> Even more interesting... you can work around this by inheriting
>> base.eclass explicitly before e.g. unpacker.eclass, something like
>>
>> inherit base unpacker games
>>
>> => unpacker_src_unpack() is carried out by default (and the ebuild
>> breaks if someone thinks the base.eclass is useless and removes it)
>>
>> inherit unpacker games
>>
>> => unpacker_src_unpack is not carried out by default although
>> games.eclass does not directly overwrite it
>>
>> If you think any of this is sensible...
>>
> 
> Almost forgot, of course this does not work if you expect
> unpacker_src_unpacker() to run:
> inherit unpacker games base
> 
> as well as
> inherit unpacker base games
> 
> however
> inherit games unpacker base
> 
> will work.
> 
> And now... guess why the games herd made it a policy to always inherit
> games.eclass last. Because of the unpredictability of eclasses and that
> they may randomly add exported phase functions. It's a bit paranoid, but
> understandable, since we don't have any real rules here.
> 
> So in the end 3 eclasses all tell you "inherit me last! really!". Good
> luck with figuring out how to make a gnome game with python and multilib
> support work together. I can predict the days such a review would take
> in #gentoo-sunrise. Not less than 3.
> 
Would it be feasible to add a repoman check for situations like this,
where the behavior of a phase is dependent on inherit order? If so, it
seems reasonable to me to require explicit calls to eclass functions in
these cases to make it clear what's being called when.

Chris Reffett

Re: [gentoo-dev] rfc: calling all eclass phase functions by default

[hasufell]

Chris Reffett:
> On 8/18/2014 8:56 AM, hasufell wrote:
>> hasufell:
>>>
>>> Even more interesting... you can work around this by inheriting
>>> base.eclass explicitly before e.g. unpacker.eclass, something like
>>>
>>> inherit base unpacker games
>>>
>>> => unpacker_src_unpack() is carried out by default (and the ebuild
>>> breaks if someone thinks the base.eclass is useless and removes it)
>>>
>>> inherit unpacker games
>>>
>>> => unpacker_src_unpack is not carried out by default although
>>> games.eclass does not directly overwrite it
>>>
>>> If you think any of this is sensible...
>>>
>>
>> Almost forgot, of course this does not work if you expect
>> unpacker_src_unpacker() to run:
>> inherit unpacker games base
>>
>> as well as
>> inherit unpacker base games
>>
>> however
>> inherit games unpacker base
>>
>> will work.
>>
>> And now... guess why the games herd made it a policy to always inherit
>> games.eclass last. Because of the unpredictability of eclasses and that
>> they may randomly add exported phase functions. It's a bit paranoid, but
>> understandable, since we don't have any real rules here.
>>
>> So in the end 3 eclasses all tell you "inherit me last! really!". Good
>> luck with figuring out how to make a gnome game with python and multilib
>> support work together. I can predict the days such a review would take
>> in #gentoo-sunrise. Not less than 3.
>>
> Would it be feasible to add a repoman check for situations like this,
> where the behavior of a phase is dependent on inherit order? If so, it
> seems reasonable to me to require explicit calls to eclass functions in
> these cases to make it clear what's being called when.
> 

It would be feasible to turn off this implicit behavior. Adding repoman
checks for these kind of things will just yell out the pitfalls instead
of actually fixing them. People will still have to read through all the
eclasses in order to know what to do. We need something you can rely on.
EXPORT_FUNCTIONS line in the ebuild is currently not reliable, because
we allow indirect exports.

So we are back to the last proposal of williamh.

Re: [gentoo-dev] rfc: calling all eclass phase functions by default

[Rich Freeman]

On Mon, Aug 18, 2014 at 8:21 AM, Sergey Popov <pinkbyte@gentoo.org> wrote:
> 18.08.2014 14:44, Rich Freeman пишет:
>> On Mon, Aug 18, 2014 at 4:54 AM, Sergey Popov <pinkbyte@gentoo.org> wrote:
>>> 17.08.2014 01:54, William Hubbs пишет:
>>>>
>>>> # Foo and bar both have src_unpack and src_install functions.
>>>> # we want foo's src_unpack and bar's src_install:
>>>>
>>>> ECLASS_PHASES="foo_src_unpack
>>>>       bar_src_install"
>>>
>>> You have my strong opposition on such change as well. It will turn
>>> ebuilds into unreadable and undpredictable mess, please do not do that
>>>
>>
>> I'm not sure I follow your complaint.  He is talking about adding one
>> line to an ebuild.  I'm not sure how that is unreadable, and the
>> algorithm you quoted looks fairly predictable to me as well.
>>
>> Certainly it is less convenient than not having to do anything to pull
>> in eclass-defined phase functions, and it requires ebuilds to be
>> updated when eclasses are updated to add new phase functions.  That
>> could be problematic for cases like KDE/X11/etc where you have a large
>> collection of short ebuilds with all the logic in an eclass.
>>
>> I just want to make sure I'm understanding your concern in case there
>> is a new issue being raised.
>
> What's bad with overriding needed functions and saying which exported
> functions(from eclasses) to execute and in which order?
>
> Is this approach flaw? In which ways?

Ok, I was misunderstanding your original comment.  You're advocating
just having ebuilds explicitly call phase functions from eclasses
then, and not automatically inheriting them?  Your objection was to
the ECLASS_PHASES concept, and not to the principle of eliminating
automatic inheritance of phase functions?

Please let me know if I'm still misunderstanding you.  There are a lot
of potential ways to go with this so it isn't always clear what part
of a proposal is being objected to.

Rich

Re: [gentoo-dev] rfc: calling all eclass phase functions by default

[Rich Freeman]

On Mon, Aug 18, 2014 at 8:56 AM, hasufell <hasufell@gentoo.org> wrote:
>
> So in the end 3 eclasses all tell you "inherit me last! really!". Good
> luck with figuring out how to make a gnome game with python and multilib
> support work together. I can predict the days such a review would take
> in #gentoo-sunrise. Not less than 3.
>

And hence my point about when not to use wraparound eclasses:
We get into trouble when we use wraparound eclasses:
1.  With a broad assortment of ebuilds.  (games, check, multilib,
check, python, check)
2.  With many individual ebuild maintainers. (games, check (though
we've apparently been trying hard to reduce the number of people
willing to deal with them), multilib, check, python, check)
3.  For things like languages or genres. (games, check, python, check)
4.  In situations where multiple wraparound eclasses could apply. (the
example above - really just the result of #1-3)

Really, gnome is the only thing in your list that might justify a
wraparound eclass.  Even then, I'd use it more for the case of the
hundred packages or so that are part of gnome proper, and not every
package that somehow integrates with gnome where gnome isn't the
upstream. Gnome really should have a utility eclass for broad package
support, augmented by a wraparound (that perhaps uses the utility
eclass) for the large number of gnome-proper packages.

Thanks for your real-world examples.

--
Rich

Re: [gentoo-dev] rfc: calling all eclass phase functions by default

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1507 bytes --]

Dnia 2014-08-18, o godz. 09:22:46
Chris Reffett <creffett@gentoo.org> napisał(a):

> On 8/18/2014 8:56 AM, hasufell wrote:
> > Almost forgot, of course this does not work if you expect
> > unpacker_src_unpacker() to run:
> > inherit unpacker games base
> > 
> > as well as
> > inherit unpacker base games
> > 
> > however
> > inherit games unpacker base
> > 
> > will work.
> > 
> > And now... guess why the games herd made it a policy to always inherit
> > games.eclass last. Because of the unpredictability of eclasses and that
> > they may randomly add exported phase functions. It's a bit paranoid, but
> > understandable, since we don't have any real rules here.
> > 
> > So in the end 3 eclasses all tell you "inherit me last! really!". Good
> > luck with figuring out how to make a gnome game with python and multilib
> > support work together. I can predict the days such a review would take
> > in #gentoo-sunrise. Not less than 3.
> > 
> Would it be feasible to add a repoman check for situations like this,
> where the behavior of a phase is dependent on inherit order? If so, it
> seems reasonable to me to require explicit calls to eclass functions in
> these cases to make it clear what's being called when.

Right now, we have no kind of repoman for eclasses. If you have time to
work on such a thing, please do. Otherwise, all we can do is put more
checks in ebuilds but that triggers the warning for the wrong people...

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] rfc: calling all eclass phase functions by default

[Chris Reffett]

On August 18, 2014 11:11:56 AM EDT, "Michał Górny" <mgorny@gentoo.org> wrote:
>Dnia 2014-08-18, o godz. 09:22:46
>Chris Reffett <creffett@gentoo.org> napisał(a):
>
>> On 8/18/2014 8:56 AM, hasufell wrote:
>> > Almost forgot, of course this does not work if you expect
>> > unpacker_src_unpacker() to run:
>> > inherit unpacker games base
>> > 
>> > as well as
>> > inherit unpacker base games
>> > 
>> > however
>> > inherit games unpacker base
>> > 
>> > will work.
>> > 
>> > And now... guess why the games herd made it a policy to always
>inherit
>> > games.eclass last. Because of the unpredictability of eclasses and
>that
>> > they may randomly add exported phase functions. It's a bit
>paranoid, but
>> > understandable, since we don't have any real rules here.
>> > 
>> > So in the end 3 eclasses all tell you "inherit me last! really!".
>Good
>> > luck with figuring out how to make a gnome game with python and
>multilib
>> > support work together. I can predict the days such a review would
>take
>> > in #gentoo-sunrise. Not less than 3.
>> > 
>> Would it be feasible to add a repoman check for situations like this,
>> where the behavior of a phase is dependent on inherit order? If so,
>it
>> seems reasonable to me to require explicit calls to eclass functions
>in
>> these cases to make it clear what's being called when.
>
>Right now, we have no kind of repoman for eclasses. If you have time to
>work on such a thing, please do. Otherwise, all we can do is put more
>checks in ebuilds but that triggers the warning for the wrong people...

I was thinking more ebuild-side. Example: my ebuild inherits both cmake-utils and games eclasses, and I don't explicitly define src_compile, repoman full could pop up a warning like "src_compile is ambiguous between cmake-utils_src_compile and games_src_compile, please explicitly define this phase and call the appropriate eclass function." I imagine that this would pop up a lot of warnings, but I feel like it would improve readability and make it explicit what should be going on where. I concede that it could make a lot more boilerplate code in ebuilds, so that's a potential issue, mostly just throwing out an idea here.

Chris Reffett

Re: [gentoo-dev] rfc: calling all eclass phase functions by default

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2811 bytes --]

Dnia 2014-08-18, o godz. 15:37:26
Chris Reffett <creffett@gentoo.org> napisał(a):

> 
> 
> On August 18, 2014 11:11:56 AM EDT, "Michał Górny" <mgorny@gentoo.org> wrote:
> >Dnia 2014-08-18, o godz. 09:22:46
> >Chris Reffett <creffett@gentoo.org> napisał(a):
> >
> >> On 8/18/2014 8:56 AM, hasufell wrote:
> >> > Almost forgot, of course this does not work if you expect
> >> > unpacker_src_unpacker() to run:
> >> > inherit unpacker games base
> >> > 
> >> > as well as
> >> > inherit unpacker base games
> >> > 
> >> > however
> >> > inherit games unpacker base
> >> > 
> >> > will work.
> >> > 
> >> > And now... guess why the games herd made it a policy to always
> >inherit
> >> > games.eclass last. Because of the unpredictability of eclasses and
> >that
> >> > they may randomly add exported phase functions. It's a bit
> >paranoid, but
> >> > understandable, since we don't have any real rules here.
> >> > 
> >> > So in the end 3 eclasses all tell you "inherit me last! really!".
> >Good
> >> > luck with figuring out how to make a gnome game with python and
> >multilib
> >> > support work together. I can predict the days such a review would
> >take
> >> > in #gentoo-sunrise. Not less than 3.
> >> > 
> >> Would it be feasible to add a repoman check for situations like this,
> >> where the behavior of a phase is dependent on inherit order? If so,
> >it
> >> seems reasonable to me to require explicit calls to eclass functions
> >in
> >> these cases to make it clear what's being called when.
> >
> >Right now, we have no kind of repoman for eclasses. If you have time to
> >work on such a thing, please do. Otherwise, all we can do is put more
> >checks in ebuilds but that triggers the warning for the wrong people...
> 
> I was thinking more ebuild-side. Example: my ebuild inherits both cmake-utils and games eclasses, and I don't explicitly define src_compile, repoman full could pop up a warning like "src_compile is ambiguous between cmake-utils_src_compile and games_src_compile, please explicitly define this phase and call the appropriate eclass function." I imagine that this would pop up a lot of warnings, but I feel like it would improve readability and make it explicit what should be going on where. I concede that it could make a lot more boilerplate code in ebuilds, so that's a potential issue, mostly just throwing out an idea here.

People will be unhappy about it. Well, I will be unhappy about it.
Maybe if we had different eclasses... but with what we have now, I'd
rather order eclasses properly than redefine each phase.

Maybe we could agree on something lighter, say, that src_configure()
through src_install() should come from the same eclass.

Also, please wrap your mails.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] rfc: calling all eclass phase functions by default

[hasufell]

Chris Reffett:
> 
> 
> On August 18, 2014 11:11:56 AM EDT, "Michał Górny" <mgorny@gentoo.org> wrote:
>> Dnia 2014-08-18, o godz. 09:22:46
>> Chris Reffett <creffett@gentoo.org> napisał(a):
>>
>>> On 8/18/2014 8:56 AM, hasufell wrote:
>>>> Almost forgot, of course this does not work if you expect
>>>> unpacker_src_unpacker() to run:
>>>> inherit unpacker games base
>>>>
>>>> as well as
>>>> inherit unpacker base games
>>>>
>>>> however
>>>> inherit games unpacker base
>>>>
>>>> will work.
>>>>
>>>> And now... guess why the games herd made it a policy to always
>> inherit
>>>> games.eclass last. Because of the unpredictability of eclasses and
>> that
>>>> they may randomly add exported phase functions. It's a bit
>> paranoid, but
>>>> understandable, since we don't have any real rules here.
>>>>
>>>> So in the end 3 eclasses all tell you "inherit me last! really!".
>> Good
>>>> luck with figuring out how to make a gnome game with python and
>> multilib
>>>> support work together. I can predict the days such a review would
>> take
>>>> in #gentoo-sunrise. Not less than 3.
>>>>
>>> Would it be feasible to add a repoman check for situations like this,
>>> where the behavior of a phase is dependent on inherit order? If so,
>> it
>>> seems reasonable to me to require explicit calls to eclass functions
>> in
>>> these cases to make it clear what's being called when.
>>
>> Right now, we have no kind of repoman for eclasses. If you have time to
>> work on such a thing, please do. Otherwise, all we can do is put more
>> checks in ebuilds but that triggers the warning for the wrong people...
> 
> I was thinking more ebuild-side. Example: my ebuild inherits both cmake-utils and games eclasses, and I don't explicitly define src_compile, repoman full could pop up a warning like "src_compile is ambiguous between cmake-utils_src_compile and games_src_compile, please explicitly define this phase and call the appropriate eclass function." I imagine that this would pop up a lot of warnings, but I feel like it would improve readability and make it explicit what should be going on where. I concede that it could make a lot more boilerplate code in ebuilds, so that's a potential issue, mostly just throwing out an idea here.
> 
> Chris Reffett
> 

I don't want to code against warning tools, but against a reliable API.

That said, EXPORT_FUNCTIONS in eclasses should be definite and
non-recursive. Currently, people have to track down the actual exported
functions themselves through the endless list of indirect inheritance
which may:
* randomly change
* be highly dependant on the inherit order in the eclass and of those
indirectly inheriting others...

So to pick up the proposal again, I think this could make sense:
* disable exported functions from indirectly inherited eclasses
* have eclass authors do these things explicitly, so they have to export
ALL functions themselves and may have to adjust their eclasses, as in:
games_src_prepare() { base_src_prepare ; } (I know that base.eclass is
deprecated, this is an example)
* include the exported functions automatically in the generated eclass
manpages

Re: [gentoo-dev] rfc: calling all eclass phase functions by default

[Sergey Popov]

[-- Attachment #1: Type: text/plain, Size: 1946 bytes --]

18.08.2014 16:56, hasufell пишет:
> hasufell:
>>
>> Even more interesting... you can work around this by inheriting
>> base.eclass explicitly before e.g. unpacker.eclass, something like
>>
>> inherit base unpacker games
>>
>> => unpacker_src_unpack() is carried out by default (and the ebuild
>> breaks if someone thinks the base.eclass is useless and removes it)
>>
>> inherit unpacker games
>>
>> => unpacker_src_unpack is not carried out by default although
>> games.eclass does not directly overwrite it
>>
>> If you think any of this is sensible...
>>
> 
> Almost forgot, of course this does not work if you expect
> unpacker_src_unpacker() to run:
> inherit unpacker games base
> 
> as well as
> inherit unpacker base games
> 
> however
> inherit games unpacker base
> 
> will work.
> 
> And now... guess why the games herd made it a policy to always inherit
> games.eclass last. Because of the unpredictability of eclasses and that
> they may randomly add exported phase functions. It's a bit paranoid, but
> understandable, since we don't have any real rules here.
> 
> So in the end 3 eclasses all tell you "inherit me last! really!". Good
> luck with figuring out how to make a gnome game with python and multilib
> support work together. I can predict the days such a review would take
> in #gentoo-sunrise. Not less than 3.
> 

As i said early - always override necessary functions if you want
non-default behaviour. Proposed solution with variable just add syntax
sugar, doing the same thing.

As for you as sunrise reviewer. I am member of proxy maintainers, i am
also reviewing ebuilds from users. And they usually understands inherit
logic very well, even in non-trivial cases.

So, your expirience just differs with mine, not more, not less.

-- 
Best regards, Sergey Popov
Gentoo developer
Gentoo Desktop Effects project lead
Gentoo Proxy maintainers project lead


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 538 bytes --]

Re: [gentoo-dev] rfc: calling all eclass phase functions by default

[Sergey Popov]

[-- Attachment #1: Type: text/plain, Size: 3727 bytes --]

19.08.2014 00:23, hasufell пишет:
> Chris Reffett:
>>
>>
>> On August 18, 2014 11:11:56 AM EDT, "Michał Górny" <mgorny@gentoo.org> wrote:
>>> Dnia 2014-08-18, o godz. 09:22:46
>>> Chris Reffett <creffett@gentoo.org> napisał(a):
>>>
>>>> On 8/18/2014 8:56 AM, hasufell wrote:
>>>>> Almost forgot, of course this does not work if you expect
>>>>> unpacker_src_unpacker() to run:
>>>>> inherit unpacker games base
>>>>>
>>>>> as well as
>>>>> inherit unpacker base games
>>>>>
>>>>> however
>>>>> inherit games unpacker base
>>>>>
>>>>> will work.
>>>>>
>>>>> And now... guess why the games herd made it a policy to always
>>> inherit
>>>>> games.eclass last. Because of the unpredictability of eclasses and
>>> that
>>>>> they may randomly add exported phase functions. It's a bit
>>> paranoid, but
>>>>> understandable, since we don't have any real rules here.
>>>>>
>>>>> So in the end 3 eclasses all tell you "inherit me last! really!".
>>> Good
>>>>> luck with figuring out how to make a gnome game with python and
>>> multilib
>>>>> support work together. I can predict the days such a review would
>>> take
>>>>> in #gentoo-sunrise. Not less than 3.
>>>>>
>>>> Would it be feasible to add a repoman check for situations like this,
>>>> where the behavior of a phase is dependent on inherit order? If so,
>>> it
>>>> seems reasonable to me to require explicit calls to eclass functions
>>> in
>>>> these cases to make it clear what's being called when.
>>>
>>> Right now, we have no kind of repoman for eclasses. If you have time to
>>> work on such a thing, please do. Otherwise, all we can do is put more
>>> checks in ebuilds but that triggers the warning for the wrong people...
>>
>> I was thinking more ebuild-side. Example: my ebuild inherits both cmake-utils and games eclasses, and I don't explicitly define src_compile, repoman full could pop up a warning like "src_compile is ambiguous between cmake-utils_src_compile and games_src_compile, please explicitly define this phase and call the appropriate eclass function." I imagine that this would pop up a lot of warnings, but I feel like it would improve readability and make it explicit what should be going on where. I concede that it could make a lot more boilerplate code in ebuilds, so that's a potential issue, mostly just throwing out an idea here.
>>
>> Chris Reffett
>>
> 
> I don't want to code against warning tools, but against a reliable API.
> 
> That said, EXPORT_FUNCTIONS in eclasses should be definite and
> non-recursive. Currently, people have to track down the actual exported
> functions themselves through the endless list of indirect inheritance
> which may:
> * randomly change
> * be highly dependant on the inherit order in the eclass and of those
> indirectly inheriting others...
> 
> So to pick up the proposal again, I think this could make sense:
> * disable exported functions from indirectly inherited eclasses
> * have eclass authors do these things explicitly, so they have to export
> ALL functions themselves and may have to adjust their eclasses, as in:
> games_src_prepare() { base_src_prepare ; } (I know that base.eclass is
> deprecated, this is an example)
> * include the exported functions automatically in the generated eclass
> manpages
> 

That's proposal make more sense. Even if it will bring such short and
wrapper functions, it may be more compliant and ease things for PM itself.

Usually i do not agree with your proposals, but that's one in it's
current definition is really get +1 from me!

-- 
Best regards, Sergey Popov
Gentoo developer
Gentoo Desktop Effects project lead
Gentoo Proxy maintainers project lead


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 538 bytes --]