[gentoo-dev] new `usex` helper

[gentoo-dev] new `usex` helper

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 442 bytes --]

i keep writing little helpers like this in ebuilds:
usex() { use $1 && echo ${2:-yes} || echo ${3:-no} ; }

this is so i can do:
	export some_var=$(usex some_flag)
and get it set to "yes" or "no"

or if i want something a little different, i can do:
	export some_var=$(usex some_flag true false)
	export some_var=$(usex some_flag y n)

useful enough for EAPI ?  or should i just stick it into eutils.eclass ?  OR 
BOTH !?
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] new `usex` helper

[Alec Warner]

On Tue, Sep 13, 2011 at 2:56 PM, Mike Frysinger <vapier@gentoo.org> wrote:
> i keep writing little helpers like this in ebuilds:
> usex() { use $1 && echo ${2:-yes} || echo ${3:-no} ; }

usex...you naughty boy.

>
> this is so i can do:
>        export some_var=$(usex some_flag)
> and get it set to "yes" or "no"

If the intent is to use it for logic:

export some_var=$(usex some_flag)

if [[ $some_var == yes ]]; then
 # buttsex
fi

Then I recommend making true / false the default and then doing

if $some_var; then
  # buttsex
fi

If you are using it more like use_enable then...thats ok I guess ;p

-A

>
> or if i want something a little different, i can do:
>        export some_var=$(usex some_flag true false)
>        export some_var=$(usex some_flag y n)
>
> useful enough for EAPI ?  or should i just stick it into eutils.eclass ?  OR
> BOTH !?
> -mike
>

Re: [gentoo-dev] new `usex` helper

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 707 bytes --]

On Tuesday, September 13, 2011 18:01:25 Alec Warner wrote:
> On Tue, Sep 13, 2011 at 2:56 PM, Mike Frysinger wrote:
> > this is so i can do:
> >        export some_var=$(usex some_flag)
> > and get it set to "yes" or "no"
> 
> If the intent is to use it for logic:
> 
> export some_var=$(usex some_flag)
> 
> if [[ $some_var == yes ]]; then
>  # buttsex
> fi

that is not the intent

> Then I recommend making true / false the default and then doing
> 
> if $some_var; then
>   # buttsex
> fi

the point is to use it to construct vars that get passed to scripts like econf 
or programs like emake

	ac_cv_some_header=$(usex foo) \
	econf ...

	emake USE_POOP=$(usex poo)
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] new `usex` helper

[Brian Harring]

On Tue, Sep 13, 2011 at 06:13:10PM -0400, Mike Frysinger wrote:
> On Tuesday, September 13, 2011 18:01:25 Alec Warner wrote:
> > On Tue, Sep 13, 2011 at 2:56 PM, Mike Frysinger wrote:
> > > this is so i can do:
> > >        export some_var=$(usex some_flag)
> > > and get it set to "yes" or "no"
> > 
> > If the intent is to use it for logic:
> > 
> > export some_var=$(usex some_flag)
> > 
> > if [[ $some_var == yes ]]; then
> >  # buttsex
> > fi
> 
> that is not the intent
> 
> > Then I recommend making true / false the default and then doing
> > 
> > if $some_var; then
> >   # buttsex
> > fi
> 
> the point is to use it to construct vars that get passed to scripts like econf 
> or programs like emake
> 
> 	ac_cv_some_header=$(usex foo) \
> 	econf ...
> 
> 	emake USE_POOP=$(usex poo)

Making it overridable seems wiser-

usex() {
	local flag="$1"
	local tval=${2-yes}
	local fval=${3-no}
	if use $flag; then
		echo "${tval}"
	else
		echo "${fval}"
	fi
}

While a bit longer, we likely can gut most of the use_* logic to 
use that, and it makes it easier to deal w/ the situations where a 
configure's options always assume --enable-blah thus don't export the 
option, but *do* export a --disable-blah.

That way we can shift away from
$(use blah && use_with blah)
to
$(usex blah --with-blah '')

Or that's the intent at least.
~brian	

Re: [gentoo-dev] new `usex` helper

[Donnie Berkholz]

[-- Attachment #1: Type: text/plain, Size: 354 bytes --]

On 17:56 Tue 13 Sep     , Mike Frysinger wrote:
> useful enough for EAPI ?  or should i just stick it into eutils.eclass 
> ?  OR BOTH !?

I prefer to avoid EAPI whenever possible, as it just makes things slower 
and more complex.

-- 
Thanks,
Donnie

Donnie Berkholz
Council Member / Sr. Developer
Gentoo Linux
Blog: http://dberkholz.com

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] new `usex` helper

[Brian Harring]

On Tue, Sep 13, 2011 at 09:02:28PM -0500, Donnie Berkholz wrote:
> On 17:56 Tue 13 Sep     , Mike Frysinger wrote:
> > useful enough for EAPI ?  or should i just stick it into eutils.eclass 
> > ?  OR BOTH !?
> 
> I prefer to avoid EAPI whenever possible, as it just makes things slower 
> and more complex.

Exactly the wrong approach; it winds up with master 
repositories/overlays cloning the functionality all over the damn 
place.

Do both.  Specifically, do it right- get it into the format (so 
it can be relied on and isn't adhoc BS that proceeded EAPI), then 
push a compat implementation into eclasses for usage by EAPI's less 
than 5.

You get the feature now, and it's sorted properly for moving forward.  
Not that complex.

And yes I'm tired of people bitching about compatibility.  I recall a 
similar group of folk bitching about the lack of compatibility prior 
to EAPI... it's annoying.
</rant>

~brian

Re: [gentoo-dev] new `usex` helper

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 772 bytes --]

On Tuesday, September 13, 2011 19:08:09 Brian Harring wrote:
> Making it overridable seems wiser-
> 
> usex() {
> 	local flag="$1"
> 	local tval=${2-yes}
> 	local fval=${3-no}
> 	if use $flag; then
> 		echo "${tval}"
> 	else
> 		echo "${fval}"
> 	fi
> }

i dont get it.  mine already does exactly this, just in one line.
usex() { use $1 && echo ${2:-yes} || echo ${3:-no} ; }

> While a bit longer, we likely can gut most of the use_* logic to
> use that, and it makes it easier to deal w/ the situations where a
> configure's options always assume --enable-blah thus don't export the
> option, but *do* export a --disable-blah.

yeah, i thought about replacing use_{with,enable} with usex, but we'd have to 
extend usex() a little bit more
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] new `usex` helper

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 392 bytes --]

On Tuesday, September 13, 2011 22:02:28 Donnie Berkholz wrote:
> On 17:56 Tue 13 Sep     , Mike Frysinger wrote:
> > useful enough for EAPI ?  or should i just stick it into eutils.eclass
> > ?  OR BOTH !?
> 
> I prefer to avoid EAPI whenever possible, as it just makes things slower
> and more complex.

should be easy to do both:
[[ ${EAPI} == [01234] ]] && usex() { ... }
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] new `usex` helper

[Brian Harring]

On Tue, Sep 13, 2011 at 10:45:27PM -0400, Mike Frysinger wrote:
> On Tuesday, September 13, 2011 19:08:09 Brian Harring wrote:
> > Making it overridable seems wiser-
> > 
> > usex() {
> > 	local flag="$1"
> > 	local tval=${2-yes}
> > 	local fval=${3-no}
> > 	if use $flag; then
> > 		echo "${tval}"
> > 	else
> > 		echo "${fval}"
> > 	fi
> > }
> 
> i dont get it.  mine already does exactly this, just in one line.
> usex() { use $1 && echo ${2:-yes} || echo ${3:-no} ; }

Err.  Mines prettier?

*cough*

Only real difference is ${2:-yes} versus ${2-yes}; the latter should 
be used so that `usex flag '' '--disable-some-feature'` is usable.


> > While a bit longer, we likely can gut most of the use_* logic to
> > use that, and it makes it easier to deal w/ the situations where a
> > configure's options always assume --enable-blah thus don't export the
> > option, but *do* export a --disable-blah.
> 
> yeah, i thought about replacing use_{with,enable} with usex, but we'd have to 
> extend usex() a little bit more

Only extension I can think of is adding a prefix/postfix... which 
frankly seems a bit too much.  Anything else you were looking for?

To be clear, I'm more interested in this from the standpoint of making 
econf invocations simpler- simplifying use_enable/use_with in the PM 
isn't a huge concern to me since they're already pretty bloody 
straightforward at this point.

~brian

Re: [gentoo-dev] new `usex` helper

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 1443 bytes --]

On Tuesday, September 13, 2011 23:04:06 Brian Harring wrote:
> On Tue, Sep 13, 2011 at 10:45:27PM -0400, Mike Frysinger wrote:
> > On Tuesday, September 13, 2011 19:08:09 Brian Harring wrote:
> > > While a bit longer, we likely can gut most of the use_* logic to
> > > use that, and it makes it easier to deal w/ the situations where a
> > > configure's options always assume --enable-blah thus don't export the
> > > option, but *do* export a --disable-blah.
> > 
> > yeah, i thought about replacing use_{with,enable} with usex, but we'd
> > have to extend usex() a little bit more
> 
> Only extension I can think of is adding a prefix/postfix... which
> frankly seems a bit too much.  Anything else you were looking for?

i dont think the postfix is needed for use_{enable,with}:
usex() { use $1 && echo ${2:-yes} || echo ${3:-no} ; }
use_enable() { usex $1 --enable-${2:-$1}${3:+=}$3 --disable-${2:-$1} ; }
use_with() { usex $1 --with-${2:-$1}${3:+=}$3 --without-${2:-$1} ; }

although adding it to usex is cheap, simplifies the other helpers a little, 
and might get used in more creative ways i cant imagine right now:
usex() { use $1 && echo ${2:-yes}$4 || echo ${3:-no}$5 ; }
use_enable() { usex $1 --{en,dis}able-${2:-$1} "${3:+=}$3" ; }
use_with() { usex $1 --with{,out}-${2:-$1} "${3:+=}$3"; }

so unless there's any other feedback, i'll open an EAPI bug on the topic and 
merge it to eutils.eclass
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] new `usex` helper

[Brian Harring]

On Tue, Sep 13, 2011 at 11:43:48PM -0400, Mike Frysinger wrote:
> On Tuesday, September 13, 2011 23:04:06 Brian Harring wrote:
> > On Tue, Sep 13, 2011 at 10:45:27PM -0400, Mike Frysinger wrote:
> > > On Tuesday, September 13, 2011 19:08:09 Brian Harring wrote:
> > > > While a bit longer, we likely can gut most of the use_* logic to
> > > > use that, and it makes it easier to deal w/ the situations where a
> > > > configure's options always assume --enable-blah thus don't export the
> > > > option, but *do* export a --disable-blah.
> > > 
> > > yeah, i thought about replacing use_{with,enable} with usex, but we'd
> > > have to extend usex() a little bit more
> > 
> > Only extension I can think of is adding a prefix/postfix... which
> > frankly seems a bit too much.  Anything else you were looking for?
> 
> i dont think the postfix is needed for use_{enable,with}:
> usex() { use $1 && echo ${2:-yes} || echo ${3:-no} ; }
> use_enable() { usex $1 --enable-${2:-$1}${3:+=}$3 --disable-${2:-$1} ; }
> use_with() { usex $1 --with-${2:-$1}${3:+=}$3 --without-${2:-$1} ; }
> 
> although adding it to usex is cheap, simplifies the other helpers a little, 
> and might get used in more creative ways i cant imagine right now:
> usex() { use $1 && echo ${2:-yes}$4 || echo ${3:-no}$5 ; }
> use_enable() { usex $1 --{en,dis}able-${2:-$1} "${3:+=}$3" ; }
> use_with() { usex $1 --with{,out}-${2:-$1} "${3:+=}$3"; }
> 
> so unless there's any other feedback, i'll open an EAPI bug on the topic and 
> merge it to eutils.eclass

Might want to get EAPI sorted before it lands in eutils...

Either way, have at it, preferably less dense implementations however 
;)
~harring

Re: [gentoo-dev] new `usex` helper

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 2356 bytes --]

On Tue, 13 Sep 2011 21:02:28 -0500
Donnie Berkholz <dberkholz@gentoo.org> wrote:
> On 17:56 Tue 13 Sep     , Mike Frysinger wrote:
> > useful enough for EAPI ?  or should i just stick it into
> > eutils.eclass ?  OR BOTH !?
> 
> I prefer to avoid EAPI whenever possible, as it just makes things
> slower and more complex.

Sticking it in an EAPI *shouldn't* be slow and more complex. There are
three reasons why it is, and they should all be within Gentoo's
ability to solve.

The first reason is that when we did what was then called EAPI 3,
several Council members refused to put in more than one hour's work
every month. To get an EAPI out quickly, we need Council members who
are prepared to do a bit of homework, and to read proposals before a
meeting and to comment on mailing lists rather than only bringing up
questions (most of which have already been answered on the lists) at
meetings. That shouldn't be too much to ask, and if it is, Council
members should be prepared to delegate.

The second is that it's impossible to get an accurate estimate from the
Portage people for how long it will take to implement something. We
were assured before the then-EAPI-3 proposals were submitted to the
Council that all would be easy and quick to implement in Portage. We
were told after approval that implementation would take a month, when
it took a year to get just partial implementations of some features.
This needs to be addressed -- to a certain extent we can drop features,
but EAPI 4 currently has nasty problems (prefix flag needs to be in
IUSE, and people are refusing to do that) due to one of two
interdependent features being dropped (strict IUSE) without the
standard wording for the other (use dependency defaults) being changed.

The third is that there are a few people intent on preventing any new
EAPI from ever happening. The solution here is to tell them that the
decision has been made, that Gentoo is going to use EAPIs and a
specification whether they like it or not, and that they can either help
or keep quiet. No-one has the time to deal with a small group of
individuals who pop up and yell "PMS sucks! EAPIs are bad! Portage is
reality! Code to an implementation not a standard!" every time anyone
asks for a new feature.

None of this should be difficult.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] new `usex` helper

[Ulrich Mueller]

>>>>> On Tue, 13 Sep 2011, Mike Frysinger wrote:

> On Tuesday, September 13, 2011 19:08:09 Brian Harring wrote:
>> Making it overridable seems wiser-
>> 
>> usex() {
>> 	local flag="$1"
>> 	local tval=${2-yes}
>> 	local fval=${3-no}
>> 	if use $flag; then
>> 		echo "${tval}"
>> 	else
>> 		echo "${fval}"
>> 	fi
>> }

Looks verbose. ;-)

> i dont get it.  mine already does exactly this, just in one line.
> usex() { use $1 && echo ${2:-yes} || echo ${3:-no} ; }

You should omit the colons though. ${2-yes} and ${3-no} will allow for
an explicit empty string as argument, whereas the :- variants won't.

Also quotes around the echo arguments can't harm.

Ulrich

Re: [gentoo-dev] new `usex` helper

[Mike Frysinger]

On Wed, Sep 14, 2011 at 02:02, Ulrich Mueller wrote:
> On Tue, 13 Sep 2011, Mike Frysinger wrote:
>> usex() { use $1 && echo ${2:-yes} || echo ${3:-no} ; }
>
> You should omit the colons though. ${2-yes} and ${3-no} will allow for
> an explicit empty string as argument, whereas the :- variants won't.

in my original use, i wanted the :- behavior.  but i can see how the -
behavior would be generally more flexible.
-mike

Re: [gentoo-dev] new `usex` helper

[Mike Frysinger]

landed https://bugs.gentoo.org/382963
-mike

Re: [gentoo-dev] new `usex` helper

[Donnie Berkholz]

[-- Attachment #1: Type: text/plain, Size: 709 bytes --]

On 19:14 Tue 13 Sep     , Brian Harring wrote:
> On Tue, Sep 13, 2011 at 09:02:28PM -0500, Donnie Berkholz wrote:
> > On 17:56 Tue 13 Sep     , Mike Frysinger wrote:
> > > useful enough for EAPI ?  or should i just stick it into eutils.eclass 
> > > ?  OR BOTH !?
> > 
> > I prefer to avoid EAPI whenever possible, as it just makes things slower 
> > and more complex.
> 
> Exactly the wrong approach; it winds up with master 
> repositories/overlays cloning the functionality all over the damn 
> place.

Why are people cloning anything if it's in eutils.eclass in gentoo-x86?

-- 
Thanks,
Donnie

Donnie Berkholz
Council Member / Sr. Developer
Gentoo Linux
Blog: http://dberkholz.com

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] new `usex` helper

[Donnie Berkholz]

[-- Attachment #1: Type: text/plain, Size: 1179 bytes --]

On 06:34 Wed 14 Sep     , Ciaran McCreesh wrote:
> On Tue, 13 Sep 2011 21:02:28 -0500
> Donnie Berkholz <dberkholz@gentoo.org> wrote:
> > On 17:56 Tue 13 Sep     , Mike Frysinger wrote:
> > > useful enough for EAPI ?  or should i just stick it into
> > > eutils.eclass ?  OR BOTH !?
> > 
> > I prefer to avoid EAPI whenever possible, as it just makes things
> > slower and more complex.
> 
> Sticking it in an EAPI *shouldn't* be slow and more complex. There are
> three reasons why it is, and they should all be within Gentoo's
> ability to solve.

[..reasons..]

I'm glad we agree about its current state, whatever the reasons. 

While I wish I could do something about them, I can't at all in some 
cases or quickly in others. No matter what, it's going to be a slow 
process to change them, and I'd like to keep things like this from 
blocking on that. I'm hoping to fix some of the council-related issues 
with an updated GLEP 39 (WIP), but even being on the council doesn't 
help much in making others focus on productive issues.


-- 
Thanks,
Donnie

Donnie Berkholz
Council Member / Sr. Developer
Gentoo Linux
Blog: http://dberkholz.com

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] new `usex` helper

[Brian Harring]

On Wed, Sep 14, 2011 at 02:16:41PM -0500, Donnie Berkholz wrote:
> On 19:14 Tue 13 Sep     , Brian Harring wrote:
> > On Tue, Sep 13, 2011 at 09:02:28PM -0500, Donnie Berkholz wrote:
> > > On 17:56 Tue 13 Sep     , Mike Frysinger wrote:
> > > > useful enough for EAPI ?  or should i just stick it into eutils.eclass 
> > > > ?  OR BOTH !?
> > > 
> > > I prefer to avoid EAPI whenever possible, as it just makes things slower 
> > > and more complex.
> > 
> > Exactly the wrong approach; it winds up with master 
> > repositories/overlays cloning the functionality all over the damn 
> > place.
> 
> Why are people cloning anything if it's in eutils.eclass in gentoo-x86?

There are more repositories than just gentoo-x86, and overlay is *not* 
the only configuration in use.

In the old days of the PM only handling a single overlay stack, what 
you're suggesting would be less heinous- heinous in detail, but 
pragmatic in reality.  These days it's a regressive approach- 
requiring everyone to slave gentoo-x86 isn't sane, nor is avoiding 
eapi (resulting in people having to duplicate code into each 
repository stack).

~harring

Re: [gentoo-dev] new `usex` helper

[Donnie Berkholz]

[-- Attachment #1: Type: text/plain, Size: 1667 bytes --]

On 17:29 Wed 14 Sep     , Brian Harring wrote:
> On Wed, Sep 14, 2011 at 02:16:41PM -0500, Donnie Berkholz wrote:
> > On 19:14 Tue 13 Sep     , Brian Harring wrote:
> > > On Tue, Sep 13, 2011 at 09:02:28PM -0500, Donnie Berkholz wrote:
> > > > On 17:56 Tue 13 Sep     , Mike Frysinger wrote:
> > > > > useful enough for EAPI ?  or should i just stick it into eutils.eclass 
> > > > > ?  OR BOTH !?
> > > > 
> > > > I prefer to avoid EAPI whenever possible, as it just makes things slower 
> > > > and more complex.
> > > 
> > > Exactly the wrong approach; it winds up with master 
> > > repositories/overlays cloning the functionality all over the damn 
> > > place.
> > 
> > Why are people cloning anything if it's in eutils.eclass in gentoo-x86?
> 
> There are more repositories than just gentoo-x86, and overlay is *not* 
> the only configuration in use.

Who else besides you is using any other configuration? Should we really 
give a crap about the 0.001% population with some weird setup when we're 
trying to improve things for the 99.999% one?

> In the old days of the PM only handling a single overlay stack, what 
> you're suggesting would be less heinous- heinous in detail, but 
> pragmatic in reality.  These days it's a regressive approach- 
> requiring everyone to slave gentoo-x86 isn't sane, nor is avoiding 
> eapi (resulting in people having to duplicate code into each 
> repository stack).

I don't know many people who aren't using gentoo-x86 or a repo that 
pulls in changes directly from it.

-- 
Thanks,
Donnie

Donnie Berkholz
Council Member / Sr. Developer
Gentoo Linux
Blog: http://dberkholz.com

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] new `usex` helper

[Brian Harring]

On Thu, Sep 15, 2011 at 10:00:19PM -0500, Donnie Berkholz wrote:
> On 17:29 Wed 14 Sep     , Brian Harring wrote:
> > On Wed, Sep 14, 2011 at 02:16:41PM -0500, Donnie Berkholz wrote:
> > > On 19:14 Tue 13 Sep     , Brian Harring wrote:
> > > > On Tue, Sep 13, 2011 at 09:02:28PM -0500, Donnie Berkholz wrote:
> > > > > On 17:56 Tue 13 Sep     , Mike Frysinger wrote:
> > > > > > useful enough for EAPI ?  or should i just stick it into eutils.eclass 
> > > > > > ?  OR BOTH !?
> > > > > 
> > > > > I prefer to avoid EAPI whenever possible, as it just makes things slower 
> > > > > and more complex.
> > > > 
> > > > Exactly the wrong approach; it winds up with master 
> > > > repositories/overlays cloning the functionality all over the damn 
> > > > place.
> > > 
> > > Why are people cloning anything if it's in eutils.eclass in gentoo-x86?
> > 
> > There are more repositories than just gentoo-x86, and overlay is *not* 
> > the only configuration in use.
> 
> Who else besides you is using any other configuration? Should we really 
> give a crap about the 0.001% population with some weird setup when we're 
> trying to improve things for the 99.999% one?

Specious argument; the point of controllable stacking was to avoid the 
issue of overlay's forcing their eclasses upon gentoo-x86 ebuilds 
(which may not support those modified eclasses) via the old 
PORTDIR_OVERLAY behaviour.  This is why multiple repositories have 
layout.conf master definitions- to explicitly mark that they 
require/consume a seperate repo.

What you're basically proposing is a variation of the "push format 
definitions into a central tree, and require everyone to use that 
central tree".  This discussion has come and gone; I say that being 
one of the folks who was *pushing for the repository solution*.  The 
problem there is it fundamentally enables (more forces) fragmentation.

Realistically I assume you're taking the stance "EAPI gets in the way, 
lets do away with it"- if so, well, out with it, and I'll dredge up 
the old logs/complaints that lead to EAPI.


> > In the old days of the PM only handling a single overlay stack, what 
> > you're suggesting would be less heinous- heinous in detail, but 
> > pragmatic in reality.  These days it's a regressive approach- 
> > requiring everyone to slave gentoo-x86 isn't sane, nor is avoiding 
> > eapi (resulting in people having to duplicate code into each 
> > repository stack).
> 
> I don't know many people who aren't using gentoo-x86 or a repo that 
> pulls in changes directly from it.

rephrase please; either you're saying "everyone uses gentoo-x86" which 
is sort of true (funtoo/chrome aren't necessarily riding HEAD/trunk 
however which means things can get ugly for derivative repository 
usage), but still ignores the situations where people have overlays 
w/ developmental eclasses that they need to selectively control where 
it overrides (which is where the notion of repo stacking comes into 
play).
~brian

Re: [gentoo-dev] new `usex` helper

[Donnie Berkholz]

[-- Attachment #1: Type: text/plain, Size: 2567 bytes --]

On 02:06 Fri 16 Sep     , Brian Harring wrote:
> Specious argument; the point of controllable stacking was to avoid the 
> issue of overlay's forcing their eclasses upon gentoo-x86 ebuilds 
> (which may not support those modified eclasses) via the old 
> PORTDIR_OVERLAY behaviour.  This is why multiple repositories have 
> layout.conf master definitions- to explicitly mark that they 
> require/consume a seperate repo.

So let me get this straight — instead, you want overlay users to 
maintain a copy of every eclass they use, which will almost 
automatically become outdated and stale because it won't track the 
gentoo-x86 version?

> What you're basically proposing is a variation of the "push format 
> definitions into a central tree, and require everyone to use that 
> central tree".  This discussion has come and gone; I say that being 
> one of the folks who was *pushing for the repository solution*.  The 
> problem there is it fundamentally enables (more forces) fragmentation.

I think Gentoo will always provide one or a set of "official" central 
repositories, that's pretty much the definition of a distribution. So 
yes, I'm saying that generally useful stuff could go into one of these 
repositories, which (in the multi-repo case) could be like the eclass 
metaphor for repos; others would depend on it implicitly or explicitly.

> Realistically I assume you're taking the stance "EAPI gets in the way, 
> lets do away with it"- if so, well, out with it, and I'll dredge up 
> the old logs/complaints that lead to EAPI.

I see EAPI as a nice thing for standardizing features that are 
implemented in the PM so they work identically across portage, pkgcore, 
and paludis. But I don't think that implementing things in the PM when 
they could go in an eclass is automatically the best choice. It 
dramatically slows down the speed of iteration, prototyping, and bug 
fixing.

> rephrase please; either you're saying "everyone uses gentoo-x86" which 
> is sort of true (funtoo/chrome aren't necessarily riding HEAD/trunk 
> however which means things can get ugly for derivative repository 
> usage), but still ignores the situations where people have overlays w/ 
> developmental eclasses that they need to selectively control where it 
> overrides (which is where the notion of repo stacking comes into 
> play).

I think I explained above, but let me know if that's not the case.

-- 
Thanks,
Donnie

Donnie Berkholz
Council Member / Sr. Developer
Gentoo Linux
Blog: http://dberkholz.com

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] new `usex` helper

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1024 bytes --]

On Fri, 16 Sep 2011 07:30:14 -0500
Donnie Berkholz <dberkholz@gentoo.org> wrote:

> > Realistically I assume you're taking the stance "EAPI gets in the
> > way, lets do away with it"- if so, well, out with it, and I'll
> > dredge up the old logs/complaints that lead to EAPI.
> 
> I see EAPI as a nice thing for standardizing features that are 
> implemented in the PM so they work identically across portage,
> pkgcore, and paludis. But I don't think that implementing things in
> the PM when they could go in an eclass is automatically the best
> choice. It dramatically slows down the speed of iteration,
> prototyping, and bug fixing.

What is more important is that it takes the code further from devs.
I like to see the code I use, and be able to do anything about it if
necessary. Not to see a spec and three different implementation, of
which two use random hacks which I can't do anything about unless I
start to implement PM-specific anti-hacks in my code.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] new `usex` helper

[Alec Warner]

On Fri, Sep 16, 2011 at 7:04 AM, Michał Górny <mgorny@gentoo.org> wrote:
> On Fri, 16 Sep 2011 07:30:14 -0500
> Donnie Berkholz <dberkholz@gentoo.org> wrote:
>
>> > Realistically I assume you're taking the stance "EAPI gets in the
>> > way, lets do away with it"- if so, well, out with it, and I'll
>> > dredge up the old logs/complaints that lead to EAPI.
>>
>> I see EAPI as a nice thing for standardizing features that are
>> implemented in the PM so they work identically across portage,
>> pkgcore, and paludis. But I don't think that implementing things in
>> the PM when they could go in an eclass is automatically the best
>> choice. It dramatically slows down the speed of iteration,
>> prototyping, and bug fixing.
>
> What is more important is that it takes the code further from devs.
> I like to see the code I use, and be able to do anything about it if
> necessary. Not to see a spec and three different implementation, of
> which two use random hacks which I can't do anything about unless I
> start to implement PM-specific anti-hacks in my code.

Just as an aside, every package mangler in Gentoo is open source. I
don't see why you can't 'see' the code it is using. Now you might say
'ahhh C++ makes my eyes bleed' (as an aside, go read versionator
eclass ;p) or 'eww portage is ugly' but every time I hear it I am less
convinced that it is a good excuse.

>
> --
> Best regards,
> Michał Górny
>

Re: [gentoo-dev] new `usex` helper

[Brian Harring]

On Fri, Sep 16, 2011 at 07:30:14AM -0500, Donnie Berkholz wrote:
> On 02:06 Fri 16 Sep     , Brian Harring wrote:
> > Specious argument; the point of controllable stacking was to avoid the 
> > issue of overlay's forcing their eclasses upon gentoo-x86 ebuilds 
> > (which may not support those modified eclasses) via the old 
> > PORTDIR_OVERLAY behaviour.  This is why multiple repositories have 
> > layout.conf master definitions- to explicitly mark that they 
> > require/consume a seperate repo.
> 
> So let me get this straight — instead, you want overlay users to 
> maintain a copy of every eclass they use, which will almost 
> automatically become outdated and stale because it won't track the 
> gentoo-x86 version?

Where did I say that?

layout.conf exists to allow repo's to explicitly state what they need- 
this means we can have individual overlay stacks, instead of having 
gentoo-x86, overlay1, overlay2, overlay3, with that as a single stack 
(including eclass lookup), it can be broken out as individual stacks.  

This limits the eclass affect for a repo to just what is explicitly 
configured.  This is a good thing.  This is controllable in addition.

What I said from the getgo and you're missing is that pushing EAPI 
implementation into the tree and ignoring EAPI, or having this notion 
that every repository must automatically use gentoo-x86 (pushing the 
format into the tree) is /wrong/; aside from meaning that the format 
definition can now *vary*, which is great for wasting dev time and 
screwing up compatibility, it opens up tweaking/customizing that 
functionality- aka, fragmentation/divergence.  If we did the sort of 
thing you're basically pushing for, how long do you think it would be 
till funtoo added support for a new archive format to unpack?  That's 
a *simple*, and *likely* example of how this can diverge.

Further, doing as you propose means we're flat out, shit out of luck 
/ever/ distributing a usable cache for standalone repositories.  If 
they're bound to the changes of another repository, distributing a 
cache in parallel is pointless (and not doable in current form since 
metadata/cache lacks necessary eclass validation data for overlay 
inheritance).

Fact is, gentoo-x86 has a lot of usable eclass in it, but it's not 
required to be used.  Anything trying to *force* that is very short 
sighted and forgetting history.

You want new EAPI functionality that is bash only?  Awesome, eclass 
compatibility, and EAPI; don't just jam it into an eclass and say 
"EAPI is slow/annoying and I don't want to do it".  Do both, everyones 
happy.

~harring, cranky at revisiting the same arguments over and over

Re: [gentoo-dev] new `usex` helper

[Zac Medico]

On 09/16/2011 02:06 AM, Brian Harring wrote:
> On Thu, Sep 15, 2011 at 10:00:19PM -0500, Donnie Berkholz wrote:
>> On 17:29 Wed 14 Sep     , Brian Harring wrote:
>>> On Wed, Sep 14, 2011 at 02:16:41PM -0500, Donnie Berkholz wrote:
>>>> On 19:14 Tue 13 Sep     , Brian Harring wrote:
>>>>> On Tue, Sep 13, 2011 at 09:02:28PM -0500, Donnie Berkholz wrote:
>>>>>> On 17:56 Tue 13 Sep     , Mike Frysinger wrote:
>>>>>>> useful enough for EAPI ?  or should i just stick it into eutils.eclass 
>>>>>>> ?  OR BOTH !?
>>>>>>
>>>>>> I prefer to avoid EAPI whenever possible, as it just makes things slower 
>>>>>> and more complex.
>>>>>
>>>>> Exactly the wrong approach; it winds up with master 
>>>>> repositories/overlays cloning the functionality all over the damn 
>>>>> place.
>>>>
>>>> Why are people cloning anything if it's in eutils.eclass in gentoo-x86?
>>>
>>> There are more repositories than just gentoo-x86, and overlay is *not* 
>>> the only configuration in use.
>>
>> Who else besides you is using any other configuration? Should we really 
>> give a crap about the 0.001% population with some weird setup when we're 
>> trying to improve things for the 99.999% one?
> 
> Specious argument; the point of controllable stacking was to avoid the 
> issue of overlay's forcing their eclasses upon gentoo-x86 ebuilds 
> (which may not support those modified eclasses) via the old 
> PORTDIR_OVERLAY behaviour.  This is why multiple repositories have 
> layout.conf master definitions- to explicitly mark that they 
> require/consume a seperate repo.
> 
> What you're basically proposing is a variation of the "push format 
> definitions into a central tree, and require everyone to use that 
> central tree".  This discussion has come and gone; I say that being 
> one of the folks who was *pushing for the repository solution*.  The 
> problem there is it fundamentally enables (more forces) fragmentation.
> 
> Realistically I assume you're taking the stance "EAPI gets in the way, 
> lets do away with it"- if so, well, out with it, and I'll dredge up 
> the old logs/complaints that lead to EAPI.
> 
> 
>>> In the old days of the PM only handling a single overlay stack, what 
>>> you're suggesting would be less heinous- heinous in detail, but 
>>> pragmatic in reality.  These days it's a regressive approach- 
>>> requiring everyone to slave gentoo-x86 isn't sane, nor is avoiding 
>>> eapi (resulting in people having to duplicate code into each 
>>> repository stack).
>>
>> I don't know many people who aren't using gentoo-x86 or a repo that 
>> pulls in changes directly from it.
> 
> rephrase please; either you're saying "everyone uses gentoo-x86" which 
> is sort of true (funtoo/chrome aren't necessarily riding HEAD/trunk 
> however which means things can get ugly for derivative repository 
> usage), but still ignores the situations where people have overlays 
> w/ developmental eclasses that they need to selectively control where 
> it overrides (which is where the notion of repo stacking comes into 
> play).
> ~brian

As I've mentioned in bug #380391 [1], a possible hybrid approach would
be to distribute an eclass library that can be installed as a dependency
of the package manager. This would provide benefits similar to those of
using eclasses from gentoo-x86, while avoiding the introduction of a
dependencies on either gentoo-x86 or package manager implementations.

[1] https://bugs.gentoo.org/show_bug.cgi?id=380391#c3
-- 
Thanks,
Zac

Re: [gentoo-dev] new `usex` helper

[Donnie Berkholz]

[-- Attachment #1: Type: text/plain, Size: 2651 bytes --]

On 13:43 Fri 16 Sep     , Brian Harring wrote:
> What I said from the getgo and you're missing is that pushing EAPI 
> implementation into the tree and ignoring EAPI, or having this notion 
> that every repository must automatically use gentoo-x86 (pushing the 
> format into the tree) is /wrong/;

I'm not necessarily requiring that every repository must automatically 
use gentoo-x86 — just the ones that want to use features in an eclass 
distributed with gentoo-x86. It sounds to me like the logical 
consequence of what you're saying is that every useful function that's 
now or could eventually be in an eclass must instead be incorporated 
into EAPI. I guess I just don't see where you're drawing the line.

What I'm suggesting is that we should add useful stuff to eclasses by 
default. If people want to use that stuff, they can stack on the 
gentoo-x86 repo and inherit the eclass. I don't know why EAPI needs to 
come into it at all.

> aside from meaning that the format definition can now *vary*, which is 
> great for wasting dev time and screwing up compatibility, it opens up 
> tweaking/customizing that functionality- aka, 
> fragmentation/divergence.

People doing that outside gentoo-x86 should do it the same way as ones 
within it, by bumping the eclass to a new unique name. Ideally one 
that's not just a numeric value so it won't conflict with ours, in the 
same way as EAPI naming.

> If we did the sort of thing you're basically pushing for, how long do 
> you think it would be till funtoo added support for a new archive 
> format to unpack?  That's a *simple*, and *likely* example of how this 
> can diverge.

So, what I'm getting out of this is that we should make it harder for 
derivative distributions to innovate? Why should I care if they want to 
do stuff with new archive formats?

> Further, doing as you propose means we're flat out, shit out of luck 
> /ever/ distributing a usable cache for standalone repositories.  If 
> they're bound to the changes of another repository, distributing a 
> cache in parallel is pointless (and not doable in current form since 
> metadata/cache lacks necessary eclass validation data for overlay 
> inheritance).

Not much different from other cross-repository dependencies; you have to 
keep everything in lockstep because who knows what other people will do 
with their repos. Maybe people would want to distribute their own copies 
of forked dependent repositories too, I haven't thought much about it.

-- 
Thanks,
Donnie

Donnie Berkholz
Council Member / Sr. Developer
Gentoo Linux
Blog: http://dberkholz.com

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] new `usex` helper

[Brian Harring]

On Sat, Sep 17, 2011 at 10:59:08PM -0500, Donnie Berkholz wrote:
> On 13:43 Fri 16 Sep     , Brian Harring wrote:
> > What I said from the getgo and you're missing is that pushing EAPI 
> > implementation into the tree and ignoring EAPI, or having this notion 
> > that every repository must automatically use gentoo-x86 (pushing the 
> > format into the tree) is /wrong/;
> 
> I'm not necessarily requiring that every repository must automatically 
> use gentoo-x86 ??? just the ones that want to use features in an eclass 
> distributed with gentoo-x86. It sounds to me like the logical 
> consequence of what you're saying is that every useful function that's 
> now or could eventually be in an eclass must instead be incorporated 
> into EAPI. I guess I just don't see where you're drawing the line.

Drawing it back to what spawned it; usex.  This isn't git.eclass, this 
isn't svn.eclass, nor is it any of the other complex eclass 
functionality.  It's a core function that benefits the rest and 
should be in EAPI.

That function is pretty clearly destined for EAPI, hence this long 
thread where you're bitching "avoid EAPI, do eclass only" and I'm 
bitching back "that's stupid for xyz reasons, do a compat eclass and 
push it into the next EAPI".


> What I'm suggesting is that we should add useful stuff to eclasses by 
> default. If people want to use that stuff, they can stack on the 
> gentoo-x86 repo and inherit the eclass. I don't know why EAPI needs to 
> come into it at all.

Stacking on gentoo-x86 means you get *everything* for that repository.  
If all you want is a random function out of eutils, that's a *lot* of 
uncontrolled change to have intermixed with your overlays eclasses 
(even worse, if the eclasses truly overlay into gentoo-x86, that 
introduces compatibility issues there too).

It's a matter of change control; using the unpack example again, if 
I'm maintaining a 6 month old snapshot for production deployment, 
which is preferable for getting an updated unpack function:

1) unpack is in eclasses; do I sync gentoo-x86 eclasses into my 
snapshot?  Do I just cherry pick the updated function out, into my own 
trees?

2) if it's part of a new EAPI, do I cherry-pick the update for my $PM, 
and then use that EAPI in my ebuilds that needs that new 
functionality?

#1 is exactly the sort of scenario where you'll start getting 
copy/pasting; you can state "well they should just update", but that's 
completely unrealistic.  Every production deployment of gentoo (or 
derived from) that I've seen has snapshot'd periodically for 
stabilization reasons.


> > aside from meaning that the format definition can now *vary*, which is 
> > great for wasting dev time and screwing up compatibility, it opens up 
> > tweaking/customizing that functionality- aka, 
> > fragmentation/divergence.
> 
> People doing that outside gentoo-x86 should do it the same way as ones 
> within it, by bumping the eclass to a new unique name. Ideally one 
> that's not just a numeric value so it won't conflict with ours, in the 
> same way as EAPI naming.

They should, but api compatibility of an eclass *can* be fluid- in the 
past it was a locked API purely because of portage environment saving 
issues.  That's been resolved, there isn't any strict requirement that 
the eclass maintain API compat now.  You're trying to rely on people 
doing best practices- for format building blocks 
(use_with/usex/unpack/etc), that's not sane.


> > If we did the sort of thing you're basically pushing for, how long do 
> > you think it would be till funtoo added support for a new archive 
> > format to unpack?  That's a *simple*, and *likely* example of how this 
> > can diverge.
> 
> So, what I'm getting out of this is that we should make it harder for 
> derivative distributions to innovate? Why should I care if they want to 
> do stuff with new archive formats?

If funtoo wants their own unpack, awesome.  Shove it into an eclass 
under a different name.  If they want to modify unpack /directly/, 
they better damn well change the EAPI the ebuild uses.

prefix, kde, hell even a crazy python attempt have all had custom 
EAPI's built out for exactly scenarios like this.

The point of EAPI is that it's an agreed to format.  Embrace/extending 
it hasn't intentionally occured yet since the versioning is 
*designed* to allow for people to cut their own formats as needs be 
for experimentation.


> > Further, doing as you propose means we're flat out, shit out of luck 
> > /ever/ distributing a usable cache for standalone repositories.  If 
> > they're bound to the changes of another repository, distributing a 
> > cache in parallel is pointless (and not doable in current form since 
> > metadata/cache lacks necessary eclass validation data for overlay 
> > inheritance).
> 
> Not much different from other cross-repository dependencies; you have to 
> keep everything in lockstep because who knows what other people will do 
> with their repos. Maybe people would want to distribute their own copies 
> of forked dependent repositories too, I haven't thought much about it.

Think that through; you're suggesting "shove it all in gentoo-x86"; 
that increases the lockstep requirements.


I suspect an easier way to wrap this thread up is if you just state 
what you want for backwards compatibility- in a seperate thread you 
already proposed basically locking out systems older than 6 months, 
and this discussion (moreso the "eapi slows our development" subtext) 
is along the same lines.

Layout what you think it should be, how you think EAPI should be 
managed (what goes in, what doesn't, etc), how derivatives should be 
handled, how we'll deal w/ installations/derivative distros that grab 
snapshots of the tree and run from that (chromeos is a public 
example, I've seen multiple private deployments using the same 
approach), and what /you/ think it should be rather than sniping at 
the examples I've been giving why things are the way they are.

As is, this thread isn't really going anywhere- better to just skip 
ahead to what you think it should be rather than random arguments 
about what it is now.

~harring

Re: [gentoo-dev] new `usex` helper

[Donnie Berkholz]

[-- Attachment #1: Type: text/plain, Size: 6116 bytes --]

On 04:22 Sun 18 Sep     , Brian Harring wrote:
> On Sat, Sep 17, 2011 at 10:59:08PM -0500, Donnie Berkholz wrote:
> > On 13:43 Fri 16 Sep     , Brian Harring wrote:
> > > What I said from the getgo and you're missing is that pushing EAPI 
> > > implementation into the tree and ignoring EAPI, or having this notion 
> > > that every repository must automatically use gentoo-x86 (pushing the 
> > > format into the tree) is /wrong/;
> > 
> > I'm not necessarily requiring that every repository must automatically 
> > use gentoo-x86 ??? just the ones that want to use features in an eclass 
> > distributed with gentoo-x86. It sounds to me like the logical 
> > consequence of what you're saying is that every useful function that's 
> > now or could eventually be in an eclass must instead be incorporated 
> > into EAPI. I guess I just don't see where you're drawing the line.
> 
> Drawing it back to what spawned it; usex.  This isn't git.eclass, this 
> isn't svn.eclass, nor is it any of the other complex eclass 
> functionality.  It's a core function that benefits the rest and should 
> be in EAPI.

OK, so the implication of what you're saying is that everything in 
eutils.eclass, base.eclass, toolchain-funcs.eclass, flag-o-matic.eclass, 
versionator.eclass, multilib.eclass, prefix.eclass, savedconfig.eclass, 
and maybe even linux-info.eclass and autotools{,-utils}.eclass should go 
into EAPI. All that stuff is pretty generally useful features.

> > What I'm suggesting is that we should add useful stuff to eclasses 
> > by default. If people want to use that stuff, they can stack on the 
> > gentoo-x86 repo and inherit the eclass. I don't know why EAPI needs 
> > to come into it at all.
> 
> Stacking on gentoo-x86 means you get *everything* for that repository.  
> If all you want is a random function out of eutils, that's a *lot* of 
> uncontrolled change to have intermixed with your overlays eclasses 
> (even worse, if the eclasses truly overlay into gentoo-x86, that 
> introduces compatibility issues there too).

Yeah, it seems like the ability to do partial stacking would be nice. 
Perhaps to do so, one wouldn't stack by default but would have a way to 
specify cross-repo dependencies (including eclasses) or file-specific 
UUIDs of some sort.

> > > aside from meaning that the format definition can now *vary*, 
> > > which is great for wasting dev time and screwing up compatibility, 
> > > it opens up tweaking/customizing that functionality- aka, 
> > > fragmentation/divergence.
> > 
> > People doing that outside gentoo-x86 should do it the same way as 
> > ones within it, by bumping the eclass to a new unique name. Ideally 
> > one that's not just a numeric value so it won't conflict with ours, 
> > in the same way as EAPI naming.
> 
> They should, but api compatibility of an eclass *can* be fluid- in the 
> past it was a locked API purely because of portage environment saving 
> issues.  That's been resolved, there isn't any strict requirement that 
> the eclass maintain API compat now.  You're trying to rely on people 
> doing best practices- for format building blocks 
> (use_with/usex/unpack/etc), that's not sane.

Which still pisses me off. It's like a shared library changing its API 
without bumping SONAME. That makes me wonder whether there's some way we 
could "enforce" it, at least on the level of repoman or test suites to 
examine whether the public interface changed, perhaps by generating a 
cache of the interfaces and comparing to them.

> I suspect an easier way to wrap this thread up is if you just state 
> what you want for backwards compatibility- in a seperate thread you 
> already proposed basically locking out systems older than 6 months, 
> and this discussion (moreso the "eapi slows our development" subtext) 
> is along the same lines.

Actually Zac said that, and I was trying to clarify if that's really 
what he was saying. 9-12 months is more my feeling, as it gets extremely 
difficult to maintain Gentoo systems without guru-level knowledge around 
there. (Spoken as someone who still gets support emails from a couple of 
previous positions.)

While I would like there to be a "proper" way to express repo-level 
dependencies, properly announce deprecation and updates (e.g. to bash 4) 
in advance as a feature integrated with the PM, and so on, I don't think 
we should block our ability to move forward on these things. The 
situation is essentially the same as it's always been, but our old 
solution is no longer considered good enough and we don't have a new one 
in place, so we're just sitting here.

> Layout what you think it should be,

Layout of what, exactly?

> how you think EAPI should be managed (what goes in, what doesn't, 
> etc),

If it can go in an eclass without strange contortions, put it there.

> how derivatives should be handled,

With white gloves. More seriously, people making derivatives should have 
maximal freedom to experiment, and I'm guessing most of them don't want 
to deal with modifying 3 different PMs written at least partially in 3 
different languages. They'd rather instantaneously support things in the 
same language they use for everything else.

> how we'll deal w/ installations/derivative distros that grab 
> snapshots of the tree and run from that (chromeos is a public example, 
> I've seen multiple private deployments using the same approach),

Being explicit about our level of support is what we need to do, no 
matter what that level is, so external groups can figure out how to deal 
with that timeline. Our current council-mandated standard is a year, as 
Ulm mentioned in another post, but I'm not aware of any of our 
documentation that actually says this.

It would obviously be good to flood our users with communication 
(website, -announce, news items, etc) before anything that broke even 
just the oldest installations.

-- 
Thanks,
Donnie

Donnie Berkholz
Council Member / Sr. Developer
Gentoo Linux
Blog: http://dberkholz.com

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] new `usex` helper

[Brian Harring]

On Sun, Sep 18, 2011 at 10:16:46PM -0500, Donnie Berkholz wrote:
> On 04:22 Sun 18 Sep     , Brian Harring wrote:
> > On Sat, Sep 17, 2011 at 10:59:08PM -0500, Donnie Berkholz wrote:
> > > On 13:43 Fri 16 Sep     , Brian Harring wrote:
> > > > What I said from the getgo and you're missing is that pushing EAPI 
> > > > implementation into the tree and ignoring EAPI, or having this notion 
> > > > that every repository must automatically use gentoo-x86 (pushing the 
> > > > format into the tree) is /wrong/;
> > > 
> > > I'm not necessarily requiring that every repository must automatically 
> > > use gentoo-x86 ??? just the ones that want to use features in an eclass 
> > > distributed with gentoo-x86. It sounds to me like the logical 
> > > consequence of what you're saying is that every useful function that's 
> > > now or could eventually be in an eclass must instead be incorporated 
> > > into EAPI. I guess I just don't see where you're drawing the line.
> > 
> > Drawing it back to what spawned it; usex.  This isn't git.eclass, this 
> > isn't svn.eclass, nor is it any of the other complex eclass 
> > functionality.  It's a core function that benefits the rest and should 
> > be in EAPI.
> 
> OK, so the implication of what you're saying is that everything in 
> eutils.eclass, base.eclass, toolchain-funcs.eclass, flag-o-matic.eclass, 
> versionator.eclass, multilib.eclass, prefix.eclass, savedconfig.eclass, 
> and maybe even linux-info.eclass and autotools{,-utils}.eclass should go 
> into EAPI. All that stuff is pretty generally useful features.

Approximately... yes.  Some of that (base.eclass for example) is EAPI 
compatibility that can't be folded in (would require retroactively 
addding to an EAPI), others aren't yet stabilized (true multilib for 
example, seems to still be under discussion), etc.

You get the jist.

> > > What I'm suggesting is that we should add useful stuff to eclasses 
> > > by default. If people want to use that stuff, they can stack on the 
> > > gentoo-x86 repo and inherit the eclass. I don't know why EAPI needs 
> > > to come into it at all.
> > 
> > Stacking on gentoo-x86 means you get *everything* for that repository.  
> > If all you want is a random function out of eutils, that's a *lot* of 
> > uncontrolled change to have intermixed with your overlays eclasses 
> > (even worse, if the eclasses truly overlay into gentoo-x86, that 
> > introduces compatibility issues there too).
> 
> Yeah, it seems like the ability to do partial stacking would be nice. 
> Perhaps to do so, one wouldn't stack by default but would have a way to 
> specify cross-repo dependencies (including eclasses) or file-specific 
> UUIDs of some sort.

We can specify cross repository dependencies (this repo needs 
that repo) already via layout.conf; partial gets a bit messy, but may 
be useful at the user level.


> > > > aside from meaning that the format definition can now *vary*, 
> > > > which is great for wasting dev time and screwing up compatibility, 
> > > > it opens up tweaking/customizing that functionality- aka, 
> > > > fragmentation/divergence.
> > > 
> > > People doing that outside gentoo-x86 should do it the same way as 
> > > ones within it, by bumping the eclass to a new unique name. Ideally 
> > > one that's not just a numeric value so it won't conflict with ours, 
> > > in the same way as EAPI naming.
> > 
> > They should, but api compatibility of an eclass *can* be fluid- in the 
> > past it was a locked API purely because of portage environment saving 
> > issues.  That's been resolved, there isn't any strict requirement that 
> > the eclass maintain API compat now.  You're trying to rely on people 
> > doing best practices- for format building blocks 
> > (use_with/usex/unpack/etc), that's not sane.
> 
> Which still pisses me off. It's like a shared library changing its API 
> without bumping SONAME.

I think you're viewing this wrong; if we're talking about openssl 
breaking API/ABI w/out bumping soname, sure.  It's one component that 
is distributed alone, but consumed by others.  There is no way to 
atomically deploy the new API at the same time as all of the consumers 
being updated for it.

That is /not/ the case of gentoo-x86; eclasses for us are equivalent 
to bundled libs.  Overlay stacking just makes it possible for a third 
party component to reach in and use what is effectively an internal 
lib.

This is even more true for repositories other than gentoo-x86; 
gentoo-x86 is just in the weird position of being both public and 
private.


> That makes me wonder whether there's some way we 
> could "enforce" it, at least on the level of repoman or test suites to 
> examine whether the public interface changed, perhaps by generating a 
> cache of the interfaces and comparing to them.

Not without some doc/annotation of each function.  The fact functions 
are basically jump targets, w/ the function doing shift's/$@ to get at 
it's args makes this very, very hard to introspect and validate.  

Tests are the realistic option, or parsing an annotation defining the 
functions prototype, caching that, and comparing it across versions.

That's realistically a seperate discussion- one snagging betelgeuse 
(and that libbash gSoC student) might result in some benefit.


> > I suspect an easier way to wrap this thread up is if you just state 
> > what you want for backwards compatibility- in a seperate thread you 
> > already proposed basically locking out systems older than 6 months, 
> > and this discussion (moreso the "eapi slows our development" subtext) 
> > is along the same lines.
> 
> Actually Zac said that, and I was trying to clarify if that's really 
> what he was saying. 9-12 months is more my feeling, as it gets extremely 
> difficult to maintain Gentoo systems without guru-level knowledge around 
> there. (Spoken as someone who still gets support emails from a couple of 
> previous positions.)
> 
> While I would like there to be a "proper" way to express repo-level 
> dependencies, properly announce deprecation and updates (e.g. to bash 4) 
> in advance as a feature integrated with the PM, and so on, I don't think 
> we should block our ability to move forward on these things.

We're not really blocked, frankly.  For repository format changes, 
while we lack versioning... it can be done.  There just hasn't been 
enough of a reason to do it, *yet* (it's been on my todo for a while 
since there is some stuff I want that requires it).

For deploying EAPI, we're able to do it pretty quickly if we wanted 
to; where we're boned is in profiles, and that will /not/ change.  
There is no magic solution to it- from a data structure standpoint, 
either we have to duplicate chunks of the profiles for backwards 
compatibility, or we have to keep it very low in EAPI deps.

That's just the fact of life for profiles; it's a central data struct, 
as such it's fairly bound to LCD requirements.


> The 
> situation is essentially the same as it's always been, but our old 
> solution is no longer considered good enough and we don't have a new one 
> in place, so we're just sitting here.

Not really; the old solution was "deploy, then have to wait 6 months 
for small stuff, year for big stuff".

For format changes, we can move *much* faster- 2 months basically 
(month of portage sitting in unstable, allowing unstable ebuilds to 
use the EAPI, then portage stabled in the second month effectively 
stabling the EAPI).

For the profiles, we can move forward at the same rate if desired in 
terms of using the new functionality, although for avoiding lock out 
(preserving the EAPI upgrade pathway) duplication may be required or a 
slower rate of EAPI consumption.

Those times are accurate- that's what we *could* do.  EAPI versions 
being cut slower is a whole other issue (council having to sign off, 
bribing portage to add stuff, etc), and that issue should be addressed 
directly- any compat mechanism will run into the same issue there, so 
the mechanism isn't the issue.


> > how derivatives should be handled,
> 
> With white gloves. More seriously, people making derivatives should have 
> maximal freedom to experiment, and I'm guessing most of them don't want 
> to deal with modifying 3 different PMs written at least partially in 3 
> different languages. They'd rather instantaneously support things in the 
> same language they use for everything else.

Full experimentation has typically required modifying the PM to extend 
functionality- you're focusing just on bash bits.  Bash bits are easy- 
you can hack that at the eclass level w/out an EAPI half the time.  
Frankly, that's not the stuff people get hung up on.

Realistically, the interesting stuff that people want have been more 
than just bash.

Simple example, you can't do USE deps, slot deps, repository deps, 
iuse defaults, etc, w/out modifying the PM.  That's just the way it 
is, as such you choose your preferred PM, get the custom eapi going, 
if it's more than a one off prototype you bribe others to support it, 
they do.

I don't see that changing; while folks can complain about it, there 
are hard technical reasons this is how it is- those issues have to be 
solved (no amount of complaining can solve 'em) ;)

~harring

Re: [gentoo-dev] new `usex` helper

[Donnie Berkholz]

[-- Attachment #1: Type: text/plain, Size: 2401 bytes --]

On 14:20 Tue 20 Sep     , Brian Harring wrote:
> On Sun, Sep 18, 2011 at 10:16:46PM -0500, Donnie Berkholz wrote:
> > OK, so the implication of what you're saying is that everything in 
> > eutils.eclass, base.eclass, toolchain-funcs.eclass, 
> > flag-o-matic.eclass, versionator.eclass, multilib.eclass, 
> > prefix.eclass, savedconfig.eclass, and maybe even linux-info.eclass 
> > and autotools{,-utils}.eclass should go into EAPI. All that stuff is 
> > pretty generally useful features.
> 
> Approximately... yes.  Some of that (base.eclass for example) is EAPI 
> compatibility that can't be folded in (would require retroactively 
> addding to an EAPI), others aren't yet stabilized (true multilib for 
> example, seems to still be under discussion), etc.
> 
> You get the jist.

Yep, and I'm completely on the opposite end of the spectrum. =)

> > > They should, but api compatibility of an eclass *can* be fluid- in 
> > > the past it was a locked API purely because of portage environment 
> > > saving issues.  That's been resolved, there isn't any strict 
> > > requirement that the eclass maintain API compat now.  You're 
> > > trying to rely on people doing best practices- for format building 
> > > blocks (use_with/usex/unpack/etc), that's not sane.
> > 
> > Which still pisses me off. It's like a shared library changing its 
> > API without bumping SONAME.
> 
> I think you're viewing this wrong; if we're talking about openssl 
> breaking API/ABI w/out bumping soname, sure.  It's one component that 
> is distributed alone, but consumed by others.  There is no way to 
> atomically deploy the new API at the same time as all of the consumers 
> being updated for it.
> 
> That is /not/ the case of gentoo-x86; eclasses for us are equivalent 
> to bundled libs.  Overlay stacking just makes it possible for a third 
> party component to reach in and use what is effectively an internal 
> lib.

Not really, because when you update a bundled lib you actually make your 
whole app compile with it. People change the APIs of eclasses and then 
just let every internal consumer (ebuilds in gentoo-x86) break. Maybe if 
we put the burden on the one who changed the API, like the Linux kernel 
model, it would bother me less.

-- 
Thanks,
Donnie

Donnie Berkholz
Council Member / Sr. Developer
Gentoo Linux
Blog: http://dberkholz.com

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] new `usex` helper

[Alec Warner]

On Wed, Sep 21, 2011 at 6:11 AM, Donnie Berkholz <dberkholz@gentoo.org> wrote:
> On 14:20 Tue 20 Sep     , Brian Harring wrote:
>> On Sun, Sep 18, 2011 at 10:16:46PM -0500, Donnie Berkholz wrote:
>> > OK, so the implication of what you're saying is that everything in
>> > eutils.eclass, base.eclass, toolchain-funcs.eclass,
>> > flag-o-matic.eclass, versionator.eclass, multilib.eclass,
>> > prefix.eclass, savedconfig.eclass, and maybe even linux-info.eclass
>> > and autotools{,-utils}.eclass should go into EAPI. All that stuff is
>> > pretty generally useful features.
>>
>> Approximately... yes.  Some of that (base.eclass for example) is EAPI
>> compatibility that can't be folded in (would require retroactively
>> addding to an EAPI), others aren't yet stabilized (true multilib for
>> example, seems to still be under discussion), etc.
>>
>> You get the jist.
>
> Yep, and I'm completely on the opposite end of the spectrum. =)
>
>> > > They should, but api compatibility of an eclass *can* be fluid- in
>> > > the past it was a locked API purely because of portage environment
>> > > saving issues.  That's been resolved, there isn't any strict
>> > > requirement that the eclass maintain API compat now.  You're
>> > > trying to rely on people doing best practices- for format building
>> > > blocks (use_with/usex/unpack/etc), that's not sane.
>> >
>> > Which still pisses me off. It's like a shared library changing its
>> > API without bumping SONAME.
>>
>> I think you're viewing this wrong; if we're talking about openssl
>> breaking API/ABI w/out bumping soname, sure.  It's one component that
>> is distributed alone, but consumed by others.  There is no way to
>> atomically deploy the new API at the same time as all of the consumers
>> being updated for it.
>>
>> That is /not/ the case of gentoo-x86; eclasses for us are equivalent
>> to bundled libs.  Overlay stacking just makes it possible for a third
>> party component to reach in and use what is effectively an internal
>> lib.
>
> Not really, because when you update a bundled lib you actually make your
> whole app compile with it. People change the APIs of eclasses and then
> just let every internal consumer (ebuilds in gentoo-x86) break. Maybe if
> we put the burden on the one who changed the API, like the Linux kernel
> model, it would bother me less.

I think people do this for three reasons.

1) There are no refactoring tools that I know of for bash.
2) There exist some package maintainers that will yell at you if you
touch their packages for any reason.
3) Breaking things means they get fixed.

We have this notify -> deprecate -> break workflow; I actually don't
mind it (but only because I've seen it used elsewhere.)

-A
>
> --
> Thanks,
> Donnie
>
> Donnie Berkholz
> Council Member / Sr. Developer
> Gentoo Linux
> Blog: http://dberkholz.com
>

Re: [gentoo-dev] new `usex` helper

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 228 bytes --]

since there's been no new feedback in a while, i'll add this to eutils.eclass 
in a while:
usex() { use "$1" && echo "${2-yes}$4" || echo "${3-no}$5" ; }

then once it hits the PMS, i'll put EAPI wrapping around it.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] new `usex` helper

[Donnie Berkholz]

[-- Attachment #1: Type: text/plain, Size: 1535 bytes --]

On 09:37 Wed 21 Sep     , Alec Warner wrote:
> On Wed, Sep 21, 2011 at 6:11 AM, Donnie Berkholz <dberkholz@gentoo.org> wrote:
> > Not really, because when you update a bundled lib you actually make 
> > your whole app compile with it. People change the APIs of eclasses 
> > and then just let every internal consumer (ebuilds in gentoo-x86) 
> > break. Maybe if we put the burden on the one who changed the API, 
> > like the Linux kernel model, it would bother me less.
> 
> I think people do this for three reasons.
> 
> 1) There are no refactoring tools that I know of for bash.
> 2) There exist some package maintainers that will yell at you if you
> touch their packages for any reason.

To refer to the Linux model again, you send patches to the maintainers, 
and they just commit them. This is much less effort than figuring out to 
handle some incomprehensible change to an already weird eclass and then 
sorting out how to deal with it across 20 or 30 packages.

> 3) Breaking things means they get fixed.
> 
> We have this notify -> deprecate -> break workflow; I actually don't
> mind it (but only because I've seen it used elsewhere.)

I do, because I don't have time to deal with other people breaking my 
packages, whether they're in gentoo-x86, the science overlay, or my 
personal one. I've got more important things to deal with, within Gentoo 
and in the rest of my life.

-- 
Thanks,
Donnie

Donnie Berkholz
Council Member / Sr. Developer
Gentoo Linux
Blog: http://dberkholz.com

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] new `usex` helper

[Alec Warner]

On Thu, Sep 22, 2011 at 5:41 PM, Donnie Berkholz <dberkholz@gentoo.org> wrote:
> On 09:37 Wed 21 Sep     , Alec Warner wrote:
>> On Wed, Sep 21, 2011 at 6:11 AM, Donnie Berkholz <dberkholz@gentoo.org> wrote:
>> > Not really, because when you update a bundled lib you actually make
>> > your whole app compile with it. People change the APIs of eclasses
>> > and then just let every internal consumer (ebuilds in gentoo-x86)
>> > break. Maybe if we put the burden on the one who changed the API,
>> > like the Linux kernel model, it would bother me less.
>>
>> I think people do this for three reasons.
>>
>> 1) There are no refactoring tools that I know of for bash.
>> 2) There exist some package maintainers that will yell at you if you
>> touch their packages for any reason.
>
> To refer to the Linux model again, you send patches to the maintainers,
> and they just commit them. This is much less effort than figuring out to
> handle some incomprehensible change to an already weird eclass and then
> sorting out how to deal with it across 20 or 30 packages.

In my experience maintainers do not 'just commit my patches.' But
perhaps I'm crazy.

>
>> 3) Breaking things means they get fixed.
>>
>> We have this notify -> deprecate -> break workflow; I actually don't
>> mind it (but only because I've seen it used elsewhere.)
>
> I do, because I don't have time to deal with other people breaking my
> packages, whether they're in gentoo-x86, the science overlay, or my
> personal one. I've got more important things to deal with, within Gentoo
> and in the rest of my life.

You don't have time to fix breakages but you have time to do code reviews?

I put a code review system in the infra survey because I think
bugzilla is a poor way to track changes to the tree and there is not a
good way to get feedback or track updates. I think if we had
streamlined tools for this I'd be less concerned about mailing out
changes to 60 or 100 or 1000 packages and having any sane idea which
packages were fixed, which were pending, which packages I had not
heard from someone in a while.. etc...

-A

>
> --
> Thanks,
> Donnie
>
> Donnie Berkholz
> Council Member / Sr. Developer
> Gentoo Linux
> Blog: http://dberkholz.com
>

Re: [gentoo-dev] new `usex` helper

[Donnie Berkholz]

[-- Attachment #1: Type: text/plain, Size: 906 bytes --]

On 18:04 Thu 22 Sep     , Alec Warner wrote:
> On Thu, Sep 22, 2011 at 5:41 PM, Donnie Berkholz <dberkholz@gentoo.org> wrote:
> > I do, because I don't have time to deal with other people breaking 
> > my packages, whether they're in gentoo-x86, the science overlay, or 
> > my personal one. I've got more important things to deal with, within 
> > Gentoo and in the rest of my life.
> 
> You don't have time to fix breakages but you have time to do code 
> reviews?

If someone else I trust makes the changes and tells me they work, I can 
review code in an email on my cell phone in 15 seconds. I have to be on 
my single dev laptop with Gentoo repos and have a good 15 *minutes* to 
spare to figure out all the changes required, make them, test them, and 
do the commit.

-- 
Thanks,
Donnie

Donnie Berkholz
Council Member / Sr. Developer
Gentoo Linux
Blog: http://dberkholz.com

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]