[gentoo-dev] [PATCH] eutils: Warn on built_with_use usage

[gentoo-dev] [PATCH] eutils: Warn on built_with_use usage

[Brian Harring]

At this point, the functionality built_with_use provides should
be covered near or more likely, in full, but USE deps in EAPI2 and
EAPI4; thus warn on usage.

While this may be a bit annoying, this is the only major consumer
left at this point that knows about /var/db/pkg layout; once that's
gone, alternative VDB formats can occur (meaning faster package
manager operations, or at least the potential if implemented sanely).

This patch is a rough first stab at the wording (english isn't
exactly my forte) to use for warning; better/clearer warning text
would be appreciated.
---
 eutils.eclass | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/eutils.eclass b/eutils.eclass
index 8bfc2bc..7f5a616 100644
--- a/eutils.eclass
+++ b/eutils.eclass
@@ -1196,7 +1196,9 @@ built_with_use() {
 	fi
 
 	local missing_action="die"
+	local missing_was_set=false
 	if [[ $1 == "--missing" ]] ; then
+		missing_was_set=true
 		missing_action=$2
 		shift ; shift
 		case ${missing_action} in
@@ -1205,6 +1207,17 @@ built_with_use() {
 		esac
 	fi
 
+	if ! has $EAPI 0 1 2 3; then
+		eqawarn "built_with_use should not be used in $EAPI; use USE deps."
+	elif has $EAPI 2 3; then
+		if [[ $hidden == yes ]] || $missing_was_set; then
+			eqawarn "built_with_use in EAPI=$EAPI without --missing or --hidden usage, should use USE deps instead."
+		else
+			eqawarn "built_with_use should not be used; upgrade to EAPI=4 instead"
+		fi
+	fi
+
+
 	local opt=$1
 	[[ ${opt:0:1} = "-" ]] && shift || opt="-a"
 
-- 
1.7.12

[gentoo-dev] Re: [PATCH] eutils: Warn on built_with_use usage

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 737 bytes --]

On Sunday 16 September 2012 03:51:04 Brian Harring wrote:
> +	if ! has $EAPI 0 1 2 3; then
> +		eqawarn "built_with_use should not be used in $EAPI; use USE deps."
> +	elif has $EAPI 2 3; then
> +		if [[ $hidden == yes ]] || $missing_was_set; then
> +			eqawarn "built_with_use in EAPI=$EAPI without --missing or --
hidden
> usage, should use USE deps instead." +		else
> +			eqawarn "built_with_use should not be used; upgrade to EAPI=4 
instead"
> +		fi
> +	fi

i'd do:
	case ${EAPI:-0} in
	# No support in these EAPIs, so don't warn.
	0|1) ;;
	# Maybe warn as some functionality exist.
	2|3) [[...]] && eqawarn "..." ;;
	# Assume EAPI=4 or newer where all functionality exists.
	*) eqawarn "..." ;;
	esac
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[gentoo-dev] Re: [PATCH] eutils: Warn on built_with_use usage

[Brian Harring]

On Sun, Sep 16, 2012 at 10:10:47PM -0400, Mike Frysinger wrote:
> On Sunday 16 September 2012 03:51:04 Brian Harring wrote:
> > +	if ! has $EAPI 0 1 2 3; then
> > +		eqawarn "built_with_use should not be used in $EAPI; use USE deps."
> > +	elif has $EAPI 2 3; then
> > +		if [[ $hidden == yes ]] || $missing_was_set; then
> > +			eqawarn "built_with_use in EAPI=$EAPI without --missing or --
> hidden
> > usage, should use USE deps instead." +		else
> > +			eqawarn "built_with_use should not be used; upgrade to EAPI=4 
> instead"
> > +		fi
> > +	fi
> 
> i'd do:
> 	case ${EAPI:-0} in
> 	# No support in these EAPIs, so don't warn.
> 	0|1) ;;
> 	# Maybe warn as some functionality exist.
> 	2|3) [[...]] && eqawarn "..." ;;
> 	# Assume EAPI=4 or newer where all functionality exists.
> 	*) eqawarn "..." ;;
> 	esac

I'd be fine w/ it; worth noting, that was a 4am patch, so I'm not 
claiming perfect implementatoin there. :)

My main focus here is switching built_with_use to actively nagging 
people to stop using it; this includes nagging EAPI0/1 users of it.

Sans the implementation details, anyone got complaints with the 
intent?
~brian

Re: [gentoo-dev] Re: [PATCH] eutils: Warn on built_with_use usage

[Ralph Sennhauser]

On Sun, 16 Sep 2012 19:41:14 -0700
Brian Harring <ferringb@gmail.com> wrote:

> On Sun, Sep 16, 2012 at 10:10:47PM -0400, Mike Frysinger wrote:
> > On Sunday 16 September 2012 03:51:04 Brian Harring wrote:
> > > +	if ! has $EAPI 0 1 2 3; then
> > > +		eqawarn "built_with_use should not be used in
> > > $EAPI; use USE deps."
> > > +	elif has $EAPI 2 3; then
> > > +		if [[ $hidden == yes ]] || $missing_was_set; then
> > > +			eqawarn "built_with_use in EAPI=$EAPI
> > > without --missing or --
> > hidden
> > > usage, should use USE deps instead." +		else
> > > +			eqawarn "built_with_use should not be
> > > used; upgrade to EAPI=4 
> > instead"
> > > +		fi
> > > +	fi
> > 
> > i'd do:
> > 	case ${EAPI:-0} in
> > 	# No support in these EAPIs, so don't warn.
> > 	0|1) ;;
> > 	# Maybe warn as some functionality exist.
> > 	2|3) [[...]] && eqawarn "..." ;;
> > 	# Assume EAPI=4 or newer where all functionality exists.
> > 	*) eqawarn "..." ;;
> > 	esac
> 
> I'd be fine w/ it; worth noting, that was a 4am patch, so I'm not 
> claiming perfect implementatoin there. :)
> 
> My main focus here is switching built_with_use to actively nagging 
> people to stop using it; this includes nagging EAPI0/1 users of it.
> 
> Sans the implementation details, anyone got complaints with the 
> intent?

How about raising the EAPI baseline from 0 to 2 - ie. every package may
use EAPI 2; not the same as deprecating 0 1 - and do:

    case ${EAPI:-0} in
        0|1|2|3|4) eqawarn "From <date> onwards this will die" ;;
        *) die ... ;;
    esac

as EAPI 2 supports the --missing case via constructs as:

    || (
        >=foo/bar-1
        <foo/bar-1[baz]
    )

Almost all affected packages can be bumped straight to 4 anyway and
so use the improved syntax.

The aim would be to get rid of built_with_use not only in a distant
future. The corresponding bug [1] is from 2009 and can't be fixed 

[1] https://bugs.gentoo.org/show_bug.cgi?id=261562

> ~brian
> 

Re: [gentoo-dev] Re: [PATCH] eutils: Warn on built_with_use usage

[Ralph Sennhauser]

On Mon, 17 Sep 2012 08:45:22 +0200
Ralph Sennhauser <sera@gentoo.org> wrote:

> The aim would be to get rid of built_with_use not only in a distant
> future. The corresponding bug [1] is from 2009 and can't be fixed 

... without something like "increasing EAPI baseline".

Re: [gentoo-dev] Re: [PATCH] eutils: Warn on built_with_use usage

[Gregory M. Turner]

> My main focus here is switching built_with_use to actively nagging
 > people to stop using it; this includes nagging EAPI0/1 users of it.
> Sans the implementation details, anyone got complaints with the
> intent?

I have a concern about it, yes.  But, maybe there's a good answer to my 
concern, so please consider this a friendly ebuild development question 
disguised as a complaint :)

Unless I'm missing something, it seems that once we deprive the ebuild 
developer of this feature, there is no simple, supported way to retrieve 
the information except to depend on it.

The issue is that calculating dependencies is not the only reason we 
might want to know if a package was built with a particular USE-flag, 
and if we get rid of built_with_use, we literally cut ourselves off from 
retrieving this information in any officially sanctioned way (except to 
DEPEND on it, which may not be semantically correct).

I can think of all kinds of legitimate reasons we might want to know if 
the installed such-and-such package was built with so-and-so use-flags 
without depending on it.  i.e.:

  o if the current gcc falls within a certain range of version
    numbers and was built with graphite, we are going to trigger
    a compiler bug.  Suppose that there is no graphite support
    or dependency in ${P}, and that we can apply a patch which will work
    around the bug, but at a performance cost in ${P} we'd rather not
    pay unless we have to.

  o We need to modify a Makefile based on how a package we
    BDEPEND on was built -- but suppose there is no BDEPEND
    /limitation/ to enforce -- in other words, either way, our package
    will build, and there is no correlating reverse dependency to worry
    about at runtime.

Such needs are fairly unusual, but they do come up in real life.

My concern is that this will lead to people doing things like:

  o cut-pasting the old implementation of built_with_use into ebuilds,
    -- but that implementation will break if the portage database layout
    changes

  o creating bogus one-off use-flags as a way of performing these queries
    (and, thanks to the upcoming requirement that USE flags
    always appear in IUSE, exposing those flags to the end-user, perhaps
    with some confusing description like "whether such-and-such was
    built with so-and-so").

  o creating BDEPENDs of -- and sketchy parsers for -- portage-utils or
    similar suites, just to ask this question.

Admittedly, it's hard to prevent people from doing

   built-with-use foo/bar baz || die "${P} needs foo/bar with baz"

since, once upon a time, that was SOP, and we'd have to parse the bash 
code or something to qa warn for it automatically.

But any number of similar prohibitions are simply documented in the 
developer handbook, including this one.

Am I missing something, here?  I kinda think we should go the opposite 
direction and un-deprecate the API.  It seems like we are cutting off 
our nose to spite our face here.

-gmt

Re: [gentoo-dev] Re: [PATCH] eutils: Warn on built_with_use usage

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 312 bytes --]

On Mon, 17 Sep 2012 00:58:02 -0700
"Gregory M. Turner" <gmt@malth.us> wrote:
> Unless I'm missing something, it seems that once we deprive the
> ebuild developer of this feature, there is no simple, supported way
> to retrieve the information except to depend on it.

has_version.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Re: [PATCH] eutils: Warn on built_with_use usage

[Ralph Sennhauser]

On Mon, 17 Sep 2012 00:58:02 -0700
"Gregory M. Turner" <gmt@malth.us> wrote:

> 
> > My main focus here is switching built_with_use to actively nagging
>  > people to stop using it; this includes nagging EAPI0/1 users of it.
> > Sans the implementation details, anyone got complaints with the
> > intent?
> 
> I have a concern about it, yes.  But, maybe there's a good answer to
> my concern, so please consider this a friendly ebuild development
> question disguised as a complaint :)
> 
> Unless I'm missing something, it seems that once we deprive the
> ebuild developer of this feature, there is no simple, supported way
> to retrieve the information except to depend on it.
> 
> The issue is that calculating dependencies is not the only reason we 
> might want to know if a package was built with a particular USE-flag, 
> and if we get rid of built_with_use, we literally cut ourselves off
> from retrieving this information in any officially sanctioned way
> (except to DEPEND on it, which may not be semantically correct).
> 
> I can think of all kinds of legitimate reasons we might want to know
> if the installed such-and-such package was built with so-and-so
> use-flags without depending on it.  i.e.:
> 
>   o if the current gcc falls within a certain range of version
>     numbers and was built with graphite, we are going to trigger
>     a compiler bug.  Suppose that there is no graphite support
>     or dependency in ${P}, and that we can apply a patch which will
> work around the bug, but at a performance cost in ${P} we'd rather not
>     pay unless we have to.
> 
>   o We need to modify a Makefile based on how a package we
>     BDEPEND on was built -- but suppose there is no BDEPEND
>     /limitation/ to enforce -- in other words, either way, our package
>     will build, and there is no correlating reverse dependency to
> worry about at runtime.
> 
> Such needs are fairly unusual, but they do come up in real life.
> 
> My concern is that this will lead to people doing things like:
> 
>   o cut-pasting the old implementation of built_with_use into ebuilds,
>     -- but that implementation will break if the portage database
> layout changes
> 
>   o creating bogus one-off use-flags as a way of performing these
> queries (and, thanks to the upcoming requirement that USE flags
>     always appear in IUSE, exposing those flags to the end-user,
> perhaps with some confusing description like "whether such-and-such
> was built with so-and-so").
> 
>   o creating BDEPENDs of -- and sketchy parsers for -- portage-utils
> or similar suites, just to ask this question.
> 
> Admittedly, it's hard to prevent people from doing
> 
>    built-with-use foo/bar baz || die "${P} needs foo/bar with baz"
> 
> since, once upon a time, that was SOP, and we'd have to parse the
> bash code or something to qa warn for it automatically.
> 
> But any number of similar prohibitions are simply documented in the 
> developer handbook, including this one.
> 
> Am I missing something, here?  I kinda think we should go the
> opposite direction and un-deprecate the API.  It seems like we are
> cutting off our nose to spite our face here.
> 
> -gmt
> 

has_version foo/bar[baz] can be used in EAPI 2 and later.

Re: [gentoo-dev] Re: [PATCH] eutils: Warn on built_with_use usage

[Gregory M. Turner]

On 9/17/2012 1:00 AM, Ciaran McCreesh wrote:
> On Mon, 17 Sep 2012 00:58:02 -0700
> "Gregory M. Turner" <gmt@malth.us> wrote:
>> Unless I'm missing something, it seems that once we deprive the
>> ebuild developer of this feature, there is no simple, supported way
>> to retrieve the information except to depend on it.
>
> has_version.

On 9/17/2012 1:10 AM, Ralph Sennhauser wrote:> On Mon, 17 Sep 2012 
00:58:02 -0700
 >
 > has_version foo/bar[baz] can be used in EAPI 2 and later.
 >

oh, duh... I guess I was just conflating the two, sorry for the noise :)

-gmt

Re: [gentoo-dev] Re: [PATCH] eutils: Warn on built_with_use usage

[Brian Harring]

On Mon, Sep 17, 2012 at 08:45:22AM +0200, Ralph Sennhauser wrote:
> On Sun, 16 Sep 2012 19:41:14 -0700
> Brian Harring <ferringb@gmail.com> wrote:
> 
> > On Sun, Sep 16, 2012 at 10:10:47PM -0400, Mike Frysinger wrote:
> > > On Sunday 16 September 2012 03:51:04 Brian Harring wrote:
> > > > +	if ! has $EAPI 0 1 2 3; then
> > > > +		eqawarn "built_with_use should not be used in
> > > > $EAPI; use USE deps."
> > > > +	elif has $EAPI 2 3; then
> > > > +		if [[ $hidden == yes ]] || $missing_was_set; then
> > > > +			eqawarn "built_with_use in EAPI=$EAPI
> > > > without --missing or --
> > > hidden
> > > > usage, should use USE deps instead." +		else
> > > > +			eqawarn "built_with_use should not be
> > > > used; upgrade to EAPI=4 
> > > instead"
> > > > +		fi
> > > > +	fi
> > > 
> > > i'd do:
> > > 	case ${EAPI:-0} in
> > > 	# No support in these EAPIs, so don't warn.
> > > 	0|1) ;;
> > > 	# Maybe warn as some functionality exist.
> > > 	2|3) [[...]] && eqawarn "..." ;;
> > > 	# Assume EAPI=4 or newer where all functionality exists.
> > > 	*) eqawarn "..." ;;
> > > 	esac
> > 
> > I'd be fine w/ it; worth noting, that was a 4am patch, so I'm not 
> > claiming perfect implementatoin there. :)
> > 
> > My main focus here is switching built_with_use to actively nagging 
> > people to stop using it; this includes nagging EAPI0/1 users of it.
> > 
> > Sans the implementation details, anyone got complaints with the 
> > intent?
> 
> How about raising the EAPI baseline from 0 to 2 - ie. every package may
> use EAPI 2; not the same as deprecating 0 1 - and do:
> 
>     case ${EAPI:-0} in
>         0|1|2|3|4) eqawarn "From <date> onwards this will die" ;;
>         *) die ... ;;
>     esac
> 
> as EAPI 2 supports the --missing case via constructs as:
> 
>     || (
>         >=foo/bar-1
>         <foo/bar-1[baz]
>     )

I'd rather be more aggressive on this one, actually; either way, for 
herds/devs, a full scan of the tree was done to identify what invokes 
bulit_with_use whether directly, or indirectly via invoking a function 
that does.

Resultant logs/tree is at 
http://dev.gentoo.org/~ferringb/built_with_use-cleanup/ .

For the 'env' files that are nestled away in there, that's a pkgcore 
dump of the environment of the ebuild post sourcing for ease of 
tracing how built_with_use was actually invoked.

Sidenote, if in looking at the env dumps you see something that looks 
like it shouldn't be saved, let me know- I'm generally pretty anal 
about trying to ensure nothing pkgcore related is accessible by 
ebuilds/eclasses, nor saved to the env.


> Almost all affected packages can be bumped straight to 4 anyway and
> so use the improved syntax.

~11% already are EAPI4, just triggered via eclass pathways.

Either way, herds, please take a look- the views should make it easy 
for y'all to trace down the offenses and deal with them.

~harring

Re: [gentoo-dev] Re: [PATCH] eutils: Warn on built_with_use usage

[Brian Harring]

On Tue, Sep 18, 2012 at 02:35:42PM -0600, Ryan Hill wrote:
> > On Mon, Sep 17, 2012 at 08:45:22AM +0200, Ralph Sennhauser wrote:
> 
> > > Almost all affected packages can be bumped straight to 4 anyway and
> > > so use the improved syntax.
> 
> toolchain_src_compile: EAPI=0: count: 38
> 
> I'm not sure this can change any time soon. :/

Honestly, if it was knocked down to <50 left in the tree, I'd be fine 
with it.

Once reaching that level, the following could be done:

1) mangling built_with_use adding a an explicit whitelist of callers, 
making everything else a die.
2) keep the existing /var/db/pkg format aware code in place.
3) Expand the function w/ manager aware code as necessary to deal w/ 
said managers potentially shifting to new vdb forms.

I doubt ciaran will like #3, but if we can get it down to .15% of the 
tree at that point pragmatism should rule the day; as long as the PM 
has alternate ways to query the info without accessing the VDB 
directly (even if those ways are outside PMS), I'm fine with it, and I 
expect most folk would be.

~harring

[gentoo-dev] Re: [PATCH] eutils: Warn on built_with_use usage

[Ryan Hill]

[-- Attachment #1: Type: text/plain, Size: 453 bytes --]

> On Mon, Sep 17, 2012 at 08:45:22AM +0200, Ralph Sennhauser wrote:

> > Almost all affected packages can be bumped straight to 4 anyway and
> > so use the improved syntax.

toolchain_src_compile: EAPI=0: count: 38

I'm not sure this can change any time soon. :/


-- 
gcc-porting
toolchain, wxwidgets          we were never more here, expanse getting broader
@ gentoo.org                          but bigger boats been done by less water

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[gentoo-dev] Re: [PATCH] eutils: Warn on built_with_use usage

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 216 bytes --]

On Sunday 16 September 2012 22:41:14 Brian Harring wrote:
> Sans the implementation details, anyone got complaints with the
> intent?

sounds like no, so i'll probably pound something out once i finish perf :p
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]