[gentoo-dev] eclass error-handling post-EAPI4

[gentoo-dev] eclass error-handling post-EAPI4

[Gregory M. Turner]

I'm cooking up some eclass functions and it occurred to me that perhaps 
I had some responsibility regarding EAPI4's new error handling 
semantics.  After looking into it, it seems that, superficially, the 
answer to my question is "no, the EAPI4 changes only apply to ebuild 
helpers."

Even so, this got me thinking: if ebuild helpers are going to warn(*) in 
EAPI<4 and die in EAPI>=4 (unless somebody used "nonfatal"), then, those 
error-handling semantics are going to bubble up through my eclass to its 
heirs (assuming my eclass uses helpers).

The problem I see with that is that a consumer of my eclass function is 
effectively required to anticipate these EAPI helper-function outcomes, 
which is fair-enough if s/he's consuming them directly, but pretty 
unreasonable if they're just consuming my eclass....  In effect, this 
seems to create a situation where they can't know what to expect, 
without wading through my code, unless I go in and document "in EAPI4+ 
it does X, but in EAPI3, it does Y".  Which, let's face it, most eclass 
authors probably won't get around to doing.

For example, suppose poop.eclass has a function "epoop" which performs 
the following steps:

   o Downloads the latest "post" from the
     http://www.unnecessaryquotes.com/ "website" using "wget"
   o Strips out the top 3 posts from piped wget output using
     a bunch of sed and awk magic.  Saves each post-processed post as
     "${S}/$(mktemp -u $( printf %100s |tr " " "X" )).xml" and
     remembers these filenames in three hard-coded variables.
   o Installs the posts as documentation using dodoc statements
   o Returns "$(( $(date +'%s') % 3 ))"

Let's say I started writing this code before I knew anything about EAPI 
4, and the current epoop documentation states that:

  o errors in the wget and postprocessing steps will result in a return
    code of '3'
  o errors from dodoc will result in a return code of '3' + the
    return-code of dodoc
  o epoop() succeeds every third second and otherwise fails, returning
    the number of seconds ago that the API could have been invoked
    successfully as the error code (WTF: this is tounge-in-cheek:
    $(( $(date +'%s') % 3 )) is the current UNIX time in
    seconds, modulo 3, an extremely pointless and arbitrary success
    criterion for my hypothetical function that will cause it to
    randomly fail 66% of the times it's invoked and runs to completion.
    If you don't find this amusing, that's OK; it's not terribly
    important to the matter at hand.)

Well, now I have an awkward situation.  In EAPI's 0-3, the function 
works as documented.  In EAPI 4+, however, epoop() will die if any error 
occurs in dodoc.  I can fix this by replacing the dodoc code -- let's 
say it originally was coded like:

   dodoc ${docfile1} || { r=$?; eerror dodoc1; return $((r+3)); }
   dodoc ${docfile2} || { r=$?; eerror dodoc2; return $((r+3)); }
   dodoc ${docfile3} || { r=$?; eerror dodoc3; return $((r+3)); }

-- with code like:

   .
   .
   if has "${EAPI:-0}" 0 1 2 3 ; then
           nonfatal dodoc ${docfile2}
   else
           dodoc ${docfile2}
   fi || { r=$?; eerror dodoc2; return $((r+3)); }
   .
   .

but I will have to do something like this for each of the three dodoc 
statements.  Seems like a real PITA.

Obviously, I've chosen a very silly example, and perhaps I've 
cherry-picked a solution that illustrates my point -- but it's not hard 
to imagine having to make fairly significant changes to an eclass in 
order to preserve its pre-EAPI4 semantics in EAPI4+.

The conclusion I'm tempted to draw is that it makes more sense to bring 
my eclass error-handling behavior into line with the EAPI-specific 
behavior of helper-functions than it does to gum up helper usage in my 
eclass with a bunch of EAPI-specific error-handling spaghetti-code.

i.e., returning to epoop: instead of the above solution, I might leave 
the dodoc code alone, and instead implement something vaguely like 
__helpers_die, i.e.:

   # note: stupidly clever -- not how i'd implement this irl
   _poop_die_maybe() {
           local ret="$?" eapi="${EAPI:-0}"
           eapi=${eapi::1}
           (( (3-eapi)*(PORTAGE_NONFATAL-1) > 0 )) && die "$@"
           eerror "$@"
           return ${ret} # pass through pre-invocation $? unchanged
   }

With this in place, it's not terribly difficult to bring the 
error-handling/EAPI semantics of epoop() into line with those of the 
helper functions across all EAPIs.  For example, where we had

   return $(( $(date +'%s') % 3 ))

we might now put

   ret="$(( $(date +'%s') % 3 ))"
   (( ret == 0 )) || \
           _poop_die_maybe "epoop: invoked ${ret} seconds too late"
   return "${ret}"

Or, if it was the last statement in the epoop function, just:

   ( exit $(( $(date +'%s') % 3 )) ) || \
           _poop_die_maybe "epoop: invoked $? seconds too late"

I hope my gist isn't getting lost in the particulars of this silly 
example.  My point is: EAPI4 changes the behavior of eclasses if those 
eclasses consume helper functions, which could lead to some pretty 
muddled error-handling semantics without some kind of clarification or 
reimplementation.  Rather than dance around EAPI4 in our eclasses, 
perhaps it's more elegant to try to approximate the helper-function 
error handling semantics at the eclass level, across all EAPI's, which 
would mean dying for "any kind of error" in EAPI4+ and warning and 
returning failure codes in EAPI3-.

Stated more strongly, perhaps EAPI auto-die effectively sets us on a 
course where eventually "everything else" really ought to work the same 
way as the PMS mandates for helpers, avoiding an otherwise confusing 
situation where ebuild authors constantly need to second-guess eclass 
behavior: "yes, but specifically how does it behave in EAPI4+/3-, so I 
can put die() in the right places?"

If so, this raises a second question: should portage provide some kind 
of facility to uh.. facilitate the needs of eclass authors looking to 
implement these error-handling patterns, rather than leave the correct 
implementation of something like _epoop_die_maybe() "as an exercise to 
the coder"?

  ... does anyone see a better approach?  Perhaps this a non-issue for 
some reason I'm not grokking?  I'd love to hear your thoughts -- the 
exact "right" answer isn't so clear to me, and, frankly, I'd like to 
move on to my eclass writing.

Thanks,

-gmt

--------

* Hey, can someone explain why, when __helpers_die decides die()ing is 
not called-for (due either to EAPI<4 or PORTAGE_NONFATAL==1), it 
unceremoniously dumps the error message to stderr, rather than, i.e., 
calling one of the __elog_base-consuming functions?  I really don't get 
that.

Re: [gentoo-dev] eclass error-handling post-EAPI4

[Zac Medico]

On 10/19/2012 07:23 AM, Gregory M. Turner wrote:
> I'm cooking up some eclass functions and it occurred to me that perhaps
> I had some responsibility regarding EAPI4's new error handling
> semantics.  After looking into it, it seems that, superficially, the
> answer to my question is "no, the EAPI4 changes only apply to ebuild
> helpers."
> 
> Even so, this got me thinking: if ebuild helpers are going to warn(*) in
> EAPI<4 and die in EAPI>=4 (unless somebody used "nonfatal"), then, those
> error-handling semantics are going to bubble up through my eclass to its
> heirs (assuming my eclass uses helpers).

Regardless of EAPI, don't call the helpers that die in EAPI 4 unless you
want the function to die when the helpers fail, and use helper || die so
it behaves the same regardless of EAPI. It's okay to die in older EAPIs,
as long as you're not changing the behavior of a previously existing
eclass function. Previously existing eclass functions that only start to
die in EAPI 4 are okay, since they only start to die when the ebuild
developer bumps the EAPI.
-- 
Thanks,
Zac

Re: [gentoo-dev] eclass error-handling post-EAPI4

[Gregory M. Turner]

Thanks for your reply, but I still have some concerns about this.

On 10/19/2012 8:11 AM, Zac Medico wrote:
> Regardless of EAPI, don't call the helpers that die in EAPI 4 unless you
> want the function to die when the helpers fail, and use helper || die so
> it behaves the same regardless of EAPI.

Took me a while, but I think I see why this is correct, now (mostly -- 
see below).  The source of my confusion was a mistaken assumption that 
die() would not respect PORTAGE_NONFATAL.  But isolated-functions.sh has 
roughly:

   die()
   {
           [[ ${PORTAGE_NONFATAL} == 1 ]] && {
                   echo "${FUNCNAME[1]}: WARNING: $*" >&2
                   return 1
           }
           .
           .

So -- correct me if I'm wrong -- this all works as follows:

   o helpers always handle errors by remembering the exit code of some
     error-prone operation, and, if that exit code is nonzero, invoking
     __helpers_die with some description of the problem, and finally
     returning the remembered exit code.

     In EAPI[0-3], this will dump the error text to stderr and return
     the remembered exit code.

     In >=EAPI4, this will call isolated-functions.sh's die(), which will
     swoop in and terminate everything before it has the chance to
     go any further, effectively ensuring that __helpers_die never
     returns -- /unless/ PORTAGE_NONFATAL is 1, in which case, it will
     prepend "__helpers_die:" to the error message, but otherwise
     behave as in EAPI[0-3].

   o die incantations in an ebuild or eclass will either directly or
     indirectly call die() in isolated-functions.sh, which still works
     as described above when used in these contexts -- which is to say,
     that die is nonfatal when PORTAGE_NONFATAL is set.

   o PORTAGE_NONFATAL is only supposed to be manipulated by means of
     the nonfatal API, which is only callable in >=EAPI4, which means
     that unless somebody "cheated" by directly messing with the
     variable, in EAPI[0-3], die() never returns (like "exit")

> It's okay to die in older EAPIs, as long as you're not changing the
 > behavior of a previously existing eclass function.

ACK

> Previously existing eclass functions that only start to
> die in EAPI 4 are okay, since they only start to die when the ebuild
> developer bumps the EAPI.

ACK

If I indeed understand the facts correctly, I'm still a bit 
uncomfortable with your advice to just use "helper || die" in eclass 
code.  It seems to me that if I follow this recipe, >=EAPI4 kinda works 
OK, but EAPI[0-3] doesn't.

Specifically, an EAPI[0-3] ebuild author will not have any means at his 
or her disposal to handle errors that occur in eclass utility functions. 
  The error-handling semantics of eclass utility functions would be like 
EAPI4, except without nonfatal around to provide relief from auto-die.

EAPI[0-3] conventions encourage the habit of writing ebuild code like 
normal bash script code, which is to say, that authors expect to be able 
to write code like

   invoke_fn || ... # handle error, probably by die()ing but maybe not

But eclasses that auto-die violate this expectation and create a 
situation where ebuild authors must be mindful of whether a particular 
function is a helper-function or an eclass utility function to correctly 
anticipate whether auto-death will occur.

This seems to be by design and may not be so horrible as I make it sound 
-- for example, it would be pretty weird to expect eclass code written 
before EAPI4 to suddenly start working differently in EAPI[0-3].

But should we really write new eclass code to behave this way as well?

Worse, eclasses authored in the pre-EAPI4 era ubiquitously assume that 
die() never returns, and do things like:

   efoo() {
           [[ ${EBUILD_PHASE} != "prepare" ]] &&
                   die "efoo can only be used during src_prepare"
           .
           .
           # stuff that isn't safe if ${EBUILD_PHASE} != "prepare"
           .
           .
   }

Since, in EAPI4, no means is provided to specify that a particular 
invocation of die() is unconditionally terminal, (except directly 
manipulating PORTAGE_NONFATAL, which doesn't seem to be encouraged), the 
only non-encapsulation-breaking non-EAPI-specific solution would be to 
re-code every presumed-terminal use of die() to look something like:

   failure_prone_action
   ret=$?
   if [[ ${ret} != 0 ]] ; then
           die "Error message"
           return ${ret}
   fi

I can imagine no reasonable way to avoid this without using aliases... 
or perhaps some construct like:

   do_or_die() {
           diemsg="$1"
           shift
           "$@" || { ret=$?; die "${diemsg}"; return ${ret}; }
   }

   efoo() {
           do_or_die "error message 1" \
                   failure_prone_action1 arg1 arg2 arg3 && \
           safe_action1 arg1 && \
           safe_action2 arg1 arg2 && \
           do_or_die "error message 2" \
                   failure_prone_action2 arg1 arg2 && \
           .
           .

Hopefully I'm missing something here...?

-gmt

Re: [gentoo-dev] eclass error-handling post-EAPI4

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 442 bytes --]

On Sat, 20 Oct 2012 03:52:49 -0700
"Gregory M. Turner" <gmt@malth.us> wrote:
> Took me a while, but I think I see why this is correct, now (mostly
> -- see below).  The source of my confusion was a mistaken assumption
> that die() would not respect PORTAGE_NONFATAL.

The source of your confusion is more the impression that there is such
a thing as PORTAGE_NONFATAL. You should be reading the spec, not code.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] eclass error-handling post-EAPI4

[Zac Medico]

On 10/20/2012 03:52 AM, Gregory M. Turner wrote:
> If I indeed understand the facts correctly, I'm still a bit
> uncomfortable with your advice to just use "helper || die" in eclass
> code.  It seems to me that if I follow this recipe, >=EAPI4 kinda works
> OK, but EAPI[0-3] doesn't.
> 
> Specifically, an EAPI[0-3] ebuild author will not have any means at his
> or her disposal to handle errors that occur in eclass utility functions.
>  The error-handling semantics of eclass utility functions would be like
> EAPI4, except without nonfatal around to provide relief from auto-die.
> 
> EAPI[0-3] conventions encourage the habit of writing ebuild code like
> normal bash script code, which is to say, that authors expect to be able
> to write code like
> 
>   invoke_fn || ... # handle error, probably by die()ing but maybe not
> 
> But eclasses that auto-die violate this expectation and create a
> situation where ebuild authors must be mindful of whether a particular
> function is a helper-function or an eclass utility function to correctly
> anticipate whether auto-death will occur.
> 
> This seems to be by design and may not be so horrible as I make it sound
> -- for example, it would be pretty weird to expect eclass code written
> before EAPI4 to suddenly start working differently in EAPI[0-3].
> 
> But should we really write new eclass code to behave this way as well?

Yeah, I think we should. If the eclass function is doing something that
might result in an undesirable die, then it can try to avoid the die by
first doing a sanity check. For example, it can check that a given file
path exists before passing it to the helper, and skip calling the helper
if the file path doesn't exist.

> Worse, eclasses authored in the pre-EAPI4 era ubiquitously assume that
> die() never returns, and do things like:
> 
>   efoo() {
>           [[ ${EBUILD_PHASE} != "prepare" ]] &&
>                   die "efoo can only be used during src_prepare"
>           .
>           .
>           # stuff that isn't safe if ${EBUILD_PHASE} != "prepare"
>           .
>           .
>   }
> 
> Since, in EAPI4, no means is provided to specify that a particular
> invocation of die() is unconditionally terminal, (except directly
> manipulating PORTAGE_NONFATAL, which doesn't seem to be encouraged), the
> only non-encapsulation-breaking non-EAPI-specific solution would be to
> re-code every presumed-terminal use of die() to look something like:
> 
>   failure_prone_action
>   ret=$?
>   if [[ ${ret} != 0 ]] ; then
>           die "Error message"
>           return ${ret}
>   fi
> 
> I can imagine no reasonable way to avoid this without using aliases...
> or perhaps some construct like:
> 
>   do_or_die() {
>           diemsg="$1"
>           shift
>           "$@" || { ret=$?; die "${diemsg}"; return ${ret}; }
>   }
> 
>   efoo() {
>           do_or_die "error message 1" \
>                   failure_prone_action1 arg1 arg2 arg3 && \
>           safe_action1 arg1 && \
>           safe_action2 arg1 arg2 && \
>           do_or_die "error message 2" \
>                   failure_prone_action2 arg1 arg2 && \
>           .
>           .
> 
> Hopefully I'm missing something here...?

I never really liked the nonfatal helper. If you don't want a helper to
die, then I'd suggest to use sanity checks before calling the helper,
and avoid calling the helper if it seems like it will trigger an
unwanted die.
-- 
Thanks,
Zac

Re: [gentoo-dev] eclass error-handling post-EAPI4

[Gregory M. Turner]

On 10/20/2012 4:05 AM, Ciaran McCreesh wrote:
> On Sat, 20 Oct 2012 03:52:49 -0700
> "Gregory M. Turner" <gmt@malth.us> wrote:
>> Took me a while, but I think I see why this is correct, now (mostly
>> -- see below).  The source of my confusion was a mistaken assumption
>> that die() would not respect PORTAGE_NONFATAL.
>
> The source of your confusion is more the impression that there is such
> a thing as PORTAGE_NONFATAL. You should be reading the spec, not code.

Seriously?  OK, let's try:

   http://dev.gentoo.org/~ulm/pms/4/pms.html#x1-13000012.3.3.1

and

   http://dev.gentoo.org/~ulm/pms/4/pms.html#x1-13500012.3.3.6

do not comport with the actual behavior of portage.

Specifically, they very strongly suggest that nonfatal does not 
influence the behavior of die().  So depending on one's reading of the 
situation, either the PMS is misspecified or portage is broken.  Meanwhile,

 
http://dev.gentoo.org/~zmedico/portage/doc/ch05s03s05.html#package-ebuild-eapi-4-helpers-die-nonfatal

is dusting so much under the carpet that I can't possibly do my job as 
an eclass author without reverse-engineering, dry-run code analysis, or 
asking the SME's here on the mailing list how this is supposed to work 
in practice.

I've already asked the mailing list and so far the closest thing I have 
to an answer is Zac's response.  Although I appreciate his help, I've 
explained why his response doesn't fully address my concerns.  If you 
don't trust my dry-run analysis then we can try the reverse-engineering 
approach.  Give this ebuild a try:

---->8----
# Copyright 1999-2012 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: $

EAPI=4
DESCRIPTION="Foo"
SLOT="0"
KEYWORDS="~x86 ~amd64 ~x86-linux ~amd64-linux"
IUSE=""
DEPEND=""
RDEPEND="${DEPEND}"
utility_fn() { die "uhoh"; }
src_unpack() { mkdir -p "${S}"; }
src_prepare() { nonfatal utility_fn; }
src_compile() { echo "yaaay" > foo.txt; }
src_install() { dodoc foo.txt; }
----8<----

As you can see, my dry-run analysis appears to have been correct.

There is no documentation concerning best practices for eclass authors 
post-EAPI4, so I can't just RTFM -- there is no FM.  Furthermore, it 
appears that EAPI4 has broken a great many eclasses when invoked with 
the new nonfatal operator.  I guess I can just file a bug for that since 
so far nobody seems too concerned by it, but that still doesn't help me 
to write my eclass.

-gmt

Re: [gentoo-dev] eclass error-handling post-EAPI4

[Zac Medico]

On 10/20/2012 02:24 PM, Gregory M. Turner wrote:
> On 10/20/2012 4:05 AM, Ciaran McCreesh wrote:
>> On Sat, 20 Oct 2012 03:52:49 -0700
>> "Gregory M. Turner" <gmt@malth.us> wrote:
>>> Took me a while, but I think I see why this is correct, now (mostly
>>> -- see below).  The source of my confusion was a mistaken assumption
>>> that die() would not respect PORTAGE_NONFATAL.
>>
>> The source of your confusion is more the impression that there is such
>> a thing as PORTAGE_NONFATAL. You should be reading the spec, not code.
> 
> Seriously?  OK, let's try:
> 
>   http://dev.gentoo.org/~ulm/pms/4/pms.html#x1-13000012.3.3.1
> 
> and
> 
>   http://dev.gentoo.org/~ulm/pms/4/pms.html#x1-13500012.3.3.6
> 
> do not comport with the actual behavior of portage.
> 
> Specifically, they very strongly suggest that nonfatal does not
> influence the behavior of die().  So depending on one's reading of the
> situation, either the PMS is misspecified or portage is broken.

I think portage's die should be fixed so that it dies regardless of the
nonfatal helper.

> Meanwhile,
> 
> 
> http://dev.gentoo.org/~zmedico/portage/doc/ch05s03s05.html#package-ebuild-eapi-4-helpers-die-nonfatal
> 
> 
> is dusting so much under the carpet that I can't possibly do my job as
> an eclass author without reverse-engineering, dry-run code analysis, or
> asking the SME's here on the mailing list how this is supposed to work
> in practice.
> 
> I've already asked the mailing list and so far the closest thing I have
> to an answer is Zac's response.  Although I appreciate his help, I've
> explained why his response doesn't fully address my concerns.  If you
> don't trust my dry-run analysis then we can try the reverse-engineering
> approach.  Give this ebuild a try:
> 
> ---->8----
> # Copyright 1999-2012 Gentoo Foundation
> # Distributed under the terms of the GNU General Public License v2
> # $Header: $
> 
> EAPI=4
> DESCRIPTION="Foo"
> SLOT="0"
> KEYWORDS="~x86 ~amd64 ~x86-linux ~amd64-linux"
> IUSE=""
> DEPEND=""
> RDEPEND="${DEPEND}"
> utility_fn() { die "uhoh"; }
> src_unpack() { mkdir -p "${S}"; }
> src_prepare() { nonfatal utility_fn; }
> src_compile() { echo "yaaay" > foo.txt; }
> src_install() { dodoc foo.txt; }
> ----8<----
> 
> As you can see, my dry-run analysis appears to have been correct.
> 
> There is no documentation concerning best practices for eclass authors
> post-EAPI4, so I can't just RTFM -- there is no FM.  Furthermore, it
> appears that EAPI4 has broken a great many eclasses when invoked with
> the new nonfatal operator.  I guess I can just file a bug for that since
> so far nobody seems too concerned by it, but that still doesn't help me
> to write my eclass.

As far as I can tell, nonfatal was only intend ed to work with package
manager built-in helpers. I don't think it's a good idea to try and
extend it to eclass functions.
-- 
Thanks,
Zac

Re: [gentoo-dev] eclass error-handling post-EAPI4

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1590 bytes --]

On Sat, 20 Oct 2012 14:24:29 -0700
"Gregory M. Turner" <gmt@malth.us> wrote:
> On 10/20/2012 4:05 AM, Ciaran McCreesh wrote:
> > On Sat, 20 Oct 2012 03:52:49 -0700
> > "Gregory M. Turner" <gmt@malth.us> wrote:
> >> Took me a while, but I think I see why this is correct, now (mostly
> >> -- see below).  The source of my confusion was a mistaken
> >> assumption that die() would not respect PORTAGE_NONFATAL.
> >
> > The source of your confusion is more the impression that there is
> > such a thing as PORTAGE_NONFATAL. You should be reading the spec,
> > not code.
> 
> Seriously?  OK, let's try:
> 
>    http://dev.gentoo.org/~ulm/pms/4/pms.html#x1-13000012.3.3.1
> 
> and
> 
>    http://dev.gentoo.org/~ulm/pms/4/pms.html#x1-13500012.3.3.6
> 
> do not comport with the actual behavior of portage.
> 
> Specifically, they very strongly suggest that nonfatal does not 
> influence the behavior of die().  So depending on one's reading of
> the situation, either the PMS is misspecified or portage is broken.

They don't just suggest it. It's stated outright in the description of
nonfatal:

    Executes the remainder of its arguments as a command,
    preserving the exit status. If this results in a command being
    called that would normally abort the build process due to a failure
    (but not due to an explicit die or assert call), instead a non-zero
    exit status shall be returned. Only in EAPIs listed in table 12.5
    as supporting nonfatal.

So if 'nonfatal die' doesn't die with Portage, you've found a bug.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] eclass error-handling post-EAPI4

[Gregory M. Turner]

On 10/20/2012 11:16 AM, Zac Medico wrote:
> On 10/20/2012 03:52 AM, Gregory M. Turner wrote:
>> EAPI[0-3] conventions encourage the habit of writing ebuild code like
>>
>>    invoke_fn || ... # handle error, probably by die()ing but maybe not
>>
>> But eclasses that auto-die violate this expectation and create a
>> situation where ebuild authors must be mindful of whether a particular
>> function is a helper-function or an eclass utility function to correctly
>> anticipate whether auto-death will occur.
>>
>>should we really write new eclass code to behave this way as well?
>
> Yeah, I think we should. If the eclass function is doing something that
> might result in an undesirable die, then it can try to avoid the die by
> first doing a sanity check. For example, it can check that a given file
> path exists before passing it to the helper, and skip calling the helper
> if the file path doesn't exist.

That seems reasonable -- in practice, I don't find myself needing to 
invoke helpers very much at all in eclasses, nor does it seem too 
terribly frequent in the existing eclasses (dodir is pretty common, but 
unlikely to fail unless something is seriously wrong anyhow).

Would it be feasible to lift the >=EAPI4 restriction on nonfatal()?  I 
guess the problem would be that old portage's wouldn't have it.

Anyhow, given that eclasses can probably avoid auto-die() if they need 
to, this leaves the issue of nonfatal allowing die to return to 
callers... is there really any reason that die() needs to work this way? 
  If helpers always used __helpers_die() instead of die(), then couldn't 
die() be changed to ignore PORTAGE_NONFATAL?

It almost seems like the portage code was designed for nonfatal only to 
apply to helper invocations, and then the change to die() was added as 
an afterthought, to support the broader range of usages suggested 
(rather vaguely, I'd say) by the language in the PMS, ... yet somehow, 
the PMS ends up strongly suggesting -- I think most people would read 
the language as an explicit specification -- that ebuilds can rely on 
die() to terminate regardless of any usage of nonfatal.

-gmt

-gmt

Re: [gentoo-dev] eclass error-handling post-EAPI4

[Zac Medico]

On 10/20/2012 03:51 PM, Gregory M. Turner wrote:
> On 10/20/2012 11:16 AM, Zac Medico wrote:
>> On 10/20/2012 03:52 AM, Gregory M. Turner wrote:
>>> EAPI[0-3] conventions encourage the habit of writing ebuild code like
>>>
>>>    invoke_fn || ... # handle error, probably by die()ing but maybe not
>>>
>>> But eclasses that auto-die violate this expectation and create a
>>> situation where ebuild authors must be mindful of whether a particular
>>> function is a helper-function or an eclass utility function to correctly
>>> anticipate whether auto-death will occur.
>>>
>>> should we really write new eclass code to behave this way as well?
>>
>> Yeah, I think we should. If the eclass function is doing something that
>> might result in an undesirable die, then it can try to avoid the die by
>> first doing a sanity check. For example, it can check that a given file
>> path exists before passing it to the helper, and skip calling the helper
>> if the file path doesn't exist.
> 
> That seems reasonable -- in practice, I don't find myself needing to
> invoke helpers very much at all in eclasses, nor does it seem too
> terribly frequent in the existing eclasses (dodir is pretty common, but
> unlikely to fail unless something is seriously wrong anyhow).
> 
> Would it be feasible to lift the >=EAPI4 restriction on nonfatal()?  I
> guess the problem would be that old portage's wouldn't have it.

No. The closest thing you can do is drop support for older EAPIs in your
eclass (just die in global scope at the beginning of the eclass if the
EAPI is unsupported).

> Anyhow, given that eclasses can probably avoid auto-die() if they need
> to, this leaves the issue of nonfatal allowing die to return to
> callers... is there really any reason that die() needs to work this way?
>  If helpers always used __helpers_die() instead of die(), then couldn't
> die() be changed to ignore PORTAGE_NONFATAL?

Yes. It's a bug, as Ciaran said.

> It almost seems like the portage code was designed for nonfatal only to
> apply to helper invocations, and then the change to die() was added as
> an afterthought, to support the broader range of usages suggested

You can see here that the special die() behavior was added in the same
commit when nonfatal() was added:

http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=d8d45379b52f5330c6cec616d675706eb0ba2599

> (rather vaguely, I'd say) by the language in the PMS, ... yet somehow,
> the PMS ends up strongly suggesting -- I think most people would read
> the language as an explicit specification -- that ebuilds can rely on
> die() to terminate regardless of any usage of nonfatal.

Yeah, I think portage's special die() behavior is clearly a bug.
-- 
Thanks,
Zac

Re: [gentoo-dev] eclass error-handling post-EAPI4

[Gregory M. Turner]

On 10/20/2012 4:07 PM, Zac Medico wrote:
> On 10/20/2012 03:51 PM, Gregory M. Turner wrote:
>> Anyhow, given that eclasses can probably avoid auto-die() if they need
>> to, this leaves the issue of nonfatal allowing die to return to
>> callers... is there really any reason that die() needs to work this way?
>>   If helpers always used __helpers_die() instead of die(), then couldn't
>> die() be changed to ignore PORTAGE_NONFATAL?
>
> Yes. It's a bug, as Ciaran said.

For anyone following this discussion, just a note that I've created a 
bug for this issue:

https://bugs.gentoo.org/show_bug.cgi?id=439356

A proposed patch is attached to the bug.

-gmt