[gentoo-dev] Making dobin, doexe, doins, doman, dodoc die by default

[gentoo-dev] Making dobin, doexe, doins, doman, dodoc die by default

[Stefan Schweizer]

Hi,

This came up in Bug 138792 [dobin etc. should automatically die on failure]
It needs more discussion on the mailing lists.
Some excerpts from the bug: The proposal from Paul Bredbury:
"Hi, I propose that the following ebuild commands themselves *die* on
failure, because the vast majority of the time the emerge might as well be
considered a failure if such commands fail.
dobin, dosbin, doexe"

Jason Stubbs called for consistency .. i.e making doman and dodoc also die
when nothing the file does not exist. A simple workaround in case an ebuild
is broken: [ -f xxx ] && dodoc xxx

SpanKY complained that he cannot set a custom die message then. But this is
not needed here, since every do* command can be clearly identified by the
argument and the directory it will be installed to.
Also as Paul suggested something like that would be possible for a custom
die message:
        if [[ -n "${DIE_MSG}" ]] ; then
                echo "!!! ${DIE_MSG}"
        fi

So, because we want no broken ebuilds and we want consistency I propose to
change this and fix possible problems. They are imo QA problems and should
be fixed.

So what do you think?

- Stefan

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Making dobin, doexe, doins, doman, dodoc die by default

[Aron Griffis]

Stefan Schweizer wrote:  [Wed Jul 12 2006, 01:37:44PM EDT]
> This came up in Bug 138792 [dobin etc. should automatically die on failure]

Since do* would become functions in this case, you'll have to fix the
few ebuilds that use them on the RHS of xargs.

    grep -r --include \*.ebuild -E 'xargs do(bin|exe|ins|man|doc)' .

Assuming the list is relatively short, it should be acceptable to
convert these to something like:

    doman $(find man -name '*.?.gz')

The speedtouch ebuild is screwy anyway.  There's no reason for the
echo.

Aron
-- 
gentoo-dev@gentoo.org mailing list

[gentoo-dev] Re: Making dobin, doexe, doins, doman, dodoc die by default

[Stefan Schweizer]

Aron Griffis wrote:
> Stefan Schweizer wrote:  [Wed Jul 12 2006, 01:37:44PM EDT]
>> This came up in Bug 138792 [dobin etc. should automatically die on
>> failure]
> 
> Since do* would become functions in this case, you'll have to fix the
> few ebuilds that use them on the RHS of xargs.
> 
>     grep -r --include \*.ebuild -E 'xargs do(bin|exe|ins|man|doc)' .

./local/layman/hanno-xgl/media-libs/mesa/mesa-6.5.1_pre20060620.ebuild: find
${S}/lib* -name '*_dri.so' | xargs doexe
./local/layman/hanno-xgl/media-libs/mesa/mesa-6.5.1_pre20060627.ebuild: find
${S}/lib* -name '*_dri.so' | xargs doexe
./media-libs/mesa/mesa-6.5-r3.ebuild:   find ${S}/lib* -name '*_dri.so' |
xargs doexe
./media-libs/mesa/mesa-6.4.2-r2.ebuild: find ${S}/lib* -name '*_dri.so' |
xargs doexe
./app-emulation/vmware-gsx-console/vmware-gsx-console-3.2.0.14497.ebuild:      
find man -name \*.\?.gz | xargs doman

> Assuming the list is relatively short, it should be acceptable to
> convert these to something like:
> 
>     doman $(find man -name '*.?.gz')

I just pinged MattM about vmware-gsx-console, and the x11 guys about mesa
and some minutes ago fixed timidity-eawpatches and speedtouch.

Thanks :)

- Stefan

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Making dobin, doexe, doins, doman, dodoc die by default

[Jakub Moc]

[-- Attachment #1: Type: text/plain, Size: 821 bytes --]

Stefan Schweizer wrote:
> Jason Stubbs called for consistency .. i.e making doman and dodoc also die
> when nothing the file does not exist. A simple workaround in case an ebuild
> is broken: [ -f xxx ] && dodoc xxx

Uh... Sorry but it's pretty hard to imagine something more annoying than
an ebuild that dies after a couple of hours compile just because
upstream decided to rename Changelog.txt to ChangeLog.txt and noone
noticed  during version bump, or because someone made a typo there. Fail
to see any benefit from this... :S Ditto for manpages.


-- 
Best regards,

 Jakub Moc
 mailto:jakub@gentoo.org
 GPG signature:
 http://subkeys.pgp.net:11371/pks/lookup?op=get&search=0xCEBA3D9E
 Primary key fingerprint: D2D7 933C 9BA1 C95B 2C95  B30F 8717 D5FD CEBA 3D9E

 ... still no signature   ;)


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Making dobin, doexe, doins, doman, dodoc die by default

[Henrik Brix Andersen]

[-- Attachment #1: Type: text/plain, Size: 630 bytes --]

On Wed, Jul 12, 2006 at 11:11:01PM +0200, Jakub Moc wrote:
> Uh... Sorry but it's pretty hard to imagine something more annoying than
> an ebuild that dies after a couple of hours compile just because
> upstream decided to rename Changelog.txt to ChangeLog.txt and noone
> noticed  during version bump, or because someone made a typo there. Fail
> to see any benefit from this... :S Ditto for manpages.

How could that slip through the initial testing of the ebuild
performed by the developer doing the version bump?

./Brix
-- 
Henrik Brix Andersen <brix@gentoo.org>
Gentoo Metadistribution | Mobile computing herd

[-- Attachment #2: Type: application/pgp-signature, Size: 211 bytes --]

Re: [gentoo-dev] Making dobin, doexe, doins, doman, dodoc die by default

[Diego 'Flameeyes' Pettenò]

[-- Attachment #1: Type: text/plain, Size: 795 bytes --]

On Wednesday 12 July 2006 23:21, Henrik Brix Andersen wrote:
> How could that slip through the initial testing of the ebuild
> performed by the developer doing the version bump?
I think that is the point, during testing. If I'm testing the version bump of 
something that takes 2 hours to build, I'll be happy to see a big fat warning 
about it, but not that happy to see it fail entirely.

Also, we already discussed in the past that there are cases in which dodoc is 
called within eclasses with a series of names that might or might not exists, 
because they are mostly standard, and left to dodoc to find the ones good and 
the ones not.

-- 
Diego "Flameeyes" Pettenò - http://farragut.flameeyes.is-a-geek.org/
Gentoo/Alt lead, Gentoo/FreeBSD, Video, AMD64, Sound, PAM, KDE

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Making dobin, doexe, doins, doman, dodoc die by default

[Steve Dibb]

Henrik Brix Andersen wrote:
> On Wed, Jul 12, 2006 at 11:11:01PM +0200, Jakub Moc wrote:
>   
>> Uh... Sorry but it's pretty hard to imagine something more annoying than
>> an ebuild that dies after a couple of hours compile just because
>> upstream decided to rename Changelog.txt to ChangeLog.txt and noone
>> noticed  during version bump, or because someone made a typo there. Fail
>> to see any benefit from this... :S Ditto for manpages.
>>     
>
> How could that slip through the initial testing of the ebuild
> performed by the developer doing the version bump?
Well, it could happen while testing an ebuild. :)  I'd be pretty ticked 
if I were testing Qt and I didn't realize they did change the doc files 
around before doing a test run.

Besides that though, imho, a simple function with a boolean return type 
shouldn't kill the script executing it.  Throw a warning, yes, but not 
stop everything.

Steve
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Making dobin, doexe, doins, doman, dodoc die by default

[John Myers]

[-- Attachment #1: Type: text/plain, Size: 625 bytes --]

On Wednesday 12 July 2006 14:36, Steve Dibb wrote:
> Well, it could happen while testing an ebuild. :)  I'd be pretty ticked
> if I were testing Qt and I didn't realize they did change the doc files
> around before doing a test run.
>
> Besides that though, imho, a simple function with a boolean return type
> shouldn't kill the script executing it.  Throw a warning, yes, but not
> stop everything.
Then you could have a IM_TESTING_STUFF_DONT_DIE_ON_DODOC variable to override 
the die or something

-- 
# Just a user; my opinion doesn't matter here...
# electronerd, the electronerdian from electronerdia
#

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Making dobin, doexe, doins, doman, dodoc die by default

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 666 bytes --]

On Wednesday 12 July 2006 18:12, John Myers wrote:
> On Wednesday 12 July 2006 14:36, Steve Dibb wrote:
> > Well, it could happen while testing an ebuild. :)  I'd be pretty ticked
> > if I were testing Qt and I didn't realize they did change the doc files
> > around before doing a test run.
> >
> > Besides that though, imho, a simple function with a boolean return type
> > shouldn't kill the script executing it.  Throw a warning, yes, but not
> > stop everything.
>
> Then you could have a IM_TESTING_STUFF_DONT_DIE_ON_DODOC variable to
> override the die or something

FEATURES=stricter

i find this feature useful for all the extra anal people out there
-mike

[-- Attachment #2: Type: application/pgp-signature, Size: 827 bytes --]

Re: [gentoo-dev] Making dobin, doexe die by default and doins, doman, dodoc warn initially

[Daniel Black]

[-- Attachment #1: Type: text/plain, Size: 1959 bytes --]

On Thursday 13 July 2006 07:36, Steve Dibb wrote:
> Henrik Brix Andersen wrote:
> > On Wed, Jul 12, 2006 at 11:11:01PM +0200, Jakub Moc wrote:
> >> Uh... Sorry but it's pretty hard to imagine something more annoying than
> >> an ebuild that dies after a couple of hours compile just because
> >> upstream decided to rename Changelog.txt to ChangeLog.txt and noone
> >> noticed  during version bump, or because someone made a typo there. Fail
> >> to see any benefit from this... :S Ditto for manpages.
> >
> > How could that slip through the initial testing of the ebuild
> > performed by the developer doing the version bump?
>
> Well, it could happen while testing an ebuild. :)  I'd be pretty ticked
> if I were testing Qt and I didn't realize they did change the doc files
> around before doing a test run.

there is always,

1. oh golly I made a mistake
2. rm -rf /var/tmp/portage/{package}/image
3. (vi|emacs|nano) {package}.ebuild
4. FEATURES="noauto" ebuild  {package}.ebuild install
5. victory

sure the may be some packages that don't like going through an install twice 
but I'd hope most of the bigger ones would oblige happily.

A FEATURE like dowarn as an alternate to what John Myers said maybe be good 
for the time conscious dev.

> Besides that though, imho, a simple function with a boolean return type
> shouldn't kill the script executing it.  Throw a warning, yes, but not
> stop everything.
>
> Steve

As a general implementation strategy I think portage should
initially:
die on dobin, doexe

warn on
doins, doman, doexe

After the QA stuff is fixed in the tree, and the general feeling that mass 
hysteria won't break out because of it, change the warns to a die.

ECLASSES

if these use doins/man/doc then the should probably check them before 
installing:
[ -f ${doc} ] && dodoc ${doc}



-- 
Daniel Black <dragonheart@gentoo.org>
Gentoo Crypto/dev-embedded/Forensics/NetMon

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Making dobin, doexe die by default and doins, doman, dodoc warn initially

[Diego 'Flameeyes' Pettenò]

[-- Attachment #1: Type: text/plain, Size: 848 bytes --]

On Thursday 13 July 2006 02:26, Daniel Black wrote:
> 4. FEATURES="noauto" ebuild  {package}.ebuild install
FEATURES=noauto has broken at least two times in different ways with portage 
2.1 pre-releases.
Same as broke FEATURES=digest and FEATURES=autoaddcvs or whatever that was.

I won't rely on those FEATURES because nobody has a clear idea of what they 
should and what they should not do.

> if these use doins/man/doc then the should probably check them before
> installing:
> [ -f ${doc} ] && dodoc ${doc}
dodoc of 30 files of which some might not exists take less runtime than 30 
checks for files and following dodoc file-by-file.

Better a dodoc $(ls ${doc} 2>/dev/null) probably

-- 
Diego "Flameeyes" Pettenò - http://farragut.flameeyes.is-a-geek.org/
Gentoo/Alt lead, Gentoo/FreeBSD, Video, AMD64, Sound, PAM, KDE

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Making dobin, doexe die by default and doins, doman, dodoc warn initially

[Ciaran McCreesh]

On Thu, 13 Jul 2006 02:41:26 +0200 "Diego 'Flameeyes' Pettenò"
<flameeyes@gentoo.org> wrote:
| > if these use doins/man/doc then the should probably check them
| > before installing:
| > [ -f ${doc} ] && dodoc ${doc}
|
| dodoc of 30 files of which some might not exists take less runtime
| than 30 checks for files and following dodoc file-by-file.

Uh... Why on earth is the runtime speed for one-off src_install things
being discussed? A few hundredths of a second in non-global code is
utterly irrelevant. The far more important issue is what's more
readable.

-- 
Ciaran McCreesh
Mail            : ciaran dot mccreesh at blueyonder.co.uk


-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Making dobin, doexe die by default and doins, doman, dodoc warn initially

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 181 bytes --]

On Wednesday 12 July 2006 20:26, Daniel Black wrote:
> there is always,

not for joe blow who just wants to use Gentoo, the implementation details of 
portage be damned
-mike

[-- Attachment #2: Type: application/pgp-signature, Size: 827 bytes --]

Re: [gentoo-dev] Making dobin, doexe, doins, doman, dodoc die by default

[Donnie Berkholz]

[-- Attachment #1: Type: text/plain, Size: 822 bytes --]

Henrik Brix Andersen wrote:
> On Wed, Jul 12, 2006 at 11:11:01PM +0200, Jakub Moc wrote:
>> Uh... Sorry but it's pretty hard to imagine something more annoying than
>> an ebuild that dies after a couple of hours compile just because
>> upstream decided to rename Changelog.txt to ChangeLog.txt and noone
>> noticed  during version bump, or because someone made a typo there. Fail
>> to see any benefit from this... :S Ditto for manpages.
> 
> How could that slip through the initial testing of the ebuild
> performed by the developer doing the version bump?

It couldn't, but this change isn't being made with a bump. We haven't
been assured those doing the change right now are going to test every
package using doman, dodoc, etc.. Packages will probably start failing
left and right.

Thanks,
Donnie


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 252 bytes --]

Re: [gentoo-dev] Making dobin, doexe, doins, doman, dodoc die by default

[Ciaran McCreesh]

On Wed, 12 Jul 2006 14:46:32 -0700 Donnie Berkholz
<dberkholz@gentoo.org> wrote:
| It couldn't, but this change isn't being made with a bump. We haven't
| been assured those doing the change right now are going to test every
| package using doman, dodoc, etc.. Packages will probably start failing
| left and right.

EAPI.

-- 
Ciaran McCreesh
Mail            : ciaran dot mccreesh at blueyonder.co.uk


-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Making dobin, doexe, doins, doman, dodoc die by default

[Ciaran McCreesh]

On Wed, 12 Jul 2006 23:11:01 +0200 Jakub Moc <jakub@gentoo.org> wrote:
| Stefan Schweizer wrote:
| > Jason Stubbs called for consistency .. i.e making doman and dodoc
| > also die when nothing the file does not exist. A simple workaround
| > in case an ebuild is broken: [ -f xxx ] && dodoc xxx
| 
| Uh... Sorry but it's pretty hard to imagine something more annoying
| than an ebuild that dies after a couple of hours compile just because
| upstream decided to rename Changelog.txt to ChangeLog.txt and noone
| noticed  during version bump, or because someone made a typo there.
| Fail to see any benefit from this... :S Ditto for manpages.

Uh... Presumably, the person doing the version bump would have, you
know... Tested the ebuild before committing it. Standards haven't
slipped *that* far have they?

-- 
Ciaran McCreesh
Mail            : ciaran dot mccreesh at blueyonder.co.uk


-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Making dobin, doexe, doins, doman, dodoc die by default

[Aron Griffis]

[-- Attachment #1: Type: text/plain, Size: 719 bytes --]

Jakub Moc wrote:  [Wed Jul 12 2006, 05:11:01PM EDT]
> Uh... Sorry but it's pretty hard to imagine something more annoying
> than an ebuild that dies after a couple of hours compile just
> because upstream decided to rename Changelog.txt to ChangeLog.txt
> and noone noticed  during version bump, or because someone made
> a typo there. Fail to see any benefit from this... :S Ditto for
> manpages.

The point is to catch the problems at version-bump time or typo-time
rather than put a broken ebuild into portage.  IMHO this change can
mean nothing but goodness for the overall quality of the portage tree.

Surely you don't go around version-bumping and typo-ing without
testing prior to commit, right?

Regards,
Aron

[-- Attachment #2: Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] Making dobin, doexe, doins, doman, dodoc die by default

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 863 bytes --]

On Wednesday 12 July 2006 13:37, Stefan Schweizer wrote:
> SpanKY complained that he cannot set a custom die message then. But this is
> not needed here, since every do* command can be clearly identified by the
> argument and the directory it will be installed to.

except for the times where the do funcs can install the same file in different 
code paths

> Also as Paul suggested something like that would be possible for a custom
> die message:
>         if [[ -n "${DIE_MSG}" ]] ; then
>                 echo "!!! ${DIE_MSG}"
>         fi

ugh, ugly
DIE_MSG="failed to install root bins" dosbin fooie

dosbin fooie || die "failed to install root bins"

you tell me which is already a clean/common standard in the tree

> They are imo QA problems and should be fixed.

crazy idea, grep your /var/log/portage/ logs and file bugs
-mike

[-- Attachment #2: Type: application/pgp-signature, Size: 827 bytes --]

Re: [gentoo-dev] Making dobin, doexe, doins, doman, dodoc die by default

[Aron Griffis]

Vapier wrote:  [Thu Jul 13 2006, 11:32:39PM EDT]
> On Wednesday 12 July 2006 13:37, Stefan Schweizer wrote:
> > SpanKY complained that he cannot set a custom die message then. But this is
> > not needed here, since every do* command can be clearly identified by the
> > argument and the directory it will be installed to.
> 
> except for the times where the do funcs can install the same file in different 
> code paths

Maybe fixup die() to include the call stack w/ function line numbers?

I can't tell from your reply whether you dislike do* dying
automatically, or if you just don't like some implementation details.
Which is it?

Aron

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Making dobin, doexe, doins, doman, dodoc die by default

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 956 bytes --]

On Monday 17 July 2006 11:14, Aron Griffis wrote:
> Vapier wrote:  [Thu Jul 13 2006, 11:32:39PM EDT]
>
> > On Wednesday 12 July 2006 13:37, Stefan Schweizer wrote:
> > > SpanKY complained that he cannot set a custom die message then. But
> > > this is not needed here, since every do* command can be clearly
> > > identified by the argument and the directory it will be installed to.
> >
> > except for the times where the do funcs can install the same file in
> > different code paths
>
> Maybe fixup die() to include the call stack w/ function line numbers?
>
> I can't tell from your reply whether you dislike do* dying
> automatically, or if you just don't like some implementation details.
> Which is it?

at this point i'm against it due to the loss of information/granularity in the 
die message ... and utilizing DIE_MSG is garbage

if die can be made more useful, then i'd be for the do* functions calling die 
for me
-mike

[-- Attachment #2: Type: application/pgp-signature, Size: 827 bytes --]

Re: [gentoo-dev] Making dobin, doexe, doins, doman, dodoc die by default

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 1149 bytes --]

On Mon, Jul 17, 2006 at 06:00:04PM -0400, Mike Frysinger wrote:
> at this point i'm against it due to the loss of information/granularity in the 
> die message ... and utilizing DIE_MSG is garbage
> 
> if die can be made more useful, then i'd be for the do* functions calling die 
> for me
> -mike

How about changing the internals of dobin etc to behave like this
(puesdo-code):

in ebuild:
dobin xyz

in dobin code:
(existing stuff)
[ (check failure) ] && die "Failure: dobin $@"

So if 'dobin xyz' fails, you get a die, with the message of
"Failure: dobin xyz"

This hooks nicely into the existing stacktrace:

!!! ERROR: dev-lang/php-5.1.4-r1 failed.
Call stack:
  ebuild.sh, line XXX:   Called dyn_install
  ebuild.sh, line XXX:   Called src_install
  php-5.1.4-r1.ebuild, line XXX:   Called php5_1-sapi_src_install
  php5_1-sapi.eclass, line XXX:   Called dobin
  dobin, line XXX: called die

Additionally, keep DIE_MSG support for anybody that wants to override the
default messages.

-- 
Robin Hugh Johnson
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

[gentoo-dev] Re: Making dobin, doexe, doins, doman, dodoc die by default

[Sven Köhler]

[-- Attachment #1: Type: text/plain, Size: 1022 bytes --]

> This came up in Bug 138792 [dobin etc. should automatically die on failure]
> It needs more discussion on the mailing lists.
> Some excerpts from the bug: The proposal from Paul Bredbury:
> "Hi, I propose that the following ebuild commands themselves *die* on
> failure, because the vast majority of the time the emerge might as well be
> considered a failure if such commands fail.
> dobin, dosbin, doexe"
> 
> So what do you think?

Since it does not seem to be too popular, to let "dobin, etc." die, i
would suggest writing a big fat red "QA: command 'dobin xyz' failed" on
the screen, so that even the last user notices that. Perhaps also
writing those QA-warnings to emerge.log?

Is that aready done at the moment? If not, i would suggest that - and
actually i would rather like that "dobin, etc." die on error.

Well, but some people here claim, that the ebuilds are not tested that
much, before they go into portage. Well, that's another decision: what
should gentoo's quality be like?




[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 251 bytes --]

Re: [gentoo-dev] Re: Making dobin, doexe, doins, doman, dodoc die by default

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 1035 bytes --]

On Tue, 2006-07-18 at 20:49 +0200, Sven Köhler wrote:
> Well, but some people here claim, that the ebuilds are not tested that
> much, before they go into portage. Well, that's another decision: what
> should gentoo's quality be like?

Actually, it was more a discussion of older ebuilds.  If I put an ebuild
into the tree today, using one of the eclasses that uses lots of "do*"
stuff and relies on it not being an error condition today, then the
change is made at a later date, how am I going to know that my ebuild
suddenly broke?  As you can guess, the answer is "when you get a bug
report" which definitely isn't the optimal answer.

I'm refraining from the "quality" discussion simply because we're pretty
well aware that we have two pretty distant camps within Gentoo these
days and I'd prefer not start another of the "stable vs. ricer" threads
while I'm busy working on release stuff... :P

-- 
Chris Gianelloni
Release Engineering - Strategic Lead
x86 Architecture Team
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]