[gentoo-dev] The usefulness of test in FEATURES

[gentoo-dev] The usefulness of test in FEATURES

[Maurice van der Pot]

[-- Attachment #1: Type: text/plain, Size: 1053 bytes --]

Hey people,

A lot of ebuilds still fail when "test" is in FEATURES. 

It's understandable that fixing this is a low priority thing, but what 
I would like to propose is to either fix the tests or disable them.
The latter would be the thing to do for devs who are currently closing
bugs about tests with WONTFIX or similar. 

By keeping tests that fail enabled in one ebuild, perfectly good tests 
of any other ebuild are rendered useless because it becomes almost
impossible to upgrade a system with "test" in FEATURES.

If fixing the tests is not WONTFIX but rather something way down on your 
todo list, I would also recommend disabling the tests in the ebuild. 
A bug report could be used to track these bugs, but at least it would 
not bother so many people while it is still unsolved.

I'd like to know what you think of this.

Thanks,
Maurice.

-- 
Maurice van der Pot

Gentoo Linux Developer   griffon26@gentoo.org     http://www.gentoo.org
Creator of BiteMe!       griffon26@kfk4ever.com   http://www.kfk4ever.com


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] The usefulness of test in FEATURES

[Rumen Yotov]

Maurice van der Pot wrote:

>Hey people,
>
>A lot of ebuilds still fail when "test" is in FEATURES. 
>
>It's understandable that fixing this is a low priority thing, but what 
>I would like to propose is to either fix the tests or disable them.
>The latter would be the thing to do for devs who are currently closing
>bugs about tests with WONTFIX or similar. 
>
>By keeping tests that fail enabled in one ebuild, perfectly good tests 
>of any other ebuild are rendered useless because it becomes almost
>impossible to upgrade a system with "test" in FEATURES.
>
>If fixing the tests is not WONTFIX but rather something way down on your 
>todo list, I would also recommend disabling the tests in the ebuild. 
>A bug report could be used to track these bugs, but at least it would 
>not bother so many people while it is still unsolved.
>
>I'd like to know what you think of this.
>
>Thanks,
>Maurice.
>
>  
>
Hi,
Some time ago used 'maketest' (now 'test') in my FEATURES-list, but
after some test-failing packages (not many and BUGS filed on them)
switched it off, as IMHO fixing 'tests' seems to be a task more up to
package-developers (so gentoo-maintainers send it upstream for fixing).
Think (tell your experiences here) that the responses may be very
different (depending on the people). The package usually works OK, so a
test-failing may be due to for example some USE-flags, CFLAGS (using
hardened profile in my former case) or other causes (WONTFIX from upstream).
If the number of fixed packages is too small or it takes too long to fix
it's not very useful feature, i just don't know.
Just sharing my experiences here.
PS: a day or two ago mentioned that i'm using per-package FEATURES in
'/etc/portage/package.features-file' ("-collision-protect") so was
thinking about doing the same for "test" (for half a day "test" is ON in
my 'make.conf'). Think of putting it OFF for failing packages while
still keeping it ON globally. Also will like the get some opinions.
Rumen
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] The usefulness of test in FEATURES

[Jason Stubbs]

[-- Attachment #1: Type: text/plain, Size: 635 bytes --]

On Sunday 01 May 2005 00:30, Rumen Yotov wrote:
> PS: a day or two ago mentioned that i'm using per-package FEATURES in
> '/etc/portage/package.features-file' ("-collision-protect") so was
> thinking about doing the same for "test" (for half a day "test" is ON in
> my 'make.conf'). Think of putting it OFF for failing packages while
> still keeping it ON globally. Also will like the get some opinions.

/etc/portage/package.features-file? Based on Ned's bashrc hack, I'm guessing? 
"collision-protect" is one of those features that can't be disabled in this 
way. "test" on the other hand can be.

Regards,
Jason Stubbs

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] The usefulness of test in FEATURES

[Rumen Yotov]

Jason Stubbs wrote:

>On Sunday 01 May 2005 00:30, Rumen Yotov wrote:
>  
>
>>PS: a day or two ago mentioned that i'm using per-package FEATURES in
>>'/etc/portage/package.features-file' ("-collision-protect") so was
>>thinking about doing the same for "test" (for half a day "test" is ON in
>>my 'make.conf'). Think of putting it OFF for failing packages while
>>still keeping it ON globally. Also will like the get some opinions.
>>    
>>
>
>/etc/portage/package.features-file? Based on Ned's bashrc hack, I'm guessing? 
>"collision-protect" is one of those features that can't be disabled in this 
>way. "test" on the other hand can be.
>
>Regards,
>Jason Stubbs
>  
>
Hi Jason,
Your guess is correct. Thanks for the info.
Rumen
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] The usefulness of test in FEATURES

[Maurice van der Pot]

[-- Attachment #1: Type: text/plain, Size: 917 bytes --]

On Sat, Apr 30, 2005 at 06:30:05PM +0300, Rumen Yotov wrote:
> a day or two ago mentioned that i'm using per-package FEATURES in
> '/etc/portage/package.features-file' ("-collision-protect") so was
> thinking about doing the same for "test" (for half a day "test" is ON in
> my 'make.conf'). Think of putting it OFF for failing packages while
> still keeping it ON globally. Also will like the get some opinions.

It would be a shame if every user would have to find out for himself
which packages need "-test". Additionally, when the tests are fixed, 
the package will still not be using them. It would be nice if something
like package.features could become not only part of portage, but also 
of the profiles.

Maurice.

-- 
Maurice van der Pot

Gentoo Linux Developer   griffon26@gentoo.org     http://www.gentoo.org
Creator of BiteMe!       griffon26@kfk4ever.com   http://www.kfk4ever.com


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] The usefulness of test in FEATURES

[Jason Stubbs]

[-- Attachment #1: Type: text/plain, Size: 997 bytes --]

On Sunday 01 May 2005 02:32, Maurice van der Pot wrote:
> On Sat, Apr 30, 2005 at 06:30:05PM +0300, Rumen Yotov wrote:
> > a day or two ago mentioned that i'm using per-package FEATURES in
> > '/etc/portage/package.features-file' ("-collision-protect") so was
> > thinking about doing the same for "test" (for half a day "test" is ON in
> > my 'make.conf'). Think of putting it OFF for failing packages while
> > still keeping it ON globally. Also will like the get some opinions.
>
> It would be a shame if every user would have to find out for himself
> which packages need "-test". Additionally, when the tests are fixed,
> the package will still not be using them. It would be nice if something
> like package.features could become not only part of portage, but also
> of the profiles.

package.features is not part of portage, nor will it be for some time to come 
due to the issues outlined in my other response. RESTRICT="test" is viable 
though.

Regards,
Jason Stubbs

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] The usefulness of test in FEATURES

[Jason Stubbs]

[-- Attachment #1: Type: text/plain, Size: 1104 bytes --]

On Sunday 01 May 2005 02:45, Jason Stubbs wrote:
> On Sunday 01 May 2005 02:32, Maurice van der Pot wrote:
> > On Sat, Apr 30, 2005 at 06:30:05PM +0300, Rumen Yotov wrote:
> > > a day or two ago mentioned that i'm using per-package FEATURES in
> > > '/etc/portage/package.features-file' ("-collision-protect") so was
> > > thinking about doing the same for "test" (for half a day "test" is ON
> > > in my 'make.conf'). Think of putting it OFF for failing packages while
> > > still keeping it ON globally. Also will like the get some opinions.
> >
> > It would be a shame if every user would have to find out for himself
> > which packages need "-test". Additionally, when the tests are fixed,
> > the package will still not be using them. It would be nice if something
> > like package.features could become not only part of portage, but also
> > of the profiles.
>
> package.features is not part of portage, nor will it be for some time to
> come due to the issues outlined in my other response. RESTRICT="test" is
> viable though.

More than viable, it's already implemented...

Regards,
Jason Stubbs

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

[gentoo-dev] Re: The usefulness of test in FEATURES

[R Hill]

Maurice van der Pot wrote:
> It's understandable that fixing this is a low priority thing, but what 
> I would like to propose is to either fix the tests or disable them.
> The latter would be the thing to do for devs who are currently closing
> bugs about tests with WONTFIX or similar. 
> 
> If fixing the tests is not WONTFIX but rather something way down on your 
> todo list, I would also recommend disabling the tests in the ebuild. 
> A bug report could be used to track these bugs, but at least it would 
> not bother so many people while it is still unsolved.

Well, in my experience as a user that has made it a priority to report 
broken test suites when I come across them (w/ patches attached whenever 
I can), I would argue that if you take the maintainers that close test 
failures as WONTFIX, and add to them the maintainers who don't care one 
way or another, and have them all disable test in their packages, you 
might as well not even have the test feature at all. :P

> By keeping tests that fail enabled in one ebuild, perfectly good tests 
> of any other ebuild are rendered useless because it becomes almost
> impossible to upgrade a system with "test" in FEATURES.

It's not so bad.  I keep a running list of which packages fail their 
tests on me and it's usually under half a dozen at any given time.  I 
have about another half dozen open in bugzilla but most of them have 
patches included.

I can definitely understand why test failures are a low-priority thing. 
  They're a pain in the ass and it's not like you need yet another thing 
to maintain.  But IMHO it's better to leave them enabled for some sap 
like me to fix one day down the line than to disable them now and lose 
that opportunity.

Maybe a way of lessening the annoyance of test failures would be having 
a way to resume the build at the install phase.  I'm thinking of 
something similar the touch ${BUILDDIR}/.compiled trick.  as it is, if 
you remove test from FEATURES, touch .tested, and then 'ebuild 
foo.ebuild install' the tests still run.  This is especially frustrating 
when you've just spent 6 hours compiling a package to have it fail 
because of sandboxing.

--de.

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: The usefulness of test in FEATURES

[Georgi Georgiev]

maillog: 30/04/2005-13:43:42(-0600): R Hill types
> Maybe a way of lessening the annoyance of test failures would be having 
> a way to resume the build at the install phase.  I'm thinking of 
> something similar the touch ${BUILDDIR}/.compiled trick.  as it is, if 
> you remove test from FEATURES, touch .tested, and then 'ebuild 
> foo.ebuild install' the tests still run.  This is especially frustrating 
> when you've just spent 6 hours compiling a package to have it fail 
> because of sandboxing.

You probably also need to edit ${T}/environment for the ebuild and
remove test from FEATURES there as well. In fact, this is *the* location
that you should probably touch.

-- 
-*   Georgi Georgiev   -* Winning isn't everything, but losing isn't   -*
*-    chutz@gg3.net    *- anything.                                    *-
-*  +81(90)2877-8845   -*                                              -*
-- 
gentoo-dev@gentoo.org mailing list

[gentoo-dev] Re: The usefulness of test in FEATURES

[R Hill]

Georgi Georgiev wrote:
> maillog: 30/04/2005-13:43:42(-0600): R Hill types
> 
>>Maybe a way of lessening the annoyance of test failures would be having 
>>a way to resume the build at the install phase.  I'm thinking of 
>>something similar the touch ${BUILDDIR}/.compiled trick.  as it is, if 
>>you remove test from FEATURES, touch .tested, and then 'ebuild 
>>foo.ebuild install' the tests still run.  This is especially frustrating 
>>when you've just spent 6 hours compiling a package to have it fail 
>>because of sandboxing.
> 
> 
> You probably also need to edit ${T}/environment for the ebuild and
> remove test from FEATURES there as well. In fact, this is *the* location
> that you should probably touch.

Cool.  So this is as simple as "sed -i -e 's@\(^FEATURES.*\) 
test\(.*$\)@\1\2@g' /var/tmp/portage/pkgname/temp/environment" then.

Thanks. =]

--de.


-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: The usefulness of test in FEATURES

[Jason Stubbs]

[-- Attachment #1: Type: text/plain, Size: 569 bytes --]

On Sunday 01 May 2005 04:43, R Hill wrote:
> Maybe a way of lessening the annoyance of test failures would be having
> a way to resume the build at the install phase.  I'm thinking of
> something similar the touch ${BUILDDIR}/.compiled trick.  as it is, if
> you remove test from FEATURES, touch .tested, and then 'ebuild
> foo.ebuild install' the tests still run.  This is especially frustrating
> when you've just spent 6 hours compiling a package to have it fail
> because of sandboxing.

Fixed so that testing is not run when .tested exists.

Regards,
Jason Stubbs

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]