Re: [gentoo-dev] tests

[Philipp Riegger <lists@anderedomain.de>]

On 02.05.2007, at 02:32, Marius Mauch wrote:

> a) cost (in terms of runtime, resource usage, additional deps)

Tools for this could be implemented in the package manager. The  
package has to be installed and tested by the developer, so if  
portage would show the times for each stage or the times for the test  
and the rest or something like that, the developer could get an idea:  
If test time is smaller than build time (or less than half of  
complete time), than it's not that much cost. It test time is less  
then 1 hour (or whatever), than it's not that much cost. In any other  
case it's much cost.

> b) effectiveness (does a failing/working test mean the package is
> broken/working?)

To figure this out before releasing a package to the tree might be  
lots of work. so this could be figured out later. If there are bugs  
about tests failing, try to reproduce it or ask the reporter to do  
some tests if everything is working as expected.

> c) importance (is there a realistic chance for the test to be useful?)

This can easily be decided, as mentioned in other posts (scientific  
packages, core packages, cryptographic packages,...)

> d) correctness (does the test match the implementation? overlaps a bit
> with effectiveness)

This might be a lot of work. I think this cannot be tested in a sane  
way for every package. So it's probably up to the maintainer/herd or  
upstream to decide if he/they sould take care of this

Philipp


-- 
gentoo-dev@gentoo.org mailing list