From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from lists.gentoo.org ([140.105.134.102] helo=robin.gentoo.org) by nuthatch.gentoo.org with esmtp (Exim 4.62) (envelope-from ) id 1HjCcc-0006Sz-Nk for garchives@archives.gentoo.org; Wed, 02 May 2007 11:03:47 +0000 Received: from robin.gentoo.org (localhost [127.0.0.1]) by robin.gentoo.org (8.14.0/8.14.0) with SMTP id l42B1iae001757; Wed, 2 May 2007 11:01:44 GMT Received: from mail.bawue.net (phoenix.bawue.net [193.7.176.60]) by robin.gentoo.org (8.14.0/8.14.0) with ESMTP id l42AveKO025410 for ; Wed, 2 May 2007 10:57:40 GMT Received: from [193.166.89.200] (b136a.mtalo.tontut.fi [193.166.89.200]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.bawue.net (Postfix) with ESMTP id 2C6E3BB4ED for ; Wed, 2 May 2007 12:55:00 +0200 (CEST) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-dev@gentoo.org Reply-to: gentoo-dev@lists.gentoo.org Mime-Version: 1.0 (Apple Message framework v752.2) In-Reply-To: <20070502013220.7a3ae9a4@sheridan.genone.homeip.net> References: <200705011508.57220.peper@gentoo.org> <20070502013220.7a3ae9a4@sheridan.genone.homeip.net> Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: <7399E7E2-C7CD-4132-A728-6A61EE910D2E@anderedomain.de> Content-Transfer-Encoding: 7bit From: Philipp Riegger Subject: Re: [gentoo-dev] tests Date: Wed, 2 May 2007 13:54:53 +0300 To: gentoo-dev@lists.gentoo.org X-Mailer: Apple Mail (2.752.2) X-Archives-Salt: fcae751d-bca2-4176-b002-5831385a6795 X-Archives-Hash: e6269fe4eba2e0b5d00bdb147f285c8b On 02.05.2007, at 02:32, Marius Mauch wrote: > a) cost (in terms of runtime, resource usage, additional deps) Tools for this could be implemented in the package manager. The package has to be installed and tested by the developer, so if portage would show the times for each stage or the times for the test and the rest or something like that, the developer could get an idea: If test time is smaller than build time (or less than half of complete time), than it's not that much cost. It test time is less then 1 hour (or whatever), than it's not that much cost. In any other case it's much cost. > b) effectiveness (does a failing/working test mean the package is > broken/working?) To figure this out before releasing a package to the tree might be lots of work. so this could be figured out later. If there are bugs about tests failing, try to reproduce it or ask the reporter to do some tests if everything is working as expected. > c) importance (is there a realistic chance for the test to be useful?) This can easily be decided, as mentioned in other posts (scientific packages, core packages, cryptographic packages,...) > d) correctness (does the test match the implementation? overlaps a bit > with effectiveness) This might be a lot of work. I think this cannot be tested in a sane way for every package. So it's probably up to the maintainer/herd or upstream to decide if he/they sould take care of this Philipp -- gentoo-dev@gentoo.org mailing list