From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from lists.gentoo.org ([140.105.134.102] helo=robin.gentoo.org) by nuthatch.gentoo.org with esmtp (Exim 4.62) (envelope-from ) id 1HkRhw-0004pa-KY for garchives@archives.gentoo.org; Sat, 05 May 2007 21:22:25 +0000 Received: from robin.gentoo.org (localhost [127.0.0.1]) by robin.gentoo.org (8.14.0/8.14.0) with SMTP id l45LKJ6e026536; Sat, 5 May 2007 21:20:19 GMT Received: from smtp.gentoo.org (smtp.gentoo.org [140.211.166.183]) by robin.gentoo.org (8.14.0/8.14.0) with ESMTP id l45LIGoR024210 for ; Sat, 5 May 2007 21:18:17 GMT Received: from localhost (localhost [127.0.0.1]) by smtp.gentoo.org (Postfix) with ESMTP id 4E93D64D92 for ; Sat, 5 May 2007 21:18:16 +0000 (UTC) X-Virus-Scanned: amavisd-new at gentoo.org X-Spam-Score: -0.939 X-Spam-Level: X-Spam-Status: No, score=-0.939 required=5.5 tests=[AWL=-0.939] Received: from smtp.gentoo.org ([127.0.0.1]) by localhost (smtp.gentoo.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id D+Jei8VyKj51 for ; Sat, 5 May 2007 21:18:14 +0000 (UTC) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by smtp.gentoo.org (Postfix) with ESMTP id A73C964549 for ; Sat, 5 May 2007 21:18:11 +0000 (UTC) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1HkRdl-00068r-5X for gentoo-dev@gentoo.org; Sat, 05 May 2007 23:18:05 +0200 Received: from static24-72-114-155.yorkton.accesscomm.ca ([24.72.114.155]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sat, 05 May 2007 23:18:05 +0200 Received: from dirtyepic by static24-72-114-155.yorkton.accesscomm.ca with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sat, 05 May 2007 23:18:05 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: gentoo-dev@lists.gentoo.org From: Ryan Hill Subject: [gentoo-dev] Re: tests Date: Sat, 05 May 2007 15:17:52 -0600 Message-ID: References: <200705011508.57220.peper@gentoo.org> Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-dev@gentoo.org Reply-to: gentoo-dev@lists.gentoo.org Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 X-Complaints-To: usenet@sea.gmane.org X-Gmane-NNTP-Posting-Host: static24-72-114-155.yorkton.accesscomm.ca User-Agent: Thunderbird 2.0.0.4pre (X11/20070504) In-Reply-To: <200705011508.57220.peper@gentoo.org> Sender: news Content-Transfer-Encoding: quoted-printable X-MIME-Autoconverted: from 8bit to quoted-printable by robin.gentoo.org id l45LKJ8b026536 X-Archives-Salt: e6c9f350-4826-467a-bdb6-39f55a9b0a6e X-Archives-Hash: e8755a09ec0da8e9e1050fb9201889d0 Piotr Jaroszy=C5=84ski wrote: > Hello, >=20 > There was some discussion about forcing/not forcing tests in EAPI-1, bu= t there=20 > was clearly no compromise. Imho, tests are very important and thus I wa= nt to=20 > discuss them a little more, but in more sensible fashion. >=20 > Firstly each test can be(not all categories are mutually exclusive): > - not existant > - non-functional > - not runnable from ebuild > - useful but unreasonable resource-wise > - useful and reasonable resource-wise > - necessary > - known to partially fail but with a way of skipping failing tests > - known to partially fail but with no easy way of skipping failing test= s > Is that list comprehensive? I've been running with FEATURES=3Dtest for a long time now. Here's some of the more interesting cases: - fail only on little/big-endian archs - fail only with/without root privs - fail only if dependencies are / are not compiled with certain optional support - fail only with GCC >=3D4 - are expected to fail and are only meant as a regression test - take 3 minutes on x86 and 3 hours on mips - fail on hardened - fail with/without new tar versions - fail with/without new flex versions (etc.) - fail if a kernel component is a module instead of built-in - fail if certain environment variables are set - fail if compiled with certain (safe) CFLAGS/CXXFLAGS Can we qualify each of these into one of your categories? (NB: I realize there are solutions for each of these examples. I'm pointing out that not only is the situation not black and white, it often ranges in the ultra-violet.) > Secondly we must answer the question how precisely we want to distingui= sh=20 > them, so users/dev can choose which categories of tests they want to ru= n.=20 > What comes to mind is: > - run all tests > - run only necessary tests > - run only reasonable tests > - don't run tests at all > Again, is that list comprehensive? - run only tests that don't require extra deps - run only tests that work on hardened - run only tests that work on my arch > Please don't post solutions unless we figure out which options we reall= y want=20 > to deliver. Sorry. (neener neener) ;) Would people accept running src_test() by default only on packages in the system set? There are some that we might want to turn off - glibc, gcc, binutils, autoconf, and automake are on my current short list. coreutils is also a lot of fun. db takes six hours. Anyone, however, who is of the opinion that tests for their packages are so important that they should never be skipped, and who is willing to deal with the bug reports that will undeniably be generated as a result, should IMO be allowed to shoot themselves in the foot, while those uninterested can go about their business without further interruption. In no way should this be tied to EAPI. --=20 where to now? if i had to guess dirtyepic gentoo org i'm afraid to say antarctica's next 9B81 6C9F E791 83BB 3AB3 5B2D E625 A073 8379 37E8 (0x837937E8) --=20 gentoo-dev@gentoo.org mailing list