From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from lists.gentoo.org ([140.105.134.102] helo=robin.gentoo.org) by nuthatch.gentoo.org with esmtp (Exim 4.43) id 1ECNK8-0001fS-2z for garchives@archives.gentoo.org; Mon, 05 Sep 2005 20:12:12 +0000 Received: from robin.gentoo.org (localhost [127.0.0.1]) by robin.gentoo.org (8.13.4/8.13.4) with SMTP id j85K8C1C001062; Mon, 5 Sep 2005 20:08:12 GMT Received: from myrddraal.demon.co.uk (myrddraal.demon.co.uk [62.49.28.63]) by robin.gentoo.org (8.13.4/8.13.4) with ESMTP id j85K6PFC020632 for ; Mon, 5 Sep 2005 20:06:25 GMT Received: from mogheiden.gnqs.org (mogheiden [192.168.0.20]) by myrddraal.demon.co.uk (8.13.3/8.12.10) with ESMTP id j85KD911011338 for ; Mon, 5 Sep 2005 21:13:09 +0100 Subject: Re: [gentoo-dev] tentative x86 arch team glep From: Stuart Herbert To: gentoo-dev@lists.gentoo.org In-Reply-To: <200509042012.38859.morfic@gentoo.org> References: <20050904143711.GD23576@dst.grantgoodyear.org> <20050904215931.53b9db51@snowdrop.home> <1125870764.11364.152.camel@mogheiden.gnqs.org> <200509042012.38859.morfic@gentoo.org> Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="=-5sHxVuec5+iu9LswznF4" Organization: Gentoo Linux Project Date: Mon, 05 Sep 2005 21:09:28 +0100 Message-Id: <1125950968.10666.72.camel@mogheiden.gnqs.org> Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-dev@gentoo.org Reply-to: gentoo-dev@lists.gentoo.org Mime-Version: 1.0 X-Mailer: Evolution 2.2.1.1 X-Archives-Salt: 2b644935-113c-46c7-a494-e9741ad0b11b X-Archives-Hash: 11f73d7f5c6d448d260f56a302e2a84f --=-5sHxVuec5+iu9LswznF4 Content-Type: text/plain Content-Transfer-Encoding: quoted-printable On Sun, 2005-09-04 at 20:12 -0500, Daniel Goller wrote: > sounds like you suggest to trick ~arch users into testing "unripe"=20 > ebuilds/bumps/versions by sending it into ~arch to get the testing done w= hile=20 > someone in a chroot would be much better equipped for doing the testing w= ith? No. =20 You've got to look at it in the context of the environment we work in. The vast majority of our packages do not come with test strategies, comprehensive test scripts, automated test cases that provide a large coverage of the code base, or any other QA tools that a career software QA person would expect / desire. We don't have activities in place to fill this gap - it's probably beyond our very limited resources anyhow. We don't require (and don't cover in the quizzes) devs to have any training or experience of deterministic software testing, or of regression testing. It is not our policy to require devs to write regression tests for every valid bug posted in Bugzilla (or for any bug at all, for that matter). A package maintainer does what testing he can, and there are times when we'd all wish the maintainer had done more :) But there will always be the need for a wider audience to be encouraged to test the package. A good rule of thumb is that a QA budget should be 40% of the development budget. We don't have the manpower to come anywhere near that. We have only a fraction of the resources of Debian or RedHat - there comes a point where we have to make up the difference *somehow*. For better or for worse, historically in Gentoo we've done it by turning to our users. Many *users* look on package.masked packages as being dangerous to install, but are much more willing to run ~arch packages. If you mask a version of a popular package, you'll get a lot of correspondence asking you when you'll unmask it, but you won't get much testing feedback to go with it. Once the package moves to ~arch, the amount of feedback improves substantially. If a package maintainer believes that a package WORKSFORME *after due diligence*, then he's not only entitled to move it to ~arch, but he's got no reason not to. I've been through this first hand over the last 14 months with PHP 5. It's a big package, one that I use all day every working day, and a topic I co-wrote the official certification study guide for. It's fair to say that it's a package I know a lot about, and that others in the community recognise I know well. But, with over 100 separate features to enable and disable, even over 14 months there are large parts of the package that I won't have tested in depth, and there will always be features that I'll never touch. I kept PHP5 masked for those 14 months, and (as Jakub and others can confirm) most of the feedback has been limited to "unmask that puppy" (sometimes put in stronger terms ;-) There were some bugs from users who had found issues, but not many. Rather than unmask the packages before they were read, I changed to another approach. I moved the work out of Portage into an overlay instead. This worked well. It has attracted a bunch of regulars to #gentoo-apache who have spent the last few months finding the bugs that existed, and making sure that they're fixed and stay fixed. It looks likely that we'll get some new devs out of that too :) Overlays are easy for larger pieces of work like PHP, Java, Gnome/Gentopia, but they'll always be small packages where an overlay feel like too much effort. They may not be the answer to everything. If we'd seen through the policy that every package has to belong to a herd, then we could organise overlays by herd - and maybe leave it up to the arch teams to import "stable" packages from the overlays into the Portage tree proper. Best regards, Stu --=20 Stuart Herbert stuart@gentoo.org Gentoo Developer http://www.gentoo.org/ http://stu.gnqs.org/diary/ GnuGP key id# F9AFC57C available from http://pgp.mit.edu Key fingerprint =3D 31FB 50D4 1F88 E227 F319 C549 0C2F 80BA F9AF C57C -- --=-5sHxVuec5+iu9LswznF4 Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) iD8DBQBDHKX4DC+AuvmvxXwRAjKdAKCezzzqEeq8T78zgfqZQhJKm6c0RwCgnLso c3VXctXBDBL0GMMUhAp7qyo= =U5hG -----END PGP SIGNATURE----- --=-5sHxVuec5+iu9LswznF4-- -- gentoo-dev@gentoo.org mailing list