From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (unknown [208.92.234.80]) by finch.gentoo.org (Postfix) with ESMTP id 176E01381FA for ; Sun, 11 May 2014 22:29:41 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 1454FE0A6C; Sun, 11 May 2014 22:29:40 +0000 (UTC) Received: from michel.telenet-ops.be (michel.telenet-ops.be [195.130.137.88]) by pigeon.gentoo.org (Postfix) with ESMTP id 3DEDCE0A62 for ; Sun, 11 May 2014 22:29:39 +0000 (UTC) Received: from gentoo.org ([94.226.55.127]) by michel.telenet-ops.be with bizsmtp id 0mVe1o00D2khLEN06mVetd; Mon, 12 May 2014 00:29:38 +0200 Date: Mon, 12 May 2014 00:29:25 +0200 From: Tom Wijsman To: gentoo-project@lists.gentoo.org Cc: hasufell@gentoo.org Subject: Re: [gentoo-project] Re: Call For Agenda Items - 13 May 2014 Message-ID: <20140512002925.6c649433@gentoo.org> In-Reply-To: <536FE7C4.2090403@gentoo.org> References: <536D2231.6030808@gentoo.org> <536E1FA7.5050704@gentoo.org> <2731252.LOkG5ql5OK@localhost> <536FE7C4.2090403@gentoo.org> X-Mailer: Claws Mail 3.9.0 (GTK+ 2.24.23; x86_64-pc-linux-gnu) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Project discussion list X-BeenThere: gentoo-project@lists.gentoo.org Reply-To: gentoo-project@lists.gentoo.org Mime-Version: 1.0 Content-Type: multipart/signed; micalg=PGP-SHA1; boundary="Sig_/hB7k4Q7XslodE1RldxWtuos"; protocol="application/pgp-signature" X-Archives-Salt: 16d77c56-0a45-4638-88d4-d3262c54b221 X-Archives-Hash: 2c79102a66313a80be83a1078c41f892 --Sig_/hB7k4Q7XslodE1RldxWtuos Content-Type: multipart/mixed; boundary="MP_/jFo_fL6jcgldNf16Sa+kCIS" --MP_/jFo_fL6jcgldNf16Sa+kCIS Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Content-Disposition: inline On Sun, 11 May 2014 21:12:36 +0000 hasufell wrote: =20 > Anyway, last time I spoke with the QA lead, he said that QA has > currently enough manpower. >=20 > It's a little bit confusing. Please ask him as to why, he's not reading this ML atm; while I could guess random reasons, I'm not entirely sure so it confuses me as well. =46rom my viewpoint, more people want to sign up for QA; at which point, I would personally question at which size the QA team becomes too large... ... although we have a Portage team of that size; from what I recall from there, is that it becomes nasty to organize meetings that way. > What I am pissed about are the arguments other people have given (not > you), not the missing tinderbox... really. I appreciate every hour > people put in gentoo. It isn't about "you didn't get enough stuff > done", at all. +1 > It is about some comments that reveal the way QA (or some parts of it) > thinks about itself. Nothing more, nothing less. Sincerely, the QA team had a rough time figuring that out; given that we started from a clean slate with nothing to refer to, or in other words we weren't mentored into a QA role. For an insight of how I think about QA; please really consider to read the attachment, it is an intact version of the application that I've written towards the Gentoo Council. It features some topics that I think might be an interesting read: - What did the python-exec blocker learn us? - How to indicate regressions much faster? - What do I think the new Gentoo QA team should look like? - What can we do to make the new Gentoo QA team more effective? - What other work is there to do? - Comments & References But I'd like to know your view on it, I have two (or four) questions: - How do you want the QA team to be (or not be)? - What parts do you think we do wrong (or right)? Feel free to reflect the application and recent QA work, I'm all ears. > Something about that needs to change, IMO. And I don't necessarily > mean a regrouping of members or something similar. We already tried > that, didn't we? Let's not make it a habit. > > It's sad that you have to yell out that loud before people actually > listen. But the fact is... you have to. > > In the end, the blame is on the guy who yelled, not on the people who > didn't listen, because CoC doesn't really cover the latter. +1 --=20 With kind regards, Tom Wijsman (TomWij) Gentoo Developer E-mail address : TomWij@gentoo.org GPG Public Key : 6D34E57D GPG Fingerprint : C165 AF18 AB4C 400B C3D2 ABF0 95B2 1FCD 6D34 E57D --MP_/jFo_fL6jcgldNf16Sa+kCIS Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename=TomWij_QA_Application.txt Hello Gentoo Council members As you are looking for developers for the new Gentoo QA team, I want you to know that I am interested in contributing and have time for it at least the next one and a half year. I share you my thoughts and plan: =3D=3D What did the python-exec blocker learn us? =3D=3D We need to (re)organize our QA. It is important to gain a clear understanding of what is happening and to evaluate the resolution and results. A package move was done as a resolution; but, it turned out to result in that blocker. The lack of a QA team noticing this, the lack of them helping the maintainer and the lack of testing this is what made this easily slip by and hit everyone. As the blocker appeared in multiple support mediums, it took time until we came to a solution for this blocker. Identification and awareness of such user experience regressions should be much faster. It was a long time until news was brought out and a resolution arrived. Ideally our process should be that we are prepared or can prevent that; for situations where FUD regarding a particular solution arises, proper help and testing should point out which solution is the way to pursue. But, preventing everything is a dream; so, ... =3D=3D How to indicate regressions much faster? =3D=3D We can react earlier by automatically gathering output and/or logs from Bugzilla, Forums and IRC; which allows us to automatically identify newly developing as well as reoccurring problems. Since it requires quite some processing, I think this project needs more than one person, or at least I think so until further study shows how easy this is to implement; I thus wonder if the rest of the QA team would be interested in helping out on such an effort. =3D=3D What do I think the new Gentoo QA team should look like? =3D=3D Gentoo QA ... - should be neutral or positive. We should put user and developer experience in front and definitely be forgiving for mostly harmless mistakes that only happen once or twice; - should only intervene if it is absolutely necessary, or when contributing results is making Gentoo Linux and its content, services and processes of better quality; - should meet at least on a monthly basis (or more often), like the Gentoo Council, to keep focus on QA and discuss so we can progress; - should mark their actions with QA to make it clear it are QA actions; - should do different tasks than Gentoo Council and Gentoo ComRel, as we are not decisive and also shouldn't and cannot give out warnings. I would act this way and hope the rest would do so as well. :) =3D=3D What can we do to make the new Gentoo QA team more effective? =3D=3D For the new Gentoo QA team to become more effective, we should put extra focus on obtaining good metric reports the first one or two weeks. For example; after starting the Bug Cleaners project, I came up with an idea that we also need QA metrics that indicate which packages are unmaintained but assigned to other people than maintainer-needed@g.o. A first metric is to check for the packages with the most amount of bugs, for which I wrote a script [1] which gives us output [2]; as you can see, this first version rather gives a list that shows the most popular packages that the most bugs are filed for. It does this by obtaining the open bug lists [3], then processing and filters them to have a list of ATOMs with associated bug ID, then count them. This metric is different from what we want to achieve, we want to find unmaintained packages. So, to improve this metric I plan to take the commit history into account which should effectively tell us "the most popular packages (on Bugzilla) for which the least commits happened". This last metric matches the description of "unmaintained" much better. For the Portage tree we already have AutoRepoman [4][5] by Patrick, which we need to put to good use as there is work to be done there. =3D=3D What other work is there to do? =3D=3D - Look for QA issues that bother users every day on the support mediums and look if we can relieve or resolve that burden. - Cleaning obsolete entries in profiles/, for example in package.mask there are a lot of entries that have been there for a long time; they are either no longer present or are lingering on QA issues. - Decrease the amount of unmaintained packages by looking (together with PR / Recruiters / ...) into ways we can bring in more manpower, as well as look if we can improve the processes used for this. - Improve documentation, policy and other reading material where missing information results in confusion or inconsistency; based on reached consensus by the developers and/or the Gentoo Council. - Indicate long standing QA related issues and discuss with the QA team towards a resolution, such that we can move on and progress. - Evaluating GLEPs to see if they are still relevant, renewing sentences that became out of date, ensuring an author or herd that can be contacted is listed (marking retired devs as retired); reporting / submitting patches to the authors and/or Gentoo Council. - Write a version of AutoRepoman that would process commits as they happen, which allows for a quicker response. We should run them side-by-side, as some commits like eclass changes can result in a lot of computation time as a lot of ebuilds would need to be checked. (AutoRepoman fits this last purpose better as it prevents doing the same computation multiple times, when the eclass is touched twice) - ... =3D=3D Comments =3D=3D Thank you very much for reading my application. Feel free to contact me. PS: For a quick summary you can read every first paragraph sentence. PPS: Personally I might also help or fork Portage [and its repoman and QA warnings] and/or pkgcore if the drop in development continues. While in the long run alternatives are made, in the short run we need to keep Portage alive; Gentoo without an alive PM could result in bad quality. =3D=3D References =3D=3D [1]: Gist - Obtain the packages with the most amount of open bugs. https://gist.github.com/TomWij/d1ebf743360352ebb2bd [2]: Gist - Example output of the above Gist. https://gist.github.com/TomWij/93432255142b5fb81998 [3]: Gentoo Bugzilla - Bot policies http://bugs.gentoo.org/bots.html=20 [4]: Patrick's playground - AutoRepoman http://gentooexperimental.org/~patrick/weblog/archives/2013-07.html [5]: Patrick's hosting - Repoman output for each category http://packages.gentooexperimental.org/repoman-checks/ =20 --MP_/jFo_fL6jcgldNf16Sa+kCIS-- --Sig_/hB7k4Q7XslodE1RldxWtuos Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQEcBAEBAgAGBQJTb/nGAAoJEPWZc8roOL/QyUUH/AjwyOWCqZy6ZDWx7H2C14RW 09wnfPcpRLXvJmjDCkN29bi0d1SvzbKVuK2Jor4+wQqqTsuEbmDbqrHlyk3IuGnn awQSI0lyqSGq2clGmDTJ+hw47RgFWDflGrzcWltj3IwV995GUljzOPhFOxv4ZPri vEE3ILVJyhYn+BTPKwNLVnvw9YZnZyanzKHSQxKrwlmNWrBHfchx4zbg1vWTN8t0 5nu5sAII7Qs7B/MO+PykZll88GFXDdN0GJiyvwFym/6Jr/gn4HMf+ckrypf3J6IM pS+R7mvmcqBaUPQGylBYuFeOIYLC/DAqiHGnJq/g/C8EfycQ6cDAj7QbAGLMJDI= =BMiK -----END PGP SIGNATURE----- --Sig_/hB7k4Q7XslodE1RldxWtuos--