From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) by finch.gentoo.org (Postfix) with ESMTP id C23D51381F3 for ; Wed, 19 Jun 2013 11:31:54 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 81D51E0A07; Wed, 19 Jun 2013 11:31:47 +0000 (UTC) Received: from smtp.gentoo.org (smtp.gentoo.org [140.211.166.183]) (using TLSv1 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id D98C7E09FF for ; Wed, 19 Jun 2013 11:31:46 +0000 (UTC) Received: from marga.jer-c2.orkz.net (D4B2706A.static.ziggozakelijk.nl [212.178.112.106]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: jer) by smtp.gentoo.org (Postfix) with ESMTPSA id 97C6C33E5EF for ; Wed, 19 Jun 2013 11:31:45 +0000 (UTC) Date: Wed, 19 Jun 2013 13:31:39 +0200 From: Jeroen Roovers To: gentoo-project@lists.gentoo.org Subject: Re: [gentoo-project] You have been researched (again) Message-ID: <20130619133139.2ddafacc@marga.jer-c2.orkz.net> In-Reply-To: <51C18BB1.8010109@gentoo.org> References: <20130619000101.526e0aaf@marga.jer-c2.orkz.net> <20130619010228.GA26921@gengoff.local.grandmasfridge.org> <51C18BB1.8010109@gentoo.org> X-Mailer: Claws Mail 3.9.1 (GTK+ 2.24.17; i686-pc-linux-gnu) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Project discussion list X-BeenThere: gentoo-project@lists.gentoo.org Reply-To: gentoo-project@lists.gentoo.org Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Archives-Salt: 76cfceeb-560c-4b64-9302-862809904122 X-Archives-Hash: e00d78c555405493ae468c7fef8c7c5b On Wed, 19 Jun 2013 12:45:05 +0200 Thomas Kahle wrote: > On 06/19/2013 03:02 AM, Aaron W. Swenson wrote: > > On 2013-06-19 00:01, Jeroen Roovers wrote: > >> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > >> - - - David Garcia, Marcelo Serrano Zanetti and Frank Schweitzer > >> Chair of Systems Design =E2=80=93 www.sg.ethz.ch =E2=80=93 ETH Zurich > >> > >> [...] > >> > >> http://arxiv.org/abs/1306.3612 > >> > >> > >> Merry reading, > >> jer > >> > >=20 > > So, there ya go. Start taking happy pills and be more positive and > > motivating! > >=20 >=20 > I've not read their new paper but the first one was just ridiculous. > They drew an edge from A to B if A reassigned a bug to B and > intepreted this as "A knows that B is an expert on the subject". > That is just ridiculous since A is usually a bug wrangler and B is to > be found in metadata. For instance the emacs team which gets very > few bugs almost never appears as B (and certainly not A). This > introduces all sorts of bias. I think it would be fair to say that > their first study had very low predictive power and the effects they > saw were created by their method. Good to (finally) see some critical review on the "laymen's" side of the subject matter. I didn't vent any opinion on the last paper since, you know, they're doing science in their niche and I'm not going to tell them how to do it - social research has its fundamental problems and I have my own to deal with. In reading the second paper on the alleged decline of Gentoo's bug tracker performance, I went back to the first one, and I don't really see how performance degraded and never returned to form after that cataclysmic event in ~2008. I think it's fair to say things are done very differently now and that lots of bug reports now don't get CLOSED/INVALID, REOPENED, wrongly assigned, reassigned, re-reported, redefined, entangled with unrelated but similar bugs, and DUPLICATEd so much any more, so I've asked the researchers some questions[1] on this issue of measuring performance[2]. I should also point out that doing social science properly is rather tricky, and that not getting entangled in the historiography of the subject matter is important. Lots of things went on in 2008 - good and bad things, and the community was under a lot of stress. And while I am at it: do note that nobody was interviewed this time around. The second paper regards Gentoo's poor bug tracker performance after Alice's retirement as a well-established fact found in the first paper. Regards, jer [1] And within hours I have three replies from the same person in my inbox! I'll probably follow up on this. [2] Frankly it stings a little to read about my vain efforts to improve bug wrangling, but conversely I might simply blame the research metrics. :)