From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id 4B31C13832E for ; Wed, 10 Aug 2016 01:28:49 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 727BEE0BB0; Wed, 10 Aug 2016 01:28:46 +0000 (UTC) Received: from smtp.gentoo.org (smtp.gentoo.org [140.211.166.183]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id 966D1E0AD2 for ; Wed, 10 Aug 2016 01:28:45 +0000 (UTC) Received: from professor-x (S010634bdfa9ecf80.vc.shawcable.net [96.49.31.57]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: dolsen) by smtp.gentoo.org (Postfix) with ESMTPSA id DDF6B340B1E for ; Wed, 10 Aug 2016 01:28:43 +0000 (UTC) Date: Tue, 9 Aug 2016 18:28:41 -0700 From: Brian Dolbec To: gentoo-project@lists.gentoo.org Subject: Re: [gentoo-project] Call for agenda items - Council meeting 2016-08-14 Message-ID: <20160809182841.470ecf6b.dolsen@gentoo.org> In-Reply-To: References: <2e11e445-c25b-b7f2-def1-99aed92308b6@gentoo.org> <20160804162443.GA7048@whubbs1.gaikai.biz> <20160804231224.7b7462168f1d23e88fe4135c@gentoo.org> <20160804222234.GA8357@whubbs1.gaikai.biz> <20160805022658.GA15727@linux1> <20160805142859.GA19008@linux1> <20160805153658.GA11058@whubbs1.gaikai.biz> <52993bd4-afc9-197e-acda-96db413e6608@gentoo.org> <20160809173255.0ddfa090@katipo2.lan> <20160809220545.6f1c7dd3@katipo2.lan> <20160809074105.37e9ba17.dolsen@gentoo.org> Organization: Gentoo Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Project discussion list X-BeenThere: gentoo-project@lists.gentoo.org Reply-To: gentoo-project@lists.gentoo.org MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Archives-Salt: 75dd725b-2198-413d-90d0-32f27972fe64 X-Archives-Hash: 95e7fde0be6865cddd52fd58e4345c6f On Wed, 10 Aug 2016 06:45:16 +0530 Pallav Agarwal wrote: > On Tue, Aug 9, 2016 at 8:11 PM, Brian Dolbec > wrote: > > > On Tue, 9 Aug 2016 22:05:45 +1200 > > Kent Fredric wrote: > > > > > On Tue, 9 Aug 2016 01:59:35 -0400 > > > Rich Freeman wrote: > > > > > > > While I think your proposal is a great one, I think this is > > > > actually the biggest limitation. A lot of our packages (most?) > > > > don't actually have tests that can be run on every build (most > > > > don't have tests, some have tests that take forever to run or > > > > can't be used on a clean install). > > > > > > IMHO, That's not "ideal", but we don't need idealism to be useful > > > here. > > > > > > Tests passing give one kind of useful kind of quality test. > > > > > > But "hey, it compiles" gives useful data in itself. > > > > > > By easy counter example, "it doesn't compile" is in itself useful > > > information ( and the predominant supply of bugs filed are > > > compilation failures ). > > > > > > Hell, sometimes I hit a compile failure and I just go "eeh, I'll > > > look into it next week". How many people are doing the same? > > > > > > The beauty of the automated datapoint is it doesn't have to be > > > "awesome quality" to be useful, its just guidance for further > > > investigation. > > > > While runtime testing doesn't HAVE to be extensive, we do want > > > > somebody to at least take a glance at it. > > > > > > Indeed, I'm not hugely in favour of abolishing manual > > > stabilization entirely, but sometimes it just gets to a point > > > where its a bit beyond a joke with requests languishing untouched > > > for months. > > > > > > If there was even data saying "hey, look, its obvious this isn't > > > ready for stabilization", we could *remove* or otherwise mark for > > > postponement stabilization requests that were failing due to > > > crowd-source metrics. > > > > > > This means it can also be used to focus existing stabilization > > > efforts to reduce the number of things being thrown in the face > > > of manual stabilizers. > > > > > > > > > > > If everything you're proposing is just on top of what we're > > > > already doing, then we have the issue that people aren't > > > > keeping up with the current workload, and even if that report > > > > is ultra-nice it is actually one more step than we have today. > > > > The workload would only go down if a machine could look at the > > > > report and stabilize things without input at least some of the > > > > time. > > > > > > Indeed, it would require the crowd service to be automated, and > > > the relevant usage of the data as automated as possible, and > > > humans would only go looking at the data when interested. > > > > > > For instance, when somebody manually files a stable request, some > > > watcher could run off and scour the reports in a given window and > > > comment "Warning: Above threshold failure rates for target in last > > > n-days, proceed with caution", and it would only enhance the > > > existing stabilization workflow. > > > > This whole thing you are proposing has been a past stats project > > many times in GSOC for Gentoo. The last time, it produced a decent > > system that was functional and __NEVER__ got deployed and turned > > on. It ran for several years on the gentoo GSOC student server > > (vulture). It never gained traction with the infra team due to > > lack of infra resources and infra personnel to maintain it. > > > > Perhaps with the new hardware recently purchased to replace the > > failed server from earlier this year, we should have some hardware > > resources. If you can dedicate some time to work on the code which > > I'm sure will need some updating now, I would help as well (not > > that I already can't keep up with all the project coding I'm > > involved with). > > > > This is of course if we can get a green light from our infra team > > to be able to implement a stats vm on the new ganeti system. > > > > We will also need some help from security people to ensure the > > system is secure, nginx/lightttp configuration, etc... > > > > So, are you up for it? Any Gentoo dev willing to help admin such a > > system, please reply with your area of expertise and ability to > > help. > > > > Maybe we can finally get a working and deployed stats system. > > > > -- > > Brian Dolbec > > > > > The similar GSoC project this year is in fact my project, named > [Continuous Stabilization]. I would be very interested to know what I > can do to ensure that the system is deployed and used this time. Yes, I am familiar with your project. The best way to ensure that it is set up and used is to become a gentoo developer after GSOC is over and keep working on your code and pushing to get it installed and running on a server. The stats project when we can get it going could be used to assist your code and help determine the stable candidates. I don't remember if it got implemented, but I proposed an optional/configurable method for the stats project to make users aware of a questionnaire about a pkg they have installed. That way a maintainer can ask some questions of users of the pkg to help determine any issues that might prevent stabilization (all anonymous of course, if the user chooses). -- Brian Dolbec