From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) by finch.gentoo.org (Postfix) with ESMTP id 5EF521384B4 for ; Tue, 8 Dec 2015 00:05:36 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 9D73621C0A5; Tue, 8 Dec 2015 00:05:28 +0000 (UTC) Received: from mail-io0-f171.google.com (mail-io0-f171.google.com [209.85.223.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id B485621C015 for ; Tue, 8 Dec 2015 00:05:27 +0000 (UTC) Received: by iofh3 with SMTP id h3so7759020iof.3 for ; Mon, 07 Dec 2015 16:05:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=scriptkitty-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:content-type; bh=5GAOWbN9jonuAlLG0HC5qbxyCzHZJF7wNZDo2WU9Kgg=; b=lGtZV31QWQGObSnKAj1LB6kzDZuJGG9mjoDXJAR7CVVxEy+WYQivNzv3WQMTO5xPjD 9NungLeKaUeKXA8w1Kb9+UdiQsO8GUf4AqbN88hd6CXWee3Lz3kt1Vbrf+fp4/89VU9r /h0DgJJ9VLZitBjTUGJMGoiwWriM+vH+RuBBxGz3pssvjRnHvPwuiZPok8hzA2SaICEG yxLaiPkT4AhjvBxqy+A8B7THj5txBd5rv+WvJamiStuvT7b2/O90dPhNhtDYmFK5xJsR WjMHlLyv/kE4gXafqPWnDLkGlBoqs5FnRl/6HgUP5BRekAtdLL4Hi3CTPfjZOT7jkWKS 8Twg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=5GAOWbN9jonuAlLG0HC5qbxyCzHZJF7wNZDo2WU9Kgg=; b=ItcfumvAgRXeFgYB6NUp0t1lEBvkZGmR793wOYyhDH0s8car3B0Z8ZYBldCsCa6Ktd dGTLIgHvMHX0LQBfpaLafSatOETN2LFrz0DhjxfKVBj18d7POzmLhjVdud0XHbcxHrTy wiLoDqJyZ+qZUCElHLAPTjodj4LcJBY0wJoAS8VTAOvy6jo+rgXM/z1T4zb0+aobitKS cvlOvwiaUMVaW1uiiodlLDT318NH3a4kWYn/7wEbocUeHs+HYbPh4PdxLOEOAep9P0lt I1TpEWD4h1NuUQe/tSX4TiHh8e5ShYZbf4XJh+uD4udp7K3Ai9GkZP/sp+ynWnc6+yMg dBvg== X-Gm-Message-State: ALoCoQm7qecXpGG8J1xRgPAecVnukaMCXSPf3CAxTINAfyA+rYmSYNbDLqYqMljLDq3Tc4HdQY8zs2xqySt/MR57rnFLcghRNg== Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-dev@lists.gentoo.org Reply-to: gentoo-dev@lists.gentoo.org MIME-Version: 1.0 X-Received: by 10.107.17.13 with SMTP id z13mr1286993ioi.97.1449533126735; Mon, 07 Dec 2015 16:05:26 -0800 (PST) Sender: antarus@scriptkitty.com Received: by 10.79.79.8 with HTTP; Mon, 7 Dec 2015 16:05:26 -0800 (PST) X-Originating-IP: [2620:0:1000:3001:10ed:811a:a06d:4e4f] In-Reply-To: <20151206153611.2a132d2c.mgorny@gentoo.org> References: <20151206153611.2a132d2c.mgorny@gentoo.org> Date: Mon, 7 Dec 2015 16:05:26 -0800 X-Google-Sender-Auth: 481mO2wbfcmihKlp1axNdAFKoQ4 Message-ID: Subject: Re: [gentoo-dev] RFC: automatically mailing people on pkgcheck problems with their packages From: Alec Warner To: Gentoo Dev Content-Type: multipart/alternative; boundary=001a113ecf321c11da052657bba2 X-Archives-Salt: 6cb01eb9-b016-4d80-bf22-7a77121647dd X-Archives-Hash: 6fb21b8b21542e1c149daf5e74eceb58 --001a113ecf321c11da052657bba2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Sun, Dec 6, 2015 at 6:36 AM, Micha=C5=82 G=C3=B3rny = wrote: > Hello, > Hi! > > As you have seen multiple times, I'm running a minimalistic CI service > for Gentoo that checks the repository for major issues using pkgcheck. > So far it's automation is limited to sending a mail to dedicated > gentoo-automated-testing@lists.gentoo.org mailing list on breakage > changes. From there, I compare the results to recent git log and mail > the developers at fault, pointing out the bad commit. > > A few developers have already subscribed to the mailing list to check > if they haven't caused any new breakages and fix them quickly. For > others, it's pretty much just me caring to check, which also means that > when I'm not around things are left broken. > > So this sort of brings up a point of responsibility. > Automating the blaming process has been suggested multiple times > already but I so far considered it not worth the effort. Mostly because > many of the issues are indirect, and trying to automatically figure > them out from combination of the pkgcheck report and recent commits > would be hard, and could cause false positives. For example, some of > the depgraph breakages happen because of package.mask changes -- > figuring that out automatically wouldn't be easy, and the script could > blame an irrelevant commit in the package. > > However, it was suggested recently that I could make it mail > the maintainers of the affected packages. Even though most often it's > not them who are at fault, it was suggested that they'd prefer to know > that their packages are broken. > I think there are a few issues: 1) Not everyone cares. I think you can either go for an opt-in approach (hard..you need to keep state) or offer clear opt-out / filtering instructions (link in the bottom of the email that points at the opt-out instructions on wiki.) Either decision will piss people off; I wouldn't fret it as long as you pick one. 2) Unclear ownership of the problem. One guy makes a commit, 100 packages break. Who is responsible? Its really murky. This is really the toughest problem to me. 3) Problems are not stateless (e.g. many are transient as they are fixed later by developers.) Is the email I got 8 hours ago still relevant? What we normally see in items like this is a framework to manage "incidents". So what you might see is an incident App. The CI infrastructure detects a problem and opens an incident. At incident open, you trigger a notification (said email). Typically incidents can be claimed (a human takes ownership and fixes the incident) or perhaps a future run of the automation detects that the incident is fixed and closes the incident. The problem of course with 3 is that you are very much re-inventing a bunch of functionality that is already in bugzilla; which leads to the argument of 'why not open bugs for breakages' ;) -A > > So what do you think? Would it be fine to mail the package maintainers > whenever their packages break? Would it be a problem if I just CC-ed > all the maintainers on the gentoo-automated-testing mails? Please note > that the breakages are catched per-package, and the script wouldn't be > able to respect restrict=3D"" or hand-written maintainer descriptions ;-)= . > > -- > Best regards, > Micha=C5=82 G=C3=B3rny > > --001a113ecf321c11da052657bba2 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
On S= un, Dec 6, 2015 at 6:36 AM, Micha=C5=82 G=C3=B3rny <mgorny@gentoo.org&= gt; wrote:
Hello,
=

Hi!
=C2=A0
gentoo-automat= ed-testing@lists.gentoo.org mailing list on breakage
changes. From there, I compare the results to recent git log and mail
the developers at fault, pointing out the bad commit.

A few developers have already subscribed to the mailing list to check
if they haven't caused any new breakages and fix them quickly. For
others, it's pretty much just me caring to check, which also means that=
when I'm not around things are left broken.


So this sort of brings up a point of r= esponsibility.

=C2=A0
Automating the blaming process has been suggested multiple times
already but I so far considered it not worth the effort. Mostly because
many of the issues are indirect, and trying to automatically figure
them out from combination of the pkgcheck report and recent commits
would be hard, and could cause false positives. For example, some of
the depgraph breakages happen because of package.mask changes --
figuring that out automatically wouldn't be easy, and the script could<= br> blame an irrelevant commit in the package.

However, it was suggested recently that I could make it mail
the maintainers of the affected packages. Even though most often it's not them who are at fault, it was suggested that they'd prefer to know<= br> that their packages are broken.

I think= there are a few issues:

1) Not everyone cares. I = think you can either go for an opt-in approach (hard..you need to keep stat= e) or offer clear opt-out / filtering instructions (link in the bottom of t= he email that points at the opt-out instructions on wiki.) Either decision = will piss people off; I wouldn't fret it as long as you pick one.
=

2) Unclear ownership of the problem. One guy makes a co= mmit, 100 packages break. Who is responsible? Its really murky. This is rea= lly the toughest problem to me.

3) Problems are no= t stateless (e.g. many are transient as they are fixed later by developers.= ) Is the email I got 8 hours ago still relevant? What we normally see in it= ems like this is a framework to manage "incidents". So what you m= ight see is an incident App. The CI infrastructure detects a problem and op= ens an incident. At incident open, you trigger a notification (said email).= Typically incidents can be claimed (a human takes ownership and fixes the = incident) or perhaps a future run of the automation detects that the incide= nt is fixed and closes the incident.

The problem o= f course with 3 is that you are very much re-inventing a bunch of functiona= lity that is already in bugzilla; which leads to the argument of 'why n= ot open bugs for breakages' ;)

-A
= =C2=A0

So what do you think? Would it be fine to mail the package maintainers
whenever their packages break? Would it be a problem if I just CC-ed
all the maintainers on the gentoo-automated-testing mails? Please note
that the breakages are catched per-package, and the script wouldn't be<= br> able to respect restrict=3D"" or hand-written maintainer descript= ions ;-).

--
Best regards,
Micha=C5=82 G=C3=B3rny
<http://dev.gentoo.org/~mgorny/>

--001a113ecf321c11da052657bba2--