[gentoo-dev] [v5] Planning for automatic assignment computation of bugs

[gentoo-dev] [v5] Planning for automatic assignment computation of bugs

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 6811 bytes --]

In the spirit of finishing projects that I started long ago, and have
been improving with age (hopefully we don't end up with any vinegar), I
bring you yet another post about automatic bug assignment.

Hopefully if this is accepted now, we can see about the integration needed when
Bugzilla3 comes online.

Previously:
===========
http://thread.gmane.org/gmane.linux.gentoo.devel/48485 [v1]
http://thread.gmane.org/gmane.linux.gentoo.devel/49601 [v2]
http://thread.gmane.org/gmane.linux.gentoo.devel/57159 [v3]
http://thread.gmane.org/gmane.linux.gentoo.devel/58575 [v4]
http://thread.gmane.org/gmane.linux.gentoo.devel/59286 [v4a]

Known prototypes:
=================
http://dev.gentoo.org/~rbu/assign.py
- I believe this prototype is fully up to date, it just takes
  herds/metadata.xml as well as a string, and gives you the output the
  commandline.
  rbu: can you confirm that your prototype matches the proposal?

How is it integrated into workflow?
===================================
(This was raised by halcy0n at the v3 review, and replaces the previous section
on triggering).
1. Bugs are still filed as normal, with the same default assignees
   (bug-wranglers most commonly).
2. In the wrangling process, we add a new button to the page, labelled:
   "Suggest assignment".
3. Optionally, the wrangler edits the summary line right now, but does not hit
   'Submit' yet.
4. The button causes a (javascript) query back to the server (also
   available as a web service for other usage). The only data sent is the
   'Summary' string.
5. Javascript then appends the server results into the "Additional
   Comments" box: a suggested assignee and suggested CC values, with logic
   as to why.
6. The wrangler can copy and paste the data into the fields, editing
   further as desired.

(Putting the data into the comments allows the assignment tool to
explain WHY it decided certain actions).

Alternatively, the above can be accomplished with pybugz integration.

Assignment/CC computing:
========================
Step 1 - Summary line processing
--------------------------------
1. If the summary line contains a package atom for a package that
   exists, use the metadata.xml for that package.
2. If the summary line contains a package atom for a package that does
   not exist, but a category that does exist, use the metadata.xml for
   that category.
3. Process ALL atoms in the summary line, using any after the first for CC
   only. (new in v4)

Step 2 - Metadata.xml contains only a herd
------------------------------------------
1. Take the herd element, and look up the herd in herds.xml to convert
   to an email address. This email address must be a valid bugzilla
   account.
2. This email is treated as an implicit maintainer element after this
   point. "<maintainer><email>${HERD_EMAIL}</email></maintainer>"
[See notes]

Step 3 - <maintainer> element
-----------------------------
1. Add the maintainer element to an ordered list, in the order they are
   present in the file.
2. If an element appears more than once, the later element overrides the
   earlier element. (This provides a route when the herd is assigned,
   but does not wish to receive email for a specific package).
3. If a maintainer element contains the non-default 'ignoreauto=1' 
   attribute AND a non-empty description element (describing why this
   maintainer should not be contacted), delete it from the list.

Step 4 - Assignment
-------------------
1. Use the first email in the ordered list as the assignee.
2. Place all remaining emails in the CC list.
3. Include a short comment about the processing results.

Notes:
------
1. For handling <herd>no-herd</herd>, we should add an entry into herds.xml to
   catch it (maintainer-needed <at> g.o). Every herd listed in an ebuild MUST be in
   herds.xml.
2. Herds that do not wish to be contacted for specific bugs should add an
   maintainer element stating that (and use 'ignoreauto' on the element).
   This case however should be very rare, as the package probably doesn't
   belong in the herd if the herd doesn't care about it.
3. If "no-herd" is listed as a herd AND the metadata contains another herd or
   maintainer element, drop the "no-herd" entry from computation.
4. If you want the default assignment to go to a maintainer, and NOT the herd, 
   move the <herd> element further down in the metadata.xml!

Differences to prior bug-wranglers policy
=========================================
(now included because every time I posted, somebody raised it. See [v4a]).
- Bug wranglers (http://www.gentoo.org/proj/en/qa/bug-wranglers/)
  "When the file lists multiple entries, then you assign the bug to the first
   maintainer, and CC the other maintainer(s) and herd(s)."
- The root problem with this is that various herds and maintainers want
  different behaviors on their bugs.
  1. Assign to herd, CC all others (eg: GNOME, base-system)
  2. Assign to first maintainer, CC herd and others (eg: net-mail)
- Instead, the automatic policy would be summarized for the bug-wranglers
  policy as follows:
  "When the file lists multiple entries (either herds or maintainers), then you
   assign the bug to the first non-ignorable entry, and CC the other entries."
- Using the order of herds and maintainers in the metadata.xml would be
  required even as a fallback to other optional XML attributes, and 
  additionally makes sense when you just read the metadata.

Effects on metadata.xml syntax
==============================
- Category metadata is now permitted to have herd and maintainer elements.
- New attribute under maintainer, 'ignoreauto'.

--- metadata.dtd	2007-11-20 10:07:33.000000000 -0800
+++ metadata.dtd.new	2008-10-18 23:22:55.000000000 -0700
@@ -1,7 +1,7 @@
 <!ELEMENT packages ( pkgmetadata* )>

 <!-- Metadata for a category -->
-<!ELEMENT catmetadata ( (longdescription)* )>
+<!ELEMENT catmetadata ( (herd|maintainer|longdescription)* )>
 <!ATTLIST catmetadata pkgname CDATA "">

 <!-- Metadata for a package -->
@@ -13,6 +13,10 @@

   <!-- One tag for each maintainer of a package, multiple allowed-->
   <!ELEMENT maintainer ( email, (description| name)* )>
+	  <!-- ignoreauto attribute specifies that a maintainer should not be used
+		   for automatic assignment computing. It must be combined with a
+		   non-empty description element. -->
+      <!ATTLIST maintainer ignoreauto CDATA "0">

   <!-- A long description of the package in freetext-->
   <!ELEMENT longdescription (#PCDATA|pkg)* >

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee & Infrastructure Lead
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 330 bytes --]

Re: [gentoo-dev] [v5] Planning for automatic assignment computation of bugs

[Sebastian Pipping]

On 05/03/10 23:49, Robin H. Johnson wrote:
> Known prototypes:
> =================
> http://dev.gentoo.org/~rbu/assign.py
> - I believe this prototype is fully up to date, it just takes
>   herds/metadata.xml as well as a string, and gives you the output the
>   commandline.
>   rbu: can you confirm that your prototype matches the proposal?

I have fork of that laying around here:
http://git.goodpoint.de/?p=eassign.git;a=summary

When rbu and I last talked about it we came to the conclusion that the
tool serves several distinct use cases ..

  - backend use by bugzilla
  - command line use on package X, even without a bug
  - security team usage (forgot the details)

.. and that they need to be split smartly to keep each working in a
shared codebase.  python modules and three entry point scripts on it
could work.

I know that neither rbu nor me have resources for the code of this atm,
maybe idl0r has?  I'd like to be notified on progress, though.

Two regex related things I'd like to append:

 - I've been using this alternative regex for but title extraction
   elsewhere before, seems to work well:

     package_pattern =
re.compile('\\b(?P<category>[a-z0-9_+-]+-[a-z0-9_+-]+)/(?P<package>[a-z0-9_+-]+(?:[a-z_+]+[0-9]+|[a-z0-9_+-]+[a-z+]+))')

 - If you stick with the original pattern, this patch may still help:

http://git.goodpoint.de/?p=eassign.git;a=commitdiff;h=7ad3cd607589aba7a1528dbd0744a942d18065eb


> How is it integrated into workflow?
> ===================================
> [..]
> 5. Javascript then appends the server results into the "Additional
>    Comments" box: a suggested assignee and suggested CC values, with logic
>    as to why.

assign.py does not explain reasoning, yet.


> Assignment/CC computing:
> ========================
> Step 1 - Summary line processing
> --------------------------------
> [..]
> 3. Process ALL atoms in the summary line, using any after the first for CC
>    only. (new in v4)

I'm unsure about the handling of order:
it wouldn't work well for "dependency foo/bar missing on foo2/bar2".


> Step 2 - Metadata.xml contains only a herd
> ------------------------------------------
> 1. Take the herd element, and look up the herd in herds.xml to convert
>    to an email address. This email address must be a valid bugzilla
>    account.

This may need a replacement map.  Think media-video versus video,
perl-bugs versus perl, etc.


Best,



Sebastian

Re: [gentoo-dev] [v5] Planning for automatic assignment computation of bugs

[Robin H. Johnson]

On Thu, May 06, 2010 at 04:06:39AM +0200, Sebastian Pipping wrote:
> .. and that they need to be split smartly to keep each working in a
> shared codebase.  python modules and three entry point scripts on it
> could work.
+1 on that.

>      package_pattern =
> re.compile('\\b(?P<category>[a-z0-9_+-]+-[a-z0-9_+-]+)/(?P<package>[a-z0-9_+-]+(?:[a-z_+]+[0-9]+|[a-z0-9_+-]+[a-z+]+))')
- Fails to match category 'virtual'.
- That PV matching looks really fishy.
- I would probably just steal the pkgatom stuff from pkgcore/Portage, as
  that's already a fairly canonical regex.

>  - If you stick with the original pattern, this patch may still help:
> http://git.goodpoint.de/?p=eassign.git;a=commitdiff;h=7ad3cd607589aba7a1528dbd0744a942d18065eb
I think most of this might have already from from one the above.

> > How is it integrated into workflow?
> > ===================================
> > [..]
> > 5. Javascript then appends the server results into the "Additional
> >    Comments" box: a suggested assignee and suggested CC values, with logic
> >    as to why.
> assign.py does not explain reasoning, yet.
Ok, major todo on adding it, see below for why.

> > Assignment/CC computing:
> > ========================
> > Step 1 - Summary line processing
> > --------------------------------
> > [..]
> > 3. Process ALL atoms in the summary line, using any after the first for CC
> >    only. (new in v4)
> 
> I'm unsure about the handling of order:
> it wouldn't work well for "dependency foo/bar missing on foo2/bar2".
That's part of why the reasoning output was needed.
The full output for that should be something like:
=========
Combined:
---------
To: $ADDR1
CC: $ADDR2,$ADDR3
Ignoring: $ADDR4, $ADDR5

For foo/bar:
------------
To: ...
CC: ...
Ignoring: ...

For foo2/bar2:
--------------
To: ...
CC: ...
Ignoring: ...
=========
(optionally with some CLI options to say give only the combined
version).

> > Step 2 - Metadata.xml contains only a herd
> > ------------------------------------------
> > 1. Take the herd element, and look up the herd in herds.xml to convert
> >    to an email address. This email address must be a valid bugzilla
> >    account.
> This may need a replacement map.  Think media-video versus video,
> perl-bugs versus perl, etc.
I'm saying the replacement map IS from the herds.xml data. For herds
that have multiple emails (normal mail vs. bugzilla address), we really
need to find a way to document both of them in herds.xml.

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee & Infrastructure Lead
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

Re: [gentoo-dev] [v5] Planning for automatic assignment computation of bugs

[Sebastian Pipping]

On 05/06/10 06:27, Robin H. Johnson wrote:
>>      package_pattern =
>> re.compile('\\b(?P<category>[a-z0-9_+-]+-[a-z0-9_+-]+)/(?P<package>[a-z0-9_+-]+(?:[a-z_+]+[0-9]+|[a-z0-9_+-]+[a-z+]+))')
> - Fails to match category 'virtual'.

True.

> - That PV matching looks really fishy.

It's not meant to match versions at all.  Could it be you misread the regex?

Best,



Sebastian