[gentoo-dev] [v4] Planning for automatic assignment computation of bugs

[gentoo-dev] [v4] Planning for automatic assignment computation of bugs

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 5080 bytes --]

Now into it's Nth great year, I bring you the fourth edition of the
automatic assignment proposal. </truman-show>

Previously:
http://thread.gmane.org/gmane.linux.gentoo.devel/48485 [v1]
http://thread.gmane.org/gmane.linux.gentoo.devel/49601 [v2]
http://thread.gmane.org/gmane.linux.gentoo.devel/57159 [v3]

Bleeding edge prototype:
http://dev.gentoo.org/~rbu/assign.py
MANY thanks to rbu.

If there are no further changes or objections at this time, it would be
nice to implement this by the end of November.

How is it integrated into workflow?
===================================
(This was raised by halcy0n at the v3 review, and replaces the previous section
on triggering).
1. Bugs are still filed as normal, with the same default assignees
   (bug-wranglers most commonly).
2. In the wrangling process, we add a new button to the page, labelled:
   "Suggest assignment".
3. Optionally, the wrangler edits the summary line right now, but does not hit
   'Submit' yet.
4. The button causes a (javascript) query back to the server (also
   available as a web service for other usage). The only data sent is the
   'Summary' string.
5. Javascript then appends the server results into the "Additional
   Comments" box: a suggested assignee and suggested CC values, with logic
   as to why.
6. The wrangler can copy and paste the data into the fields, editing
   further as desired.

(Putting the data into the comments allows the assignment tool to
explain WHY it decided certain actions).

Assignment/CC computing:
========================
Step 1 - Summary line processing
--------------------------------
1. If the summary line contains a package atom for a package that
   exists, use the metadata.xml for that package.
2. If the summary line contains a package atom for a package that does
   not exist, but a category that does exist, use the metadata.xml for
   that category.
3. Process ALL atoms in the summary line, using any after the first for CC
   only. (new in v4)

Step 2 - Metadata.xml contains only a herd
------------------------------------------
1. Take the herd element, and look up the herd in herds.xml to convert
   to an email address. This email address must be a valid bugzilla
   account.
2. This email is treated as an implicit maintainer element after this
   point. "<maintainer><email>${HERD_EMAIL}</email></maintainer>"
[See notes]

Step 3 - <maintainer> element
-----------------------------
1. Add the maintainer element to an ordered list, in the order they are
   present in the file.
2. If an element appears more than once, the later element overrides the
   earlier element. (This provides a route when the herd is assigned,
   but does not wish to receive email for a specific package).
3. If a maintainer element contains the non-default 'ignoreauto=1' 
   attribute AND a non-empty role element (describing why this maintainer 
   should not be contacted), delete it from the list.

Step 4 - Assignment
-------------------
1. Use the first email in the ordered list as the assignee.
2. Place all remaining emails in the CC list.
3. Include a short comment about the processing results.

Notes:
------
1. For handling <herd>no-herd</herd>, we should add an entry into herds.xml to
   catch it (maintainer-needed <at> g.o). Every herd listed in an ebuild MUST be in
   herds.xml.
2. Herds that do not wish to be contacted for specific bugs should add an
   maintainer element stating that (and use 'ignoreauto' on the element).
   This case however should be very rare, as the package probably doesn't
   belong in the herd if the herd doesn't care about it.
3. If you want the default assignment to go to a maintainer, and NOT the herd, 
   move the <herd> element further down in the metadata.xml!

Effects on metadata.xml syntax
==============================
- Category metadata is now permitted to have herd and maintainer elements.
- New attribute under maintainer, 'ignoreauto'.

--- metadata.dtd	2007-11-20 10:07:33.000000000 -0800
+++ metadata.dtd.new	2008-10-18 23:22:55.000000000 -0700
@@ -1,7 +1,7 @@
 <!ELEMENT packages ( pkgmetadata* )>
 
 <!-- Metadata for a category -->
-<!ELEMENT catmetadata ( (longdescription)* )>
+<!ELEMENT catmetadata ( (herd|maintainer|longdescription)* )>
 <!ATTLIST catmetadata pkgname CDATA "">
 
 <!-- Metadata for a package -->
@@ -13,6 +13,10 @@
 
   <!-- One tag for each maintainer of a package, multiple allowed-->
   <!ELEMENT maintainer ( email, (description| name)* )>
+	  <!-- ignoreauto attribute specifies that a maintainer should not be used
+		   for automatic assignment computing. It must be combined with a
+		   non-empty description element. -->
+      <!ATTLIST maintainer ignoreauto CDATA "0">
 
   <!-- A long description of the package in freetext-->
   <!ELEMENT longdescription (#PCDATA|pkg)* >

-- 
Robin Hugh Johnson
Gentoo Linux Developer & Infra Guy
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 329 bytes --]

Re: [gentoo-dev] [v4] Planning for automatic assignment computation of bugs

[Robert Buchholz]

[-- Attachment #1: Type: text/plain, Size: 2608 bytes --]

On Sunday 19 October 2008, Robin H. Johnson wrote:
> Now into it's Nth great year, I bring you the fourth edition of the
> automatic assignment proposal. </truman-show>

Yay!

> 5. Javascript then appends the server results into the "Additional
>    Comments" box: a suggested assignee and suggested CC values, with
> logic as to why.
> 6. The wrangler can copy and paste the data into the fields, editing
>    further as desired.

It would be nice if we had a checkbox "Accept Changes" or of 
that "Suggest" button would fill in the fields itself. Two more copy 
and paste actions less.


> 2. If the summary line contains a package atom for a package that
> does not exist, but a category that does exist, use the metadata.xml
> for that category.

We have three alternatives here:
If at least one valid atom is found, but other atoms are present that 
only have an existing category...

(1) ignore metadata for that category
(2) treat category as regular metadata
(3) append categtory metadata to the end of maintainer list

For example, "dev-java/ibm-jkd-bin breaks with 
x11-base/xorg-server-1.3.0.0"

With the typo in ibm-jdk-bin, the ordered list of maintainers to 
assign/cc would* be
(1) x11@gentoo.org
(2) java@gentoo.org,x11@gentoo.org
(3) x11@gentoo.org,java@gentoo.org

* if java herd maintained dev-java category


> 3. If a maintainer element contains the non-default 'ignoreauto=1'
> attribute AND a non-empty role element (describing why this
> maintainer should not be contacted), delete it from the list.

The role element is not present for maintainers in metadata.xml, only in 
herds.xml. Should we leave this out, or do you mean the description 
element?

> 1. For handling <herd>no-herd</herd>, we should add an entry into
> herds.xml to catch it (maintainer-needed <at> g.o). Every herd listed
> in an ebuild MUST be in herds.xml.

I agree for consistency reasons, the "no-herd" should be listed on 
herds.xml. However, it should not implicate maintainer-needed, as a lot 
of maintained ebuilds carry no-herd and all maintainer-needed ebuilds 
carry a "maintainer-needed@gentoo.org" maintainer in their own 
metadata.xml

> Effects on metadata.xml syntax
> ==============================
> - Category metadata is now permitted to have herd and maintainer
> elements. - New attribute under maintainer, 'ignoreauto'.

Just to add a rationale here: This entry is used to assign / cc to 
ebuild submissions, i.e. "Add dev-perl/Some-CPAN-Module to the tree" 
could figure out perl@gentoo.org automatically. 



Robert

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 835 bytes --]

Re: [gentoo-dev] [v4] Planning for automatic assignment computation of bugs

[Jeremy Olexa]

Robin H. Johnson wrote:
> Notes:
> ------
> 1. For handling <herd>no-herd</herd>, we should add an entry into herds.xml to
>    catch it (maintainer-needed <at> g.o). Every herd listed in an ebuild MUST be in
>    herds.xml.

As rbu pointed out, this is slightly incorrect. Most of my ebuilds 
contain no-herd. ;) As do others.

> 2. Herds that do not wish to be contacted for specific bugs should add an
>    maintainer element stating that (and use 'ignoreauto' on the element).
>    This case however should be very rare, as the package probably doesn't
>    belong in the herd if the herd doesn't care about it.
> 3. If you want the default assignment to go to a maintainer, and NOT the herd, 
>    move the <herd> element further down in the metadata.xml!

AFAIK, "default" now it is assign to maintainer and cc herd and nearly 
always herd is higher up in the file. So, [nearly] all ebuilds will have 
to change metadata.xml

OT, Do the changes to the bugzie interface block bugzilla 3 migration?

Re: [gentoo-dev] [v4] Planning for automatic assignment computation of bugs

[Ulrich Mueller]

>>>>> On Sat, 18 Oct 2008, Robin H Johnson wrote:

> 3. If you want the default assignment to go to a maintainer, and NOT
>    the herd, move the <herd> element further down in the metadata.xml!

I disagree about this point. IMHO the procedure described in
<http://www.gentoo.org/proj/en/qa/bug-wranglers/> makes more sense:

# When the file [i.e., metadata.xml] lists multiple entries, then you
# assign the bug to the first maintainer, and CC the other
# maintainer(s) and herd(s).

> 2. This email is treated as an implicit maintainer element after this
>    point. "<maintainer><email>${HERD_EMAIL}</email></maintainer>"

Explicit maintainer elements should have precedence over implicit ones.

Ulrich

Re: [gentoo-dev] [v4] Planning for automatic assignment computation of bugs

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 2735 bytes --]

On Sun, Oct 19, 2008 at 03:29:46PM +0200, Robert Buchholz wrote:
> > 5. Javascript then appends the server results into the "Additional
> >    Comments" box: a suggested assignee and suggested CC values, with
> > logic as to why.
> > 6. The wrangler can copy and paste the data into the fields, editing
> >    further as desired.
> It would be nice if we had a checkbox "Accept Changes" or of 
> that "Suggest" button would fill in the fields itself. Two more copy 
> and paste actions less.
It's much harder to edit in the CC and assigned-to input boxes than in
the comment area.

> > 2. If the summary line contains a package atom for a package that
> > does not exist, but a category that does exist, use the metadata.xml
> > for that category.
> We have three alternatives here:
> If at least one valid atom is found, but other atoms are present that 
> only have an existing category...
> 
> (1) ignore metadata for that category
> (2) treat category as regular metadata
> (3) append categtory metadata to the end of maintainer list
> 
> For example, "dev-java/ibm-jkd-bin breaks with 
> x11-base/xorg-server-1.3.0.0"
> 
> With the typo in ibm-jdk-bin, the ordered list of maintainers to 
> assign/cc would* be
> (1) x11@gentoo.org
> (2) java@gentoo.org,x11@gentoo.org
> (3) x11@gentoo.org,java@gentoo.org
> 
> * if java herd maintained dev-java category
#2 is the best result here, and I think most summary sentences will be
structured such that the broken thing is the first atom, so we shouldn't
re-order the category meta to the end.

> > 3. If a maintainer element contains the non-default 'ignoreauto=1'
> > attribute AND a non-empty role element (describing why this
> > maintainer should not be contacted), delete it from the list.
> The role element is not present for maintainers in metadata.xml, only in 
> herds.xml. Should we leave this out, or do you mean the description 
> element?
Sorry, I did mean description. My diff to metadata.dtd was right, just
this text was wrong.

> > 1. For handling <herd>no-herd</herd>, we should add an entry into
> > herds.xml to catch it (maintainer-needed <at> g.o). Every herd listed
> > in an ebuild MUST be in herds.xml.
> I agree for consistency reasons, the "no-herd" should be listed on 
> herds.xml. However, it should not implicate maintainer-needed, as a lot 
> of maintained ebuilds carry no-herd and all maintainer-needed ebuilds 
> carry a "maintainer-needed@gentoo.org" maintainer in their own 
> metadata.xml
I'll handle this in the other subthread.

-- 
Robin Hugh Johnson
Gentoo Linux Developer & Infra Guy
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 329 bytes --]

Re: [gentoo-dev] [v4] Planning for automatic assignment computation of bugs

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 1643 bytes --]

On Sun, Oct 19, 2008 at 08:47:15AM -0500, Jeremy Olexa wrote:
> Robin H. Johnson wrote:
>> Notes:
>> ------
>> 1. For handling <herd>no-herd</herd>, we should add an entry into 
>> herds.xml to
>>    catch it (maintainer-needed <at> g.o). Every herd listed in an ebuild 
>> MUST be in
>>    herds.xml.
> As rbu pointed out, this is slightly incorrect. Most of my ebuilds contain 
> no-herd. ;) As do others.
Ok, I realize that I missed a case here, that was mentioned in the very
first proposal. 

>> 2. Herds that do not wish to be contacted for specific bugs should add an
>>    maintainer element stating that (and use 'ignoreauto' on the element).
>>    This case however should be very rare, as the package probably doesn't
>>    belong in the herd if the herd doesn't care about it.
>> 3. If you want the default assignment to go to a maintainer, and NOT the 
>> herd,    move the <herd> element further down in the metadata.xml!
> AFAIK, "default" now it is assign to maintainer and cc herd and nearly 
> always herd is higher up in the file. So, [nearly] all ebuilds will have to 
> change metadata.xml
How about the following variation to rules:
3. If the herd is "no-herd" AND the metadata contains another herd or
   maintainer element, drop the "no-herd" entry from computation.

I'll answer about the default in the other thread.

> OT, Do the changes to the bugzie interface block bugzilla 3 migration?
Unknown, but I would suspect not.

-- 
Robin Hugh Johnson
Gentoo Linux Developer & Infra Guy
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 329 bytes --]

Re: [gentoo-dev] [v4] Planning for automatic assignment computation of bugs

[Alec Warner]

On Sat, Oct 18, 2008 at 11:01 PM, Robin H. Johnson <robbat2@gentoo.org> wrote:
> Now into it's Nth great year, I bring you the fourth edition of the
> automatic assignment proposal. </truman-show>
>
> Previously:
> http://thread.gmane.org/gmane.linux.gentoo.devel/48485 [v1]
> http://thread.gmane.org/gmane.linux.gentoo.devel/49601 [v2]
> http://thread.gmane.org/gmane.linux.gentoo.devel/57159 [v3]
>
> Bleeding edge prototype:
> http://dev.gentoo.org/~rbu/assign.py
> MANY thanks to rbu.
>
> If there are no further changes or objections at this time, it would be
> nice to implement this by the end of November.
>
> How is it integrated into workflow?
> ===================================
> (This was raised by halcy0n at the v3 review, and replaces the previous section
> on triggering).
> 1. Bugs are still filed as normal, with the same default assignees
>   (bug-wranglers most commonly).
> 2. In the wrangling process, we add a new button to the page, labelled:
>   "Suggest assignment".
> 3. Optionally, the wrangler edits the summary line right now, but does not hit
>   'Submit' yet.
> 4. The button causes a (javascript) query back to the server (also
>   available as a web service for other usage). The only data sent is the
>   'Summary' string.
> 5. Javascript then appends the server results into the "Additional
>   Comments" box: a suggested assignee and suggested CC values, with logic
>   as to why.
> 6. The wrangler can copy and paste the data into the fields, editing
>   further as desired.
>
> (Putting the data into the comments allows the assignment tool to
> explain WHY it decided certain actions).
>
> Assignment/CC computing:
> ========================
> Step 1 - Summary line processing
> --------------------------------
> 1. If the summary line contains a package atom for a package that
>   exists, use the metadata.xml for that package.
> 2. If the summary line contains a package atom for a package that does
>   not exist, but a category that does exist, use the metadata.xml for
>   that category.
> 3. Process ALL atoms in the summary line, using any after the first for CC
>   only. (new in v4)
>
> Step 2 - Metadata.xml contains only a herd
> ------------------------------------------
> 1. Take the herd element, and look up the herd in herds.xml to convert
>   to an email address. This email address must be a valid bugzilla
>   account.

What if my herd email address is different from my bugzie address?
Can I have both in herds.xml?  What if my herd address *isn't* a
bugzie account, will the world end?  How can we automatically detect
when developers make mistakes in editing herds.xml?

> 2. This email is treated as an implicit maintainer element after this
>   point. "<maintainer><email>${HERD_EMAIL}</email></maintainer>"
> [See notes]
>
> Step 3 - <maintainer> element
> -----------------------------
> 1. Add the maintainer element to an ordered list, in the order they are
>   present in the file.
> 2. If an element appears more than once, the later element overrides the
>   earlier element. (This provides a route when the herd is assigned,
>   but does not wish to receive email for a specific package).
> 3. If a maintainer element contains the non-default 'ignoreauto=1'
>   attribute AND a non-empty role element (describing why this maintainer
>   should not be contacted), delete it from the list.
>
> Step 4 - Assignment
> -------------------
> 1. Use the first email in the ordered list as the assignee.
> 2. Place all remaining emails in the CC list.
> 3. Include a short comment about the processing results.
>
> Notes:
> ------
> 1. For handling <herd>no-herd</herd>, we should add an entry into herds.xml to
>   catch it (maintainer-needed <at> g.o). Every herd listed in an ebuild MUST be in
>   herds.xml.

You and I both know this is not going to be true.  Complicated
solution; make Repoman do it.  Certainly it is the 'correct' thing to
do; however I don't expect it to get implemented or deployed quickly.
Hacky solution: run script on osprey that tries to validate tree
metadata against herds.xml and annoy herds who forgot to add
themselves.

> 2. Herds that do not wish to be contacted for specific bugs should add an
>   maintainer element stating that (and use 'ignoreauto' on the element).
>   This case however should be very rare, as the package probably doesn't
>   belong in the herd if the herd doesn't care about it.
> 3. If you want the default assignment to go to a maintainer, and NOT the herd,
>   move the <herd> element further down in the metadata.xml!
>
> Effects on metadata.xml syntax
> ==============================
> - Category metadata is now permitted to have herd and maintainer elements.
> - New attribute under maintainer, 'ignoreauto'.
>
> --- metadata.dtd        2007-11-20 10:07:33.000000000 -0800
> +++ metadata.dtd.new    2008-10-18 23:22:55.000000000 -0700
> @@ -1,7 +1,7 @@
>  <!ELEMENT packages ( pkgmetadata* )>
>
>  <!-- Metadata for a category -->
> -<!ELEMENT catmetadata ( (longdescription)* )>
> +<!ELEMENT catmetadata ( (herd|maintainer|longdescription)* )>
>  <!ATTLIST catmetadata pkgname CDATA "">
>
>  <!-- Metadata for a package -->
> @@ -13,6 +13,10 @@
>
>   <!-- One tag for each maintainer of a package, multiple allowed-->
>   <!ELEMENT maintainer ( email, (description| name)* )>
> +         <!-- ignoreauto attribute specifies that a maintainer should not be used
> +                  for automatic assignment computing. It must be combined with a
> +                  non-empty description element. -->
> +      <!ATTLIST maintainer ignoreauto CDATA "0">
>
>   <!-- A long description of the package in freetext-->
>   <!ELEMENT longdescription (#PCDATA|pkg)* >
>
> --
> Robin Hugh Johnson
> Gentoo Linux Developer & Infra Guy
> E-Mail     : robbat2@gentoo.org
> GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85
>

Re: [gentoo-dev] [v4] Planning for automatic assignment computation of bugs

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 3186 bytes --]

On Sun, Oct 19, 2008 at 03:49:35PM +0200, Ulrich Mueller wrote:
> >>>>> On Sat, 18 Oct 2008, Robin H Johnson wrote:
> 
> > 3. If you want the default assignment to go to a maintainer, and NOT
> >    the herd, move the <herd> element further down in the metadata.xml!
> 
> I disagree about this point. IMHO the procedure described in
> <http://www.gentoo.org/proj/en/qa/bug-wranglers/> makes more sense:
> 
> # When the file [i.e., metadata.xml] lists multiple entries, then you
> # assign the bug to the first maintainer, and CC the other
> # maintainer(s) and herd(s).
See now, here I disagree. If you review the v1 proposal, there was a LOT
of resistance to your assignment procedure there, primarily from teams
where this produced a lot of spam.

The package belongs to the team if a herd element exists, and maintainer is the
person who usually fixes it the most, and is the best person to ask for
detailed package questions.

Some devs have gotten so annoyed about the duplicate spam in the past, that
they've taken the herd out of the metadata.xml, replacing it with no-herd.

It's also a LOT easier to search for bugs assigned to the team than having to
search for bugs assigned to each of the maintainer with the team in the CC,
because the team may be in the CC for other reasons, producing lots of noise.

> > 2. This email is treated as an implicit maintainer element after this
> >    point. "<maintainer><email>${HERD_EMAIL}</email></maintainer>"
> Explicit maintainer elements should have precedence over implicit ones.
How much precedence? This also causes problems when you have multiple atoms on
the summary line.

Here's a contrived example:
Summary: "x11-base/xorg-server-1.5.2 fails to compile when net-misc/openssh-5.1_p1 is installed - errant headers"
(I added both donnie and vapier just for the example).

x11-base/xorg-server:
<herd>x11</herd>
<maintainer><email>dberkholz@gentoo.org</email></maintainer>

net-misc/openssh:
<herd>base-system</herd>
<maintainer>
  <email>robbat2@gentoo.org</email>
  <description>LPK issues. Only assign if it's a direct LPK issue, I'm on base-system for everything else.</description>
</maintainer>
<maintainer><email>vapier@gentoo.org</email></maintainer>

It SHOULD be assigned to x11, and CC to base-system, unless the description
also mentions it being specific to LPK (OpenSSH key storage in LDAP), in which
case I should be CC or assigned to as well.

Possible orders, with the atoms being processed in order:
1. x11, (dberkholz, base-system, vapier)
2. dberkholz, (x11, base-system, vapier)
3. x11, (dberkholz, vapier, base-system)
4. dberkholz, (x11, vapier, base-system)

I push for #1 as the most correct. If multiple assignees were possible, I'd
even say this order was better:
(x11, base-system), (dberkholz, vapier)

The GNOME guys have lots of similar cases to the openssh metadata, where one
team member is the actual maintainer listed, but the herd is present as well,
and they want 

-- 
Robin Hugh Johnson
Gentoo Linux Developer & Infra Guy
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 329 bytes --]

Re: [gentoo-dev] [v4] Planning for automatic assignment computation of bugs

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 1448 bytes --]

On Sun, Oct 19, 2008 at 12:32:44PM -0700, Alec Warner wrote:
> What if my herd email address is different from my bugzie address?
> Can I have both in herds.xml?  What if my herd address *isn't* a
> bugzie account, will the world end?
I don't know any herd where the herd email is not the same as the bugzie
email. Why would this case arise anyway? The mail aliases reside on dev,
and the duplication doesn't make sense.

> How can we automatically detect when developers make mistakes in
> editing herds.xml?
There is a validation CGI in Bugzie, I created it for when somebody (I
forgot who) was checking all of the metadata and herd emails previously,
which we should probably automated.

> > 1. For handling <herd>no-herd</herd>, we should add an entry into herds.xml to
> >   catch it (maintainer-needed <at> g.o). Every herd listed in an ebuild MUST be in
> >   herds.xml.
> You and I both know this is not going to be true.  Complicated
> solution; make Repoman do it.  Certainly it is the 'correct' thing to
> do; however I don't expect it to get implemented or deployed quickly.
> Hacky solution: run script on osprey that tries to validate tree
> metadata against herds.xml and annoy herds who forgot to add
> themselves.
Yes, automation is useful here.

-- 
Robin Hugh Johnson
Gentoo Linux Developer & Infra Guy
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 329 bytes --]

Re: [gentoo-dev] [v4] Planning for automatic assignment computation of bugs

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 1515 bytes --]

On Sun, Oct 19, 2008 at 12:43:05PM -0700, Robin H. Johnson wrote:
> On Sun, Oct 19, 2008 at 03:49:35PM +0200, Ulrich Mueller wrote:
> > >>>>> On Sat, 18 Oct 2008, Robin H Johnson wrote:
> > 
> > > 3. If you want the default assignment to go to a maintainer, and NOT
> > >    the herd, move the <herd> element further down in the metadata.xml!
> > 
> > I disagree about this point. IMHO the procedure described in
> > <http://www.gentoo.org/proj/en/qa/bug-wranglers/> makes more sense:
> > 
> > # When the file [i.e., metadata.xml] lists multiple entries, then you
> > # assign the bug to the first maintainer, and CC the other
> > # maintainer(s) and herd(s).
> See now, here I disagree. If you review the v1 proposal, there was a LOT
> of resistance to your assignment procedure there, primarily from teams
> where this produced a lot of spam.
As an addenda, from v1, different teams and developers DO want different
behaviors:
1. Assign to herd, CC all others (eg: GNOME, base-system)
2. Assign to first maintainer, CC herd and others (eg: net-mail)

That was deliberately why I had logic about using the order in the
metadata.xml file, with the addition that later duplicate entries of an
email would override the first one.

Your generic rule of (assign to first non-herd maintainer, CC rest)
doesn't fit all of the cases.

-- 
Robin Hugh Johnson
Gentoo Linux Developer & Infra Guy
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 329 bytes --]

Re: [gentoo-dev] [v4] Planning for automatic assignment computation of bugs

[Donnie Berkholz]

[-- Attachment #1: Type: text/plain, Size: 728 bytes --]

On 23:01 Sat 18 Oct     , Robin H. Johnson wrote:
> 5. Javascript then appends the server results into the "Additional
>    Comments" box: a suggested assignee and suggested CC values, with logic
>    as to why.
> 6. The wrangler can copy and paste the data into the fields, editing
>    further as desired.
> 
> (Putting the data into the comments allows the assignment tool to
> explain WHY it decided certain actions).

I think this is only useful as debugging information when you think it's 
doing something wrong, and default behavior should be the one requiring 
minimal effort on the wrangler's part.

-- 
Thanks,
Donnie

Donnie Berkholz
Developer, Gentoo Linux
Blog: http://dberkholz.wordpress.com

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: [gentoo-dev] [v4] Planning for automatic assignment computation of bugs

[Thilo Bangert]

[-- Attachment #1: Type: text/plain, Size: 1712 bytes --]

"Robin H. Johnson" <robbat2@gentoo.org> said:
> On Sun, Oct 19, 2008 at 12:32:44PM -0700, Alec Warner wrote:
> > What if my herd email address is different from my bugzie address?
> > Can I have both in herds.xml?  What if my herd address *isn't* a
> > bugzie account, will the world end?
>
> I don't know any herd where the herd email is not the same as the
> bugzie email. Why would this case arise anyway? The mail aliases reside
> on dev, and the duplication doesn't make sense.

the gentopia guys seem to have this annoying scheme: herd email is 
utopia@gentoo.org while bugzi account is gentopia@gentoo.org

>
> > How can we automatically detect when developers make mistakes in
> > editing herds.xml?
>
> There is a validation CGI in Bugzie, I created it for when somebody (I
> forgot who) was checking all of the metadata and herd emails
> previously, which we should probably automated.

for me. i have started using it today.. thanks!

>
> > > 1. For handling <herd>no-herd</herd>, we should add an entry into
> > > herds.xml to catch it (maintainer-needed <at> g.o). Every herd
> > > listed in an ebuild MUST be in herds.xml.
> >
> > You and I both know this is not going to be true.  Complicated
> > solution; make Repoman do it.  Certainly it is the 'correct' thing to
> > do; however I don't expect it to get implemented or deployed quickly.
> > Hacky solution: run script on osprey that tries to validate tree
> > metadata against herds.xml and annoy herds who forgot to add
> > themselves.
>
> Yes, automation is useful here.

very few packages do not have a herd; no ebuilds have a wrong herd listed, 
but there are tons of other errors in our metadata...

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: [gentoo-dev] [v4] Planning for automatic assignment computation of bugs

[Robert Buchholz]

[-- Attachment #1: Type: text/plain, Size: 1114 bytes --]

On Sunday 19 October 2008, Robin H. Johnson wrote:
> Now into it's Nth great year, I bring you the fourth edition of the
> automatic assignment proposal. </truman-show>
>
> Previously:
> http://thread.gmane.org/gmane.linux.gentoo.devel/48485 [v1]
> http://thread.gmane.org/gmane.linux.gentoo.devel/49601 [v2]
> http://thread.gmane.org/gmane.linux.gentoo.devel/57159 [v3]
>
> Bleeding edge prototype:
> http://dev.gentoo.org/~rbu/assign.py
> MANY thanks to rbu.
>
> If there are no further changes or objections at this time, it would
> be nice to implement this by the end of November.

I updated the assign.py with suggestions from this thread, in 
particular:

* maintainer/herd elements in categories' metadata.xml are respected
* Anything that looks like a CAT/PN in the string is caught if it is
  either a valid CAT/PN or valid CAT
* Order in metadata.xml is what dictates assignee, i.e.: The first herd
  or maintainer listed in the metadata.xml of the first CAT or CAT/PN
  is who is assignee, all other follow as CC.

What are the next steps for this to go forward?


Robert

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 835 bytes --]

Re: [gentoo-dev] [v4] Planning for automatic assignment computation of bugs

[Mike Auty]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Robert Buchholz wrote:
> * Order in metadata.xml is what dictates assignee, i.e.: The first herd
>   or maintainer listed in the metadata.xml of the first CAT or CAT/PN
>   is who is assignee, all other follow as CC.

Great work, just wanted to double check the ordering though?

According to [1], "When the file lists multiple entries, then you assign
the bug to the first maintainer, and CC the other maintainer(s) and
herd(s)."  So it looks as though the file should go through the
maintainers first and herds second?  Should be pretty easy to fix...

Keep up the good work!  5:)

Mike  5:)

[1] http://www.gentoo.org/proj/en/qa/bug-wranglers/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (GNU/Linux)

iEYEARECAAYFAklg650ACgkQu7rWomwgFXpokwCgmocuZFPeNu6x9qXOml+DtI/a
3i0An0tGDd1va7VthoWrWyTyald9erxX
=ksov
-----END PGP SIGNATURE-----

Re: [gentoo-dev] [v4] Planning for automatic assignment computation of bugs

[Jeroen Roovers]

On Sun, 04 Jan 2009 17:02:21 +0000
Mike Auty <ikelos@gentoo.org> wrote:

> According to [1], "When the file lists multiple entries, then you
> assign the bug to the first maintainer, and CC the other
> maintainer(s) and herd(s)."  So it looks as though the file should go
> through the maintainers first and herds second?  Should be pretty
> easy to fix...

I spotted that too but didn't remember putting it in black and white. :)

The order ("first maintainer as assignee" or "first maintainer/herd as
assignee") is open to discussion and I think this is the proper forum to
have that discussion.


Kind regards,
     jer

Re: [gentoo-dev] [v4] Planning for automatic assignment computation of bugs

[Mike Auty]

[-- Attachment #1: Type: text/plain, Size: 1188 bytes --]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jeroen Roovers wrote:
> I spotted that too but didn't remember putting it in black and white. :)
> 
> The order ("first maintainer as assignee" or "first maintainer/herd as
> assignee") is open to discussion and I think this is the proper forum to
> have that discussion.

It seems sensible to me.  I would've thought that being more specific
would surely be better?  Splitting them up means those who are mostly in
charge of a package see it easily, and it's also then easier to spot
packages that only have a herd, rather than them getting lost in all the
packages that do have individual maintainers...

I've attached a quick patch that should fix up assign.py to add all the
herds on the end.  Since the order of the second item onwards doesn't
matter, all herds are added at the end.  If we do need an ordering (like
maintainer1, herd1, maintainer2, maintainer3, herd2) then it'll need a
more complex patch...

Hope this helps,

Mike  5:)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (GNU/Linux)

iEYEARECAAYFAklg7p8ACgkQu7rWomwgFXoDwACcCDBD5Dj/F//7Bxq+zIB1/GPZ
UHQAnA82J6UxuHva7uEXmEL9wuNDMkIk
=SRmT
-----END PGP SIGNATURE-----

[-- Attachment #2: assign-patch.py --]
[-- Type: text/plain, Size: 1030 bytes --]

diff --git a/assign.py b/assign.py
index 82d894b..c7c2c59 100644
--- a/assign.py
+++ b/assign.py
@@ -54,6 +54,7 @@ def get_pkg_cat(string):
 def get_maintainer_for(directory):
     """ returns a priority-sorted list of maintainers for a given CAT or CAT/PN """
     cc = []
+    hcc = []
     try:
         if not heXML:
             globals()['heXML'] = et.parse(HERDS)
@@ -65,7 +66,7 @@ def get_maintainer_for(directory):
                     if thisherd.findtext("name") == elem.text:
                         herdmail = thisherd.findtext("email")
                         if herdmail:
-                            cc.append(herdmail)
+                            hcc.append(herdmail)
             elif elem.tag == "maintainer":
                 email = elem.findtext("email")
                 if not email:
@@ -75,6 +76,7 @@ def get_maintainer_for(directory):
                         cc.remove(email)
                 else:
                     cc.append(email)
+        cc.extend(hcc)
 
     except Exception:
         pass

[-- Attachment #3: assign-patch.py.sig --]
[-- Type: application/octet-stream, Size: 72 bytes --]

Re: [gentoo-dev] [v4] Planning for automatic assignment computation of bugs

[Robert Buchholz]

[-- Attachment #1: Type: text/plain, Size: 2165 bytes --]

On Sunday 04 January 2009, Mike Auty wrote:
> Jeroen Roovers wrote:
> > The order ("first maintainer as assignee" or "first maintainer/herd
> > as assignee") is open to discussion and I think this is the proper
> > forum to have that discussion.
>
> It seems sensible to me.  I would've thought that being more specific
> would surely be better?  Splitting them up means those who are mostly
> in charge of a package see it easily, and it's also then easier to
> spot packages that only have a herd, rather than them getting lost in
> all the packages that do have individual maintainers...
>
> I've attached a quick patch that should fix up assign.py to add all
> the herds on the end.  Since the order of the second item onwards
> doesn't matter, all herds are added at the end.  If we do need an
> ordering (like maintainer1, herd1, maintainer2, maintainer3, herd2)
> then it'll need a more complex patch...

I actually implemented it this way before (only that I had all herds 
with higher priority than all maintainers, which is the reverse of your 
patch). The outcome of the previous discussion for me was that there is 
no real consent on who should be the assignee. Robin put it this way:

On Sunday 19 October 2008, Robin H. Johnson wrote:
> As an addenda, from v1, different teams and developers DO want
> different behaviors:
> 1. Assign to herd, CC all others (eg: GNOME, base-system)
> 2. Assign to first maintainer, CC herd and others (eg: net-mail)
>
> That was deliberately why I had logic about using the order in the
> metadata.xml file, with the addition that later duplicate entries of
> an email would override the first one.
>
> Your generic rule of (assign to first non-herd maintainer, CC rest)
> doesn't fit all of the cases.

Accepting the fact that different teams have different preferences, we 
need to find a solution for them to set theirs individually. This could 
either be the order of elements in metadata.xml (and would set the 
preference on a per-package basis) or some attribute in herds.xml 
(which would be a global setting per herd, and we'd need to find a 
default).


Robert

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 835 bytes --]

Re: [gentoo-dev] [v4] Planning for automatic assignment computation of bugs

[Mike Auty]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Robert Buchholz wrote:
> Accepting the fact that different teams have different preferences, we 
> need to find a solution for them to set theirs individually. This could 
> either be the order of elements in metadata.xml (and would set the 
> preference on a per-package basis) or some attribute in herds.xml 
> (which would be a global setting per herd, and we'd need to find a 
> default).

Ok, sounds like we've got some options:

a) herds.xml per-herd priority flag (herd gets assigned)
b) metadata.xml priority element (can be opt-in or opt-out)
c) order of elements in metadata.xml

I'm personally not keen on the order of elements, since adding meaning
to the order might mean a fair number of misassignments until people fix
the metadata.xml files.

The herds.xml element isn't very specific, but if the herd-first rules
apply to the whole herd, then it's probably the least-impact solution.

Finally, if we think we'll ever need something more specific than
herds.xml, we could add an extra element.  <priority type="herd"> or
<priority type="maintainer"> could be added to the minority case (I'm
not sure which has fewer ebuilds, but if there's hard and fast rules
this should be relatively automatable).

More involved solutions could include wrapping the appropriate element
in <assignee></assignee> (what happens if there's no assignee tag), or
adding an assignee attribute to one of the herd/maintainer tags (how can
we ensure there's never two assignees).

I'm up for whatever, with a slight preference toward not relying on
ordering...

Mike  5:)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (GNU/Linux)

iEYEARECAAYFAklg/AEACgkQu7rWomwgFXolAACgoujUIQs0AYRHK+JRoOsMiO41
HMkAoIHx5re/FOiD3GQNCR7fJ7xC3ebM
=Sc4j
-----END PGP SIGNATURE-----

Re: [gentoo-dev] [v4] Planning for automatic assignment computation of bugs

[Peter Volkov]

В Вск, 04/01/2009 в 18:57 +0100, Robert Buchholz пишет:
> On Sunday 04 January 2009, Mike Auty wrote:
> > Jeroen Roovers wrote:
> > > The order ("first maintainer as assignee" or "first maintainer/herd
> > > as assignee") is open to discussion and I think this is the proper
> > > forum to have that discussion.
>
> I actually implemented it this way before (only that I had all herds 
> with higher priority than all maintainers, which is the reverse of your 
> patch).
>
> Accepting the fact that different teams have different preferences, we 
> need to find a solution for them to set theirs individually. This could 
> either be the order of elements in metadata.xml (and would set the 
> preference on a per-package basis) or some attribute in herds.xml 
> (which would be a global setting per herd, and we'd need to find a 
> default).

It looks like we really need some per-team configuration for default
assignment. Probably it's good idea to add 'weight' (or 'nice')
attribute for <herd> and <maintainer> elements both in herds.xml and
metadata.xml. Bug assignment field will be selected from the elements
with minimal weight (least nice ;)). IMO best is to assign on first
(any) maintainer in this list and on first (any) herd if there is no
maintainer elements there. If weight is defined in multiple places, per
category weight overrides weight from herd.xml and weight in
metadata.xml overrides everything. This allows easy way to define any
policy team wants but still allow maintainer to override team
preference. What do you think?

-- 
Peter.

Re: [gentoo-dev] [v4] Planning for automatic assignment computation of bugs

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 2602 bytes --]

On Sun, Jan 04, 2009 at 06:12:17PM +0000, Mike Auty wrote:
> a) herds.xml per-herd priority flag (herd gets assigned)
> b) metadata.xml priority element (can be opt-in or opt-out)
> c) order of elements in metadata.xml
> 
> I'm personally not keen on the order of elements, since adding meaning
> to the order might mean a fair number of misassignments until people fix
> the metadata.xml files.
How many metadata files have the ordering wrong to start with?
Of the packages I maintain, just looking at a handful, very few have it
bad enough that I'd bother complaining rather than just changing them.

> The herds.xml element isn't very specific, but if the herd-first rules
> apply to the whole herd, then it's probably the least-impact solution.
> 
> Finally, if we think we'll ever need something more specific than
> herds.xml, we could add an extra element.  <priority type="herd"> or
> <priority type="maintainer"> could be added to the minority case (I'm
> not sure which has fewer ebuilds, but if there's hard and fast rules
> this should be relatively automatable).
Neither set of rules is ideal. Ordering makes a lot of sense when you
just read it. Consider metadata with multiple maintainers and multiple
herds. Either you have to start assigning explicitly (requires editing
metadata.xml), or you need to fall back to ordering. If you're going to
do ordering further down, why not do it from the start and be done with
it.

For anybody that wants to complain that XML is unordered - it isn't,
consider an HTML document that is also well-formed XML and validates
against a DTD. You wouldn't want your paragraphs changing order on you.

Count of total <herd>+<mainteiner> elements and how many metadata.xml files
have the count:
1  7842
2  4958
3   290
4    35

By number of herds:
0    26
1 12720
2   359
3    19
4     1

By number of maintainers:
0  8135
1  4730
2   241
3    19

If we assume that every metadata.xml with 2 or more items is wrong, thats at
most 40% of the tree.  I say go with ordering. I think it will affect less than
10% of packages in the end, and for large swaths it won't matter (dev-perl and
dev-$LANG in general, which account for some 20% of the tree).

Also, maintainers that don't want dupe assignments (normally because they in
the herd) are going to be editing anyway, and I think that will cover a lot of
the required edit cases as well.

-- 
Robin Hugh Johnson
Gentoo Linux Developer & Infra Guy
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 329 bytes --]

Re: [gentoo-dev] [v4] Planning for automatic assignment computation of bugs

[Mike Auty]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Robin H. Johnson wrote:
> Neither set of rules is ideal. Ordering makes a lot of sense when you
> just read it. Consider metadata with multiple maintainers and multiple
> herds. Either you have to start assigning explicitly (requires editing
> metadata.xml), or you need to fall back to ordering. If you're going to
> do ordering further down, why not do it from the start and be done with
> it.

Ok, I'm convinced.  5:)  I tend not to prefer having "hidden" meanings,
but as you point out XML is ordered, and you definitely made the case
that it won't have a large impact.  Fine by me...  5:)

Mike  5:)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (GNU/Linux)

iEYEARECAAYFAkljaG4ACgkQu7rWomwgFXq53gCeMRe+n+S6N2za00zuHjrge/gw
nUIAniyeVK/RTlVUaR2Q3Eqw+5cCMIoU
=01UN
-----END PGP SIGNATURE-----

Re: [gentoo-dev] [v4] Planning for automatic assignment computation of bugs

[Tiziano Müller]

[-- Attachment #1: Type: text/plain, Size: 1549 bytes --]

Am Sonntag, den 04.01.2009, 18:06 +0100 schrieb Jeroen Roovers:
> On Sun, 04 Jan 2009 17:02:21 +0000
> Mike Auty <ikelos@gentoo.org> wrote:
> 
> > According to [1], "When the file lists multiple entries, then you
> > assign the bug to the first maintainer, and CC the other
> > maintainer(s) and herd(s)."  So it looks as though the file should go
> > through the maintainers first and herds second?  Should be pretty
> > easy to fix...
> 
> I spotted that too but didn't remember putting it in black and white. :)
> 
> The order ("first maintainer as assignee" or "first maintainer/herd as
> assignee") is open to discussion and I think this is the proper forum to
> have that discussion.

I'd say that the correct way to fix this is to fix the metadata schema
to be able to write something like this:

<maintainer>
  <team>foo</team>
</maintainer>
<maintainer>
  <dev>adev</dev>
</maintainer>
<maintainer>
  <team>bar</team>
</maintainer>

or

<maintainer type="team">foo</maintainer>
<maintainer type="dev">adev</maintainer>
<maintainer type="team">bar</maintainer>

Because having to write this:

<herd>foo</herd>
<maintainer><email>adev@gentoo.org</email></maintainer>
<herd>bar</herd>

is just nonsense.

Cheers,
Tiziano

-- 
-------------------------------------------------------
Tiziano Müller
Gentoo Linux Developer
Areas of responsibility:
  Samba, PostgreSQL, CPP, Python, sysadmin
E-Mail     : dev-zero@gentoo.org
GnuPG FP   : F327 283A E769 2E36 18D5  4DE2 1B05 6A63 AE9C 1E30

[-- Attachment #2: Dies ist ein digital signierter Nachrichtenteil --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

[gentoo-dev] Re: [v4] Planning for automatic assignment computation of bugs

[Steve Long]

Peter Volkov wrote:

> ? ???, 04/01/2009 ? 18:57 +0100, Robert Buchholz ?????:
>> Accepting the fact that different teams have different preferences, we
>> need to find a solution for them to set theirs individually. This could
>> either be the order of elements in metadata.xml (and would set the
>> preference on a per-package basis) or some attribute in herds.xml
>> (which would be a global setting per herd, and we'd need to find a
>> default).
> 
> It looks like we really need some per-team configuration for default
> assignment.
I agree, given that it's not going to affect running systems (I hope); in
the longer term, it would be nice to be able to configure by pkg, cat or
herd.

> Probably it's good idea to add 'weight' (or 'nice') 
> attribute for <herd> and <maintainer> elements both in herds.xml and
> metadata.xml. Bug assignment field will be selected from the elements
> with minimal weight (least nice ;)).
Shouldn't the 'nicest' entity take it? ;)

A simple assignToHerd="yes|no|<unset>" (or 0|1) might be easier to deal with
(otherwise you're going to have a maintenance headache with the variant
levels?) and would deal with all the use-cases afaict; a team does [eg
kde/gnome] or does not want bugs, unless the category/CP/CPV merits a
change in that policy. Obviously if none set, use the maintainer list as-is
without filtering.

Sure, it can be done by patching the tree over time, but it seems crude and
a further maintenance + bug-wrangling burden for no benefit, when the
coders are on-hand and engaged to tweak the new impl.

OFC, a rush to completion is understandable, given how long this has been in
the planning; I'd argue that's a reason to go the final ten metres.