[gentoo-dev] January 2014 QA Policy Updates

[gentoo-dev] January 2014 QA Policy Updates

[Chris Reffett]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello all,
The new QA team has completed its first meeting, and so I would like
to announce policy changes agreed upon during the meeting which are
relevant to the developer community. In the future, when a meeting
results in changed or amended policy, we will notify the community via
- -dev and -dev-announce, so there will probably be a summary email like
this coming out about once a month. Changes to policy from this meeting:

- -USE-controlled optional RDEPENDs policy clarified to say that those
dependencies are not allowed, but QA will grant exceptions for certain
circumstances (such as a package not working unless one of a set of
optional deps is installed)

- -The QA team policymaking workflow will look like the following:
* User/dev/team member brings us an issue
* Team investigates
* Team discusses at the next meeting
** If the person is violating policy, we inform them of that and
request that they stop violating policy
** If the existing policy is unclear, we update it
** If there is no existing policy, make one
If we think a developer's actions are causing problems, we may ask
them to stop/undo pending discussion by the QA team at the next meeting.

- -Rules for the QA team editing peoples' packages:
*For trivial fixes, such as repoman errors, we fix the issue and send
the developer a friendly reminder
*For large but non-critical fixes, we open a bug, wait 2 weeks, and if
it is not fixed within that time frame we make the change.
*For critical fixes, such as a problem that breaks the tree or a
package, we fix the issue and send the developer a notice about our change

- -The QA team will communicate changes to policy via emails to
gentoo-dev and gentoo-dev-announce and by updating the QA policy page
on the Gentoo Wiki.

For anyone interested, the summary of the meeting can be found at
https://wiki.gentoo.org/wiki/Project:Quality_Assurance/Meeting_Summaries
and the current set of QA policies can be found at
https://wiki.gentoo.org/wiki/Project:Quality_Assurance/Policies. If
you have any questions or concerns about these policies, feel free to
discuss them with us in #gentoo-qa or by emailing qa@gentoo.org.

Chris Reffett
Gentoo QA Lead
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iKYEARECAGYFAlLp51VfFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl
bnBncC5maWZ0aGhvcnNlbWFuLm5ldEM2NzU5RjUyMDczREJDQkVDQTBDRkE1NERC
Nzk1QThBNDI2MTgzNTQACgkQ23laikJhg1R6zwCfXY0q7Ig3d40Xq2hScLcT4Hm6
zE8AoJfIWsuV9yAKdsuxwB6JSDr8KbZY
=sheY
-----END PGP SIGNATURE-----

Re: [gentoo-dev] January 2014 QA Policy Updates

[Chris Reffett]

[-- Attachment #1: Type: text/plain, Size: 3267 bytes --]

On 01/30/2014 03:10 PM, William Hubbs wrote:
> On Thu, Jan 30, 2014 at 12:47:01AM -0500, Chris Reffett wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Hello all,
>> The new QA team has completed its first meeting, and so I would like
>> to announce policy changes agreed upon during the meeting which are
>> relevant to the developer community. In the future, when a meeting
>> results in changed or amended policy, we will notify the community via
>> - -dev and -dev-announce, so there will probably be a summary email like
>> this coming out about once a month. Changes to policy from this meeting:
>>
>> - -USE-controlled optional RDEPENDs policy clarified to say that those
>> dependencies are not allowed, but QA will grant exceptions for certain
>> circumstances (such as a package not working unless one of a set of
>> optional deps is installed)
>>
>> - -The QA team policymaking workflow will look like the following:
>> * User/dev/team member brings us an issue
>> * Team investigates
>> * Team discusses at the next meeting
>> ** If the person is violating policy, we inform them of that and
>> request that they stop violating policy
>> ** If the existing policy is unclear, we update it
>> ** If there is no existing policy, make one
>> If we think a developer's actions are causing problems, we may ask
>> them to stop/undo pending discussion by the QA team at the next meeting.
>>
>> - -Rules for the QA team editing peoples' packages:
>> *For trivial fixes, such as repoman errors, we fix the issue and send
>> the developer a friendly reminder
>> *For large but non-critical fixes, we open a bug, wait 2 weeks, and if
>> it is not fixed within that time frame we make the change.
>> *For critical fixes, such as a problem that breaks the tree or a
>> package, we fix the issue and send the developer a notice about our change
>>
>> - -The QA team will communicate changes to policy via emails to
>> gentoo-dev and gentoo-dev-announce and by updating the QA policy page
>> on the Gentoo Wiki.
>>
>> For anyone interested, the summary of the meeting can be found at
>> https://wiki.gentoo.org/wiki/Project:Quality_Assurance/Meeting_Summaries
>> and the current set of QA policies can be found at
>> https://wiki.gentoo.org/wiki/Project:Quality_Assurance/Policies. If
>> you have any questions or concerns about these policies, feel free to
>> discuss them with us in #gentoo-qa or by emailing qa@gentoo.org.
> 
> I have one.
> 
> The way I read that email, we are saying that our only policies are on
> the wiki.
> 
> Is that right, or is the DevManual stil relevant?
> 
> If it is, I would suggest that on the wiki we make it clear that our
> technical policies are all in the devmanual. Along that line, I would
> suggest moving the stabilization policy to the devmanual. I'll look for
> a place for a patch.
> 
> William
> 

That's my mistake, the devmanual is still relevant. My idea is to use
the wiki page for smaller or more specific items which probably don't go
in the devmanual (for example, policy which is about specific packages
or USE flags, or policies which just affect the QA team). Stabilization
should go to the devmanual, then.

Chris Reffett


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 393 bytes --]

Re: [gentoo-dev] January 2014 QA Policy Updates

[Peter Stuge]

Chris Reffett wrote:
> - -The QA team policymaking workflow will look like the following:
..
> If we think a developer's actions are causing problems, we may ask
> them to stop/undo pending discussion by the QA team at the next meeting.

plus

> - -Rules for the QA team editing peoples' packages:
..
> *For trivial fixes, such as repoman errors, we fix the issue and send
> the developer a friendly reminder
> *For large but non-critical fixes, we open a bug, wait 2 weeks, and if
> it is not fixed within that time frame we make the change.

sounds to me like QA is giving itself carte blanche to make any "fix"
they want as per "we think a developer's actions are causing problems"
hmm?


//Peter

Re: [gentoo-dev] January 2014 QA Policy Updates

[Alec Warner]

[-- Attachment #1: Type: text/plain, Size: 2101 bytes --]

On Thu, Jan 30, 2014 at 9:48 PM, Peter Stuge <peter@stuge.se> wrote:

> Chris Reffett wrote:
> > - -The QA team policymaking workflow will look like the following:
> ..
> > If we think a developer's actions are causing problems, we may ask
> > them to stop/undo pending discussion by the QA team at the next meeting.
>
> plus
>
> > - -Rules for the QA team editing peoples' packages:
> ..
> > *For trivial fixes, such as repoman errors, we fix the issue and send
> > the developer a friendly reminder
> > *For large but non-critical fixes, we open a bug, wait 2 weeks, and if
> > it is not fixed within that time frame we make the change.
>
> sounds to me like QA is giving itself carte blanche to make any "fix"
> they want as per "we think a developer's actions are causing problems"
> hmm?
>

To be fair, I had a long discussion with this regarding when QA has the
authority to temporarily ban a developer. This discussion came about
because on occasion, a developer and the QA team will have a disagreement
regarding a package, and QA will attempt to assert their authority. My
interpretation is that the authority of the QA team nominally comes from
the idea that we want a certain level of quality in the tree, and that we
have policies in place to facilitate that quality level.

So in a case where a developer is clearly violating an existing documented
policy, then QA does have the authority to modify their package after a
short discussion about it. In the case where policy is missing, QA does not
have a clear case of authority there. It becomes a more murky area. I've
tried to very much encourage QA to both publish the policies they want to
enforce, and automate enforcement with better tooling (repoman or
otherwise). Being transparent and consistent in enforcement of policy goes
a long way for getting developers on your side.

So in short, while one could read that passage as you did, I don't think
that is their intention. When developers 'cause problems' it should be
obvious to everyone how and why that is, and not simply obvious to
say...the QA lead.

-A


>
>
> //Peter
>
>

[-- Attachment #2: Type: text/html, Size: 2903 bytes --]

Re: [gentoo-dev] January 2014 QA Policy Updates

[Rich Freeman]

On Fri, Jan 31, 2014 at 1:14 AM, Alec Warner <antarus@gentoo.org> wrote:
>> sounds to me like QA is giving itself carte blanche to make any "fix"
>> they want as per "we think a developer's actions are causing problems"
>> hmm?
>
> So in short, while one could read that passage as you did, I don't think
> that is their intention.

Probably also worth noting that QA isn't giving itself any authority -
GLEP 48 gave them very broad authority some time ago.  QA has been
operating in the manner described for a fairly long time as well.  It
just seems like they're trying to be more transparent about it, which
I applaud.

GLEP 48 was recently amended to require QA to have its nominated lead
confirmed by the council largely because it is a position that wields
a great deal of authority.

I was really happy to see a public notice of meeting and a published
summary.  While any kind of policy-making will always involve some
level of controversy, I think that this goes a long way towards
assuring the community that they have a voice and that if policies
don't turn out well they can be adjusted.  Maybe the only thing that
changed is perceptions, but those are important.

Rich

Re: [gentoo-dev] January 2014 QA Policy Updates

[Peter Stuge]

Alec Warner wrote:
> > hmm?
> 
> To be fair, I had a long discussion with this regarding when QA has the
> authority to temporarily ban a developer.

Cool.


> In the case where policy is missing, QA does not have a clear case
> of authority there. It becomes a more murky area. I've tried to
> very much encourage QA to both publish the policies they want to
> enforce, and automate enforcement with better tooling (repoman or
> otherwise). Being transparent and consistent in enforcement of
> policy goes a long way for getting developers on your side.

Absolutely.


> So in short, while one could read that passage as you did, I don't
> think that is their intention.

To be clear, I don't think so either.


Rich Freeman wrote:
> I was really happy to see a public notice of meeting and a published
> summary.

Yes, me too!


I still think it seems like QA could essentially introduce arbitrary
new policies and 2 weeks later be expected to effect them.

Fine when everyone agrees. Not so much at other times. The
responsibility is with QA to build support among the developers, and
I agree that the transparency goes a long way!


//Peter

Re: [gentoo-dev] January 2014 QA Policy Updates

[Chris Reffett]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 01/31/2014 09:07 AM, Peter Stuge wrote:
> Alec Warner wrote:
>>> hmm?
>> 
>> To be fair, I had a long discussion with this regarding when QA
>> has the authority to temporarily ban a developer.
> 
> Cool.
> 
> 
>> In the case where policy is missing, QA does not have a clear
>> case of authority there. It becomes a more murky area. I've tried
>> to very much encourage QA to both publish the policies they want
>> to enforce, and automate enforcement with better tooling (repoman
>> or otherwise). Being transparent and consistent in enforcement
>> of policy goes a long way for getting developers on your side.
> 
> Absolutely.
> 
> 
>> So in short, while one could read that passage as you did, I
>> don't think that is their intention.
> 
> To be clear, I don't think so either.
> 
> 
> Rich Freeman wrote:
>> I was really happy to see a public notice of meeting and a
>> published summary.
> 
> Yes, me too!
> 
> 
> I still think it seems like QA could essentially introduce
> arbitrary new policies and 2 weeks later be expected to effect
> them.
> 
> Fine when everyone agrees. Not so much at other times. The 
> responsibility is with QA to build support among the developers,
> and I agree that the transparency goes a long way!
> 
> 
> //Peter
> 

Regarding the question in your first email: We will not create a
policy then immeiately use the policy as justification to to go edit
packages. The intention of the "we may ask the developer to stop" line
is for cases where we suspect that what the developer is doing is
causing a problem and would like to discuss it further. I feel that
that is well within the bounds of GLEP 48. As for the "when/how we
make and communicate fixes," I think that just about any policy we
make will fall into the middle ground you omitted of "file a bug, wait
2 weeks, fix." So no, we will not be making arbitrary fixes just
because we can.

Regarding your concern about us introducing arbitrary policies: again,
most will fall into the "file a bug" middle ground. We also are
perfectly aware that you can't expect people to change overnight, and
we will not be shouting at devs just because they didn't implement
$(new-policy) right away. We will file bugs asking for changes, and we
will try to offer suggestions or patches wherever possible to make it
easier for maintainers.

Chris Reffett
Gentoo QA Lead
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iKYEARECAGYFAlLrv0hfFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl
bnBncC5maWZ0aGhvcnNlbWFuLm5ldEM2NzU5RjUyMDczREJDQkVDQTBDRkE1NERC
Nzk1QThBNDI2MTgzNTQACgkQ23laikJhg1QZ+wCeJre5n44E9BdZcUBgZdC5DjPe
WR8AoJ1W9QuqVIFXxsVAWBO23yx+etak
=5CIT
-----END PGP SIGNATURE-----