[gentoo-project] Call for agenda items - Council meeting 2016-08-14

[gentoo-project] Call for agenda items - Council meeting 2016-08-14

[Kristian Fiskerstrand]

[-- Attachment #1.1: Type: text/plain, Size: 402 bytes --]

Dear all,

the Gentoo Council will meet again on Sunday, August 14 at 19:00 UTC
in #gentoo-council on FreeNode.

Please reply to this message on the gentoo-project list with any items
the council should put on its agenda to discuss or vote on.

-- 
Kristian Fiskerstrand
OpenPGP certificate reachable at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2016-08-14

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 2454 bytes --]

On Thu, Aug 04, 2016 at 04:15:14PM +0200, Kristian Fiskerstrand wrote:
> Dear all,
> 
> the Gentoo Council will meet again on Sunday, August 14 at 19:00 UTC
> in #gentoo-council on FreeNode.
> 
> Please reply to this message on the gentoo-project list with any items
> the council should put on its agenda to discuss or vote on.

I feel that our stable tree is so far behind on all
architectures that we are doing our stable users a disservice, so I
would like to open up a discussion here, and maybe some policy changes
at the next meeting.

Ultimately, I think we need some form of automated stabilization, e.g.
if a package version sits in ~ for 30 days and there are no blockers at
that point, the new version should go automatically to stable on all
architectures where there is a previous stable version.

I realize that automation is going to take a lot of work, so in the
meantime, I would like to discuss changes to our stabilization policies
that will get new versions of packages to stable faster.

The first issue is maintainers not filing stable requests for new
versions of packages in a timely manor. I'm not sure how to get around
this, but I feel that once a version of a package is stable, we are
doing a disservice to our stable users by not keeping stable as current
as possible.  I am as bad as anyone; it is easy to forget to file
stable requests until someone pings me or files the request
themselves.

I have heard other maintainers say specifically that they do not file
stable requests unless a user asks them to, but Again, I do not feel
comfortable with this arrangement if there is an old version of the
package in stable. Users shouldn't have to ask for newer versions to be
stabilized; this should be driven by the maintainers.

The second issue is slow arch teams. Again, by not moving packages from
~ to stable, we are doing a disservice to our stable users.

I can think of two ways we can improve our situation.

We can allow maintainers to stabilize new versions of certain types of
 packages on all arches where there is a previous version of the package stable
 without filing stable requests. This would take a significant load off
 of the arch teams.

For packages that do not fit the first group, we could require stable
requests, but allow maintainers to stabilize the new versions after a
timeout (I would propose 30 days).

What do folks think?

William


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2016-08-14

[Dirkjan Ochtman]

On Thu, Aug 4, 2016 at 6:24 PM, William Hubbs <williamh@gentoo.org> wrote:
> I can think of two ways we can improve our situation.
>
> We can allow maintainers to stabilize new versions of certain types of
>  packages on all arches where there is a previous version of the package stable
>  without filing stable requests. This would take a significant load off
>  of the arch teams.
>
> For packages that do not fit the first group, we could require stable
> requests, but allow maintainers to stabilize the new versions after a
> timeout (I would propose 30 days).
>
> What do folks think?

I very much agree that there's a problem. Interestingly, for the
stable requests I file, it seems that e.g. alpha does better than
amd64 right now, thanks to having a very dedicated arch team.

It makes a lot of sense to me to allow maintainers to stabilize
packages after 30 days or so at least if there is a previous stable
version.

Cheers,

Dirkjan

Re: [gentoo-project] Call for agenda items - Council meeting 2016-08-14

[Brian Dolbec]

[-- Attachment #1: Type: text/plain, Size: 3071 bytes --]

On Thu, 4 Aug 2016 11:24:43 -0500
William Hubbs <williamh@gentoo.org> wrote:

> On Thu, Aug 04, 2016 at 04:15:14PM +0200, Kristian Fiskerstrand wrote:
> > Dear all,
> > 
> > the Gentoo Council will meet again on Sunday, August 14 at 19:00 UTC
> > in #gentoo-council on FreeNode.
> > 
> > Please reply to this message on the gentoo-project list with any
> > items the council should put on its agenda to discuss or vote on.  
> 
> I feel that our stable tree is so far behind on all
> architectures that we are doing our stable users a disservice, so I
> would like to open up a discussion here, and maybe some policy changes
> at the next meeting.
> 
> Ultimately, I think we need some form of automated stabilization, e.g.
> if a package version sits in ~ for 30 days and there are no blockers
> at that point, the new version should go automatically to stable on
> all architectures where there is a previous stable version.
> 
> I realize that automation is going to take a lot of work, so in the
> meantime, I would like to discuss changes to our stabilization
> policies that will get new versions of packages to stable faster.
> 
> The first issue is maintainers not filing stable requests for new
> versions of packages in a timely manor. I'm not sure how to get around
> this, but I feel that once a version of a package is stable, we are
> doing a disservice to our stable users by not keeping stable as
> current as possible.  I am as bad as anyone; it is easy to forget to
> file stable requests until someone pings me or files the request
> themselves.
> 
> I have heard other maintainers say specifically that they do not file
> stable requests unless a user asks them to, but Again, I do not feel
> comfortable with this arrangement if there is an old version of the
> package in stable. Users shouldn't have to ask for newer versions to
> be stabilized; this should be driven by the maintainers.
> 
> The second issue is slow arch teams. Again, by not moving packages
> from ~ to stable, we are doing a disservice to our stable users.
> 
> I can think of two ways we can improve our situation.
> 
> We can allow maintainers to stabilize new versions of certain types of
>  packages on all arches where there is a previous version of the
> package stable without filing stable requests. This would take a
> significant load off of the arch teams.
> 
> For packages that do not fit the first group, we could require stable
> requests, but allow maintainers to stabilize the new versions after a
> timeout (I would propose 30 days).
> 
> What do folks think?
> 
> William
> 

William, there is a GSOC project underway that is creating an automated
testing system as a helper and auto-stabilization system.  You should
read over the gentoo-soc list and/or talk to the mentors and student
doing the project.

student: Pallav Agarwal <pallavagarwal07@gmail.com>
Mentors: Sébastien Fabbro <bicatali@gentoo.org>
         Nitin Agarwal <nitinagarwal3006@gmail.com>

-- 
Brian Dolbec <dolsen>


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 951 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2016-08-14

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 3689 bytes --]

On Thu, Aug 04, 2016 at 10:09:38AM -0700, Brian Dolbec wrote:
> On Thu, 4 Aug 2016 11:24:43 -0500
> William Hubbs <williamh@gentoo.org> wrote:
> 
> > On Thu, Aug 04, 2016 at 04:15:14PM +0200, Kristian Fiskerstrand wrote:
> > > Dear all,
> > > 
> > > the Gentoo Council will meet again on Sunday, August 14 at 19:00 UTC
> > > in #gentoo-council on FreeNode.
> > > 
> > > Please reply to this message on the gentoo-project list with any
> > > items the council should put on its agenda to discuss or vote on.  
> > 
> > I feel that our stable tree is so far behind on all
> > architectures that we are doing our stable users a disservice, so I
> > would like to open up a discussion here, and maybe some policy changes
> > at the next meeting.
> > 
> > Ultimately, I think we need some form of automated stabilization, e.g.
> > if a package version sits in ~ for 30 days and there are no blockers
> > at that point, the new version should go automatically to stable on
> > all architectures where there is a previous stable version.
> > 
> > I realize that automation is going to take a lot of work, so in the
> > meantime, I would like to discuss changes to our stabilization
> > policies that will get new versions of packages to stable faster.
> > 
> > The first issue is maintainers not filing stable requests for new
> > versions of packages in a timely manor. I'm not sure how to get around
> > this, but I feel that once a version of a package is stable, we are
> > doing a disservice to our stable users by not keeping stable as
> > current as possible.  I am as bad as anyone; it is easy to forget to
> > file stable requests until someone pings me or files the request
> > themselves.
> > 
> > I have heard other maintainers say specifically that they do not file
> > stable requests unless a user asks them to, but Again, I do not feel
> > comfortable with this arrangement if there is an old version of the
> > package in stable. Users shouldn't have to ask for newer versions to
> > be stabilized; this should be driven by the maintainers.
> > 
> > The second issue is slow arch teams. Again, by not moving packages
> > from ~ to stable, we are doing a disservice to our stable users.
> > 
> > I can think of two ways we can improve our situation.
> > 
> > We can allow maintainers to stabilize new versions of certain types of
> >  packages on all arches where there is a previous version of the
> > package stable without filing stable requests. This would take a
> > significant load off of the arch teams.
> > 
> > For packages that do not fit the first group, we could require stable
> > requests, but allow maintainers to stabilize the new versions after a
> > timeout (I would propose 30 days).
> > 
> > What do folks think?
> > 
> > William
> > 
> 
> William, there is a GSOC project underway that is creating an automated
> testing system as a helper and auto-stabilization system.  You should
> read over the gentoo-soc list and/or talk to the mentors and student
> doing the project.
> 
> student: Pallav Agarwal <pallavagarwal07@gmail.com>
> Mentors: Sébastien Fabbro <bicatali@gentoo.org>
>          Nitin Agarwal <nitinagarwal3006@gmail.com>

As I said, automated stabilization is where we should go in the long
term, and I'm glad someone is working on that.

What I'm interested in in this thread mostly though is how we can adjust
our current policies so that more things get stabilized until we get
that automated system working.

In other words, I want us to start moving more things into stable asap
then the automated system can start when it is in place.

Thanks,

William


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2016-08-14

[Andrew Savchenko]

[-- Attachment #1: Type: text/plain, Size: 6743 bytes --]

Hi all,

On Thu, 4 Aug 2016 11:24:43 -0500 William Hubbs wrote:
> On Thu, Aug 04, 2016 at 04:15:14PM +0200, Kristian Fiskerstrand wrote:
> > Dear all,
> > 
> > the Gentoo Council will meet again on Sunday, August 14 at 19:00 UTC
> > in #gentoo-council on FreeNode.
> > 
> > Please reply to this message on the gentoo-project list with any items
> > the council should put on its agenda to discuss or vote on.
> 
> I feel that our stable tree is so far behind on all
> architectures that we are doing our stable users a disservice, so I
> would like to open up a discussion here, and maybe some policy changes
> at the next meeting.

Thank you for caring about this issue. I had similar thoughts
myself, but was too slow on writing e-mail :) IMO stable tree has
three problems: 
1) too old packages
2) too few packages
3) stabilization often takes too long, such stable packages are
broken or buggy, while their unstable versions are fixed and work
fine. (It is not possible to fix all bugs without version or
revision bump, thus stabilization is needed to fix many bugs.)

This results in fact that many users and some devs (including
myself) do not use stable at all.

> Ultimately, I think we need some form of automated stabilization, e.g.
> if a package version sits in ~ for 30 days and there are no blockers at
> that point, the new version should go automatically to stable on all
> architectures where there is a previous stable version.

Automation should be possible only after rigorous testing. If this
will be implemented, than this may be done (as Brian pointed out in
another mail in this thread, there is an ongoing effort on this
matter as GSoC project, which is great). But I'd like to emphasize
that automated stabilization without:
a) build testing;
b) run-time testing;
c) fixing all serious open bugs affecting package being stabilized.

Automation tests will definitely help here, but I'm not sure that
packages can be fully tested without human intervention: 
- how to automatically test GUI app run-time?
- how to determine which open bugs from bugzilla are serious enough
to block stabilization, and which bugs shouldn't block the process?

IMO automation can help with build tests, maybe limited run-time
testing, but no more.

> The first issue is maintainers not filing stable requests for new
> versions of packages in a timely manor. I'm not sure how to get around
> this, but I feel that once a version of a package is stable, we are
> doing a disservice to our stable users by not keeping stable as current
> as possible.  I am as bad as anyone; it is easy to forget to file
> stable requests until someone pings me or files the request
> themselves.

We have another problem: arch teams often can't keep up with
stabilization requests, they are hanging for months, I even have
several bugs open for more than half a year. Storming arch teams
with stabilization requests will make things even worse.

> I have heard other maintainers say specifically that they do not file
> stable requests unless a user asks them to, but Again, I do not feel
> comfortable with this arrangement if there is an old version of the
> package in stable. Users shouldn't have to ask for newer versions to be
> stabilized; this should be driven by the maintainers.

I usually stabilize versions when they are too old or I have a user
request. The idea is not to stabilize as often as unstable version
is updated, since arch teams are unable to keep up even with
request rate they have now.

> The second issue is slow arch teams. Again, by not moving packages from
> ~ to stable, we are doing a disservice to our stable users.

And here comes my idea. We need an approved policy of how to remove
packages from stable (I'll name this procedure as "dekeywording"
below). Right now we only have a mention in the devmanual, that arch
teams must be informed via a bug [1] on dekeywording. But we have no
determined time frame, nor policy about reverse dependencies.

I propose to empower maintainers to remove packages from stable if
arch team can't stabilize package within 3 months. Of course, time
counts from a point when there are no blockers for stabilization;
if arch team finds a problem blocking stabilization, timer is reset
and will start again only after maintainer will fix the issue.

So if 3 months have elapsed, maintainer should notify arch team and
all reverse deps that package will be removed from stable tree. For
reverse deps this means that they will be removed from stable as
well if dependency is hard or will have some USE flag(s) stable
masked if dependency is optional.

Now we have few questions:

1) Should all reverse dependencies be just CC'ed on original
dekeywording bug or should separates bugs be created for every
reverse dep package? Apparently reverse dep tree can be quite large.

2) For how long maintainer should wait after dekeywording bugs will
be created before taking actual action? I suppose 1 month should be
same (the same as for last-rites).

3) What to do if dekeywording affects @system or other widely used
packages?

4) Of course 3+1 month waiting period can be discussed and
tuned here.

The very idea of this proposal is to create a mechanism of removing
packages from stable if arch team can't keep up with them, so we'll
have a natural selection of packages in stable.

Now one more question comes: if package was removed from stable,
will be there any policies to keep it from entering stable for a
while? Obviously flipping packages to and from stable will make no
good for both users and arch teams. Probably we should ban such
packages from being requested for stabilization for a while (e.g.
for 3 months). Of course, exceptions can be made, e.g. if such
package becomes a hard dependency of @system package an so on.

> I can think of two ways we can improve our situation.
> 
> We can allow maintainers to stabilize new versions of certain types of
>  packages on all arches where there is a previous version of the package stable
>  without filing stable requests. This would take a significant load off
>  of the arch teams.
> 
> For packages that do not fit the first group, we could require stable
> requests, but allow maintainers to stabilize the new versions after a
> timeout (I would propose 30 days).
 
If maintainer has host(s) with stable tree where package can be
tested on arch in question, then yes, this should be done.
Otherwise we'll just add undertested packages in stable which
will undermine the very idea of stable.

[1] https://devmanual.gentoo.org/keywording/index.html

Best regards,
Andrew Savchenko

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2016-08-14

[Daniel Campbell]

[-- Attachment #1.1: Type: text/plain, Size: 3673 bytes --]

On 08/04/2016 09:24 AM, William Hubbs wrote:
> On Thu, Aug 04, 2016 at 04:15:14PM +0200, Kristian Fiskerstrand wrote:
>> Dear all,
>>
>> the Gentoo Council will meet again on Sunday, August 14 at 19:00 UTC
>> in #gentoo-council on FreeNode.
>>
>> Please reply to this message on the gentoo-project list with any items
>> the council should put on its agenda to discuss or vote on.
> 
> I feel that our stable tree is so far behind on all
> architectures that we are doing our stable users a disservice, so I
> would like to open up a discussion here, and maybe some policy changes
> at the next meeting.
> 
> Ultimately, I think we need some form of automated stabilization, e.g.
> if a package version sits in ~ for 30 days and there are no blockers at
> that point, the new version should go automatically to stable on all
> architectures where there is a previous stable version.
> 
> I realize that automation is going to take a lot of work, so in the
> meantime, I would like to discuss changes to our stabilization policies
> that will get new versions of packages to stable faster.
> 
> The first issue is maintainers not filing stable requests for new
> versions of packages in a timely manor. I'm not sure how to get around
> this, but I feel that once a version of a package is stable, we are
> doing a disservice to our stable users by not keeping stable as current
> as possible.  I am as bad as anyone; it is easy to forget to file
> stable requests until someone pings me or files the request
> themselves.
> 
> I have heard other maintainers say specifically that they do not file
> stable requests unless a user asks them to, but Again, I do not feel
> comfortable with this arrangement if there is an old version of the
> package in stable. Users shouldn't have to ask for newer versions to be
> stabilized; this should be driven by the maintainers.
> 
> The second issue is slow arch teams. Again, by not moving packages from
> ~ to stable, we are doing a disservice to our stable users.
> 
> I can think of two ways we can improve our situation.
> 
> We can allow maintainers to stabilize new versions of certain types of
>  packages on all arches where there is a previous version of the package stable
>  without filing stable requests. This would take a significant load off
>  of the arch teams.
> 
> For packages that do not fit the first group, we could require stable
> requests, but allow maintainers to stabilize the new versions after a
> timeout (I would propose 30 days).
> 
> What do folks think?
> 
> William
> 
I can understand where you're coming from on this. Perhaps a more
comprehensive, approachable guide could be written for newer maintainers
like myself so we can learn a few of the skills the arch teams use to
stabilize packages. Proposed testing environments and setups (for
example, a VM dedicated to stabilization that runs on stable rather than
using the dev's main machine) would be super helpful.

Count me as one maintainer that'd love to learn how to do a better job
maintaining and getting newer versions stabilized. As it stands, I'm
almost afraid of pushing something to stable because I fear I'll miss
something or piss somebody off because I didn't perform the right ritual
beforehand or something.

Perhaps a good way to approach it is to adopt a sort of "every
maintainer is also an arch tester" ideology and get these skills passed
down/out to everyone to lessen the load of the arch teams.

-- 
Daniel Campbell - Gentoo Developer
OpenPGP Key: 0x1EA055D6 @ hkp://keys.gnupg.net
fpr: AE03 9064 AE00 053C 270C  1DE4 6F7A 9091 1EA0 55D6


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2016-08-14

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 6566 bytes --]

On Thu, Aug 04, 2016 at 11:12:24PM +0300, Andrew Savchenko wrote:
> Hi all,
> 
> On Thu, 4 Aug 2016 11:24:43 -0500 William Hubbs wrote:
> > I feel that our stable tree is so far behind on all
> > architectures that we are doing our stable users a disservice, so I
> > would like to open up a discussion here, and maybe some policy changes
> > at the next meeting.
> 
> Thank you for caring about this issue. I had similar thoughts
> myself, but was too slow on writing e-mail :) IMO stable tree has
> three problems: 
> 1) too old packages
> 2) too few packages
> 3) stabilization often takes too long, such stable packages are
> broken or buggy, while their unstable versions are fixed and work
> fine. (It is not possible to fix all bugs without version or
> revision bump, thus stabilization is needed to fix many bugs.)
 
 "too few packages" doesn't really affect things much, I'm more
 concerned about 1 and 3. If packages are not stable in the first place,
 that is because the maintainer hasn't requested stabilization, and that
 is a separate issue.

> > Ultimately, I think we need some form of automated stabilization, e.g.
> > if a package version sits in ~ for 30 days and there are no blockers at
> > that point, the new version should go automatically to stable on all
> > architectures where there is a previous stable version.
> 
> Automation should be possible only after rigorous testing. If this
> will be implemented, than this may be done (as Brian pointed out in
> another mail in this thread, there is an ongoing effort on this
> matter as GSoC project, which is great). But I'd like to emphasize
> that automated stabilization without:
> a) build testing;
> b) run-time testing;
> c) fixing all serious open bugs affecting package being stabilized.
> 
> Automation tests will definitely help here, but I'm not sure that
> packages can be fully tested without human intervention: 
> - how to automatically test GUI app run-time?
> - how to determine which open bugs from bugzilla are serious enough
> to block stabilization, and which bugs shouldn't block the process?
> 
> IMO automation can help with build tests, maybe limited run-time
> testing, but no more.

This is why there is a period for a package to be in ~ before it goes to
stable. All of this rigorous testing you are talking about should be
done by people who are running the ~ version of the package and bugs
should be reported. Automated stabilization just refers to checking to
see if there any bugs against the latest ~ version of a package that
should block stabilization, and if there are not, maybe doing a build
test then stabilizing.

> 
> > The first issue is maintainers not filing stable requests for new
> > versions of packages in a timely manor. I'm not sure how to get around
> > this, but I feel that once a version of a package is stable, we are
> > doing a disservice to our stable users by not keeping stable as current
> > as possible.  I am as bad as anyone; it is easy to forget to file
> > stable requests until someone pings me or files the request
> > themselves.
> 
> We have another problem: arch teams often can't keep up with
> stabilization requests, they are hanging for months, I even have
> several bugs open for more than half a year. Storming arch teams
> with stabilization requests will make things even worse.
 
I'm not proposing storming the arch teams with stable requests, see
below.

> > I have heard other maintainers say specifically that they do not file
> > stable requests unless a user asks them to, but Again, I do not feel
> > comfortable with this arrangement if there is an old version of the
> > package in stable. Users shouldn't have to ask for newer versions to be
> > stabilized; this should be driven by the maintainers.
> 
> I usually stabilize versions when they are too old or I have a user
> request. The idea is not to stabilize as often as unstable version
> is updated, since arch teams are unable to keep up even with
> request rate they have now.
 
 My proposal is saying that if you have a version of a package in ~,
 testing is being done, and at the end of the testing period (30 days at
 most), that new version in ~ should move to stable if there are no
 blockers. It would be up to you, the maintainer, or any users running
 the ~ version, to test and file bugs that block stabilization. These
 bugs could be detected automatically.

> > The second issue is slow arch teams. Again, by not moving packages from
> > ~ to stable, we are doing a disservice to our stable users.
> 
> And here comes my idea. We need an approved policy of how to remove
> packages from stable (I'll name this procedure as "dekeywording"
> below). Right now we only have a mention in the devmanual, that arch
> teams must be informed via a bug [1] on dekeywording. But we have no
> determined time frame, nor policy about reverse dependencies.
 
 We basically do. I don't have a link in front of me, but the council
 did make a decision allowing the removal of packages from the stable
 tree. It hasn't played out well though, because stable users expect
 that once a package is in the stable tree it will stay there until a
 newer version comes to the stable tree.

My proposal doesn't apply to all packages that are in ~, only to
packages that have a version in stable.

I'm suggesting that once a package has a version in stable, newer
versions need to be stabilized in a timely manor, e.g. 30 days at most
after they enter the tree once all of their dependencies are stable.

Here are the ways I would do this:

1. First, this assumes that all reverse dependencies of the package in
consideration are stable. It also assumes that the package in
consideration has an older version in stable.

2. if the package is all data files, or if it is written in an
interpreted language e.g. python, perl, etc., Once the testing period
has passed, the maintainer will be allowed to stabilize it on all arches
that have a stable version without a stable request.

2. For other packages , once the testing period is passed, a stable request
will be filed for the new version. Once
another 30 days has passed without a response from the arch teams, the
maintainer would be allowed to stabilize the new version on all arches
that have an older stable version.

This would satisfy the expectation that a package version in the stable
tree cannot be removed until there is a newer stable version.

William


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2016-08-14

[Rich Freeman]

On Thu, Aug 4, 2016 at 6:22 PM, William Hubbs <williamh@gentoo.org> wrote:
> On Thu, Aug 04, 2016 at 11:12:24PM +0300, Andrew Savchenko wrote:
>>
>> Thank you for caring about this issue. I had similar thoughts
>> myself, but was too slow on writing e-mail :) IMO stable tree has
>> three problems:
>> 1) too old packages
>> 2) too few packages
>> 3) stabilization often takes too long, such stable packages are
>> broken or buggy, while their unstable versions are fixed and work
>> fine. (It is not possible to fix all bugs without version or
>> revision bump, thus stabilization is needed to fix many bugs.)
>
>  "too few packages" doesn't really affect things much, I'm more
>  concerned about 1 and 3. If packages are not stable in the first place,
>  that is because the maintainer hasn't requested stabilization, and that
>  is a separate issue.

I'm less concerned with old (within reason) and few.  I think the
primary criteria has to always be that the packages are reliable.  If
somebody wants to make the tradeoff less reliability and fresher
packages they can just install testing.

>
>  My proposal is saying that if you have a version of a package in ~,
>  testing is being done, and at the end of the testing period (30 days at
>  most), that new version in ~ should move to stable if there are no
>  blockers. It would be up to you, the maintainer, or any users running
>  the ~ version, to test and file bugs that block stabilization. These
>  bugs could be detected automatically.
>

I'm mostly fine with that, but I'd add just a requirement that
somebody does a quick sanity check on an otherwise-stable system.  The
30 days of testing is really only testing against dependencies that
are in ~arch.  Granted, that will become less of a concern if all
those dependencies are also making their way to stable.

I'm not suggesting anything rigorous.  However, it would be a bit
embarrassing to stabilize a package and find that it doesn't even
build, or which has some glaring issue.  Build testing at least could
be automated, but I'm not sure if we need more than that.  I'm not
entirely opposed to loosening things up on a trial basis and seeing
how it goes.  However, if we're going to do that I wouldn't go nuts
with automation in case we decide to back off.

>
>  We basically do. I don't have a link in front of me, but the council
>  did make a decision allowing the removal of packages from the stable
>  tree. It hasn't played out well though, because stable users expect
>  that once a package is in the stable tree it will stay there until a
>  newer version comes to the stable tree.

I'd have to look up the exact decision, but it was basically left to
maintainer discretion after some time lag.  I think it is a useful
safety valve.  If the maintainer feels that the stable version is
de-facto unmaintained and is causing problems, then we're not doing
stable users any favors by just leaving it on their systems.  Just go
ahead and drop it and stable users can stick it in an overlay if they
know what they're doing, but they won't just live with some unknown
issue.

However, this shouldn't be done lightly.  This isn't for that package
that is 60 days old.  And it really should be the last resort when the
maintainer can't just stabilize it on their own (it is a bigger issue
for minor archs where hardware is less available and arch teams don't
have time to touch it).

>
> 2. if the package is all data files, or if it is written in an
> interpreted language e.g. python, perl, etc., Once the testing period
> has passed, the maintainer will be allowed to stabilize it on all arches
> that have a stable version without a stable request.

I believe there is already widespread agreement on this point.  We've
talked about mechanisms to designate these packages but if we just
want to go with maintainer discretion we might be fine.

-- 
Rich

Re: [gentoo-project] Call for agenda items - Council meeting 2016-08-14

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 3539 bytes --]

On Thu, Aug 04, 2016 at 07:25:52PM -0400, Rich Freeman wrote:
> On Thu, Aug 4, 2016 at 6:22 PM, William Hubbs <williamh@gentoo.org> wrote:

*snip*

> >
> >  My proposal is saying that if you have a version of a package in ~,
> >  testing is being done, and at the end of the testing period (30 days at
> >  most), that new version in ~ should move to stable if there are no
> >  blockers. It would be up to you, the maintainer, or any users running
> >  the ~ version, to test and file bugs that block stabilization. These
> >  bugs could be detected automatically.
> >
> 
> I'm mostly fine with that, but I'd add just a requirement that
> somebody does a quick sanity check on an otherwise-stable system.  The
> 30 days of testing is really only testing against dependencies that
> are in ~arch.  Granted, that will become less of a concern if all
> those dependencies are also making their way to stable.
 
 Repoman will complain loudly if you try to stabilize something that
 doesn't have all of its reverse dependencies stabilized, so I think we
 are safe as long as people listen to repoman. I'm not advocating
 stabilizing things with ~ reverse dependencies, just trying to find a
 way to move stabilization along better than it has been moving.

*snip*

> >
> >  We basically do. I don't have a link in front of me, but the council
> >  did make a decision allowing the removal of packages from the stable
> >  tree. It hasn't played out well though, because stable users expect
> >  that once a package is in the stable tree it will stay there until a
> >  newer version comes to the stable tree.
> 
> I'd have to look up the exact decision, but it was basically left to
> maintainer discretion after some time lag.  I think it is a useful
> safety valve.  If the maintainer feels that the stable version is
> de-facto unmaintained and is causing problems, then we're not doing
> stable users any favors by just leaving it on their systems.  Just go
> ahead and drop it and stable users can stick it in an overlay if they
> know what they're doing, but they won't just live with some unknown
> issue.
 
 If we can get the newer version stabilized, we can then remove the
 older version without breaking stable, so this then becomes a
 non-issue.

Also, getting the newer version stabilized is a more favorable approach
because you don't have to deal with breaking the depgraph, or in the
case of a package that is in the stages, if you remove the stable
version, you can break the stages for that arch.

*snip*

> >
> > 2. if the package is all data files, or if it is written in an
> > interpreted language e.g. python, perl, etc., Once the testing period
> > has passed, the maintainer will be allowed to stabilize it on all arches
> > that have a stable version without a stable request.
> 
> I believe there is already widespread agreement on this point.  We've
> talked about mechanisms to designate these packages but if we just
> want to go with maintainer discretion we might be fine.

Well, let me back up a bit on this one. We have the allarches keyword
which can be added to a stable request to let the first arch team know
to stabilize on all listed arches.

Maybe we should forget option 2, and just say that if a package version is in ~
with a stable request opened for more than 30 days with all of its
reverse dependencies stable the maintainer can stabilize that version of
the package on all arches that have a stable version.

William


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2016-08-14

[Rich Freeman]

On Thu, Aug 4, 2016 at 10:26 PM, William Hubbs <williamh@gentoo.org> wrote:
> On Thu, Aug 04, 2016 at 07:25:52PM -0400, Rich Freeman wrote:
>>
>> I'm mostly fine with that, but I'd add just a requirement that
>> somebody does a quick sanity check on an otherwise-stable system.  The
>> 30 days of testing is really only testing against dependencies that
>> are in ~arch.  Granted, that will become less of a concern if all
>> those dependencies are also making their way to stable.
>
>  Repoman will complain loudly if you try to stabilize something that
>  doesn't have all of its reverse dependencies stabilized, so I think we
>  are safe as long as people listen to repoman. I'm not advocating
>  stabilizing things with ~ reverse dependencies, just trying to find a
>  way to move stabilization along better than it has been moving.
>

This only helps if the sanity check is correct.  If a package has a
dependency on foo/bar, but it should have >=foo/bar-2, and ~arch is at
-2 and stable is at -1, then repoman will happily let you stabilize
your package even though it will break.  Spending 30 days in testing
might or might not spot the issue, it depends on whether users running
mixed keywords test it.  Since most testing users aren't running mixed
keywords they may not spot that the package breaks with bar-1.

I think we really ought to do SOME testing against the stable
dependencies.  Otherwise you're going to have the occasional breakage,
and if people wanted occasional breakage they'd be running ~arch in
the first place.

I think it makes more sense to just get rid of stable than to make it
a stale version of testing.

Are the older packages actually hurting anybody?  For the most common
arch (amd64) maintainers can just stabilize their own packages, so old
stable packages shouldn't be hurting maintainers (or if they are it is
self-inflicted...).

-- 
Rich

Re: [gentoo-project] Call for agenda items - Council meeting 2016-08-14

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 2626 bytes --]

On Fri, Aug 05, 2016 at 06:57:59AM -0400, Rich Freeman wrote:
> On Thu, Aug 4, 2016 at 10:26 PM, William Hubbs <williamh@gentoo.org> wrote:
> > On Thu, Aug 04, 2016 at 07:25:52PM -0400, Rich Freeman wrote:
> >>
> >> I'm mostly fine with that, but I'd add just a requirement that
> >> somebody does a quick sanity check on an otherwise-stable system.  The
> >> 30 days of testing is really only testing against dependencies that
> >> are in ~arch.  Granted, that will become less of a concern if all
> >> those dependencies are also making their way to stable.
> >
> >  Repoman will complain loudly if you try to stabilize something that
> >  doesn't have all of its reverse dependencies stabilized, so I think we
> >  are safe as long as people listen to repoman. I'm not advocating
> >  stabilizing things with ~ reverse dependencies, just trying to find a
> >  way to move stabilization along better than it has been moving.
> >
> 
> This only helps if the sanity check is correct.  If a package has a
> dependency on foo/bar, but it should have >=foo/bar-2, and ~arch is at
> -2 and stable is at -1, then repoman will happily let you stabilize
> your package even though it will break.  Spending 30 days in testing
> might or might not spot the issue, it depends on whether users running
> mixed keywords test it.  Since most testing users aren't running mixed
> keywords they may not spot that the package breaks with bar-1.
 
 I think if you are doing this sort of testing you need to run a
 mostly stable system. If you are running full ~, you definitely would
 miss issues like this.  I, for one, do not run full ~.

I would actually recommend for devs that they run stable on everything
except packages they maintain. If you do that, you catch issues like
this.

 *snip*

> Are the older packages actually hurting anybody?  For the most common
> arch (amd64) maintainers can just stabilize their own packages, so old
> stable packages shouldn't be hurting maintainers (or if they are it is
> self-inflicted...).

I don't have the numbers in front of me, but from what I hear recently
amd64 has become one of the more lagging architectures. I don't know if
it is because most of our devs are running full ~ and are not set up to
test against stable or if, like some I've talked to, it is just that
they prefer a second set of eyes to go over a  package before it is
stabilized.

Besides our maintainers keeping old packages around, we are doing a
disservice to our stable users by offering them old software instead of
keeping them as current as possible.

William


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2016-08-14

[Rich Freeman]

On Fri, Aug 5, 2016 at 10:28 AM, William Hubbs <williamh@gentoo.org> wrote:
>
> Besides our maintainers keeping old packages around, we are doing a
> disservice to our stable users by offering them old software instead of
> keeping them as current as possible.
>

No argument, but if you actually asked stable users I'm not convinced
that they'd prefer less-tested recent packages over well-tested older
ones.  Anything to get things fresher is good, but there probably
needs to be some kind of sanity check.

-- 
Rich

Re: [gentoo-project] Call for agenda items - Council meeting 2016-08-14

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 945 bytes --]

On Fri, Aug 05, 2016 at 10:36:41AM -0400, Rich Freeman wrote:
> On Fri, Aug 5, 2016 at 10:28 AM, William Hubbs <williamh@gentoo.org> wrote:
> >
> > Besides our maintainers keeping old packages around, we are doing a
> > disservice to our stable users by offering them old software instead of
> > keeping them as current as possible.
> >
> 
> No argument, but if you actually asked stable users I'm not convinced
> that they'd prefer less-tested recent packages over well-tested older
> ones.  Anything to get things fresher is good, but there probably
> needs to be some kind of sanity check.

To clarify what I said earlier, if you are running full ~, there is no
way you can easily test your packages against the stable tree, so you
shouldn't be stabilizing anything, no matter which arch you are using.
To stabilize packages, you should be running a mostly stable system
other than the packages you maintain.

William



[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2016-08-14

[Michael Orlitzky]

On 08/04/2016 12:24 PM, William Hubbs wrote:
> 
> We can allow maintainers to stabilize new versions of certain types of
>  packages on all arches where there is a previous version of the package stable
>  without filing stable requests. This would take a significant load off
>  of the arch teams.
> 
> For packages that do not fit the first group, we could require stable
> requests, but allow maintainers to stabilize the new versions after a
> timeout (I would propose 30 days).
> 
> What do folks think?
> 

This is a good idea; there's a lot of low-hanging fruit here. We could
write up a fairly restrictive list of exceptions and still accomplish a
lot of good.

For example, we've had an exception floating around for a while that
says that maintainers can stabilize their own x86/amd64 packages. That's
assuming they have the hardware, run a stable system themselves, and do
the testing. This sounds reasonable to me -- as the maintainer, I'm
likely to do a better job of testing a package than an arch tester will.
For certain things like web/database servers, I'm using these things in
production for weeks, and the additional build test performed by the
arch team isn't going to add any new information.

Can we at least agree on that exception? Let's codify it in

  https://bugs.gentoo.org/show_bug.cgi?id=510198

and move on from there.

Assuming we go through with that exception, here's another one to
consider: if a package would otherwise be stabilized ALLARCHES, then the
maintainer can perform the stabilization on all arches himself under
those same prior assumptions (he can test, etc). There's no point in
making 8 different people test a new version of some PHP package that
only changed pure PHP code.

Another exception would be for maintainer-needed packages. If there's a
revision that only fixes a bug, we should be able to get it stabilized
and remove the old version even though there's no maintainer.

I would like to add an exception for metadata changes, too, but honestly
I don't trust people to use it wisely. I shouldn't have to bug the arch
teams if I add a second LICENSE and revbump... maybe if this exception
is worded strongly-enough it could do more good than harm.

What I don't want to see is us changing the meaning of "stable" (cf.
making a "D" a passing grade). We should never have maintainers doing
stabilization of C programs on architectures they don't run. I'm only in
favor of exceptions that speed things up without reducing the quality of
the stable tree. This is met by the exceptions I've outlined above, if I
really am doing more thorough testing than the arch teams, or if I have
truly made a harmless metadata change.

In any case, we should still have to wait a minimum of 30 days before
moving a package stable (modulo security issues).

[gentoo-project] Re: Call for agenda items - Council meeting 2016-08-14

[Michael Palimaka]

On 06/08/16 02:11, Michael Orlitzky wrote:
> I would like to add an exception for metadata changes, too, but honestly
> I don't trust people to use it wisely. I shouldn't have to bug the arch
> teams if I add a second LICENSE and revbump... maybe if this exception
> is worded strongly-enough it could do more good than harm.

Why would one revbump for a change to LICENSE?

Re: [gentoo-project] Re: Call for agenda items - Council meeting 2016-08-14

[Michael Orlitzky]

On 08/05/2016 12:22 PM, Michael Palimaka wrote:
> On 06/08/16 02:11, Michael Orlitzky wrote:
>> I would like to add an exception for metadata changes, too, but honestly
>> I don't trust people to use it wisely. I shouldn't have to bug the arch
>> teams if I add a second LICENSE and revbump... maybe if this exception
>> is worded strongly-enough it could do more good than harm.
> 
> Why would one revbump for a change to LICENSE?
> 

Along with ACCEPT_LICENSE, the LICENSE variable affects which packages
can and cannot be installed. You need a new revision so that users who
already have the package installed will pick up the change. If the
change makes a package violate a user's ACCEPT_LICENSE, they need to know.

If you installed something whose EULA says it can hijack your webcam and
post naked pictures of you to slashdot, but it incorrectly had
LICENSE="GPL-2", wouldn't you want to find out that I corrected it?

Re: [gentoo-project] Re: Call for agenda items - Council meeting 2016-08-14

[Chí-Thanh Christopher Nguyễn]

Michael Orlitzky schrieb:
> Along with ACCEPT_LICENSE, the LICENSE variable affects which packages
> can and cannot be installed. You need a new revision so that users who
> already have the package installed will pick up the change. If the
> change makes a package violate a user's ACCEPT_LICENSE, they need to know.

I disagree. ACCEPT_LICENSE does not change the files which the ebuild 
installs on the user's filesystem, nor does it interfere with subslot 
dependency calculations like EAPI changes do. Therefore a new revision 
is not needed.

> If you installed something whose EULA says it can hijack your webcam and
> post naked pictures of you to slashdot, but it incorrectly had
> LICENSE="GPL-2", wouldn't you want to find out that I corrected it?

That is the package manager's task. I do get reminded if a package which 
I have installed is now masked by package.mask, maybe something like 
this would work for licenses too.


Best regards,
Chí-Thanh Christopher Nguyễn

Re: [gentoo-project] Re: Call for agenda items - Council meeting 2016-08-14

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1251 bytes --]

On Fri, 5 Aug 2016 13:06:51 -0400
Michael Orlitzky <mjo@gentoo.org> wrote:

> On 08/05/2016 12:22 PM, Michael Palimaka wrote:
> > On 06/08/16 02:11, Michael Orlitzky wrote:  
> >> I would like to add an exception for metadata changes, too, but honestly
> >> I don't trust people to use it wisely. I shouldn't have to bug the arch
> >> teams if I add a second LICENSE and revbump... maybe if this exception
> >> is worded strongly-enough it could do more good than harm.  
> > 
> > Why would one revbump for a change to LICENSE?
> >   
> 
> Along with ACCEPT_LICENSE, the LICENSE variable affects which packages
> can and cannot be installed. You need a new revision so that users who
> already have the package installed will pick up the change. If the
> change makes a package violate a user's ACCEPT_LICENSE, they need to know.
> 
> If you installed something whose EULA says it can hijack your webcam and
> post naked pictures of you to slashdot, but it incorrectly had
> LICENSE="GPL-2", wouldn't you want to find out that I corrected it?

Wouldn't the revbump actually cause the PM to ignore the new version
and keep to the incorrectly licensed one?

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-project] Re: Call for agenda items - Council meeting 2016-08-14

[Michael Orlitzky]

On 08/05/2016 01:19 PM, Michał Górny wrote:
> 
> Wouldn't the revbump actually cause the PM to ignore the new version
> and keep to the incorrectly licensed one?
> 

Only if the existing ebuild was stable, and the user is running a stable
system... which is the entire reason I proposed the exception in the
first place =P

Re: [gentoo-project] Call for agenda items - Council meeting 2016-08-14

[Kristian Fiskerstrand]

[-- Attachment #1.1: Type: text/plain, Size: 626 bytes --]

On 08/04/2016 10:12 PM, Andrew Savchenko wrote:
> 3) stabilization often takes too long, such stable packages are
> broken or buggy, while their unstable versions are fixed and work
> fine. (It is not possible to fix all bugs without version or
> revision bump, thus stabilization is needed to fix many bugs.)

This is a fault on the part of maintainer, bug should not be closed as
resolved fixed before it is in stable, but kept open with invcs keyword
in bugzilla

-- 
Kristian Fiskerstrand
OpenPGP certificate reachable at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2016-08-14

[Kristian Fiskerstrand]

[-- Attachment #1.1: Type: text/plain, Size: 1089 bytes --]

On 08/04/2016 06:24 PM, William Hubbs wrote:
> I feel that our stable tree is so far behind on all
> architectures that we are doing our stable users a disservice, so I
> would like to open up a discussion here, and maybe some policy changes
> at the next meeting.

Far behind isn't necessarily a problem as long as it doesn't have bugs,
in particular security related ones. Updating too often (without a good
reason) can also be annoying enough.

> 
> Ultimately, I think we need some form of automated stabilization, e.g.
> if a package version sits in ~ for 30 days and there are no blockers at
> that point, the new version should go automatically to stable on all
> architectures where there is a previous stable version.

I LOUDLY disagree. The stable tree should not be compromised by such
automation, it is already bad enough without proper use-testing in some
cases. Stable isn't only about building properly.

-- 
Kristian Fiskerstrand
OpenPGP certificate reachable at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2016-08-14

[Kristian Fiskerstrand]

[-- Attachment #1.1: Type: text/plain, Size: 1389 bytes --]

On 08/05/2016 01:25 AM, Rich Freeman wrote:
> On Thu, Aug 4, 2016 at 6:22 PM, William Hubbs <williamh@gentoo.org> wrote:
>> > On Thu, Aug 04, 2016 at 11:12:24PM +0300, Andrew Savchenko wrote:
>>> >>
>>> >> Thank you for caring about this issue. I had similar thoughts
>>> >> myself, but was too slow on writing e-mail :) IMO stable tree has
>>> >> three problems:
>>> >> 1) too old packages
>>> >> 2) too few packages
>>> >> 3) stabilization often takes too long, such stable packages are
>>> >> broken or buggy, while their unstable versions are fixed and work
>>> >> fine. (It is not possible to fix all bugs without version or
>>> >> revision bump, thus stabilization is needed to fix many bugs.)
>> >
>> >  "too few packages" doesn't really affect things much, I'm more
>> >  concerned about 1 and 3. If packages are not stable in the first place,
>> >  that is because the maintainer hasn't requested stabilization, and that
>> >  is a separate issue.
> I'm less concerned with old (within reason) and few.  I think the
> primary criteria has to always be that the packages are reliable.  If
> somebody wants to make the tradeoff less reliability and fresher
> packages they can just install testing.
> 

+1

-- 
Kristian Fiskerstrand
OpenPGP certificate reachable at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

Re: [gentoo-project] Re: Call for agenda items - Council meeting 2016-08-14

[Michael Orlitzky]

On 08/05/2016 01:11 PM, Chí-Thanh Christopher Nguyễn wrote:
> 
> I disagree. ACCEPT_LICENSE does not change the files which the ebuild 
> installs on the user's filesystem, nor does it interfere with subslot 
> dependency calculations like EAPI changes do. Therefore a new revision 
> is not needed.

Those two are sufficient for a new revision, but not necessary. Use
common sense. If not doing a revision is going to negatively affect
people, then you should do a revision.


>> If you installed something whose EULA says it can hijack your webcam and
>> post naked pictures of you to slashdot, but it incorrectly had
>> LICENSE="GPL-2", wouldn't you want to find out that I corrected it?
> 
> That is the package manager's task. I do get reminded if a package which 
> I have installed is now masked by package.mask, maybe something like 
> this would work for licenses too.
> 

The package manager's task is well-defined, and that isn't part of it.
Saying "maybe it's possible somehow some day" is not a good reason to
screw up ACCEPT_LICENSE handling with certainty right now.

To bring this back on topic. If I change LICENSE, I'm going to do it in
a new revision, because I want it to work right -- to each his own. At
the moment, doing a revision on top of a stable ebuild is not optimal,
because little bug fixes like that can remain in unstable for a year. A
carefully-worded exception could alleviate that, when the change could
in no way affect architecture stability.

Re: [gentoo-project] Call for agenda items - Council meeting 2016-08-14

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 1363 bytes --]

On Fri, Aug 05, 2016 at 07:31:33PM +0200, Kristian Fiskerstrand wrote:
> On 08/04/2016 06:24 PM, William Hubbs wrote:
> > I feel that our stable tree is so far behind on all
> > architectures that we are doing our stable users a disservice, so I
> > would like to open up a discussion here, and maybe some policy changes
> > at the next meeting.
> 
> Far behind isn't necessarily a problem as long as it doesn't have bugs,
> in particular security related ones. Updating too often (without a good
> reason) can also be annoying enough.
> 
> > 
> > Ultimately, I think we need some form of automated stabilization, e.g.
> > if a package version sits in ~ for 30 days and there are no blockers at
> > that point, the new version should go automatically to stable on all
> > architectures where there is a previous stable version.
> 
> I LOUDLY disagree. The stable tree should not be compromised by such
> automation, it is already bad enough without proper use-testing in some
> cases. Stable isn't only about building properly.

and that's why we don't commit straight to stable. people are supposed
to be testing those ~arch versions for a while before they go stable.
That testing should cover the use cases you are talking about and work
out the bugs. Once that's done, we should be able to move the package to
stable.

William



[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2016-08-14

[Kristian Fiskerstrand]

[-- Attachment #1.1: Type: text/plain, Size: 1258 bytes --]

On 08/05/2016 08:42 PM, William Hubbs wrote:
>>> > > Ultimately, I think we need some form of automated stabilization, e.g.
>>> > > if a package version sits in ~ for 30 days and there are no blockers at
>>> > > that point, the new version should go automatically to stable on all
>>> > > architectures where there is a previous stable version.
>> > 
>> > I LOUDLY disagree. The stable tree should not be compromised by such
>> > automation, it is already bad enough without proper use-testing in some
>> > cases. Stable isn't only about building properly.
> and that's why we don't commit straight to stable. people are supposed
> to be testing those ~arch versions for a while before they go stable.
> That testing should cover the use cases you are talking about and work
> out the bugs. Once that's done, we should be able to move the package to
> stable.

It was recently a discussion in #-dev that could help on this, the
automation can't be only build-testing, but if writing usage-tests
(protocol, interface access testing etc for servers etc) automation
would indeed be helpful.

-- 
Kristian Fiskerstrand
OpenPGP certificate reachable at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2016-08-14

[NP-Hardass]

[-- Attachment #1.1: Type: text/plain, Size: 1513 bytes --]

On 08/05/2016 02:45 PM, Kristian Fiskerstrand wrote:
> On 08/05/2016 08:42 PM, William Hubbs wrote:
>>>>>> Ultimately, I think we need some form of automated stabilization, e.g.
>>>>>> if a package version sits in ~ for 30 days and there are no blockers at
>>>>>> that point, the new version should go automatically to stable on all
>>>>>> architectures where there is a previous stable version.
>>>>
>>>> I LOUDLY disagree. The stable tree should not be compromised by such
>>>> automation, it is already bad enough without proper use-testing in some
>>>> cases. Stable isn't only about building properly.
>> and that's why we don't commit straight to stable. people are supposed
>> to be testing those ~arch versions for a while before they go stable.
>> That testing should cover the use cases you are talking about and work
>> out the bugs. Once that's done, we should be able to move the package to
>> stable.
> 
> It was recently a discussion in #-dev that could help on this, the
> automation can't be only build-testing, but if writing usage-tests
> (protocol, interface access testing etc for servers etc) automation
> would indeed be helpful.
> 

Isn't the src_test phase psuedo runtime testing?  I'm not sure that us
developing test suites for upstreams is a good use of our time.  I mean,
yes, it would improve the upstream projects where they accept it (which
is good for all), but I feel it detracts from what Gentoo developers are
doing (for Gentoo)

-- 
NP-Hardass


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2016-08-14

[Kristian Fiskerstrand]

[-- Attachment #1.1: Type: text/plain, Size: 694 bytes --]

On 08/05/2016 08:55 PM, NP-Hardass wrote:
> Isn't the src_test phase psuedo runtime testing?  I'm not sure that us
> developing test suites for upstreams is a good use of our time.  I mean,
> yes, it would improve the upstream projects where they accept it (which
> is good for all), but I feel it detracts from what Gentoo developers are
> doing (for Gentoo)

We (Gentoo) are putting the stable lable on something, if we include
github upstreams without proper unit tests into stable it is on us to
ensure it is sufficient quality.

-- 
Kristian Fiskerstrand
OpenPGP certificate reachable at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2016-08-14

[Marek Szuba]

[-- Attachment #1.1: Type: text/plain, Size: 1242 bytes --]

On Fri, Aug 05, 2016 at 10:36:41AM -0400, Rich Freeman wrote:

> No argument, but if you actually asked stable users I'm not convinced
> that they'd prefer less-tested recent packages over well-tested older
> ones.  Anything to get things fresher is good, but there probably
> needs to be some kind of sanity check.
My two cents on the subject. Perhaps I am odd in this but in spite of
having been messing with ebuild development I keep my Gentoo systems
mostly stable - and I definitely prefer keeping it this way. This might
perhaps have something to do with the fact I do not really need the
latest and supposedly greatest versions of all packages, for instance
the amd64 box I am writing this on has only got 15 entries (plus
dependencies of those) in my "version bump" Portage keyword files.

That said, the situation gets much worse for packages which can be found
*only* in ~arch - on the same box the "not in stable" keyword file
contains 72 entries. Only a few of these have been orphaned so I guess I
could always submit a lot of STABLEREQs but telling the truth, the
number has really discouraged me.

Bottom line: I would say we do need some way of streamlining ebuild
stabilisation.

Cheers,
-- 
MS



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2016-08-14

[Pacho Ramos]

El lun, 08-08-2016 a las 14:35 +0200, Marek Szuba escribió:
> [...]

I am not sure if this has been already suggested or not (as I have not
being able to follow all the thread :S)

My suggestion, for now would be to modify a bit the current policy: if
I don't misremember, we can drop stable keywords for arches that are
not stabilizing the package in 90 days. The problem is that it
currently cannot be done in most of the times because it's not feasible
for the maintainer to drop the keyword and *also* all the stable
keywords of reverse deps.

Hence, I would suggest to, apart of allowing the maintainers to drop
the keywords, to also allow them to stabilize that packages on that
arches when this timeout has expired 

Thanks a lot

Re: [gentoo-project] Call for agenda items - Council meeting 2016-08-14

[Jack Morgan]

[-- Attachment #1.1: Type: text/plain, Size: 1199 bytes --]



On 08/08/16 05:35, Marek Szuba wrote:
> 
> Bottom line: I would say we do need some way of streamlining ebuild
> stabilisation.

I vote we fix this problem. I'm tired of having this same discussion
ever 6 or 12 months. I'd like to see less policy discussion and more
technical solutions to the problems we face.

I propose calling for volunteers to create a new project that works on
solving our stabilization problem. I see that looking like the following:

1) project identifies the problem(s) with real data from Bugzilla and
the portage tree.

2) new project defines a technical proposal to fixing this issue, then
presents it to the developer community for feedback. This would include
defining tools needed or used

3) start working on solution + define future roadmap


All processes and policies should be on the table for negotiating in the
potential solution. If we need to reinvent the wheel, then let's do it.

To be honest, adding more policy just ends up making everyone unhappy
one way or the other.


-- 
Jack Morgan
Pub 4096R/761D8E0A 2010-09-13 Jack Morgan <jmorgan@gentoo.org>
Fingerprint = DD42 EA48 D701 D520 C2CD 55BE BF53 C69B 761D 8E0A


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2016-08-14

[Kent Fredric]

[-- Attachment #1: Type: text/plain, Size: 9553 bytes --]

On Mon, 8 Aug 2016 19:07:04 -0700
Jack Morgan <jmorgan@gentoo.org> wrote:

> On 08/08/16 05:35, Marek Szuba wrote:
> > 
> > Bottom line: I would say we do need some way of streamlining ebuild
> > stabilisation.  
> 
> I vote we fix this problem. I'm tired of having this same discussion
> ever 6 or 12 months. I'd like to see less policy discussion and more
> technical solutions to the problems we face.
> 
> I propose calling for volunteers to create a new project that works on
> solving our stabilization problem. I see that looking like the
> following:
> 
> 1) project identifies the problem(s) with real data from Bugzilla and
> the portage tree.
> 
> 2) new project defines a technical proposal to fixing this issue, then
> presents it to the developer community for feedback. This would
> include defining tools needed or used
> 
> 3) start working on solution + define future roadmap
> 
> 
> All processes and policies should be on the table for negotiating in
> the potential solution. If we need to reinvent the wheel, then let's
> do it.
> 
> To be honest, adding more policy just ends up making everyone unhappy
> one way or the other.
> 
> 

There's a potential way to garner a technical solution that somewhat
alleviates the need for such rigourous arch testers, and without
degrading the stabilisation mechanic to "blind monkey system that
stabilises based on conjecture".

I've mentioned it before ages ago on the Gentoo Dev list, somewhere.

The idea is basically to instrument portage to have an (optional)
feature that when turned on, records and submits certain facts about
every failed or successful install, with the objective being to
essentially spread the load out of what `tatt` does organically over
the participant base.

1. Firstly, make no demands of homoegenity or even sanity for a users
system to participate. Ever thing they throw at this system I'm about
to propose should be considered "valid"

2. Every time a package is installed, or attempted to be installed, the
exit of that installation is qualified in one of a number of ways:

   - installed OK without tests
   - installed OK with tests
   - failed tests
   - failed install
   - failed compile 
   - failed configure

Each of these is a single state in a single field.

3. The Name, Version, and SHA1 of the ebuild that generated the report.


4. The USE flags and any other pertinent ( and carefully selected by
Gentoo ) flags are included, each as single fields in a property set,
and decomposed into structured property lists where possible.

5. <arch> satisfaction data for the target package at the time of
installation is recorded.

eg:

   KEYWORDS="arch"  + ACCEPT_KEYWORDS="~arch" -> [ "arch(~)"  ]
   KEYWORDS="~arch" + ACCEPT_KEYWORDS="~arch" -> [ "~arch(~)" ]
   KEYWORDS="arch"  + ACCEPT_KEYWORDS="arch"  -> [ "arch"     ]
   KEYWORDS=""      + ACCEPT_KEYWORDS="**"    -> [ "(**)"     ]

This seems redundant, but this is basically suggesting "hey, if you're
insane and setting lots of different arches for accept keywords, that
would be relevant data to use to ignore your report. This data can also
be used with other data I'll mention later to isolate users with "mixed
keywording" setups.

6. For every dependency listed in *DEPEND, a dictionary/hash of

  "specified atom" -> {
     name -> resolved dependency name
     version -> version of resolved dependency
     arch -> [ satisfied arch spec as in #4 ]
     sha1 -> Some kind of SHA1 that hopefully turns up in gentoo.git
  }


is recorded in the response at the time of the result.

The "satisified arch spec" field is used to isolate anomalies in
keywording and user keyword mixing and filter out non-target reports
for stabilization data.

7. A Submitter Unique Identifier

8. Possibly a Submitter-Machine Unique Identifier.

9. The whole build log will be included compressed, verbatim.

This latter part will an independent option to the "reporting" feature,
because its a slightly more invasive privacy concern than the others,
in that, arbitrary code execution can leak private data.

Hence, people who turn this feature on have to know what they're
signing up for.

10. All of the above data is pooled and shipped as a single report, and
submitted to a "report server" and aggregated.


With all of the above, in the most native of situations, we can use
that data at very least to give us a lot more assurance than "well, 30
days passed, and nobody complained", because we'll have a paper trail
of a known countable number of successful installs, which while
not representative, are likely to still be more diverse and
reassuring of confidence than the deafening silence of no
feedback.

And in non-naive situations, the results for given versions can be
aggregated and compared, and factors that are present can be correlated
with failures statistically.

And this would give us a status board of "here's a bunch of
configurations that seem to be statisically more problematic than
others, might be worth investigating"

But there would be no burden to actually dive into the logs unless you
found clusters of failures from different sources failing under the
same scenarios ( And this is why not everyone *has* to send build logs
to be effective, just enough people have to report "x configuration
bad" and some subset of them have to provide elucidating logs ).

None of what I mention here is conceptually "new", I've just
re-explained the entire CPAN Testers model in terms relevant to Gentoo,
using Gentoo parts instead of CPAN parts.

And CPAN testers find it *very effective* at being assured they didn't
break anything: They ship a TRIAL release ( akin to our ~arch ), and
then wait a week or so while people download and test it.

And pretty much anyone can become "a tester", there's no barrier to
entry, and no requirements for membership. Just install the tools, get
yourself an ID, and start installing stuff with tests (the default),
and the tools you have will automatically fire off those reports to the
hive, and you get a big pretty matrix of "We're good here", and then
after no red results in some period, they go "hey, yep, we're good" and
ship a stable release.

Or maybe occasional pockets of "you dun goofed" where there will be a
problem you might have to look into ( sometimes those problems are
entirely invalid problems, ... this is somehow typically not an issue )

http://matrix.cpantesters.org/?dist=App-perlbrew+0.76

And you throw variants analysis into the mix and you get those other
facts compared and ranked by "Likelihood to be part of the problem"

http://analysis.cpantesters.org/solved?distv=App-perlbrew-0.76

^ you see here variant analysis found 3 common strings in the logs that
indicated a failure, and it pointed the finger directly at the failing
test as a result. And then in rank #3, you see its pointing a finger at
CPAN::Perl::Releases as "a possible problem highly correlated with
failures" with the -0.5 theta on version 2.88 

Lo and behold, automated differential analysis has found the bug: 

https://rt.cpan.org/Ticket/Display.html?id=116517

It still takes a human to 

a) decide to look
b) decide the differential factors are useful enough to pursue 
c) verify the problem manually by using the guidance given
d) manually file the bug

But the point here is we can actually build some infrastructure that
will give automated tooling some degree of assurance that "this can
probably be safely stabilized now, the testers aren't seeing any issues"

Its just also the sort of data collection that can lend itself to much
more powerful benefits as well.

The only hard parts are:

1. Making a good server to handle these reports that scales well
2. Making a good client for report generation, collection from PORTAGE
and submission
3. Getting people to turn on the feature
4. Getting enough people using the feature that the majority of the
"easy" stabilizations can happen hands-free. 

And we don't even have to do the "Fancy" parts of it now:

 Just pools of  "package:  arch = 100pass/0fail archb = 10pass/0  fail" 

Would be a great start.

Because otherwise we're relying 100% on negative feedback, and assuming
that the absence of negative feedback is positive, when the reality
might be closer that the absence of negative feedback is that the
problems were too confusing to report as an explicit bug, the problems
faced were deemed unimportant to the person in question and they gave
up before they reported it, the user encountered some other entry
barrier in reporting, ..... or maybe, nobody is actually using the
package at all, so it could actually be completely broken and nobody
notices.

And it seems entirely hap-hazard to encourage tooling that not
*builds* upon that assumption.

At least with the manual stabilization process, you can be assured that
at least one human will personally install, test, and verify a package
works in at least one situation.

With a completely automated stabilization that relies on the absence of
negative feedback to stabilize, you're *not even getting that*.

Why bother with stabilization at all if the entire thing is merely
*conjecture* ?

Even a broken, flawed stabilization workflow done by teams of people
who are bad at testing is better than a stabilization workflow
implemented on conjecture of stability :P






[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2016-08-14

[Rich Freeman]

On Tue, Aug 9, 2016 at 1:32 AM, Kent Fredric <kentnl@gentoo.org> wrote:
>
> 2. Every time a package is installed, or attempted to be installed, the
> exit of that installation is qualified in one of a number of ways:
>
>    - installed OK without tests

While I think your proposal is a great one, I think this is actually
the biggest limitation.  A lot of our packages (most?) don't actually
have tests that can be run on every build (most don't have tests, some
have tests that take forever to run or can't be used on a clean
install).

While runtime testing doesn't HAVE to be extensive, we do want
somebody to at least take a glance at it.

If everything you're proposing is just on top of what we're already
doing, then we have the issue that people aren't keeping up with the
current workload, and even if that report is ultra-nice it is actually
one more step than we have today.  The workload would only go down if
a machine could look at the report and stabilize things without input
at least some of the time.

-- 
Rich

Re: [gentoo-project] Call for agenda items - Council meeting 2016-08-14

[Kent Fredric]

[-- Attachment #1: Type: text/plain, Size: 2572 bytes --]

On Tue, 9 Aug 2016 01:59:35 -0400
Rich Freeman <rich0@gentoo.org> wrote:

> While I think your proposal is a great one, I think this is actually
> the biggest limitation.  A lot of our packages (most?) don't actually
> have tests that can be run on every build (most don't have tests, some
> have tests that take forever to run or can't be used on a clean
> install).

IMHO, That's not "ideal", but we don't need idealism to be useful here.

Tests passing give one kind of useful kind of quality test.

But "hey, it compiles" gives useful data in itself.

By easy counter example, "it doesn't compile" is in itself useful
information ( and the predominant supply of bugs filed are compilation
failures ).

Hell, sometimes I hit a compile failure and I just go "eeh, I'll look
into it next week". How many people are doing the same?

The beauty of the automated datapoint is it doesn't have to be "awesome
quality" to be useful, its just guidance for further investigation.
 
> While runtime testing doesn't HAVE to be extensive, we do want
> somebody to at least take a glance at it.

Indeed, I'm not hugely in favour of abolishing manual stabilization
entirely, but sometimes it just gets to a point where its a bit beyond
a joke with requests languishing untouched for months.

If there was even data saying "hey, look, its obvious this isn't ready
for stabilization", we could *remove* or otherwise mark for
postponement stabilization requests that were failing due to
crowd-source metrics.

This means it can also be used to focus existing stabilization efforts
to reduce the number of things being thrown in the face of manual
stabilizers.

> 
> If everything you're proposing is just on top of what we're already
> doing, then we have the issue that people aren't keeping up with the
> current workload, and even if that report is ultra-nice it is actually
> one more step than we have today.  The workload would only go down if
> a machine could look at the report and stabilize things without input
> at least some of the time.

Indeed, it would require the crowd service to be automated, and the
relevant usage of the data as automated as possible, and humans would
only go looking at the data when interested.

For instance, when somebody manually files a stable request, some
watcher could run off and scour the reports in a given window and
comment "Warning: Above threshold failure rates for target in last
n-days, proceed with caution", and it would only enhance the existing
stabilization workflow.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2016-08-14

[Brian Dolbec]

[-- Attachment #1: Type: text/plain, Size: 4008 bytes --]

On Tue, 9 Aug 2016 22:05:45 +1200
Kent Fredric <kentnl@gentoo.org> wrote:

> On Tue, 9 Aug 2016 01:59:35 -0400
> Rich Freeman <rich0@gentoo.org> wrote:
> 
> > While I think your proposal is a great one, I think this is actually
> > the biggest limitation.  A lot of our packages (most?) don't
> > actually have tests that can be run on every build (most don't have
> > tests, some have tests that take forever to run or can't be used on
> > a clean install).  
> 
> IMHO, That's not "ideal", but we don't need idealism to be useful
> here.
> 
> Tests passing give one kind of useful kind of quality test.
> 
> But "hey, it compiles" gives useful data in itself.
> 
> By easy counter example, "it doesn't compile" is in itself useful
> information ( and the predominant supply of bugs filed are compilation
> failures ).
> 
> Hell, sometimes I hit a compile failure and I just go "eeh, I'll look
> into it next week". How many people are doing the same?
> 
> The beauty of the automated datapoint is it doesn't have to be
> "awesome quality" to be useful, its just guidance for further
> investigation. 
> > While runtime testing doesn't HAVE to be extensive, we do want
> > somebody to at least take a glance at it.  
> 
> Indeed, I'm not hugely in favour of abolishing manual stabilization
> entirely, but sometimes it just gets to a point where its a bit beyond
> a joke with requests languishing untouched for months.
> 
> If there was even data saying "hey, look, its obvious this isn't ready
> for stabilization", we could *remove* or otherwise mark for
> postponement stabilization requests that were failing due to
> crowd-source metrics.
> 
> This means it can also be used to focus existing stabilization efforts
> to reduce the number of things being thrown in the face of manual
> stabilizers.
> 
> > 
> > If everything you're proposing is just on top of what we're already
> > doing, then we have the issue that people aren't keeping up with the
> > current workload, and even if that report is ultra-nice it is
> > actually one more step than we have today.  The workload would only
> > go down if a machine could look at the report and stabilize things
> > without input at least some of the time.  
> 
> Indeed, it would require the crowd service to be automated, and the
> relevant usage of the data as automated as possible, and humans would
> only go looking at the data when interested.
> 
> For instance, when somebody manually files a stable request, some
> watcher could run off and scour the reports in a given window and
> comment "Warning: Above threshold failure rates for target in last
> n-days, proceed with caution", and it would only enhance the existing
> stabilization workflow.

This whole thing you are proposing has been a past stats project many
times in GSOC for Gentoo.  The last time, it produced a decent system
that was functional and __NEVER__ got deployed and turned on.  It ran
for several years on the gentoo GSOC student server (vulture).  It
never gained traction with the infra team due to lack of infra
resources and infra personnel to maintain it.

Perhaps with the new hardware recently purchased to replace the failed
server from earlier this year, we should have some hardware resources.
If you can dedicate some time to work on the code which I'm sure will
need some updating now, I would help as well (not that I already can't
keep up with all the project coding I'm involved with).  

This is of course if we can get a green light from our infra team to be
able to implement a stats vm on the new ganeti system.

We will also need some help from security people to ensure the system
is secure, nginx/lightttp configuration, etc...

So, are you up for it?  Any Gentoo dev willing to help admin such a
system, please reply with your area of expertise and ability to help.

Maybe we can finally get a working and deployed stats system.

-- 
Brian Dolbec <dolsen>


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 951 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2016-08-14

[Kent Fredric]

[-- Attachment #1: Type: text/plain, Size: 618 bytes --]

On Tue, 9 Aug 2016 07:41:05 -0700
Brian Dolbec <dolsen@gentoo.org> wrote:

> So, are you up for it?  Any Gentoo dev willing to help admin such a
> system, please reply with your area of expertise and ability to help.
> 
> Maybe we can finally get a working and deployed stats system.

Is there anywhere other than https://wiki.gentoo.org/wiki/Gentoostats I
can read more? There's lots of dead links to a non-existent
soc.dev.gentoo.org server

I Can find the code:
https://gitweb.gentoo.org/proj/gentoostats.git/tree/ 

But I can't find really any in-depth design documents :/

Deadlinks everywhere.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2016-08-14

[Brian Dolbec]

[-- Attachment #1: Type: text/plain, Size: 1242 bytes --]

On Wed, 10 Aug 2016 03:12:08 +1200
Kent Fredric <kentnl@gentoo.org> wrote:

> On Tue, 9 Aug 2016 07:41:05 -0700
> Brian Dolbec <dolsen@gentoo.org> wrote:
> 
> > So, are you up for it?  Any Gentoo dev willing to help admin such a
> > system, please reply with your area of expertise and ability to
> > help.
> > 
> > Maybe we can finally get a working and deployed stats system.  
> 
> Is there anywhere other than https://wiki.gentoo.org/wiki/Gentoostats
> I can read more? There's lots of dead links to a non-existent
> soc.dev.gentoo.org server
> 
> I Can find the code:
> https://gitweb.gentoo.org/proj/gentoostats.git/tree/ 
> 
> But I can't find really any in-depth design documents :/
> 
> Deadlinks everywhere.


yeah, that what happens with abandoned things...

Also vulture tends to get wiped each year to be reset for the next
batch of students and projects.  

The best is probably to go thru the gentoo-soc mail list archive to get
a feel for the direction and basic design and goals.  I do remember
some of it, but I only helped with a bit of it (portage, gentoolkit
api's) plus it was 5 years ago now and my memory is not as good as it
used to be (yeah, I'm old).
-- 
Brian Dolbec <dolsen>


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 951 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2016-08-14

[Kent Fredric]

[-- Attachment #1: Type: text/plain, Size: 368 bytes --]

On Tue, 9 Aug 2016 09:15:16 -0700
Brian Dolbec <dolsen@gentoo.org> wrote:

> yeah, that what happens with abandoned things...

Is it normal for bugs to get deleted as well?

https://archives.gentoo.org/gentoo-soc/message/760cbd58a309b56f31d3697d90f44601

References 

https://bugs.gentoo.org/show_bug.cgi?id=425055

But that page says "Missing Bug ID"


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2016-08-14

[Brian Evans]

[-- Attachment #1.1: Type: text/plain, Size: 529 bytes --]

On 8/9/2016 1:09 PM, Kent Fredric wrote:
> On Tue, 9 Aug 2016 09:15:16 -0700
> Brian Dolbec <dolsen@gentoo.org> wrote:
> 
>> yeah, that what happens with abandoned things...
> 
> Is it normal for bugs to get deleted as well?
> 
> https://archives.gentoo.org/gentoo-soc/message/760cbd58a309b56f31d3697d90f44601
> 
> References 
> 
> https://bugs.gentoo.org/show_bug.cgi?id=425055

It's off by one, the bugs have been even numbers for quite a while now..

https://bugs.gentoo.org/show_bug.cgi?id=425056

Brian


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2016-08-14

[Chí-Thanh Christopher Nguyễn]

Brian Evans schrieb:
>> Is it normal for bugs to get deleted as well?
>>
>> https://archives.gentoo.org/gentoo-soc/message/760cbd58a309b56f31d3697d90f44601
>>
>> References
>>
>> https://bugs.gentoo.org/show_bug.cgi?id=425055
> It's off by one, the bugs have been even numbers for quite a while now..
>
> https://bugs.gentoo.org/show_bug.cgi?id=425056
>
> Brian
>

BTW. that was around the time when Gentoo bugzilla reshuffled servers 
and went from odd to even bug IDs. Last odd bug was #424805.


Best regards,
Chí-Thanh Christopher Nguyễn

Re: [gentoo-project] Call for agenda items - Council meeting 2016-08-14

[Ciaran McCreesh]

On Wed, 10 Aug 2016 05:09:51 +1200
Kent Fredric <kentnl@gentoo.org> wrote:
> On Tue, 9 Aug 2016 09:15:16 -0700
> Brian Dolbec <dolsen@gentoo.org> wrote:
> 
> > yeah, that what happens with abandoned things...  
> 
> Is it normal for bugs to get deleted as well?

Yes, there's a shortage of bug numbers, and sometimes infra has to
reuse bugs that used to be about something else. It does cause things
to disappear down the memory hole occasionally, but on the plus side,
it stops some embarrassing mistakes from being visible for too long.

-- 
Ciaran McCreesh

Re: [gentoo-project] Call for agenda items - Council meeting 2016-08-14

[Rich Freeman]

On Tue, Aug 9, 2016 at 1:22 PM, Ciaran McCreesh
<ciaran.mccreesh@googlemail.com> wrote:
> On Wed, 10 Aug 2016 05:09:51 +1200
> Kent Fredric <kentnl@gentoo.org> wrote:
>> On Tue, 9 Aug 2016 09:15:16 -0700
>> Brian Dolbec <dolsen@gentoo.org> wrote:
>>
>> > yeah, that what happens with abandoned things...
>>
>> Is it normal for bugs to get deleted as well?
>
> Yes, there's a shortage of bug numbers, and sometimes infra has to
> reuse bugs that used to be about something else. It does cause things
> to disappear down the memory hole occasionally, but on the plus side,
> it stops some embarrassing mistakes from being visible for too long.
>

Please tell me you're joking?

Is the even/odd thing some kind of bugzilla issue?  That seems...suboptimal...

-- 
Rich

Re: [gentoo-project] Call for agenda items - Council meeting 2016-08-14

[Kristian Fiskerstrand]

[-- Attachment #1.1: Type: text/plain, Size: 539 bytes --]

On 08/09/2016 10:08 PM, Rich Freeman wrote:
> 
> Is the even/odd thing some kind of bugzilla issue?  That seems...suboptimal...
> 

iirc it is originally a feature that allows for load distribution
between multiple backend instances. It is likely less useful in later
times when it can be distributed on RMDBS by Galera cluster to
complement the traditional master/slave setups.

-- 
Kristian Fiskerstrand
OpenPGP certificate reachable at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2016-08-14

[Kristian Fiskerstrand]

[-- Attachment #1.1: Type: text/plain, Size: 598 bytes --]

On 08/09/2016 10:08 PM, Rich Freeman wrote:
>> > Yes, there's a shortage of bug numbers, and sometimes infra has to
>> > reuse bugs that used to be about something else. It does cause things
>> > to disappear down the memory hole occasionally, but on the plus side,
>> > it stops some embarrassing mistakes from being visible for too long.
>> >
> Please tell me you're joking?

Buzilla isn't limited to our current 6 digit bug numbers :)

-- 
Kristian Fiskerstrand
OpenPGP certificate reachable at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2016-08-14

[Pallav Agarwal]

[-- Attachment #1: Type: text/plain, Size: 4339 bytes --]

On Tue, Aug 9, 2016 at 8:11 PM, Brian Dolbec <dolsen@gentoo.org> wrote:

> On Tue, 9 Aug 2016 22:05:45 +1200
> Kent Fredric <kentnl@gentoo.org> wrote:
>
> > On Tue, 9 Aug 2016 01:59:35 -0400
> > Rich Freeman <rich0@gentoo.org> wrote:
> >
> > > While I think your proposal is a great one, I think this is actually
> > > the biggest limitation.  A lot of our packages (most?) don't
> > > actually have tests that can be run on every build (most don't have
> > > tests, some have tests that take forever to run or can't be used on
> > > a clean install).
> >
> > IMHO, That's not "ideal", but we don't need idealism to be useful
> > here.
> >
> > Tests passing give one kind of useful kind of quality test.
> >
> > But "hey, it compiles" gives useful data in itself.
> >
> > By easy counter example, "it doesn't compile" is in itself useful
> > information ( and the predominant supply of bugs filed are compilation
> > failures ).
> >
> > Hell, sometimes I hit a compile failure and I just go "eeh, I'll look
> > into it next week". How many people are doing the same?
> >
> > The beauty of the automated datapoint is it doesn't have to be
> > "awesome quality" to be useful, its just guidance for further
> > investigation.
> > > While runtime testing doesn't HAVE to be extensive, we do want
> > > somebody to at least take a glance at it.
> >
> > Indeed, I'm not hugely in favour of abolishing manual stabilization
> > entirely, but sometimes it just gets to a point where its a bit beyond
> > a joke with requests languishing untouched for months.
> >
> > If there was even data saying "hey, look, its obvious this isn't ready
> > for stabilization", we could *remove* or otherwise mark for
> > postponement stabilization requests that were failing due to
> > crowd-source metrics.
> >
> > This means it can also be used to focus existing stabilization efforts
> > to reduce the number of things being thrown in the face of manual
> > stabilizers.
> >
> > >
> > > If everything you're proposing is just on top of what we're already
> > > doing, then we have the issue that people aren't keeping up with the
> > > current workload, and even if that report is ultra-nice it is
> > > actually one more step than we have today.  The workload would only
> > > go down if a machine could look at the report and stabilize things
> > > without input at least some of the time.
> >
> > Indeed, it would require the crowd service to be automated, and the
> > relevant usage of the data as automated as possible, and humans would
> > only go looking at the data when interested.
> >
> > For instance, when somebody manually files a stable request, some
> > watcher could run off and scour the reports in a given window and
> > comment "Warning: Above threshold failure rates for target in last
> > n-days, proceed with caution", and it would only enhance the existing
> > stabilization workflow.
>
> This whole thing you are proposing has been a past stats project many
> times in GSOC for Gentoo.  The last time, it produced a decent system
> that was functional and __NEVER__ got deployed and turned on.  It ran
> for several years on the gentoo GSOC student server (vulture).  It
> never gained traction with the infra team due to lack of infra
> resources and infra personnel to maintain it.
>
> Perhaps with the new hardware recently purchased to replace the failed
> server from earlier this year, we should have some hardware resources.
> If you can dedicate some time to work on the code which I'm sure will
> need some updating now, I would help as well (not that I already can't
> keep up with all the project coding I'm involved with).
>
> This is of course if we can get a green light from our infra team to be
> able to implement a stats vm on the new ganeti system.
>
> We will also need some help from security people to ensure the system
> is secure, nginx/lightttp configuration, etc...
>
> So, are you up for it?  Any Gentoo dev willing to help admin such a
> system, please reply with your area of expertise and ability to help.
>
> Maybe we can finally get a working and deployed stats system.
>
> --
> Brian Dolbec <dolsen>
>
>
The similar GSoC project this year is in fact my project, named
[Continuous Stabilization]. I would be very interested to know what I
can do to ensure that the system is deployed and used this time.

[-- Attachment #2: Type: text/html, Size: 5510 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2016-08-14

[Brian Dolbec]

On Wed, 10 Aug 2016 06:45:16 +0530
Pallav Agarwal <pallavagarwal07@gmail.com> wrote:

> On Tue, Aug 9, 2016 at 8:11 PM, Brian Dolbec <dolsen@gentoo.org>
> wrote:
> 
> > On Tue, 9 Aug 2016 22:05:45 +1200
> > Kent Fredric <kentnl@gentoo.org> wrote:
> >  
> > > On Tue, 9 Aug 2016 01:59:35 -0400
> > > Rich Freeman <rich0@gentoo.org> wrote:
> > >  
> > > > While I think your proposal is a great one, I think this is
> > > > actually the biggest limitation.  A lot of our packages (most?)
> > > > don't actually have tests that can be run on every build (most
> > > > don't have tests, some have tests that take forever to run or
> > > > can't be used on a clean install).  
> > >
> > > IMHO, That's not "ideal", but we don't need idealism to be useful
> > > here.
> > >
> > > Tests passing give one kind of useful kind of quality test.
> > >
> > > But "hey, it compiles" gives useful data in itself.
> > >
> > > By easy counter example, "it doesn't compile" is in itself useful
> > > information ( and the predominant supply of bugs filed are
> > > compilation failures ).
> > >
> > > Hell, sometimes I hit a compile failure and I just go "eeh, I'll
> > > look into it next week". How many people are doing the same?
> > >
> > > The beauty of the automated datapoint is it doesn't have to be
> > > "awesome quality" to be useful, its just guidance for further
> > > investigation.  
> > > > While runtime testing doesn't HAVE to be extensive, we do want
> > > > somebody to at least take a glance at it.  
> > >
> > > Indeed, I'm not hugely in favour of abolishing manual
> > > stabilization entirely, but sometimes it just gets to a point
> > > where its a bit beyond a joke with requests languishing untouched
> > > for months.
> > >
> > > If there was even data saying "hey, look, its obvious this isn't
> > > ready for stabilization", we could *remove* or otherwise mark for
> > > postponement stabilization requests that were failing due to
> > > crowd-source metrics.
> > >
> > > This means it can also be used to focus existing stabilization
> > > efforts to reduce the number of things being thrown in the face
> > > of manual stabilizers.
> > >  
> > > >
> > > > If everything you're proposing is just on top of what we're
> > > > already doing, then we have the issue that people aren't
> > > > keeping up with the current workload, and even if that report
> > > > is ultra-nice it is actually one more step than we have today.
> > > > The workload would only go down if a machine could look at the
> > > > report and stabilize things without input at least some of the
> > > > time.  
> > >
> > > Indeed, it would require the crowd service to be automated, and
> > > the relevant usage of the data as automated as possible, and
> > > humans would only go looking at the data when interested.
> > >
> > > For instance, when somebody manually files a stable request, some
> > > watcher could run off and scour the reports in a given window and
> > > comment "Warning: Above threshold failure rates for target in last
> > > n-days, proceed with caution", and it would only enhance the
> > > existing stabilization workflow.  
> >
> > This whole thing you are proposing has been a past stats project
> > many times in GSOC for Gentoo.  The last time, it produced a decent
> > system that was functional and __NEVER__ got deployed and turned
> > on.  It ran for several years on the gentoo GSOC student server
> > (vulture).  It never gained traction with the infra team due to
> > lack of infra resources and infra personnel to maintain it.
> >
> > Perhaps with the new hardware recently purchased to replace the
> > failed server from earlier this year, we should have some hardware
> > resources. If you can dedicate some time to work on the code which
> > I'm sure will need some updating now, I would help as well (not
> > that I already can't keep up with all the project coding I'm
> > involved with).
> >
> > This is of course if we can get a green light from our infra team
> > to be able to implement a stats vm on the new ganeti system.
> >
> > We will also need some help from security people to ensure the
> > system is secure, nginx/lightttp configuration, etc...
> >
> > So, are you up for it?  Any Gentoo dev willing to help admin such a
> > system, please reply with your area of expertise and ability to
> > help.
> >
> > Maybe we can finally get a working and deployed stats system.
> >
> > --
> > Brian Dolbec <dolsen>
> >
> >  
> The similar GSoC project this year is in fact my project, named
> [Continuous Stabilization]. I would be very interested to know what I
> can do to ensure that the system is deployed and used this time.

Yes, I am familiar with your project.

The best way to ensure that it is set up and used is to become a gentoo
developer after GSOC is over and keep working on your code and pushing
to get it installed and running on a server.

The stats project when we can get it going could be used to assist your
code and help determine the stable candidates.  I don't remember if it
got implemented, but I proposed an optional/configurable method for the
stats project to make users aware of a questionnaire about a pkg they
have installed. That way a maintainer can ask some questions of users
of the pkg to help determine any issues that might prevent
stabilization (all anonymous of course, if the user chooses).

-- 
Brian Dolbec <dolsen>