[gentoo-dev] bug queue size over time

[gentoo-dev] bug queue size over time

[Paweł Hajdan, Jr.]

[-- Attachment #1.1: Type: text/plain, Size: 125 bytes --]

Is it possible to get graphs of bugs.g.o bug queue size for certain
query (e.g. by assignee) over time?

Best,
Paweł


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 827 bytes --]

Re: [gentoo-dev] bug queue size over time

[Kent Fredric]

[-- Attachment #1: Type: text/plain, Size: 1086 bytes --]

On Mon, 19 Mar 2018 19:33:11 -0700
"Paweł Hajdan, Jr." <phajdan.jr@gentoo.org> wrote:

> Is it possible to get graphs of bugs.g.o bug queue size for certain
> query (e.g. by assignee) over time?
> 
> Best,
> Paweł
> 

The *data* is there to do it, but its a bit of a pain, you have to
extract all the individual "changed" events and then use that to reason
about each individual bugs state at a given time, and use *that* to
deduce how many open bugs there *were* at a historical moment.

And that's a lot of painful queries for the REST API.

I've done something like this before with Perl bugs[1], but again, very
painful, slow and time consuming.

This sort of thing would be *much* easier if we could have direct bulk
access to the underlying MYSQL store, but that's a tricky thing to do because:

1. Its MySQL
2. Some bugs have visibility restrictions that have to be factored for

----

Note: these pages are very browser taxing:

1: https://docs.google.com/spreadsheets/d/1mm8iYE77SRh-q2jOfKNSWHUetswEABJawp-94UyTHio/edit?usp=sharing
  





[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] bug queue size over time

[Alec Warner]

[-- Attachment #1: Type: text/plain, Size: 1732 bytes --]

On Tue, Mar 20, 2018 at 12:24 AM, Kent Fredric <kentnl@gentoo.org> wrote:

> On Mon, 19 Mar 2018 19:33:11 -0700
> "Paweł Hajdan, Jr." <phajdan.jr@gentoo.org> wrote:
>
> > Is it possible to get graphs of bugs.g.o bug queue size for certain
> > query (e.g. by assignee) over time?
> >
> > Best,
> > Paweł
> >
>
> The *data* is there to do it, but its a bit of a pain, you have to
> extract all the individual "changed" events and then use that to reason
> about each individual bugs state at a given time, and use *that* to
> deduce how many open bugs there *were* at a historical moment.
>
> And that's a lot of painful queries for the REST API.
>

I'd avoid the REST API here. If you want this data; I'd consider filing a
bug. Infra can do stuff like run nightly reports for this information and
hang them off of endpoints you can access.
This works well for public bugs; and not well for private ones.


>
> I've done something like this before with Perl bugs[1], but again, very
> painful, slow and time consuming.
>
> This sort of thing would be *much* easier if we could have direct bulk
> access to the underlying MYSQL store, but that's a tricky thing to do
> because:
>
> 1. Its MySQL
> 2. Some bugs have visibility restrictions that have to be factored for
>

In this way, you can download the (likely hundreds of megs) of JSON or
whatever, and do the sorting / filtering / timeseries work on the client
end?

Its not great I suspect, but it saves everyone hamming the database.

-A

>
> ----
>
> Note: these pages are very browser taxing:
>
> 1: https://docs.google.com/spreadsheets/d/1mm8iYE77SRh-
> q2jOfKNSWHUetswEABJawp-94UyTHio/edit?usp=sharing
>
>
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 2748 bytes --]

Re: [gentoo-dev] bug queue size over time

[Matthew Thode]

[-- Attachment #1: Type: text/plain, Size: 241 bytes --]

On 18-03-19 19:33:11, Paweł Hajdan, Jr. wrote:
> Is it possible to get graphs of bugs.g.o bug queue size for certain
> query (e.g. by assignee) over time?
> 

I suspect it's up and to the right.

-- 
Matthew Thode (prometheanfire)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] bug queue size over time

[Paweł Hajdan, Jr.]

[-- Attachment #1.1: Type: text/plain, Size: 853 bytes --]

On 19/03/2018 21:33, Alec Warner wrote:
> I'd avoid the REST API here. If you want this data; I'd consider filing a
> bug. Infra can do stuff like run nightly reports for this information and
> hang them off of endpoints you can access.
> This works well for public bugs; and not well for private ones.

Luckily, I'm not that concerned with private bugs.

Would it only collect data going forward, or does this method also
support historical backfill?

>> I've done something like this before with Perl bugs[1], but again, very
>> painful, slow and time consuming.

Ah, that looks like lots of stats.

I'm mostly interested in how many bugs were open for given assignee for
each point in time.

If you still have scripts that generated these reports and they could be
useful to compute the above, I could be interested.

Paweł


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 827 bytes --]

Re: [gentoo-dev] bug queue size over time

[James Le Cuirot]

On Mon, 19 Mar 2018 22:05:16 -0700
"Paweł Hajdan, Jr." <phajdan.jr@gentoo.org> wrote:

> On 19/03/2018 21:33, Alec Warner wrote:
> > I'd avoid the REST API here. If you want this data; I'd consider
> > filing a bug. Infra can do stuff like run nightly reports for this
> > information and hang them off of endpoints you can access.
>
> Would it only collect data going forward, or does this method also
> support historical backfill?

We used to have graphs for Java bugs from data that was collected over
time but that all broke when the categories were reorganised. I recall
it was possible to set these up through the web interface but I've
totally forgotten how.

-- 
James Le Cuirot (chewi)
Gentoo Linux Developer

Re: [gentoo-dev] bug queue size over time

[Paweł Hajdan, Jr.]

[-- Attachment #1.1: Type: text/plain, Size: 1229 bytes --]

On 20/03/2018 06:05, Paweł Hajdan, Jr. wrote:
> On 19/03/2018 21:33, Alec Warner wrote:
>> I'd avoid the REST API here. If you want this data; I'd consider filing a
>> bug. Infra can do stuff like run nightly reports for this information and
>> hang them off of endpoints you can access.
>> This works well for public bugs; and not well for private ones.
> 
> Luckily, I'm not that concerned with private bugs.
> 
> Would it only collect data going forward, or does this method also
> support historical backfill?
> 
>>> I've done something like this before with Perl bugs[1], but again, very
>>> painful, slow and time consuming.
> 
> Ah, that looks like lots of stats.
> 
> I'm mostly interested in how many bugs were open for given assignee for
> each point in time.
> 
> If you still have scripts that generated these reports and they could be
> useful to compute the above, I could be interested.

Just checking back here - what would be the best way to graph number of
bugs with given assignee, preferably with historical backfill?

I'm not necessarily looking for something ready to use right away. If
there's some work to be done or code to implement, I may be willing to
do so.

Paweł


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 827 bytes --]

Re: [gentoo-dev] bug queue size over time

[Andreas K. Huettel]

[-- Attachment #1: Type: text/plain, Size: 895 bytes --]

Am Mittwoch, 2. Mai 2018, 18:42:32 CEST schrieb Paweł Hajdan, Jr.:
> 
> Just checking back here - what would be the best way to graph number of
> bugs with given assignee, preferably with historical backfill?
> 
> I'm not necessarily looking for something ready to use right away. If
> there's some work to be done or code to implement, I may be willing to
> do so.
> 
> Paweł

Hi Pawel, 

some graphs already exist (and I'm doing it the dumb way, ask me about the 
details if interested). E.g., 
https://www.akhuettel.de/gentoo-bugs/arches.php

Feel free to send me any e-mail addresses you want on a similar page; it's 
fairly easy to add. I'm only adding fresh data, though, so if I dont have it 
yet your line will start today.

Cheers, 
Andreas

-- 
Andreas K. Hüttel
dilfridge@gentoo.org
Gentoo Linux developer
(council, toolchain, perl, libreoffice, comrel)

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

Re: [gentoo-dev] bug queue size over time

[Matt Turner]

On Wed, May 2, 2018 at 10:24 AM, Andreas K. Huettel
<dilfridge@gentoo.org> wrote:
> Am Mittwoch, 2. Mai 2018, 18:42:32 CEST schrieb Paweł Hajdan, Jr.:
>>
>> Just checking back here - what would be the best way to graph number of
>> bugs with given assignee, preferably with historical backfill?
>>
>> I'm not necessarily looking for something ready to use right away. If
>> there's some work to be done or code to implement, I may be willing to
>> do so.
>>
>> Paweł
>
> Hi Pawel,
>
> some graphs already exist (and I'm doing it the dumb way, ask me about the
> details if interested). E.g.,
> https://www.akhuettel.de/gentoo-bugs/arches.php
>
> Feel free to send me any e-mail addresses you want on a similar page; it's
> fairly easy to add. I'm only adding fresh data, though, so if I dont have it
> yet your line will start today.

Please add mips@ and x11@. Thank you!

Re: [gentoo-dev] bug queue size over time

[Andreas K. Huettel]

Am Mittwoch, 2. Mai 2018, 19:45:21 CEST schrieb Matt Turner:
> > 
> > some graphs already exist (and I'm doing it the dumb way, ask me about the
> > details if interested). E.g.,
> > https://www.akhuettel.de/gentoo-bugs/arches.php
> > 
> > Feel free to send me any e-mail addresses you want on a similar page; it's
> > fairly easy to add. I'm only adding fresh data, though, so if I dont have
> > it yet your line will start today.
> 
> Please add mips@ and x11@. Thank you!

mips: is now added to the arches page (as is m68k, sh, s390, sparc), just 
scroll down for the experimental ones
https://www.akhuettel.de/gentoo-bugs/arches.php

x11:
https://www.akhuettel.de/gentoo-bugs/x11.php


-- 
Andreas K. Hüttel
dilfridge@gentoo.org
Gentoo Linux developer 
(council, toolchain, perl, libreoffice, comrel)