[gentoo-dev] Tools to help QA

[gentoo-dev] Tools to help QA

[Stuart Herbert]

[-- Attachment #1: Type: text/plain, Size: 1785 bytes --]

Hiya,

Post net-www breakup, I'm going to put some time into writing some tools to 
help me keep an eye on what goes on in the new www-* categories.  I'm 
interested in hearing from other devs as to what useful tools we need. 

Mr_Bones_, this is your chance to have the work you do made a little easier, 
so don't be shy ;-)

Volunteers to help code them up would ofc be very much appreciated too ;-)

Technical constraints that I'm working to are:

a) the tools should be written in bash or python
b) the tools should run from the command-line

To kick off the discussion ...

My motivation is that net-www became quite the dumping ground of packages that 
were abandoned over time, or worse still were added without working with the 
relevant herd.  Trying to spot these things by hand is both silly and futile; 
automation is the only way to cope with how large Gentoo has become.

I'm interested in writing tools to check that packages have metadata.xml, that 
the metadata lists the 'right' herd, lists a primary maintainer, and that the 
primary maintainer is still a valid Gentoo developer.

I'm also interested in tools that check that the HOMEPAGE of a package is 
valid, that report on how many bugs are open against a specific package or 
set of packages, and that report on which packages in a given category have 
changed since a given date.

Best regards,
Stu
-- 
Stuart Herbert                                              stuart@gentoo.org
Gentoo Developer                                       http://www.gentoo.org/
                                                   http://stu.gnqs.org/diary/

GnuPG key id# F9AFC57C available from http://pgp.mit.edu
Key fingerprint = 31FB 50D4 1F88 E227 F319  C549 0C2F 80BA F9AF C57C
--

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Tools to help QA

[Guy Martin]

[-- Attachment #1: Type: text/plain, Size: 476 bytes --]


Hello,

Not sure if someone need this tool but I wrote it some time ago and
hasn't really been used.

http://dev.gentoo.org/~gmsoft/tools/qa/checkdup/

This tool check if one emerged file is in more than one CONTENTS file.
It's (quickly/badly:) written in C because bash wasn't fast enough and I
don't know python.

To compile, `gcc -o checkdup main.c filelist.c`.

I hope it will be usefull.

-- 
Guy Martin
Gentoo Linux - HPPA port Lead / IPv6 team
Lug Charleroi (Belgium)

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Tools to help QA

[Eric Olinger]

[-- Attachment #1: Type: text/plain, Size: 666 bytes --]

I've done some coding on similar tools and can help, if you'd like me
to. I only have a few questions.

As far as pulling information Bugzilla, whats the best way to
interface python code with Bugzilla? I tried searching but only came up
with a few dead projects and nothing really useful.

Second is there a xml or text list of all the developers to check the
maintainers in the metadata.xml files against?


-- 
Eric Olinger

Public Key: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xF90FBBC1
Key fingerprint: B678 9E22 1161 51CF 6664 7591 6767 5BDB F90F BBC1

Give a man a password, he'll log in for a day. Teach him to code, and he
will hack his way in...

[-- Attachment #2: Type: application/pgp-signature, Size: 187 bytes --]

Re: [gentoo-dev] Tools to help QA

[Stuart Herbert]

[-- Attachment #1: Type: text/plain, Size: 1654 bytes --]

On Thursday 24 June 2004 23:18, Eric Olinger wrote:
> I've done some coding on similar tools and can help, if you'd like me
> to. I only have a few questions.

Great ;-)

> As far as pulling information Bugzilla, whats the best way to
> interface python code with Bugzilla? I tried searching but only came up
> with a few dead projects and nothing really useful.

I'd assume that any interfacing with Bugzilla will have to be done via GET 
requests over http.  Google should be able to turn up some documentation on 
the URL parameters that work with Bugzilla.

I should say that, when I get the chance to talk to sj7trunks, I'm looking 
into a change to our Bugzilla that would make life a lot easier.  Basically, 
I'm interested in users being able to assign a bug against a specific package 
when the bug is submitted.  At the moment, it looks like we'd add the package 
name to the Keywords field of a bug.

> Second is there a xml or text list of all the developers to check the
> maintainers in the metadata.xml files against?

There's supposed to be one somewhere on gentoo.org, but I can't find it.  To 
be honest, I'd be happy to handle this by caching /etc/passwd off of 
dev.gentoo.org, and checking devs against that.

Best regards,
Stu
-- 
Stuart Herbert                                              stuart@gentoo.org
Gentoo Developer                                       http://www.gentoo.org/
                                                   http://stu.gnqs.org/diary/

GnuPG key id# F9AFC57C available from http://pgp.mit.edu
Key fingerprint = 31FB 50D4 1F88 E227 F319  C549 0C2F 80BA F9AF C57C
--

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Tools to help QA

[Stuart Herbert]

[-- Attachment #1: Type: text/plain, Size: 526 bytes --]

More as a reminder to me really ...

A tool to check one or more packages, and report if any of the digests are 
invalid.  

Best regards,
Stu
-- 
Stuart Herbert                                              stuart@gentoo.org
Gentoo Developer                                       http://www.gentoo.org/
                                                   http://stu.gnqs.org/diary/

GnuPG key id# F9AFC57C available from http://pgp.mit.edu
Key fingerprint = 31FB 50D4 1F88 E227 F319  C549 0C2F 80BA F9AF C57C
--

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Tools to help QA

[Kurt Lieber]

[-- Attachment #1: Type: text/plain, Size: 596 bytes --]

On Fri, Jun 25, 2004 at 12:06:07AM +0100 or thereabouts, Stuart Herbert wrote:
> I'd assume that any interfacing with Bugzilla will have to be done via GET 
> requests over http.  Google should be able to turn up some documentation on 
> the URL parameters that work with Bugzilla.

This is something of a hack and, as we found out the hard way with
packages.gentoo.org, can quickly bring bugzilla to its knees.

Depending on what you're looking to do, I'm more than happy to facilitate
direct access to the bugzilla database.  This is preferable for read-only
connections. 

--kurt

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Tools to help QA

[Christopher Korn]

a read only access to the bugzilla database would be the best thing like
kurt said. the bugzilla-get method is really not the cleanest way and a
direct-databaseaccess via php or python is much cleaner and more beautiful.

also a good tool for the QA team would be something like a feedback-option
in the portage script. i mean: if "emerge apache" fails a user should be
able to report a feedback direktly via portage to the bugzilla or better the
qa team. the qa-team then probably can do more checks and verify the
failure. 

greetings, chris

-- 
"Sie haben neue Mails!" - Die GMX Toolbar informiert Sie beim Surfen!
Jetzt aktivieren unter http://www.gmx.net/info


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Tools to help QA

[Markus Nigbur]

On Fri, June 25, 2004 8:17, Christopher Korn said:
> a read only access to the bugzilla database would be the best thing like
> kurt said. the bugzilla-get method is really not the cleanest way and a
> direct-databaseaccess via php or python is much cleaner and more
> beautiful.

Or a daily|hourly|whatever database dump you can fetch from the mirrors
and work with.

> also a good tool for the QA team would be something like a feedback-option
> in the portage script. i mean: if "emerge apache" fails a user should be
> able to report a feedback direktly via portage to the bugzilla or better
> the
> qa team. the qa-team then probably can do more checks and verify the
> failure.

That's a frequent request, but the general opinion is - bugzilla serves
all needs.

-- 
 (o_  Markus Nigbur
 //\  Gentoo Linux Developer
[ ]/_ http://www.gentoo.org


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Tools to help QA

[Christopher Korn]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="us-ascii", Size: 810 bytes --]

> Or a daily|hourly|whatever database dump you can fetch from the mirrors
> and work with.

yeah. but no text-dump if  possible. a complete databasemirror would be
better. searching in a database is much more efficient than searching
sequential in a text-file.

> That's a frequent request, but the general opinion is - bugzilla serves
> all needs.

hm. maybe. but i really think that many users are too lazy to report some
bugs and so the fixing-time for some bugs is really high because no one
know's them for a 'long' time.
but that's just my opinion.



if you need some python-code-monkeys for some of the discussed. Here I am.



-- 
+++ Jetzt WLAN-Router für alle DSL-Einsteiger und Wechsler +++
GMX DSL-Powertarife zudem 3 Monate gratis* http://www.gmx.net/dsl


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Tools to help QA

[Eric Olinger]

[-- Attachment #1: Type: text/plain, Size: 993 bytes --]


Well, I might as well throw this one to the wolfs. I still have some
work to do on it but most of the features are there and its probably due
for a name change.

QATool 

Homepage: http://evvl.rustedhalo.net/projects/gentoo-linux/qatool/

Requirements:
	pyxml

Usage:
	qatool category/package 
		or
	qatool package

Features:
	* Lists the herd
	* Lists the primary maintainer and errors if none
	* Verifies the homepage is still valid
	* If nomirror is enabled then it checks the SRC_URI
	* It checks the digests

Todo:
	* Check that the primary maintainer is still an active dev
	* Check the other listed maintainers are still active
	* List number of bugs
	* Auto-fetch files for digest verifying 
	* Add flags for turning test off


-- 
Eric Olinger

Public Key: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xF90FBBC1
Key fingerprint: B678 9E22 1161 51CF 6664 7591 6767 5BDB F90F BBC1

Give a man a password, he'll log in for a day. Teach him to code, and he
will hack his way in...

[-- Attachment #2: Type: application/pgp-signature, Size: 187 bytes --]

Re: [gentoo-dev] Tools to help QA

[Stuart Herbert]

[-- Attachment #1: Type: text/plain, Size: 1138 bytes --]

On Friday 25 June 2004 07:44, Markus Nigbur wrote:
> > also a good tool for the QA team would be something like a
> > feedback-option in the portage script. i mean: if "emerge apache" fails a
> > user should be able to report a feedback direktly via portage to the
> > bugzilla or better the
> > qa team. the qa-team then probably can do more checks and verify the
> > failure.
>
> That's a frequent request, but the general opinion is - bugzilla serves
> all needs.

I like the idea of this tool, and the idea of reporting automatically into 
bugzilla.  If we have bug voting (like bugs.kde.org), you could combine this 
to automatically produce interesting stats and reports about the problems 
that are actually occuring.

Best regards,
Stu
-- 
Stuart Herbert                                              stuart@gentoo.org
Gentoo Developer                                       http://www.gentoo.org/
                                                   http://stu.gnqs.org/diary/

GnuPG key id# F9AFC57C available from http://pgp.mit.edu
Key fingerprint = 31FB 50D4 1F88 E227 F319  C549 0C2F 80BA F9AF C57C
--

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Tools to help QA

[Michael Cummings]

[-- Attachment #1: Type: text/plain, Size: 569 bytes --]

Better late than never with my responses...


On Thu, Jun 24, 2004 at 03:18:38PM -0700, Eric Olinger wrote:
> As far as pulling information Bugzilla, whats the best way to
> interface python code with Bugzilla? I tried searching but only came up
> with a few dead projects and nothing really useful.

I'm still working on code clean ups of a perl based package that lets you
search, update, close, add, the whole 9+ yards, with bugzilla. Commandline
and tk interfaces available. Ping me offline if your interested at all, I
just have a few clean ups left to do.

-Mike

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Tools to help QA

[Rob Cakebread]

Michael Cummings wrote:
> Better late than never with my responses...
> 
> 
> On Thu, Jun 24, 2004 at 03:18:38PM -0700, Eric Olinger wrote:
> 
>>As far as pulling information Bugzilla, whats the best way to
>>interface python code with Bugzilla? I tried searching but only came up
>>with a few dead projects and nothing really useful.
> 

I did a little of this in app-portage/abeni-0.0.9, which uses 
dev-python/clientform to search, post bugs and attachments to Gentoo's 
bugzilla.

Here's a few examples:
http://dev.gentoo.org/~pythonhead/bugzilla/

> 
> I'm still working on code clean ups of a perl based package that lets you
> search, update, close, add, the whole 9+ yards, with bugzilla. Commandline
> and tk interfaces available. Ping me offline if your interested at all, I
> just have a few clean ups left to do.
> 
> -Mike




-- 
Rob Cakebread
Gentoo Linux Developer
Public Key: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x96BA679B
Key fingerprint = 5E1A 57A0 0FA6 939D 3258  8369 81C5 A17B 96BA 679B

--
gentoo-dev@gentoo.org mailing list