On wto, 2017-07-25 at 22:59 +1000, Michael Palimaka wrote:
> On 07/25/2017 07:22 AM, Sergei Trofimovich wrote:
> > 2. Q: How to make arch testing faster and easier?
> > 
> >    A: - KEYWORDREQ/STABLEREQ bugs not marked as "runtime testing
> >         required" will be automatically tested and keyworded.
> > 
> >         [handwave] automated tinderbox setup would help a lot
> >         to now upfront what fails to built, fails tests.
> 
> I've had similar thoughts about this and have already been working on
> some tooling for this.
> 
> We would need to establish exactly what criteria must be met for an
> automated test to be considered as successful.
> 
> Here's a sample report that my tool produces:
> https://dev.gentoo.org/~kensington/tinderbox/sys-apps/dbus/dbus-1.6.18-r1/df017e14-bd68-47e2-9738-554e7ba1cf10.html
> 
> In this case, would it be enough that it builds and tests pass? What
> about the QA issues? Do we need someone to review them to determine if
> they should block stabilisation, or if they're even a regression or not?
> 

There are different kinds of QA issues and they have different
significance. You can ignore some of them, some others are more
important.

GLEP 65 defines a 'eqatag' function that the checks can use to provide
machine-readable results for the checks. You should work with that
instead of parsing the verbose messages.

-- 
Best regards,
Michał Górny