Re: [gentoo-dev] Proposal for changes for the next EAPI version

[Rich Freeman <rich0@gentoo.org>]

On Tue, May 17, 2016 at 7:25 AM, Pallav Agarwal
<pallavagarwal07@gmail.com> wrote:
> Because we are already expecting an arch tester to conduct tests for the
> package. And knowing what to test is something I expect to come more
> easily from the maintainer.
>

It would come even more easily from upstream.  My point is that
upstream obviously doesn't consider it a priority, since if they did
they'd be already supplying automated build scripts and we'd already
be using them in the test phase.  Maintainers generally don't consider
them a priority either, since I imagine many existing upstream build
scripts aren't implemented, and when they don't exist maintainers
generally do not provide test plans.

Again, I'm not objecting to the creation of this feature.  I just
think that it is unlikely to have a large impact on how things get
done except perhaps for a couple packages or projects.  For those
packages/projects that do adopt this I'm sure there will be some real
benefits.

I'd also suggest having some way to break out short vs long tests.  It
might be beneficial to users to be able to run the tests on their own
systems and not just limit this to arch testing.  Obviously users are
going to be less likely to want to run test scripts that take hours to
run.  Due to the nature of Gentoo it is unlikely that the environment
used for testing by the AT is identical to the environment the
packages will be run in.  I'm not negating the value of testing prior
to stabilization, but I don't agree that it completely takes the place
of testing at time of installation and if we have automated test
scripts, why not make them available to users?

Of course, that would suggest that there is a potential benefit to be
had from putting this into an EAPI.  My suggestion would be to try to
design this so that it would be appropriate for a future EAPI, but
defer considering it for an EAPI until we see the feature getting
enough use/demand to warrant this.

-- 
Rich