[gentoo-python] Help needed with adding tests to Python packages

[gentoo-python] Help needed with adding tests to Python packages

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1823 bytes --]

Hi, everyone.

We're a fair bit behind with updating packages to support Python 3.5
[1,2]. I've been doing some testing-and-adding myself lately. Sadly, I
have to say that what was supposed to be a pretty simple experience
turned out a bumpy fix-a-lot-of-mistakes road.

Unlike what some people claim, updating PYTHON_COMPAT is not a problem.
Neither should be testing the changes. The major problem is bad quality
of multiple ebuilds that require immediate attention before python3.5
can be even added. The biggest issue is lack of tests.

While I can agree that sometimes tests in Python packages are poorly
written upstream or not present at all, there's a lot of packages where
lack of tests is purely the Gentoo maintainer's fault, and once added
they work seamlessly.

So if you have some time, please take a look at our Python packages,
especially those on the list for 3.5 testing ([1]), and enable test
support whenever possible. Even partially failing tests are better than
none.

In the matter of instructions, I would like to remind you that distutils
don't have a single method of running tests. Some upstreams support
the setuptools 'test' command, some require using specific test systems.
  Some tips regarding the common cases can be found on [3].

If you decide to add 3.5 to packages, please also test 3.6 to avoid
revisiting the package again soon. If the new implementations fail,
preferably please file a bug to ensure that we can track it.

Thanks in advance! Please also let me know if you can think of some ways
to improve our documentation.

[1]:https://qa-reports.gentoo.org/output/gpyutils/34-to-35.txt
[2]:https://qa-reports.gentoo.org/output/gpyutils/34-to-35.svg
[3]:https://wiki.gentoo.org/wiki/Project:Python/Tests

-- 
Best regards,
Michał Górny

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 963 bytes --]

Re: [gentoo-python] Help needed with adding tests to Python packages

[Mike Gilbert]

On Wed, Apr 26, 2017 at 3:06 PM, Michał Górny <mgorny@gentoo.org> wrote:
> Even partially failing tests are better than none.

While I personally agree with this, arch testers tend to give us a
hard time if the tests are failing when they process a keyword or
stable request.

Re: [gentoo-python] Help needed with adding tests to Python packages

[Patrick McLean]

On Wed, 26 Apr 2017 15:35:47 -0400
Mike Gilbert <floppym@gentoo.org> wrote:

> On Wed, Apr 26, 2017 at 3:06 PM, Michał Górny <mgorny@gentoo.org>
> wrote:
> > Even partially failing tests are better than none.  
> 
> While I personally agree with this, arch testers tend to give us a
> hard time if the tests are failing when they process a keyword or
> stable request.

Usually if there are tests that fail for reasons other than packaging
(like requirements on other distro stuff (I have encountered tests that
run 'apt-get' or 'yum') or running daemons), I try to patch them
out rather than have failing tests that can't reasonably be fixed. I
will also try to fix buggy tests, if it can be done with reasonable
effort.