[gentoo-dev] Handling installed tests

[gentoo-dev] Handling installed tests

[Sam James]

[-- Attachment #1: Type: text/plain, Size: 859 bytes --]

Hi,

I'm currently working on packaging dtrace which doesn't (at least
currently, may not ever) support running tests as non-root, but
does support handling installing them for later manual use.

This raises a question: how should we control installing such tests? How
should the user request that?

USE=test isn't suitable because:
a) I generally expect it not to mutate the image;
b) ago, rightly, based on this has a tinderbox check which looks for
added/removed files when tests are enabled.

I'd like to pick some name which is suitable for us to use elsewhere and
it's not really a package-specific issue. Thoughts?

Please assume the tests are large enough to justify conditional install
and have additional dependencies.

(This is arguably related to https://bugs.gentoo.org/867799 and so on
wrt dev-lang/python always installing tests.)

thanks,
sam

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 377 bytes --]

Re: [gentoo-dev] Handling installed tests

[Eli Schwartz]

[-- Attachment #1.1: Type: text/plain, Size: 1110 bytes --]

On 8/9/24 11:43 AM, Sam James wrote:
> Hi,
> 
> I'm currently working on packaging dtrace which doesn't (at least
> currently, may not ever) support running tests as non-root, but
> does support handling installing them for later manual use.
> 
> This raises a question: how should we control installing such tests? How
> should the user request that?
> 
> USE=test isn't suitable because:
> a) I generally expect it not to mutate the image;
> b) ago, rightly, based on this has a tinderbox check which looks for
> added/removed files when tests are enabled.
> 
> I'd like to pick some name which is suitable for us to use elsewhere and
> it's not really a package-specific issue. Thoughts?
> 
> Please assume the tests are large enough to justify conditional install
> and have additional dependencies.
> 
> (This is arguably related to https://bugs.gentoo.org/867799 and so on
> wrt dev-lang/python always installing tests.)


This reminds me of
https://wiki.gnome.org/Initiatives/GnomeGoals/InstalledTests so it's
definitely not a dtrace specific issue.


-- 
Eli Schwartz


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

Re: [gentoo-dev] Handling installed tests

[Maciej Barć]

[-- Attachment #1.1.1: Type: text/plain, Size: 1621 bytes --]

Hi Sam,

We have a "source" flag so maybe we could either use that or have 
"test-source" BUT we also have FEATURES="installsources" ...so (if I 
understand that feature correctly) we can just run test from 
/usr/src/debug/${CATEGORY}/${PF}

But IMO I'd rather clone the project and run test from the clone.

Of source the drawback of installsources & git cloning is that emerge 
does not run any modifications on the tests nor applies patches to them 
nor buildsystem/testrunner used.

W dniu 9.08.2024 o 17:43, Sam James pisze:
> Hi,
> 
> I'm currently working on packaging dtrace which doesn't (at least
> currently, may not ever) support running tests as non-root, but
> does support handling installing them for later manual use.
> 
> This raises a question: how should we control installing such tests? How
> should the user request that?
> 
> USE=test isn't suitable because:
> a) I generally expect it not to mutate the image;
> b) ago, rightly, based on this has a tinderbox check which looks for
> added/removed files when tests are enabled.
> 
> I'd like to pick some name which is suitable for us to use elsewhere and
> it's not really a package-specific issue. Thoughts?
> 
> Please assume the tests are large enough to justify conditional install
> and have additional dependencies.
> 
> (This is arguably related to https://bugs.gentoo.org/867799 and so on
> wrt dev-lang/python always installing tests.)
> 
> thanks,
> sam

-- 
Have a great day!

~ Maciej XGQT Barć

https://wiki.gentoo.org/wiki/User:Xgqt
9B0A 4C5D 02A3 B43C 9D6F D6B1 14D7 4A1F 43A6 AC3C

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 16315 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [gentoo-dev] Handling installed tests

[Eli Schwartz]

[-- Attachment #1.1: Type: text/plain, Size: 1205 bytes --]

On 8/9/24 11:59 AM, Maciej Barć wrote:
> Hi Sam,
> 
> We have a "source" flag so maybe we could either use that or have
> "test-source" BUT we also have FEATURES="installsources" ...so (if I
> understand that feature correctly) we can just run test from
> /usr/src/debug/${CATEGORY}/${PF}
> 
> But IMO I'd rather clone the project and run test from the clone.
> 
> Of source the drawback of installsources & git cloning is that emerge
> does not run any modifications on the tests nor applies patches to them
> nor buildsystem/testrunner used.


installsources is not the same thing at all. That parses ELF files to
extract .c / .cpp etc source filenames from debuginfo to save them.

This thread is asking about the idea of compiling test programs and
installing the test executables so that you can run them later. You
absolutely cannot do that with a directory of .c files, for numerous
reasons including the fact that it only saves sources for files which
get installed anyway, but also because it does not install build system
files (meson.build, Makefile, CMakeLists.txt, etc).

P.S. See my gnome documentation link, it talks about git cloning too.

-- 
Eli Schwartz


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

Re: [gentoo-dev] Handling installed tests

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 924 bytes --]

On Fri, 2024-08-09 at 16:43 +0100, Sam James wrote:
> Hi,
> 
> I'm currently working on packaging dtrace which doesn't (at least
> currently, may not ever) support running tests as non-root, but
> does support handling installing them for later manual use.
> 
> This raises a question: how should we control installing such tests? How
> should the user request that?
> 
> USE=test isn't suitable because:
> a) I generally expect it not to mutate the image;
> b) ago, rightly, based on this has a tinderbox check which looks for
> added/removed files when tests are enabled.
> 
> I'd like to pick some name which is suitable for us to use elsewhere and
> it's not really a package-specific issue. Thoughts?
> 
> Please assume the tests are large enough to justify conditional install
> and have additional dependencies.
> 

Sounds a bit related to USE=examples.

-- 
Best regards,
Michał Górny


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

Re: [gentoo-dev] Handling installed tests

[Joonas Niilola]

[-- Attachment #1.1: Type: text/plain, Size: 349 bytes --]

Hey,

On 9.8.2024 18.43, Sam James wrote:
> 
> I'd like to pick some name which is suitable for us to use elsewhere and
> it's not really a package-specific issue. Thoughts?
> 

the first simple thought that came to my mind: "install-tests" or
"install-test-files". Both are available and should be pretty
self-explanatory.

-- juippis

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 618 bytes --]

Re: [gentoo-dev] Handling installed tests

[Sam James]

[-- Attachment #1: Type: text/plain, Size: 842 bytes --]

Joonas Niilola <juippis@gentoo.org> writes:

> Hey,
>
> On 9.8.2024 18.43, Sam James wrote:
>> 
>> I'd like to pick some name which is suitable for us to use elsewhere and
>> it's not really a package-specific issue. Thoughts?
>> 
>
> the first simple thought that came to my mind: "install-tests" or
> "install-test-files". Both are available and should be pretty
> self-explanatory.

Thoughts on prefix/suffix?

"test-install", "test-install-files" are more in-keeping with USE=test,
USE=test-full, etc, but maybe that's *bad* because it's not related
at all to what the ebuild does, so we should do install-* like you've
suggested?

It's been in the back of my head so wanted to bring it up before
committing to install-*. I think it's the better option indeed but want
to check.

>
> -- juippis

thanks,
sam

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 377 bytes --]

Re: [gentoo-dev] Handling installed tests

[Joonas Niilola]

[-- Attachment #1.1: Type: text/plain, Size: 581 bytes --]

Hey,

On 18.8.2024 22.06, Sam James wrote:
> 
> Thoughts on prefix/suffix?
> 
> "test-install", "test-install-files" are more in-keeping with USE=test,
> USE=test-full, etc, but maybe that's *bad* because it's not related
> at all to what the ebuild does, so we should do install-* like you've
> suggested?
> 

My instant thought was that "install*" prefix is more accurate and
easier to understand. But "test*" makes it more consistent with other
test-related stuff. No hard opinion either way, I'd prefer "install*"
just because it's clearer to me.

-- juippis

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 618 bytes --]