[gentoo-dev] [RFC] Testing Gentoo with openQA

[gentoo-dev] [RFC] Testing Gentoo with openQA

[mudler]

[-- Attachment #1.1: Type: text/plain, Size: 944 bytes --]

Hi everyone,


This is an idea that has been floating around already, as some of you
already know, openQA [1] is a project that provides the ability to fully
test  O.S.

I'd like to bring openQA to Gentoo, and start using it for both testing,
and maybe, if promising at it seems, using it to automatize even more
our testing workflow and help stabilization processes.

I'm creating a new project for this, the aim is first to provide ebuilds
for openQA, and meanwhile writing test cases that can be used to test
Gentoo inside it. As i see it, it would be really helpful exp. regarding
testing DEs  and track back regressions as well [2], it offers also
integrations with bugzilla.

Development plan and goals will be updated to the Project page soon [3]


What do you think?


Best,

Ettore


1: http://open.qa/

2: https://openqa.opensuse.org/tests/600283

3: https://wiki.gentoo.org/wiki/Project:OpenQA



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 1006 bytes --]

Re: [gentoo-dev] [RFC] Testing Gentoo with openQA

[Francesco Riosa]

[-- Attachment #1: Type: text/plain, Size: 1516 bytes --]

2018-02-03 13:16 GMT+01:00 mudler <mudler@gentoo.org>:

> Hi everyone,
>
>
> This is an idea that has been floating around already, as some of you
> already know, openQA [1] is a project that provides the ability to fully
> test  O.S.
>

Being used to routinely build Gentoo profiles, this idea is one of the
first to pop up, indeed.


>
> I'd like to bring openQA to Gentoo, and start using it for both testing,
> and maybe, if promising at it seems, using it to automatize even more
> our testing workflow and help stabilization processes.
>

simply Great.


>
> I'm creating a new project for this, the aim is first to provide ebuilds
> for openQA, and meanwhile writing test cases that can be used to test
> Gentoo inside it. As i see it, it would be really helpful exp. regarding
> testing DEs  and track back regressions as well [2], it offers also
> integrations with bugzilla.
>

One of the first things noticed when trying to achieve full testing is that
it's a gigantic work. Suse seem to have a really good framework that
surely costed a lot to build.
How much can be reused of that work?
What are the differences, in features and building blocks expected for a
gentoo implementation?


>
> Development plan and goals will be updated to the Project page soon [3]
>
>
> What do you think?
>
>
That's a difficult objective worth the effort to achieve it


>
> Best,
>
> Ettore
>
>
> 1: http://open.qa/
>
> 2: https://openqa.opensuse.org/tests/600283
>
> 3: https://wiki.gentoo.org/wiki/Project:OpenQA
>
>
>

[-- Attachment #2: Type: text/html, Size: 2782 bytes --]

Re: [gentoo-dev] [RFC] Testing Gentoo with openQA

[mudler]

[-- Attachment #1.1: Type: text/plain, Size: 2188 bytes --]

On 04/02/2018 11:16, Francesco Riosa wrote:
> 2018-02-03 13:16 GMT+01:00 mudler <mudler@gentoo.org>:
>
>> Hi everyone,
>>
>>
>> This is an idea that has been floating around already, as some of you
>> already know, openQA [1] is a project that provides the ability to fully
>> test  O.S.
>>
> Being used to routinely build Gentoo profiles, this idea is one of the
> first to pop up, indeed.
>
>
>> I'd like to bring openQA to Gentoo, and start using it for both testing,
>> and maybe, if promising at it seems, using it to automatize even more
>> our testing workflow and help stabilization processes.
>>
> simply Great.

Glad to hear that!

>
>
>> I'm creating a new project for this, the aim is first to provide ebuilds
>> for openQA, and meanwhile writing test cases that can be used to test
>> Gentoo inside it. As i see it, it would be really helpful exp. regarding
>> testing DEs  and track back regressions as well [2], it offers also
>> integrations with bugzilla.
>>
> One of the first things noticed when trying to achieve full testing is that
> it's a gigantic work. Suse seem to have a really good framework that
> surely costed a lot to build.
> How much can be reused of that work?

I would say approx ~30% of test codebase could be reused, but this is
just my personal estimation.


> What are the differences, in features and building blocks expected for a
> gentoo implementation?

I don't see any big changes in the structure - rather of writing tests
that suits more on our needs.
But baby steps, i would be happy to see for now a test that from a
stage3 bootstraps a DE environment correctly.
What i see will be more complicated is trying to abstract tests that
expect different results from different useflags combinations.

>
>
>> Development plan and goals will be updated to the Project page soon [3]
>>
>>
>> What do you think?
>>
>>
> That's a difficult objective worth the effort to achieve it

I definitely agree

>
>
>> Best,
>>
>> Ettore
>>
>>
>> 1: http://open.qa/
>>
>> 2: https://openqa.opensuse.org/tests/600283
>>
>> 3: https://wiki.gentoo.org/wiki/Project:OpenQA
>>
>>
>>



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 1006 bytes --]