[gentoo-releng] Autobuilds go to /experimental and to /releases only when someone actually tests them?

[gentoo-releng] Autobuilds go to /experimental and to /releases only when someone actually tests them?

[Matt Turner]

Can we make autobuilds go to /experimental and then only move them to
/releases when someone actually tests them?

Looking at bugzilla and listening in #gentoo-releng, it's kind of
embarrassing how often someone downloads a Live CD only to find out
that networking is totally broken by a udev upgrade, or something to
that effect.

We don't commit version bumps straight to stable; I don't see why we
do with release media.

Matt

Re: [gentoo-releng] Autobuilds go to /experimental and to /releases only when someone actually tests them?

[Anthony G. Basile]

On 07/26/2013 12:24 AM, Matt Turner wrote:
> Can we make autobuilds go to /experimental and then only move them to
> /releases when someone actually tests them?
>
> Looking at bugzilla and listening in #gentoo-releng, it's kind of
> embarrassing how often someone downloads a Live CD only to find out
> that networking is totally broken by a udev upgrade, or something to
> that effect.
>
> We don't commit version bumps straight to stable; I don't see why we
> do with release media.
>
> Matt
>

Matt, I assume you are talking about the stages that normally go 
directly to /releases.  All of the stages I work on stay in /experimental.

--Tony

-- 
Anthony G. Basile, Ph. D.
Chair of Information Technology
D'Youville College
Buffalo, NY 14201
(716) 829-8197

Re: [gentoo-releng] Autobuilds go to /experimental and to /releases only when someone actually tests them?

[Jorge Manuel B. S. Vicetto]

[-- Attachment #1: Type: text/plain, Size: 1138 bytes --]

On Fri, Jul 26, 2013 at 4:24 AM, Matt Turner <mattst88@gentoo.org> wrote:

> Can we make autobuilds go to /experimental and then only move them to
> /releases when someone actually tests them?
>
>
Even though this sounds good and makes sense, if we don't even have enough
people to work on things now, this would only make things worse (imho).


> Looking at bugzilla and listening in #gentoo-releng, it's kind of
> embarrassing how often someone downloads a Live CD only to find out
> that networking is totally broken by a udev upgrade, or something to
> that effect.
>
>
There's no way to deal with this breaks when those committing them don't
even warn us about them.


> We don't commit version bumps straight to stable; I don't see why we
> do with release media.
>
> Matt
>
>
The alternative here is the automated testing that I've talked about with
others on FOSDEM and Prague last year, but that still needs to be done.
Amongst other testing, the idea would be to pick the latest ISO and boot it
on some virtualization platform and catch the console output to ensure it
booted successfully and not tests failed.

Regards,

Jorge

[-- Attachment #2: Type: text/html, Size: 2016 bytes --]

Re: [gentoo-releng] Autobuilds go to /experimental and to /releases only when someone actually tests them?

[Matt Turner]

On Mon, Jul 29, 2013 at 12:05 PM, Jorge Manuel B. S. Vicetto
<jmbsvicetto@gentoo.org> wrote:
>
> On Fri, Jul 26, 2013 at 4:24 AM, Matt Turner <mattst88@gentoo.org> wrote:
>>
>> Can we make autobuilds go to /experimental and then only move them to
>> /releases when someone actually tests them?
>>
>
> Even though this sounds good and makes sense, if we don't even have enough
> people to work on things now, this would only make things worse (imho).

Everyone that's expressed concerns about this plan seems to do so
based on the idea that we must move things to /releases very often. I
do not believe this to be the case.

Release media doesn't need to be the latest and greatest, it just
needs to be able to install Gentoo.

The stages aren't so much a problem, since you're already able to
configure and install software when using them, but the LiveCDs should
be stable and tested.

I specifically care that the LiveCDs are tested before being moved to
/releases. I think it's perfectly okay to leave stages in
/experimental.

>> Looking at bugzilla and listening in #gentoo-releng, it's kind of
>> embarrassing how often someone downloads a Live CD only to find out
>> that networking is totally broken by a udev upgrade, or something to
>> that effect.
>>
>
> There's no way to deal with this breaks when those committing them don't
> even warn us about them.

I think this is a separate problem and should be treated as such. We
certainly can't claim "oh, someone pushed a commit the broken udev, so
it's not our fault that the release media is broken." Whose fault it
is is immaterial to the user who just downloaded something they expect
to work from /releases and is now unable to install Gentoo.

We can work on that problem, but I don't think it's going to go away
easily. We could avoid almost all of the problems if we simply tell
users which LiveCD images are known to work.

> The alternative here is the automated testing that I've talked about with
> others on FOSDEM and Prague last year, but that still needs to be done.
> Amongst other testing, the idea would be to pick the latest ISO and boot it
> on some virtualization platform and catch the console output to ensure it
> booted successfully and not tests failed.

Some recent problems have involved linux-firmware being broken. I
don't think virtualization will help notice that, e.g., the Tigon 3
firmware is missing.

Re: [gentoo-releng] Autobuilds go to /experimental and to /releases only when someone actually tests them?

[Sebastian Pipping]

On 29.07.2013 22:38, Matt Turner wrote:
>> The alternative here is the automated testing that I've talked about with
>> others on FOSDEM and Prague last year, but that still needs to be done.
>> Amongst other testing, the idea would be to pick the latest ISO and boot it
>> on some virtualization platform and catch the console output to ensure it
>> booted successfully and not tests failed.
> 
> Some recent problems have involved linux-firmware being broken. I
> don't think virtualization will help notice that, e.g., the Tigon 3
> firmware is missing.

Ideally, you'd use virtualization and some concrete physical platforms
in parallel.  For virtualization, KVM Qemu might be quick to get
results, I think I used something like

  -append 'earlyprintk=serial,keep' -serial file:/abs/path/log/file

previously to log the full console output from inside a KVM machine.
That could be the base of a regression test suite.

Best,



Sebastian