[gentoo-dev] AMD64 Arch Testers needed urgently

[gentoo-dev] AMD64 Arch Testers needed urgently

[Michał Górny]

Hi, everyone.

It seems that we've started lacking arch testers for AMD64 architecture.
At this moment, there are already 159 bugs in amd64 backlog, and there
is no noticeable progress. New stabilization requests are usually
handled much faster by x86, sparc and hppa teams!

If you have a stable AMD64 system and would like to help arch testing,
please do! I don't know what exact rules for that are but I think [1]
could be helpful. If someone knows better, then please share.

[1]:https://wiki.gentoo.org/wiki/Project:AMD64_Arch_Testers

-- 
Best regards,
Michał Górny

Re: [gentoo-dev] AMD64 Arch Testers needed urgently

[Rich Freeman]

On Tue, Dec 12, 2017 at 1:15 PM, Michał Górny <mgorny@gentoo.org> wrote:
>
> It seems that we've started lacking arch testers for AMD64 architecture.
> At this moment, there are already 159 bugs in amd64 backlog, and there
> is no noticeable progress. New stabilization requests are usually
> handled much faster by x86, sparc and hppa teams!
>
> If you have a stable AMD64 system and would like to help arch testing,
> please do! I don't know what exact rules for that are but I think [1]
> could be helpful. If someone knows better, then please share.
>

As far as I'm aware the standing policy already exists that
maintainers can stabilize their own packages on amd64.

That said, if somebody wants to organize an AT program for amd64 they
should feel free to do so.  I could explain how things used to work at
least if that is helpful.

The old way it used to work is that ATs had to pass the ebuild quiz
and they would get editbugs privs to tag bugs as tested.  I won't
elaborate here unless there is interest...

-- 
Rich

Re: [gentoo-dev] AMD64 Arch Testers needed urgently

[Francesco Riosa]

On 12/12/2017 19:24, Rich Freeman wrote:
> On Tue, Dec 12, 2017 at 1:15 PM, Michał Górny <mgorny@gentoo.org> wrote:
>> It seems that we've started lacking arch testers for AMD64 architecture.
>> At this moment, there are already 159 bugs in amd64 backlog, and there
>> is no noticeable progress. New stabilization requests are usually
>> handled much faster by x86, sparc and hppa teams!
>>
>> If you have a stable AMD64 system and would like to help arch testing,
>> please do! I don't know what exact rules for that are but I think [1]
>> could be helpful. If someone knows better, then please share.
>>
> As far as I'm aware the standing policy already exists that
> maintainers can stabilize their own packages on amd64.

It would be interesting to discuss if proxy maintainers should be able
to stabilize too.


>
> That said, if somebody wants to organize an AT program for amd64 they
> should feel free to do so.  I could explain how things used to work at
> least if that is helpful.
>
> The old way it used to work is that ATs had to pass the ebuild quiz
> and they would get editbugs privs to tag bugs as tested.  I won't
> elaborate here unless there is interest...
>

Re: [gentoo-dev] AMD64 Arch Testers needed urgently

[R0b0t1]

On Tue, Dec 12, 2017 at 12:24 PM, Rich Freeman <rich0@gentoo.org> wrote:
> On Tue, Dec 12, 2017 at 1:15 PM, Michał Górny <mgorny@gentoo.org> wrote:
>>
>> It seems that we've started lacking arch testers for AMD64 architecture.
>> At this moment, there are already 159 bugs in amd64 backlog, and there
>> is no noticeable progress. New stabilization requests are usually
>> handled much faster by x86, sparc and hppa teams!
>>
>> If you have a stable AMD64 system and would like to help arch testing,
>> please do! I don't know what exact rules for that are but I think [1]
>> could be helpful. If someone knows better, then please share.
>>
>
> As far as I'm aware the standing policy already exists that
> maintainers can stabilize their own packages on amd64.
>
> That said, if somebody wants to organize an AT program for amd64 they
> should feel free to do so.  I could explain how things used to work at
> least if that is helpful.
>
> The old way it used to work is that ATs had to pass the ebuild quiz
> and they would get editbugs privs to tag bugs as tested.  I won't
> elaborate here unless there is interest...
>

I would like to know. But on the other hand, anyone interested in
contributing to packages they work with is likely already doing so. On
the third (and final?) hand, it may also be that there are people
looking for direction.

Cheers,
    R0b0t1

Re: [gentoo-dev] AMD64 Arch Testers needed urgently

[Thomas Deutschmann]

[-- Attachment #1.1: Type: text/plain, Size: 753 bytes --]

On 2017-12-12 19:24, Rich Freeman wrote:
> As far as I'm aware the standing policy already exists that
> maintainers can stabilize their own packages on amd64.

That's right but keep in mind that nevertheless you need a stable
system. Marking a package stable because it works on your ~arch box you
use for your daily dev work would lead the whole process ad absurdum.

And in general maintainer stabilization should be the last resort. The
person who wrote the ebuild maybe doesn't notice that the ebuild is
doing something wrong (doesn't honor CFLAGS, calls compiler directly,
not working with /bin/sh not /bin/bash ...).


-- 
Regards,
Thomas Deutschmann / Gentoo Linux Developer
C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

Re: [gentoo-dev] AMD64 Arch Testers needed urgently

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 1051 bytes --]

On Wed, Dec 13, 2017 at 01:22:04PM +0100, Thomas Deutschmann wrote:
> On 2017-12-12 19:24, Rich Freeman wrote:
> > As far as I'm aware the standing policy already exists that
> > maintainers can stabilize their own packages on amd64.
> 
> That's right but keep in mind that nevertheless you need a stable
> system. Marking a package stable because it works on your ~arch box you
> use for your daily dev work would lead the whole process ad absurdum.

In  my discussions with other developers, I've found that this is the
biggest concern. Most devs are runnning ~amd64, so they don't feel that
they can mark things stable.

> And in general maintainer stabilization should be the last resort. The
> person who wrote the ebuild maybe doesn't notice that the ebuild is
> doing something wrong (doesn't honor CFLAGS, calls compiler directly,
> not working with /bin/sh not /bin/bash ...).

In theory, this is correct. However, when maintainers don't stabilize
packages and no one else does either, our stable tree suffers.

William

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [gentoo-dev] AMD64 Arch Testers needed urgently

[Lucas Ramage]

[-- Attachment #1: Type: text/plain, Size: 2037 bytes --]

> In  my discussions with other developers, I've found that this is the
​> ​
biggest concern. Most devs are runnning ~amd64, so they don't feel that
​> ​
they can mark things stable.

W
​hat about running a stable chroot?​ Are there any tools that can be used
to automate this process?

On Wed, Dec 13, 2017 at 12:51 PM, William Hubbs <williamh@gentoo.org> wrote:

> On Wed, Dec 13, 2017 at 01:22:04PM +0100, Thomas Deutschmann wrote:
> > On 2017-12-12 19:24, Rich Freeman wrote:
> > > As far as I'm aware the standing policy already exists that
> > > maintainers can stabilize their own packages on amd64.
> >
> > That's right but keep in mind that nevertheless you need a stable
> > system. Marking a package stable because it works on your ~arch box you
> > use for your daily dev work would lead the whole process ad absurdum.
>
> In  my discussions with other developers, I've found that this is the
> biggest concern. Most devs are runnning ~amd64, so they don't feel that
> they can mark things stable.
>
> > And in general maintainer stabilization should be the last resort. The
> > person who wrote the ebuild maybe doesn't notice that the ebuild is
> > doing something wrong (doesn't honor CFLAGS, calls compiler directly,
> > not working with /bin/sh not /bin/bash ...).
>
> In theory, this is correct. However, when maintainers don't stabilize
> packages and no one else does either, our stable tree suffers.
>
> William
>



-- 
Regards,

[image: Visit online journal] <https://lramage94.github.io/>

*Lucas Ramage* / Software Engineer
ramage.lucas@openmailbox.org / (941) 404-6794

*PGP Fingerprint* / Learn More <https://emailselfdefense.fsf.org/en/>
EAE7 45DF 818D 4948 DDA7 0F44 F52A 5A96 7B9B 6FB7
<https://pgp.mit.edu/pks/lookup?op=get&search=0xF52A5A967B9B6FB7>

*Visit online journal*
http://lramage94.github.io <https://lramage94.github.io/>

[image: Github]  <https://github.com/lramage94>[image: Linkedin]
<https://www.linkedin.com/in/lramage94>

[-- Attachment #2: Type: text/html, Size: 5152 bytes --]

Re: [gentoo-dev] AMD64 Arch Testers needed urgently

[Aaron W. Swenson]

[-- Attachment #1: Type: text/plain, Size: 612 bytes --]

On 2017-12-13 13:20, Lucas Ramage wrote:
> > In  my discussions with other developers, I've found that this is the
> ​> ​
> biggest concern. Most devs are runnning ~amd64, so they don't feel that
> ​> ​
> they can mark things stable.
> 
> W
> ​hat about running a stable chroot?​ Are there any tools that can be used
> to automate this process?

Yes, a stable chroot is okay, and there is app-portage/tatt that can run
through the various USE flag combinations for you.

It’s output isn’t terribly helpful, so it takes a little while to
understand what it’s trying to tell you.

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 376 bytes --]

Re: [gentoo-dev] AMD64 Arch Testers needed urgently

[Lucas Ramage]

[-- Attachment #1: Type: text/plain, Size: 1403 bytes --]

I see, well I can setup buildbot to do that. Is there some place in
particular that I should send my test results?

On Wed, Dec 13, 2017 at 1:28 PM, Aaron W. Swenson <titanofold@gentoo.org>
wrote:

> On 2017-12-13 13:20, Lucas Ramage wrote:
> > > In  my discussions with other developers, I've found that this is the
> > ​> ​
> > biggest concern. Most devs are runnning ~amd64, so they don't feel that
> > ​> ​
> > they can mark things stable.
> >
> > W
> > ​hat about running a stable chroot?​ Are there any tools that can be used
> > to automate this process?
>
> Yes, a stable chroot is okay, and there is app-portage/tatt that can run
> through the various USE flag combinations for you.
>
> It’s output isn’t terribly helpful, so it takes a little while to
> understand what it’s trying to tell you.
>



-- 
Regards,

[image: Visit online journal] <https://lramage94.github.io/>

*Lucas Ramage* / Software Engineer
ramage.lucas@openmailbox.org / (941) 404-6794

*PGP Fingerprint* / Learn More <https://emailselfdefense.fsf.org/en/>
EAE7 45DF 818D 4948 DDA7 0F44 F52A 5A96 7B9B 6FB7
<https://pgp.mit.edu/pks/lookup?op=get&search=0xF52A5A967B9B6FB7>

*Visit online journal*
http://lramage94.github.io <https://lramage94.github.io/>

[image: Github]  <https://github.com/lramage94>[image: Linkedin]
<https://www.linkedin.com/in/lramage94>

[-- Attachment #2: Type: text/html, Size: 3848 bytes --]

Re: [gentoo-dev] AMD64 Arch Testers needed urgently

[Thomas Deutschmann]

[-- Attachment #1.1: Type: text/plain, Size: 1414 bytes --]

On 2017-12-13 18:51, William Hubbs wrote:
> In theory, this is correct. However, when maintainers don't stabilize
> packages and no one else does either, our stable tree suffers.

I agree but we have to pay attention that we don't stabilize packages at
all costs because otherwise they would never go stable.

If this is the problem then we should discuss stabilization at all. What
do people expect from something marked stable vs. reality. ;)

And in this case I would prefer a system like Debian SID -> Testing
supported by build bots. I can think of 2 variants:

a) Once maintainer files a stabilization request, a build bot will pick
up the bug and try building the package in a chroot per architecture. If
everything passes build bot will mark the version stable for the tested
architecture.

A flag or a blacklist could prevent build bot stabilization.


b) Because not all devs care about stable Gentoo, I would recommend
auto-stabilization: I.e. if a package is in the repository for x days
build bot would try to build the package and mark the package stable if
everything passes. If for some reason maintainer want to block a
specific version they could create a bug or set a flag in an already
existing bug which will cause build bot to ignore this version.


-- 
Regards,
Thomas Deutschmann / Gentoo Linux Developer
C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

Re: [gentoo-dev] AMD64 Arch Testers needed urgently

[Kent Fredric]

On Wed, 13 Dec 2017 21:58:05 +0100
Thomas Deutschmann <whissi@gentoo.org> wrote:

> b) Because not all devs care about stable Gentoo, I would recommend
> auto-stabilization: I.e. if a package is in the repository for x days
> build bot would try to build the package and mark the package stable
> if everything passes. If for some reason maintainer want to block a
> specific version they could create a bug or set a flag in an already
> existing bug which will cause build bot to ignore this version.

Slightly modified suggestion:

Add a flag called "autostabilize" with [unset], [y], [n]

Default is 'unset', and if found unset after a given time, it flips to
y and the stable bot gets queued up.

If its set to 'n', then stable bot never does anything.

This way maintainers who want to rush the stablebot on things they
consider "safe" can get ahead of the queue.


[nb: unsigned mail, pinentry currently broken due to -fPIC profile
migration]

Re: [gentoo-dev] AMD64 Arch Testers needed urgently

[Thomas Deutschmann]

[-- Attachment #1.1: Type: text/plain, Size: 2261 bytes --]

On 2017-12-14 01:58, Kent Fredric wrote:
> On Wed, 13 Dec 2017 21:58:05 +0100
> Thomas Deutschmann <whissi@gentoo.org> wrote:
> 
>> b) Because not all devs care about stable Gentoo, I would recommend
>> auto-stabilization: I.e. if a package is in the repository for x days
>> build bot would try to build the package and mark the package stable
>> if everything passes. If for some reason maintainer want to block a
>> specific version they could create a bug or set a flag in an already
>> existing bug which will cause build bot to ignore this version.
> 
> Slightly modified suggestion:
> 
> Add a flag called "autostabilize" with [unset], [y], [n]

Flag in Bugzilla?


> Default is 'unset', and if found unset after a given time, it flips to
> y and the stable bot gets queued up.
> 
> If its set to 'n', then stable bot never does anything.
> 
> This way maintainers who want to rush the stablebot on things they
> consider "safe" can get ahead of the queue.

Sounds good but the variant b was about full auto-stabilization. I.e.
the idea that we even don't require stabilization bugs anymore (like
Debian, where a package will move from SID to testing after some time if
nobody created a blocker bug).

We could auto-generate bugs but I am not sure if this is a good idea.
When mgorny set up autolinking of Github PRs there was some "bug spam"
when people lost control over Git and messed up the rebase (now
prevented via limits).

Also I am not sure how we should handle things like Gnome/KDE
stabilization which requires that a bunch of packages will be stabilized
at once. I.e. if bot detects a new ebuild of kde-frameworks/kservice
auto-stabilization is only allowed to kick in if it is a rev bump of an
already stable ebuild but shouldn't auto-start stabilization of next KDE
version on its own (at least I guess this is what we want).

But well, for the beginning we don't need the perfect solution. We can
start with an easy mode and blacklist most packages. So devs interested
can remove their packages from blacklist. And like said, build bot would
still handle stabilization bugs.


-- 
Regards,
Thomas Deutschmann / Gentoo Linux Developer
C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

Re: [gentoo-dev] AMD64 Arch Testers needed urgently

[Rich Freeman]

On Thu, Dec 14, 2017 at 5:39 AM, Thomas Deutschmann <whissi@gentoo.org> wrote:
>
> But well, for the beginning we don't need the perfect solution. We can
> start with an easy mode and blacklist most packages. So devs interested
> can remove their packages from blacklist. And like said, build bot would
> still handle stabilization bugs.
>

I think this has all been gone over in a list post previously, but it
seems like the most straightforward solution here is flags in metadata
where individual packages can be flagged for auto-stabilization.

In the beginning the system would be opt-in.  Then once we have
confidence that it is working well the flag could potentially be made
opt-out.

Maybe initially for testing you could keep the whitelist outside of
the repository, but then you need to make it straightforward for
maintainers to have their packages added to the whitelist if they
wish.  Long-term if this becomes the standard workflow it would make
sense for the flags to go inside the repository.

-- 
Rich

Re: [gentoo-dev] AMD64 Arch Testers needed urgently

[Aaron W. Swenson]

[-- Attachment #1: Type: text/plain, Size: 1222 bytes --]

On 2017-12-14 13:58, Kent Fredric wrote:
> On Wed, 13 Dec 2017 21:58:05 +0100
> Slightly modified suggestion:
> 
> Add a flag called "autostabilize" with [unset], [y], [n]
> 
> Default is 'unset', and if found unset after a given time, it flips to
> y and the stable bot gets queued up.
> 
> If its set to 'n', then stable bot never does anything.
> 
> This way maintainers who want to rush the stablebot on things they
> consider "safe" can get ahead of the queue.

Well, we kind of have that already with “Runtime testing required” (RTR).

RTR=no stablereqs are good candidates for automation.

If everything has been properly handled in the ebuild, then an emerge
should succeed. And, RTR being set to “No” indicates that there aren’t any
tests to perform other than those defined and handled within the ebuild.

Restricting the set to RTR=no would eliminate special cases needing to
be taken into consideration.

Of course, RTR being unset or set to yes should be skipped.

We could initially start with stablebot only adding a comment to a bug
that it thinks it’s safe to stabilize the subject. This would give us
some time to gain confidence in it and/or weed out the bugs.

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 376 bytes --]

Re: [gentoo-dev] AMD64 Arch Testers needed urgently

[Rich Freeman]

On Thu, Dec 14, 2017 at 7:07 AM, Aaron W. Swenson <titanofold@gentoo.org> wrote:
> On 2017-12-14 13:58, Kent Fredric wrote:
>> On Wed, 13 Dec 2017 21:58:05 +0100
>> Slightly modified suggestion:
>>
>> Add a flag called "autostabilize" with [unset], [y], [n]
>>
>> Default is 'unset', and if found unset after a given time, it flips to
>> y and the stable bot gets queued up.
>>
>> If its set to 'n', then stable bot never does anything.
>>
>> This way maintainers who want to rush the stablebot on things they
>> consider "safe" can get ahead of the queue.
>
> Well, we kind of have that already with “Runtime testing required” (RTR).
>
> RTR=no stablereqs are good candidates for automation.
>
> If everything has been properly handled in the ebuild, then an emerge
> should succeed. And, RTR being set to “No” indicates that there aren’t any
> tests to perform other than those defined and handled within the ebuild.

I'm not sure runtime testing is the only use case for preventing
auto-stabilization.

The example of KDE/Gnome was brought up where large number of packages
need to be stabilized at the same time.  I'm not sure this is actually
a factor that should prevent auto-stabilization, since these packages
should have the correct dependencies and the stabilization system
would definitely need to be dependency-aware (which would lead to them
being stabilized as a group automatically).  If these require runtime
testing then they fall into the existing use case.

Does anybody actually have a reason to prevent stabilization other
than for runtime testing, assuming that stabilization is done in a
manner where all dependencies are satisfied at all times?

Maybe runtime testing really ought to be the only reason to prevent
auto stabilization...

-- 
Rich

Re: [gentoo-dev] AMD64 Arch Testers needed urgently

[Harald Weiner]

[-- Attachment #1.1: Type: text/plain, Size: 781 bytes --]

Dear Thomas and everyone else interested!

Before re-inventing the wheel, you might take a closer look at this Google summer of code project in 2016:
https://wiki.gentoo.org/wiki/Google_Summer_of_Code/2016/Ideas/Continuous_Stabilization

I do not know how far the author got but it might be a good starting point...

Best wishes,

Harald.

>>> Thomas Deutschmann <whissi@gentoo.org> 12/13/17 9:59 PM >>>

> auto-stabilization: I.e. if a package is in the repository for x days
> build bot would try to build the package and mark the package stable if
> everything passes. If for some reason maintainer want to block a
> specific version they could create a bug or set a flag in an already
> existing bug which will cause build bot to ignore this version.





[-- Attachment #1.2: HTML --]
[-- Type: text/html, Size: 1431 bytes --]

Re: [gentoo-dev] AMD64 Arch Testers needed urgently

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 1319 bytes --]

On Wed, Dec 13, 2017 at 09:58:05PM +0100, Thomas Deutschmann wrote:

*snip*

> b) Because not all devs care about stable Gentoo, I would recommend
> auto-stabilization: I.e. if a package is in the repository for x days
> build bot would try to build the package and mark the package stable if
> everything passes. If for some reason maintainer want to block a
> specific version they could create a bug or set a flag in an already
> existing bug which will cause build bot to ignore this version.

I tend to like this better. Let's try to move away from filing stable
requests for new versions of packages once an old version is stable and
have a way to block newer versions from going stable. Maybe buildbot
could check to see if there is a bug open against the version it is
looking at, then check the keywords or severity of that bug and  use one
or the other of those to decide whether or not to skip stabilizing that version
of the package.

I think something else we should add to this is that buildbot should do
nothing if there isn't already a stable version of the package on the
architecture it is testing.

In other words, the first stabilization for a package on an architecture
should be done the way we currently do them, by filing a stable request
then using the current stabilization process.

William


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [gentoo-dev] AMD64 Arch Testers needed urgently

[R0b0t1]

[-- Attachment #1: Type: text/plain, Size: 535 bytes --]

On Wed, Dec 13, 2017 at 2:55 PM, Lucas Ramage <ramage.lucas94@gmail.com>
wrote:

> I see, well I can setup buildbot to do that. Is there some place in
> particular that I should send my test results?
>

Is this part of the point of a Tinderbox? The only problem I can see is
that the configurations being tested can be extremely hard to replicate and
lead to sporadic errors.

In response to the concerns about stability: If I run a lot of unstable
packages, would that preclude my system from being able to help?

Cheers,
     R0b0t1

[-- Attachment #2: Type: text/html, Size: 1006 bytes --]

Re: [gentoo-dev] AMD64 Arch Testers needed urgently

[Thomas Deutschmann]

[-- Attachment #1.1: Type: text/plain, Size: 466 bytes --]

On 2017-12-14 21:06, R0b0t1 wrote:
> In response to the concerns about stability: If I run a lot of unstable
> packages, would that preclude my system from being able to help?

Yes. Only clean stable systems are eligible for arch testing. That's the
whole idea of arch testing... ;)

Remember that mixed systems aren't officially supported.


-- 
Regards,
Thomas Deutschmann / Gentoo Linux Developer
C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

Re: [gentoo-dev] AMD64 Arch Testers needed urgently

[R0b0t1]

On Thu, Dec 14, 2017 at 2:13 PM, Thomas Deutschmann <whissi@gentoo.org> wrote:
> On 2017-12-14 21:06, R0b0t1 wrote:
>> In response to the concerns about stability: If I run a lot of unstable
>> packages, would that preclude my system from being able to help?
>
> Yes. Only clean stable systems are eligible for arch testing. That's the
> whole idea of arch testing... ;)
>
> Remember that mixed systems aren't officially supported.
>

It seems like lagging stability is due to a lack of resources. I do
not know a single person who would be able to run only stable
packages. They seem to move too slowly, and people switch to unstable
packages because they contain bugfixes and sometimes new features.

Could the criteria for stability be reconsidered? Mixed systems might
not be supported, but save for cases of ABI/API breakage (which can
happen when transitioning from stable->stable) I do not know why the
packages would not play well with each other. I am sure there are
examples where things have blown up, but it seems like expecting that
to be the case isn't helping.

Cheers,
     R0b0t1

Re: [gentoo-dev] AMD64 Arch Testers needed urgently

[Kristian Fiskerstrand]

[-- Attachment #1.1: Type: text/plain, Size: 1432 bytes --]

On 12/14/2017 09:21 PM, R0b0t1 wrote:
> It seems like lagging stability is due to a lack of resources. I do
> not know a single person who would be able to run only stable
> packages.

I run stable only on most of my systems.

> They seem to move too slowly, and people switch to unstable
> packages because they contain bugfixes and sometimes new features.

slow isn't necessarily a problem, as long as security fixes are handled.
There is some balancing for large performance gains, but most existing
systems are scaled based on the current estimates so it would only be
relevant for the up sizing of the server park for growth needs etc.

> 
> Could the criteria for stability be reconsidered? Mixed systems might

why would it?

> not be supported, but save for cases of ABI/API breakage (which can
> happen when transitioning from stable->stable) I do not know why the
> packages would not play well with each other. I am sure there are
> examples where things have blown up, but it seems like expecting that
> to be the case isn't helping.

There are plenty of cases where this fails in miserable ways, so thats
not a good idea (not to mention the dependency hell from it). That said,
you can have a stable chroot, or just use a VM for testing etc.

-- 
Kristian Fiskerstrand
OpenPGP keyblock reachable at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [gentoo-dev] AMD64 Arch Testers needed urgently

[Kristian Fiskerstrand]

[-- Attachment #1.1: Type: text/plain, Size: 690 bytes --]

On 12/14/2017 01:01 PM, Rich Freeman wrote:
> In the beginning the system would be opt-in.  Then once we have
> confidence that it is working well the flag could potentially be made
> opt-out.

The only place I imagine this being a good idea is for the kernel, given
the strict no break of userland policy (but even they fail from time to
time). For rest of the applications, even if we add tools to help
automate part of the stabilization, I'd very much oppose it being
automated without being initiated / acked by the maintainer.

-- 
Kristian Fiskerstrand
OpenPGP keyblock reachable at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [gentoo-dev] AMD64 Arch Testers needed urgently

[Alec Warner]

[-- Attachment #1: Type: text/plain, Size: 1726 bytes --]

On Thu, Dec 14, 2017 at 3:34 PM, Kristian Fiskerstrand <k_f@gentoo.org>
wrote:

> On 12/14/2017 01:01 PM, Rich Freeman wrote:
> > In the beginning the system would be opt-in.  Then once we have
> > confidence that it is working well the flag could potentially be made
> > opt-out.
>
> The only place I imagine this being a good idea is for the kernel, given
> the strict no break of userland policy (but even they fail from time to
> time). For rest of the applications, even if we add tools to help
> automate part of the stabilization, I'd very much oppose it being
> automated without being initiated / acked by the maintainer.
>

This reminds me a bit of advertising actually:

"Nineteenth century Philadelphia retailer John Wanamaker supposedly said
“Half the money I spend on advertising is wasted; the trouble is I don't
know which half.” "

I feel like stabilization are similar; we know many of them are likely safe
and don't need humans. But many are unsafe and require validation.
Can we tell them apart? Perhaps Rich's flag is sufficient?

Its also a risk mitigation problem. Is it better for Gentoo to have more
packages marked stable (so that stable is closer to HEAD). Or is it more
valauble
for Gentoo to have a very old stable (like debian) and then have a split
keywords system?

I'm skeptical the keywords for most packages matter, particularly on common
arches. Remember this is usually software that upstream
already tested and released; so most of the bugs would be ebuild / Gentoo
related.

-A




>
> --
> Kristian Fiskerstrand
> OpenPGP keyblock reachable at hkp://pool.sks-keyservers.net
> fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3
>
>

[-- Attachment #2: Type: text/html, Size: 2581 bytes --]

Re: [gentoo-dev] AMD64 Arch Testers needed urgently

[Thomas Deutschmann]

[-- Attachment #1.1: Type: text/plain, Size: 1309 bytes --]

On 2017-12-14 20:45, William Hubbs wrote:
> I tend to like this better. Let's try to move away from filing stable
> requests for new versions of packages once an old version is stable and
> have a way to block newer versions from going stable. Maybe buildbot
> could check to see if there is a bug open against the version it is
> looking at, then check the keywords or severity of that bug and  use one
> or the other of those to decide whether or not to skip stabilizing that version
> of the package.

How would you identify such a bug? Someone reports a bug. He/she is
using app-misc/foo-1.2-r2 at the moment. It is often not clear if the
bug affects only =app-misc/foo-1.2-r2, >=, ... What's about foo-2.0?
Maybe foo-1.1 is also affected but wasn't (yet) tested...


> In other words, the first stabilization for a package on an architecture
> should be done the way we currently do them, by filing a stable request
> then using the current stabilization process.

I am not sure if something like this should be a general rule. But like
said, maintainer could either opt-in or opt-out from auto-stabilization.
So if you think your new package is somehow special...


-- 
Regards,
Thomas Deutschmann / Gentoo Linux Developer
C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

Re: [gentoo-dev] AMD64 Arch Testers needed urgently

[Thomas Deutschmann]

[-- Attachment #1.1: Type: text/plain, Size: 750 bytes --]

On 2017-12-14 21:53, Alec Warner wrote:
> I'm skeptical the keywords for most packages matter, particularly on
> common arches. Remember this is usually software that upstream
> already tested and released; so most of the bugs would be ebuild /
> Gentoo related.

That's why I prefer Debian's SID -> Testing workflow.

If a package has multiple users, a serious problem will popup very soon.
So stabilization would be blocked.

If a package hasn't many users and a problem also affects only a special
configuration we maybe won't notice before "regular" stabilization and
after some time...

So why should we wait...


-- 
Regards,
Thomas Deutschmann / Gentoo Linux Developer
C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

Re: [gentoo-dev] AMD64 Arch Testers needed urgently

[R0b0t1]

On Thu, Dec 14, 2017 at 2:32 PM, Kristian Fiskerstrand <k_f@gentoo.org> wrote:
> On 12/14/2017 09:21 PM, R0b0t1 wrote:
>> It seems like lagging stability is due to a lack of resources. I do
>> not know a single person who would be able to run only stable
>> packages.
>
> I run stable only on most of my systems.
>

That is fine, but this thread exists because at least the OP thinks
stabilization is not happening quickly enough, likely because there
are not enough people working on it. Allowing stabilization work from
mixed systems might allow more people to help.

>> They seem to move too slowly, and people switch to unstable
>> packages because they contain bugfixes and sometimes new features.
>
> slow isn't necessarily a problem, as long as security fixes are handled.
> There is some balancing for large performance gains, but most existing
> systems are scaled based on the current estimates so it would only be
> relevant for the up sizing of the server park for growth needs etc.
>
>>
>> Could the criteria for stability be reconsidered? Mixed systems might
>
> why would it?
>

Per the question posed by OP the current state of affairs does not
seem to be working, and I have tried to point out one likely cause. If
it's hard to justify the criteria for stability then maybe the
criteria don't make sense.

>> not be supported, but save for cases of ABI/API breakage (which can
>> happen when transitioning from stable->stable) I do not know why the
>> packages would not play well with each other. I am sure there are
>> examples where things have blown up, but it seems like expecting that
>> to be the case isn't helping.
>
> There are plenty of cases where this fails in miserable ways, so thats
> not a good idea (not to mention the dependency hell from it). That said,
> you can have a stable chroot, or just use a VM for testing etc.
>

Can you be specific? Human memory is biased towards negative
experiences. If it's hard to actually describe the multitude of issues
that mixed systems cause then it is very likely mixed systems do not
cause many issues.

Personally, I have very few problems due to my mixed system, and less
than I would have on a stable system.

Cheers,
     R0b0t1

Re: [gentoo-dev] AMD64 Arch Testers needed urgently

[R0b0t1]

On Thu, Dec 14, 2017 at 4:04 PM, R0b0t1 <r030t1@gmail.com> wrote:
> On Thu, Dec 14, 2017 at 2:32 PM, Kristian Fiskerstrand <k_f@gentoo.org> wrote:
>> On 12/14/2017 09:21 PM, R0b0t1 wrote:
>>> It seems like lagging stability is due to a lack of resources. I do
>>> not know a single person who would be able to run only stable
>>> packages.
>>
>> I run stable only on most of my systems.
>>
>
> That is fine, but this thread exists because at least the OP thinks
> stabilization is not happening quickly enough, likely because there
> are not enough people working on it. Allowing stabilization work from
> mixed systems might allow more people to help.
>
>>> They seem to move too slowly, and people switch to unstable
>>> packages because they contain bugfixes and sometimes new features.
>>
>> slow isn't necessarily a problem, as long as security fixes are handled.
>> There is some balancing for large performance gains, but most existing
>> systems are scaled based on the current estimates so it would only be
>> relevant for the up sizing of the server park for growth needs etc.
>>
>>>
>>> Could the criteria for stability be reconsidered? Mixed systems might
>>
>> why would it?
>>
>
> Per the question posed by OP the current state of affairs does not
> seem to be working, and I have tried to point out one likely cause. If
> it's hard to justify the criteria for stability then maybe the
> criteria don't make sense.
>
>>> not be supported, but save for cases of ABI/API breakage (which can
>>> happen when transitioning from stable->stable) I do not know why the
>>> packages would not play well with each other. I am sure there are
>>> examples where things have blown up, but it seems like expecting that
>>> to be the case isn't helping.
>>
>> There are plenty of cases where this fails in miserable ways, so thats
>> not a good idea (not to mention the dependency hell from it). That said,
>> you can have a stable chroot, or just use a VM for testing etc.
>>
>
> Can you be specific? Human memory is biased towards negative
> experiences. If it's hard to actually describe the multitude of issues
> that mixed systems cause then it is very likely mixed systems do not
> cause many issues.
>
> Personally, I have very few problems due to my mixed system, and less
> than I would have on a stable system.
>
> Cheers,
>      R0b0t1

I'm not trying to be confrontational, but asserting an opinion is
correct without explaining why that it is so isn't really conducive to
arriving at the truth. I understand not wanting to answer if I am
completely clueless, and would like to apologize in advance for
bothering the developers.

I am not very smart, sir.

Cheers,
     R0b0t1

Re: [gentoo-dev] AMD64 Arch Testers needed urgently

[M. J. Everitt]

[-- Attachment #1.1: Type: text/plain, Size: 1529 bytes --]

On 15/12/17 01:17, R0b0t1 wrote (excerpted):
> I'm not trying to be confrontational, but asserting an opinion is
> correct without explaining why that it is so isn't really conducive to
> arriving at the truth. I understand not wanting to answer if I am
> completely clueless, and would like to apologize in advance for
> bothering the developers.
>
> I am not very smart, sir.
>
> Cheers,
>      R0b0t1
>
With all due respect, Gentoo is not renowned for spoon-feeding ...

Returning to the topic in hand, two key points strike out at me:-

1) Gentoo isn't really interested in having a 'stable' tree or it would
already be happening. As such, why not cut the Gordian knot, declare
that this is not something that will happen [soon] and let users make
their own choices. The [majority of] developers already seem to have ...

2) Whilst there has yet another fine bike-shed emerged on the subject, I
have only seen one volunteer willing or capable to actually take on
implementation of anything that has been discussed on this thread. As
such, you can talk all you like .. nothing will happen until somebody
actually *does* something .. For all the hating, I will duly credit
mgorny for producing a consistent quantity of commits across the board
in Gentoo, and whilst you may not agree with all his bitching (for want
of a better term) at least he can stand and say "well at least I did
*something* about it, whether You like it or not ...".

Damnit, there's another $2 from me .. my apologies.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-dev] AMD64 Arch Testers needed urgently

[Brian Dolbec]

On Wed, 13 Dec 2017 15:55:59 -0500
Lucas Ramage <ramage.lucas94@gmail.com> wrote:

> I see, well I can setup buildbot to do that. Is there some place in
> particular that I should send my test results?
> 
> On Wed, Dec 13, 2017 at 1:28 PM, Aaron W. Swenson
> <titanofold@gentoo.org> wrote:
> 
> > On 2017-12-13 13:20, Lucas Ramage wrote:  
> > > > In  my discussions with other developers, I've found that this
> > > > is the
> > > ​> ​  
> > > biggest concern. Most devs are runnning ~amd64, so they don't
> > > feel that ​> ​  
> > > they can mark things stable.
> > >
> > > W
> > > ​hat about running a stable chroot?​ Are there any tools that can
> > > be used to automate this process?  
> >
> > Yes, a stable chroot is okay, and there is app-portage/tatt that
> > can run through the various USE flag combinations for you.
> >
> > It’s output isn’t terribly helpful, so it takes a little while to
> > understand what it’s trying to tell you.
> >  
> 
> 
> 

Lucas, ping me on irc/email me directly.  I have been maintaining
buildbot in gentoo for awhile and my day job is working on/developing
buildbot scripts.  I am planning out some pkg bumping scripts to help
with regular pkg maintenance.  I hope to have some initial code running
during the holiday break.

There should be a bunch of shared factory code possible as there will
be some overlap in their needs.

-- 
Brian Dolbec <dolsen>

Re: [gentoo-dev] AMD64 Arch Testers needed urgently

[Michael Orlitzky]

On 12/12/2017 01:24 PM, Rich Freeman wrote:
> 
> As far as I'm aware the standing policy already exists that
> maintainers can stabilize their own packages on amd64.

https://bugs.gentoo.org/510198

is this thing on

[gentoo-dev] Re: AMD64 Arch Testers needed urgently

[Duncan]

M. J. Everitt posted on Fri, 15 Dec 2017 01:25:38 +0000 as excerpted:

> 1) Gentoo isn't really interested in having a 'stable' tree or it would
> already be happening. As such, why not cut the Gordian knot, declare
> that this is not something that will happen [soon] and let users make
> their own choices. The [majority of] developers already seem to have ...

The (general) argument for keeping stable seems to consist of two points 
which together have always (to the present at least) tended to 
overwhelmingly tip things in favor of keeping it.  Indeed, either one 
alone can do so, even when the other one is discounted for some reason 
(like some people not putting value in that point).

a) The world of FLOSS isn't a zero-sum game.  People tend to contribute 
where they have interest (whether it's theirs personally or that of their 
employer), and attempting to ban whatever "wasted effort" project in 
favor of what they "should" be contributing to seldom results in /that/ 
much more effort going to the "favored" project after all (and may result 
in less), because if people were interested in that they'd already be 
contributing to it, and the fact that they aren't, or aren't very much, 
tends to mean their interest is elsewhere, and they'll either go 
somewhere else or simply stop contributing as much if they can't 
contribute to what they were interested in, in the first place.

By this point (again, keep in mind we're talking /general/ here, the 
apparent current exception is addressed below), stable exists because 
it's of enough interest to certain contributors that it /continues/ to 
exist, and however much people such as myself (for I'm simply not 
interested in stal^Hble) might consider it a waste of time, killing it 
would be very unlikely to result in invigorating the ~arch I'm primarily 
interested in.  In fact, likely the opposite due to less people as a 
whole contributing.

That's the idealistic point, now perhaps the more practical one...

b) As a point of fact many gentoo sponsors, including those providing 
hosting and/or paying a few gentoo devs to actually do the gentoo work  
they'd very possibly be doing in their spare time anyway (even if it's 
just payment for the 20% aka one day a week open source community 
contribution that's a thing in the tech world), are primarily interested 
in gentoo-stable.

One big and public example is Google, which uses a gentoo base for its 
ChromeOS product.  I'm not privy to details, but from what I've read it 
seems they start with snapshots of gentoo stable, then stabilize them 
further, often feeding their additional patches back upstream to gentoo 
(of course as gentoo feeds stuff back upstream as well, but for some 
things, gentoo /is/ the upstream).  Tho of course if they have reason to 
they can and do pick individual ~arch packages to supplement the stable 
base as well, but the point is, they're interested in /stable/.

And google sponsors, directly or indirectly (some gentooers apparently 
work at unrelated google jobs), a number of gentoo devs, in addition to 
the testing and patches they feed back to us.

Similarly, some of those in our mirror network, for instance, are 
commercial hosters doing it because some of their customers wanted gentoo, 
and providing that local mirror for them, but usable by the public as 
well, is a nice point of convenience to keep those customers /as/ 
customers.

And that's commercial server biz, where the general rule is if it's not 
broke, don't try to "fix" it and in the process risk your view/income/etc 
stream.  So a slow stable actually works well there.


So while I'm personally very much an ~arch and even live-git-build on 
some sets of packages (kde-frameworks/plasma/apps, primarily, with 
occasional others) type of guy and don't /personally/ see much practical 
use for stal^Hble, I certainly see how keeping it makes gentoo bigger and 
better for all of us, including those of us that don't use it personally.


> 2) Whilst there has yet another fine bike-shed emerged on the subject, I
> have only seen one volunteer willing or capable to actually take on
> implementation of anything that has been discussed on this thread. As
> such, you can talk all you like .. nothing will happen until somebody
> actually *does* something ..

Given the context above, What seems to be the problem here is that the 
people that /had/ been interested in, and thus contributing to, gentoo/
amd64-stable, basically the (a) point folks above, seem to have moved on 
to other things, whether inside gentoo or out.

But the (b) point surely remains, so we have a problem.

One could argue that in that case the sponsors should sponsor amd64-
stable folks, but it's not generally that direct, and even if it were, 
getting that worked into the sponsorship pipeline would take time.


I don't have a solution (my reason for posting was to point out that it's 
not as simple as just dropping stable, not really to provide an answer I 
don't have), but can note that I believe there's two people now 
volunteered for it, and of course people have to be aware of it before 
they can realize their personal stake and thus interest in it, thus this 
thread... Even if not enough by itself, you gotta start somewhere, and 
there's at least the two interested, now...

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] AMD64 Arch Testers needed urgently

[R0b0t1]

On Thu, Dec 14, 2017 at 7:25 PM, M. J. Everitt <m.j.everitt@iee.org> wrote:
> On 15/12/17 01:17, R0b0t1 wrote (excerpted):
>> I'm not trying to be confrontational, but asserting an opinion is
>> correct without explaining why that it is so isn't really conducive to
>> arriving at the truth. I understand not wanting to answer if I am
>> completely clueless, and would like to apologize in advance for
>> bothering the developers.
>>
>> I am not very smart, sir.
>>
>> Cheers,
>>      R0b0t1
>>
> With all due respect, Gentoo is not renowned for spoon-feeding ...
>

That is exactly right, sir! I am trying my best to not impose on the
goodwill of the mailing list participants. At the same time, I feel
like I could understand an explanation if one was offered. If any
person's judgement suggests otherwise, however, surely they are
correct and no time should be wasted on such a person as myself.

> Returning to the topic in hand, two key points strike out at me:-
>
> 1) Gentoo isn't really interested in having a 'stable' tree or it would
> already be happening. As such, why not cut the Gordian knot, declare
> that this is not something that will happen [soon] and let users make
> their own choices. The [majority of] developers already seem to have ...
>

This is one of the valid conclusions, especially if the criteria for
stable packages are not changed.

> 2) Whilst there has yet another fine bike-shed emerged on the subject, I
> have only seen one volunteer willing or capable to actually take on
> implementation of anything that has been discussed on this thread. As
> such, you can talk all you like .. nothing will happen until somebody
> actually *does* something .. For all the hating, I will duly credit
> mgorny for producing a consistent quantity of commits across the board
> in Gentoo, and whilst you may not agree with all his bitching (for want
> of a better term) at least he can stand and say "well at least I did
> *something* about it, whether You like it or not ...".
>
> Damnit, there's another $2 from me .. my apologies.
>

I did some initial work trying to fix issues with erlang and rebar,
but I was unable to duplicate the errors due to how tinderboxes work.
A buildbot may not be any more repeatable depending on how it is set
up. Before any work is done I think the problem could be better
characterized, but then you have another long mailing list discussion
that people may or may not be willing to read. People want to do
something, not think about doing something.

In a lot of times that is the better option, even. But not always.

Cheers,
     R0b0t1

Re: [gentoo-dev] AMD64 Arch Testers needed urgently

[Sergey Popov]

[-- Attachment #1.1: Type: text/plain, Size: 414 bytes --]

13.12.2017 21:20, Lucas Ramage пишет:
> W​hat about running a stable chroot?​ Are there any tools that can be
> used to automate this process?

Try gentoo-chrootiez[1], it is written by our fellow gentoo developer -
slyfox. And it is damn simple to use.

[1] - https://github.com/trofi/gentoo-chrootiez

-- 
Best regards, Sergey Popov
Gentoo developer
Gentoo Quality Assurance project lead


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [gentoo-dev] AMD64 Arch Testers needed urgently

[Kent Fredric]

[-- Attachment #1: Type: text/plain, Size: 1789 bytes --]

On Thu, 14 Dec 2017 16:04:17 -0600
R0b0t1 <r030t1@gmail.com> wrote:

> Can you be specific? Human memory is biased towards negative
> experiences. If it's hard to actually describe the multitude of issues
> that mixed systems cause then it is very likely mixed systems do not
> cause many issues.

We have mixed system issues in part *BECAUSE* we use keywords
to migrate through transitions.

So the expectation is that when X gets keyworded stable, that diligence
has been performed to ensure everything else that needs to be stable at
the same time in order to retain function, is also stabilized.

Very much the case with Perl Virtuals.

The design of Perl virtuals is pretty much a bodge for Portage not having
a usable "provides" model and needing to represent certain upstream dynamics
in ways portage can't otherwise handle. ( Due to portage being "package" based
 and upstream being "component" based )


And for that to work, you pretty much need to keyword virtuals, and perl, 
and sometimes perl-core/, in flying formation.

And if you don't get those exposed to you in flying formation, portage gets
very confused and tries to do silly things.

( In my testing rigs I just don't bother with virtuals and have 
a >50 line package.provided file )

But that's just one example of how we use keywords in flying formation as
a mechanism for keeping parts working.

These all stem ultimately from the knowledge that other approaches
( eg: dependdency specifications ) all end in portage getting confused in
 dependency graphs and invoking impossible stabilization requirements.

Hence, keyword mixing is "discouraged if you want a stable working system".

If you're advanced and responsible however, go ahead, please do, and test.






[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]