Re: [gentoo-dev] Re: dropping redundant stable keywords

[Tom Wijsman <TomWij@gentoo.org>]

On Tue, 18 Feb 2014 18:31:28 +0000
"Steven J. Long" <slong@rathaus.eclipse.co.uk> wrote:

> On Fri, Feb 14, 2014 Tom Wijsman wrote:
> > On Thu, 13 Feb 2014 "Steven J. Long" wrote:
> > 
> > > > > Much better for the arch in question to field the bug, than
> > > > > tell the user there is no problem, and we don't care. That
> > > > > way you can get the user involved in stabilisation and AT via
> > > > > that bug, instead of turning them away with priggishness.
> > > > 
> > > > Problems should be visible instead of hidden, as well as
> > > > resolved. A move in work means a move in work, further
> > > > implications are yours...
> > > 
> > > Very gnomic: nothing's being hidden in the above approach.
> > 
> > Filing a bug but not telling the user about it is hiding the
> > problem.
> 
> "The bug" in the above is a bug the user on the arch in question has
> filed, which would be duped to the stabilisation bug (for the arch if
> there's more than one) so the user a) knows there's a newer version,
> which they can try out and help stabilise, and b) isn't kicked back
> with a WONTFIX, when they could be put in touch with the AT instead.
> 
> So the user filed the bug in the first place: by definition they know
> about it.

There are more users than the user that filed the bug; in order to
address all of them, and not just the user of the bug, other
forms of communication need to be taken to address those users. One of
such that came up is using the package.mask reason to do this.

> > > I can't make sense of the rest so I'll move on, noting only that
> > > it's up to the arch team, as to what _they_ decide to kick back to
> > > unstable.
> > 
> > They need manpower to decide it, which they don't have.
> 
> It's still no-one else's call, though given the approach it's easy
> enough to allow a further say 180 days after the "-* arch" marking,

As detailed before, -* has a different meaning defined by policy; if we
want to see that changed it should be brought up for a vote, otherwise
its usage in discussions like these seems to suggest to break an
existing policy. So, I read this as "arch" whenever it is brought up.

> before the maintainer is allowed to mark the package as a whole
> unstable on the arch in question, given no movement.

When the developer marks the package as "minorarch", it's already
unstable on the newer versions; thus I think I'm missing a particular
detail in your paragraph to understand what is meant here, unless you
mean that we keep track of this in a separate way.

> > > And that can work without a problem if we have a mechanism
> > > in place to relieve maintainers of those bugs.
> > 
> > Such mechanism could be to assign those bug to the arch team, this
> > idea came up at FOSDEM; it won't solve the lack of manpower, but it
> > will at least relieve the maintainers and make the problem more
> > visible.
> 
> Exactly. What the AT does with it is their problem: if they don't move
> after another 180 days, they can't complain, but there's at least a
> more formal process, and there's no row, nor even discussion needed.
> You had a chance to stabilise, we've waited, and you've still got the
> stable ebuild in your tree, but now any users on your arch are going
> to get redirected to you, so you can discuss moving the package
> forward or bumping it to unstable.

We've had discussion here and on IRC after the similar QA policy; it's
what put it in a state of discussing it again, which will happen
tomorrow evening during the Gentoo QA meeting.

> To me that makes the action something positive, rather than negative.
> Especially coupled with an indicator in the package manager output,
> similar to how unstable ebuilds are marked if you're running stable,
> or forced-rebuilds. That would spur more users to help, since they're
> using the package in question, and are on a slow arch.

+1, package.mask or so could serve well here.
 
> > > Personally I'd do it after 45 days to speed things up, and let the
> > > ATs concerned, take the bug as and when (eg turn the stabilisation
> > > into a tracker for the slow archs concerned, if there are
> > > multiple.)
> > 
> > Since this is a soft measure I've seen a lot of people want the
> > time to be longer; if it still continues to be a problem, then
> > shortening it might be a solution but I think we'll want to avoid a
> > too hard approach for now. 45 days are over before you know it...
> 
> No the whole point is that the slower archs are not losing a stable
> ebuild in their tree: so it's fine for the maintainer to wait a
> shorter time before marking it as no-longer-my-problem. It's not the
> same as dropping the ebuild altogether; it leaves the arch-tree
> intact, and still marks a new phase for interested users to
> collaborate around, while relieving the maintainer of the burden.
> 
> And if desired, that can be seen as the start of a phase toward
> bumping it to unstable, but with more notice, and a defined process.

Ah, no-longer-my-problem; yeah, seems we're having a different timeout
in mind then. I'd still would like to see this rather as a last resort
solution ("it's been a long while now, ..."), rather than a short ("yes,
I can ditch it now, ...") solution for when it is really needed; as we
just want to relieve some lingering maintenance weight, while not
moving more weight than needed to the architecture teams.

> > The part about ATs taking the bug sounds like what came up at
> > FOSDEM. +1
> 
> <snip>
> > > The wonderful thing about policy is that it reflects (or is
> > > supposed to) consensus opinion with light to contemporaneous
> > > circumstance. Since circumstances change, so too must policy be
> > > open to change or adaptation, in line with basic principles.
> > 
> > In this case -* works fine for what it intends to work; changing the
> > policy to redefine it to fit another purpose, while no longer
> > reflecting its original purpose is what we should avoid at all cost.
> 
> Utter nonsense. Back that up with some consequent to justify why we
> should "avoid it at all costs", bearing in mind the below.

Already did that in the other thread, let's not bring it up again; it
was shown that multiple words' meaning were bent to make it mean the
other thing that you want to use it for here, while that changes the
borders of its original meaning such that you can no longer it that way.

By all means, it is possible to change it with a vote; but just plain
out using it in a different way than how we defined it is inconsistent.

> > > So let's look at extending it, since there is *no* technical
> > > problem:
> > 
> > You have colons after the above quoted sentence with nothing after
> > it.
> > 
> > (An eye for an eye... :D)
> 
> Don't be so facile: the proposed extended policy is what follows.

Exactly it does; so, let's avoid the use of that reply. :)

> > > 'The redundant -* keyword is a metadata marker.
> > 
> > The keyword has a meaning, therefore it is not redundant.
> 
> It is technically redundant, since the package manager does not infer
> any existing keywords, so there is nothing to strip. As a metadata
> marker, no ofc it's not redundant: but that's what I'm proposing to
> use.

It could be towards the package manager, like quite a few other things
(eg. the exact name of a license); that doesn't imply that it's
redundant in general, as they have their specific meaning.

> > > It is used to indicate, in line with the semantic of "strip all",
> > > that the ebuild in question can only be used for the specific
> > > archs noted for one of two reasons:
> > > 
> > > 1) The package-maintainer has stabilised a newer version on at
> > > least one arch personally, the ATs for the archs listed have
> > > taken longer than XX days to test and stabilise, and the
> > > maintainer would otherwise drop the ebuild altogether.
> > > 
> > > 2) The package version is not worth trying to test on unlisted
> > > archs.'
> > 
> > (1) changes its meaning in a way that you can no longer interpret
> > the keyword solely as (2). The whole purpose of (2) is that you can
> > interpret it that you should not test it as it was found not to
> > work; (1) is different from that, as it means that there was no
> > manpower to test it. (1) would move ebuilds to -* and be
> > misinterpreted as (2).
> 
> Nope: 1) implies 2) so any existing use of the marker solely in the
> context of 2) still works. It certainly is not worth testing an old
> ebuild that would have been dropped on an unlisted arch: whoever is
> looking at the ebuild, should instead use a newer version.

1) means a package is unstable (~), 2) means a package is unusable (-),
there is no hence no implication; even if there were, this discussion is
solely about dropping it from stable. -* does more than what is needed.

"arch minorarch" -> "minorarch" on the old version can suffice, with
"arch ~minorarch" on the newer version; then what is "-* minorarch"
needed for? What is its additional benefit over what I suggest here?

> > > The policy which flows from that is:
> > > 
> > > 'In the first case, it is QA policy that a comment of the form:
> > > # STABLEREQ: https://bugs.gentoo.org/show_bug.cgi?id=NNNNNN
> > > is required on the line immediately above the KEYWORDS
> > > declaration.
> > > 
> > > This is to aid both automated identification, and human
> > > collaboration.'
> > > 
> > > (The latter explains why a URL, not a bug id, is required. We're
> > > trying to get end-users to help us, so let's make it easy for
> > > them.)
> > >
> > > I'd personally add:
> > > 'It is envisaged that the line will be added by an automated tool
> > > at some point.'
> > > 
> > > ..as well as a requirement for a bug id in the second case, but I
> > > don't know it well enough; I'd certainly want some sort of
> > > tracking.
> > 
> > This yields extra commits; comments in ebuilds are directed towards
> > maintainers, for users we should use another approach to contact
> > them.
> 
> No it keeps all the useful info in one place: the ebuild, where it's
> easy to edit and see. The commit is already implicit in dropping the
> ebuild to "-* slower arch": all the policy does is require a link to
> the original stabilisation bug, which may be a tracker if multiple
> archs are involved. However none of that is the maintainer's concern:
> all they need do, when facing this rare situation, is change the
> keywords and add a link to the bug (commenting is SOP for most unusual
> situations in ebuilds), instead of dropping the arch's stable version.

It's an extra commit to add the comment prior to stabilization.

> The link is easy to extract for an automated external to show to the
> user, like the mangler in the above, or a porcelain layer, given an
> indicator from the mangler.

This can be implemented by checking Bugzilla; there's no need for a
comment, as that leaves room for missing comments misrepresenting that
there is no stabilization when there is in fact stabilization going on.

> It's also a helluva lot less work than package.mask, as well as
> keeping all the info in the ebuild, where it's easier to handle.

Adding a generic or specific text to package.mask can be scripted;
beyond that, it is usable. package.mask is easier for QA to handle;
given that it's a single file, instead of ebuilds across the tree.

> > > It's hardly an onerous requirement, and a small price to pay: if
> > > we have a policy for how a maintainer drops an ebuild from his
> > > queue due to it being stabilised, which we can trigger scripts
> > > from, we avoid the arguments every year or so, and stop archs from
> > > being borked. We can also speed it up, since we have a mechanism
> > > in-place to support it, as opposed to ad-hoc, flawed
> > > decision-making.
> > > 
> > > The thing I think that's missing from this debate, is an
> > > acknowledgement, or an understanding, that arch-teams are all
> > > effectively working on their own variant of the shared Gentoo
> > > tree. (This includes the concept of upgrades working as smoothly
> > > as they do on other archs, at the PM level.) Similar to other
> > > portability efforts, the tree must support them in that, not make
> > > it harder.
> > > 
> > > Especially not because another developer doesn't care about the
> > > arch in question. That's *natural*, but _cannot_ be cause for
> > > dropping stable ebuilds. The only issue is to take the bugs off
> > > their hands, and we can do that with a simple tweak in metadata,
> > > so ffs let's get on and do it.
> > 
> > +1 on this thought.
> 
> Well I wish you could see that steev's solution, with a bit of
> standard commenting, fulfils both your +1s

The +1 is just on the thought part, not the entire thing.

> and doesn't lead to broken arch-trees. It also doesn't require any
> communication with a non-responsive arch, and can form the basis of a
> much more useful mode of collaboration, from where I'm sitting.

It however leads to different things and requires different things; so,
there are trade-offs to be made. Some of the other solutions put
forward also make collaboration an option; even outside of all
solutions, collaboration by joining an arch team is already possible.

> Post-facto complaints about an upstream project decision to keep
> migration code, or about the anguish of seeing an older version
> stable on some arch you don't even care about (or you wouldn't be
> dropping their last stable version) are an order-of-magnitude less
> concerning. By all means address them (up to the project in the
> first case, adjust your script in the second) but let's at least
> admit they are less of a problem than this continual argument, and
> the consequent ill-feeling brought about by lack of a defined
> action to take.

Action by the QA team was taken last month, and tomorrow another could
be made; this thread is making progress, and participating therein is
part of a preparation for the meeting that will take place tomorrow.

> The issue about confusion with multiple archs has already been
> addressed by making the original stabilisation bug a tracker for
> the N slow archs, if there are more than one.
> 
> Most of this can be automated, or is at least amenable to it. And
> that discussion is *far* more interesting than "you broke our arch"
> vs "so what, you're too slow".

-- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : TomWij@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D