[gentoo-dev] blocking mixed versions of split QT libraries

[gentoo-dev] blocking mixed versions of split QT libraries

[Alex Alexander]

QT doesn't work well when mixed versions of its core libraries are
installed. Usually an emerge -avDu world solves the problem, but some
users tend to avoid this.

For example, lets say you have parts of QT 4.4.2 on your system. QT
4.5.1 is available and a user decides to manually update qt-core, or
to install KDE which has a QT dependency on a QT library not
installed. In these cases, portage will update only a part of the
installed QT libraries, leaving the system in a mixed state.

QT based apps don't like that. Others will refuse to build, others
will fail to run.

We've managed to experimentally block partial QT upgrades by adding an
RDEPEND to all QT libraries [1] in qting-edge overlay. Portage
understands this and throws out B blocks if you try to change versions
only in parts of QT, but upgrades/downgrades fine if you do them all
at once (or use -Du world).

This "fix" also catches stale QT libraries that nothing depends on
anymore because the user has removed whatever required them (and never
ran --depclean).

Unfortunately we've got reports from paludis users stating that they
can't update QT from qting-edge anymore.

What I'd like to discuss is the following:

1) Is there a saner way to achieve our goal of doing whatever is
possible to avoid mixed QT versions?
2) Is our implementation considered correct and acceptable by the PMS guys?
3) Whats the general Gentoo Policy on mixed versions? Do we care, or
is our policy "please -Du world"?

We've also managed to achieve the same thing by adding PDEPENDs to
each QT library for any other QT libraries that depend on it:

i.e. if qt-xmlpatterns depends on qt-gui, we add the following to qt-gui:
PDEPEND="
|| ( !x11-libs/qt-xmlpatterns ~x11-libs/qt-xmlpatterns-${PV} )
"

the above (expanded for all libraries) has the same effect as the [1]
RDEPEND but looks a bit more hackish.

thanks

[1] lines 30-59
http://github.com/gentoo-qt/qting-edge/blob/master/eclass/qt4-build-edge.eclass

--
Alex Alexander || wired
Gentoo QT && KDE Herd Tester
http://www.linuxized.com

Re: [gentoo-dev] blocking mixed versions of split QT libraries

[Alistair Bush]

Alex Alexander wrote:
> QT doesn't work well when mixed versions of its core libraries are
> installed. Usually an emerge -avDu world solves the problem, but some
> users tend to avoid this.
> 
> For example, lets say you have parts of QT 4.4.2 on your system. QT
> 4.5.1 is available and a user decides to manually update qt-core, or
> to install KDE which has a QT dependency on a QT library not
> installed. In these cases, portage will update only a part of the
> installed QT libraries, leaving the system in a mixed state.
> 
> QT based apps don't like that. Others will refuse to build, others
> will fail to run.
> 
> We've managed to experimentally block partial QT upgrades by adding an
> RDEPEND to all QT libraries [1] in qting-edge overlay. Portage
> understands this and throws out B blocks if you try to change versions
> only in parts of QT, but upgrades/downgrades fine if you do them all
> at once (or use -Du world).
> 
> This "fix" also catches stale QT libraries that nothing depends on
> anymore because the user has removed whatever required them (and never
> ran --depclean).
> 
> Unfortunately we've got reports from paludis users stating that they
> can't update QT from qting-edge anymore.

From what I understand you are utilizing portages ability to
automagically resolve blockers when all blockers will be resolved within
the current command.  Agree?? or is it the fact that you are doing
!>x11-libs/qt-assistant-${PV}-r9999 that is causing the paludis problem?

I would suggest that you just tell paludis users to use --dl-blocks
discard when updating qt.  After looking at the eclass im not sure
whether it will work or not.  im assuming that discarding blocks will
just ignore everything, but I haven't tested it so can't be sure.

> 
> What I'd like to discuss is the following:
> 
> 1) Is there a saner way to achieve our goal of doing whatever is
> possible to avoid mixed QT versions?

I don't believe so, not within current ways of declaring dependencies.

> 2) Is our implementation considered correct and acceptable by the PMS guys?
> 3) Whats the general Gentoo Policy on mixed versions? Do we care, or
> is our policy "please -Du world"?

I say we should be stopping them from happening.

> 
> We've also managed to achieve the same thing by adding PDEPENDs to
> each QT library for any other QT libraries that depend on it:
> 
> i.e. if qt-xmlpatterns depends on qt-gui, we add the following to qt-gui:
> PDEPEND="
> || ( !x11-libs/qt-xmlpatterns ~x11-libs/qt-xmlpatterns-${PV} )
> "
> 
> the above (expanded for all libraries) has the same effect as the [1]
> RDEPEND but looks a bit more hackish.
> 

And I would agree with the hackish comment.

> thanks

Good work btw.

Alistair.

Re: [gentoo-dev] blocking mixed versions of split QT libraries

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1319 bytes --]

On Mon, 18 May 2009 13:04:27 +0300
Alex Alexander <alex.alexander@gmail.com> wrote:
> Unfortunately we've got reports from paludis users stating that they
> can't update QT from qting-edge anymore.

Paludis treats blocks as strong, the way Portage used to and the way
PMS defined them until we had to retroactively change it to allow
Portage's newer behaviour...

> 1) Is there a saner way to achieve our goal of doing whatever is
> possible to avoid mixed QT versions?

Not really. There's no particularly good mechanism for ensuring equal
versions of things where not everything has to be installed. The best
option I can think of is to have a meta package called, say, split-qt,
and to do all your external (not inter-qt-library) dependencies as:

    x11-libs/split-qt[gui][xmlpatterns]

and then have x11-libs/split-qt's deps be like:

    gui? ( ~x11-libs/qt-gui-${PV} )

> 2) Is our implementation considered correct and acceptable by the PMS
> guys?

The way PMS defines blockers has been rewritten to allow both what
Portage used to do and what Portage now does. It's fairly horrible, but
unfortunately Zac changed Portage's behaviour (breaking anything that
relied upon strong blockers, hence the quickly-hacked-in !! blocker
hack) without EAPI control.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] blocking mixed versions of split QT libraries

[Petteri Räty]

[-- Attachment #1: Type: text/plain, Size: 597 bytes --]

Ciaran McCreesh wrote:
> On Mon, 18 May 2009 13:04:27 +0300
> Alex Alexander <alex.alexander@gmail.com> wrote:
>> Unfortunately we've got reports from paludis users stating that they
>> can't update QT from qting-edge anymore.
> 
> Paludis treats blocks as strong, the way Portage used to and the way
> PMS defined them until we had to retroactively change it to allow
> Portage's newer behaviour...
> 

Unfortunate but what does this have to do with the original question?
EAPI 2 moved to the new soft behavior and as far as I know all qt
ebuilds are EAPI 2.

Regards,
Petteri


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 261 bytes --]

Re: [gentoo-dev] blocking mixed versions of split QT libraries

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1028 bytes --]

On Mon, 18 May 2009 17:47:52 +0300
Petteri Räty <betelgeuse@gentoo.org> wrote:
> Ciaran McCreesh wrote:
> > On Mon, 18 May 2009 13:04:27 +0300
> > Alex Alexander <alex.alexander@gmail.com> wrote:
> >> Unfortunately we've got reports from paludis users stating that
> >> they can't update QT from qting-edge anymore.
> > 
> > Paludis treats blocks as strong, the way Portage used to and the way
> > PMS defined them until we had to retroactively change it to allow
> > Portage's newer behaviour...
> 
> Unfortunate but what does this have to do with the original question?
> EAPI 2 moved to the new soft behavior and as far as I know all qt
> ebuilds are EAPI 2.

The definition of soft behaviour allows soft blockers to be treated
identically to hard blockers. We had to do it this way because
Portage's rules for soft blockers are too fuzzy and arbitrary to turn
into a formal specification -- they were a "code first, think later"
solution with which we can't do anything useful.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] blocking mixed versions of split QT libraries

[Alex Alexander]

> From what I understand you are utilizing portages ability to
> automagically resolve blockers when all blockers will be resolved within
> the current command.  Agree?? or is it the fact that you are doing
> !>x11-libs/qt-assistant-${PV}-r9999 that is causing the paludis problem?

Yes, portage's auto-resolving ability thats exactly what we're using
to make this happen.

> I would suggest that you just tell paludis users to use --dl-blocks
> discard when updating qt.  After looking at the eclass im not sure
> whether it will work or not.  im assuming that discarding blocks will
> just ignore everything, but I haven't tested it so can't be sure.

Well since there's no other obvious way to achieve the whole thing,
this seems like the only option for paludis users for now. If it
works.

>> 1) Is there a saner way to achieve our goal of doing whatever is
>> possible to avoid mixed QT versions?
>
> I don't believe so, not within current ways of declaring dependencies.

maybe the PMS guys could implement something... :D

>> 2) Is our implementation considered correct and acceptable by the PMS guys?
>> 3) Whats the general Gentoo Policy on mixed versions? Do we care, or
>> is our policy "please -Du world"?
>
> I say we should be stopping them from happening.
>
> Good work btw.

Thank you for your thoughts

--
Alex Alexander || wired
Gentoo QT && KDE Herd Tester
http://www.linuxized.com

Re: [gentoo-dev] blocking mixed versions of split QT libraries

[Alex Alexander]

On Mon, May 18, 2009 at 17:21, Ciaran McCreesh
<ciaran.mccreesh@googlemail.com> wrote:
> Not really. There's no particularly good mechanism for ensuring equal
> versions of things where not everything has to be installed. The best
> option I can think of is to have a meta package called, say, split-qt,
> and to do all your external (not inter-qt-library) dependencies as:
>
>    x11-libs/split-qt[gui][xmlpatterns]
>
> and then have x11-libs/split-qt's deps be like:
>
>    gui? ( ~x11-libs/qt-gui-${PV} )

how would that solve a user's "emerge -av1 qt-core" when a new Qt
version becomes available?

--
Alex Alexander || wired
Gentoo QT && KDE Herd Tester
http://www.linuxized.com

Re: [gentoo-dev] blocking mixed versions of split QT libraries

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 611 bytes --]

On Mon, 18 May 2009 19:42:02 +0300
Alex Alexander <alex.alexander@gmail.com> wrote:
> >    x11-libs/split-qt[gui][xmlpatterns]
> >
> > and then have x11-libs/split-qt's deps be like:
> >
> >    gui? ( ~x11-libs/qt-gui-${PV} )
> 
> how would that solve a user's "emerge -av1 qt-core" when a new Qt
> version becomes available?

It wouldn't, although it would be fixed as soon as someone tried to
install a package with a Qt dep.

Dependencies the way they are now aren't really expressive enough to
handle what you're after -- split packages are considered unrelated.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: [gentoo-dev] blocking mixed versions of split QT libraries

[Alex Alexander]

On Mon, May 18, 2009 at 19:51, Ciaran McCreesh
<ciaran.mccreesh@googlemail.com> wrote:
> It wouldn't, although it would be fixed as soon as someone tried to
> install a package with a Qt dep.
>
> Dependencies the way they are now aren't really expressive enough to
> handle what you're after -- split packages are considered unrelated.

so the RDEPEND in qting-edge seems to be the only working solution for now...

is paludis expected to behave like portage in the near future
regarding these blocks?

are there any plans to add support for these kinds of cases in the
PMS? other sets of packages could probably benefit from such a feature
as well.

thanks

--
Alex Alexander || wired
Gentoo QT && KDE Herd Tester
http://www.linuxized.com

Re: [gentoo-dev] blocking mixed versions of split QT libraries

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1331 bytes --]

On Mon, 18 May 2009 20:01:22 +0300
Alex Alexander <alex.alexander@gmail.com> wrote:
> is paludis expected to behave like portage in the near future
> regarding these blocks?

Probably not. My issue with the way Portage does soft blocks is that
it's way too arbitrary, fuzzy and ill defined.

There were plans to do blocks properly (include annotations that would
let the developer tell the package manager to point the user to a URL
explaining the block and how to resolve it) back before Zac went and
did his own thing. One of the goals was to tell the package manager
exactly what was meant by the block, allowing the package manager to
come up with a much more sensible and far less dangerous solution. If
those plans are ever revived, Paludis would support them.

> are there any plans to add support for these kinds of cases in the
> PMS? other sets of packages could probably benefit from such a feature
> as well.

I don't recall any existing discussion about such a feature (beyond me
moaning in pre-EAPI days about vim/gvim/vim-core breaking in the same
way Qt does). So... The way to start is probably by identifying the
problem in detail, and identifying other groups of packages affected by
similar issues, so we can work out what exactly it is we'd be looking
to fix.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: [gentoo-dev] blocking mixed versions of split QT libraries

[Maciej Mrozowski]

On Monday 18 of May 2009 16:52:19 Ciaran McCreesh wrote:
> On Mon, 18 May 2009 17:47:52 +0300
[snip]
> The definition of soft behaviour allows soft blockers to be treated
> identically to hard blockers. We had to do it this way because
> Portage's rules for soft blockers are too fuzzy and arbitrary to turn
> into a formal specification -- they were a "code first, think later"
> solution with which we can't do anything useful.

Not sure who is 'we' there, but Portage team already made is useful. Basic 
portage rule for soft-blocks behaviour is "no longer referenced (a'ka 'soft') 
blocked package can be uninstalled cleanly without user intervention" - it's 
well defined behaviour and possible subject of formal specification - it's 
just up to PM to implement block resolution algorithms for corner cases (those 
would not be the subject of formal specification of course, it's just an 
implementation detail like whether to apply rule like order set by '||' 
operator takes precedence over '!' block or order of block appearance in 
RDEPEND sets block precedence) - Zac did good job there saving users 
(especially KDE users) from nightmare of handling all package 
refactoring/blocks manually.

-- 
regards
MM


----------------------------------------------------------------------
Oryginalne przepisy na grilla. Zaskocz swoich gosci!
Sprawdz >>> http://link.interia.pl/f217c

Re: [gentoo-dev] blocking mixed versions of split QT libraries

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 717 bytes --]

On Mon, 18 May 2009 19:15:59 +0200
Maciej Mrozowski <reavertm@poczta.fm> wrote:
> Not sure who is 'we' there, but Portage team already made is useful.
> Basic portage rule for soft-blocks behaviour is "no longer referenced
> (a'ka 'soft') blocked package can be uninstalled cleanly without user
> intervention"

That's not in the least bit well defined, and it's also extremely
dangerous.

> Zac did good job there saving users (especially KDE users) from
> nightmare of handling all package refactoring/blocks manually.

The nightmare only existed because of abuse of that feature. Had blocks
kept their original meaning, people would not have abused them to the
same extent.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: [gentoo-dev] blocking mixed versions of split QT libraries

[Maciej Mrozowski]

On Monday 18 of May 2009 19:26:58 Ciaran McCreesh wrote:
> On Mon, 18 May 2009 19:15:59 +0200
>
> Maciej Mrozowski <reavertm@poczta.fm> wrote:
> > Not sure who is 'we' there, but Portage team already made is useful.
> > Basic portage rule for soft-blocks behaviour is "no longer referenced
> > (a'ka 'soft') blocked package can be uninstalled cleanly without user
> > intervention"
>
> That's not in the least bit well defined, and it's also extremely
> dangerous.

Please elaborate on that.
(to make it simple, let me use portage terminology below)
Everything what user should be interested in is expected to be in 'world' file 
or 'world_sets' or pulled as dependencies. This I define by "referenced". 
Everything else like things installed temporarily, no longer pulled packages, 
are subject of 'depclean'. I don't see why pruning those you consider 
extremely dangerous - especially when there are parameters like --pretend or 
--ask.
While "no longer referenced" term may be considered not fully defined as it 
does not specify the way things become not referenced anymore (as packages may 
be determined to be referenced later, during block resolution stage, but 
that's implementation specific) - the term "referenced" is defined well 
enough.
Nobody is (for now) expecting every PMS compliant package manager to return 
identical dependency graph in corner cases.

> > Zac did good job there saving users (especially KDE users) from
> > nightmare of handling all package refactoring/blocks manually.
>
> The nightmare only existed because of abuse of that feature. Had blocks
> kept their original meaning, people would not have abused them to the
> same extent.

Unfortunately in packaging of dynamically developed applications like whole 
KDE environment (with Gentoo KDE split package policy - ~250 ebuilds with 
every release) it's impossible not to 'abuse' blocks - either to handle file 
collisions issues, or removed/moved libraries (by upstream). Not sure what was 
original meaning of blocks you're referring to, either way - there is no rule 
stating ">= N uses of feature X in scope Y in time frame T is considered 
abuse" - that being said, I'm surprised you're looking for cheap excuse for 
providing no working block auto-resolution mechanism (or maybe there is some 
I'm not aware of) - it does not need to be in any Gentoo specification after 
all - just to make things easier for users.

-- 
regards
MM

----------------------------------------------------------------------
Zrob sobie prezent. Wygraj nagrode!
Sprawdz >>  http://link.interia.pl/f2176 

Re: [gentoo-dev] blocking mixed versions of split QT libraries

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 2678 bytes --]

On Mon, 18 May 2009 20:05:51 +0200
Maciej Mrozowski <reavertm@poczta.fm> wrote:
> > That's not in the least bit well defined, and it's also extremely
> > dangerous.
> 
> Please elaborate on that.

With Portage's soft blocks, there's no guarantee that your blocks will
do anything at all. Soft blocks are ignored if "they'll be fixed
later", but then there's no guarantee that later will be reached.

> Everything else like things installed temporarily, no longer pulled
> packages, are subject of 'depclean'. I don't see why pruning those
> you consider extremely dangerous - especially when there are
> parameters like --pretend or --ask.

It's unrealistic to assume that depclean's going to be accurate at
every given moment, especially given Portage's massively overoptimistic
treatment of slots. It's also a very bad idea to remove packages
without the user explicitly giving permission to do so.

> > > Zac did good job there saving users (especially KDE users) from
> > > nightmare of handling all package refactoring/blocks manually.
> >
> > The nightmare only existed because of abuse of that feature. Had
> > blocks kept their original meaning, people would not have abused
> > them to the same extent.
> 
> Unfortunately in packaging of dynamically developed applications like
> whole KDE environment (with Gentoo KDE split package policy - ~250
> ebuilds with every release) it's impossible not to 'abuse' blocks -
> either to handle file collisions issues, or removed/moved libraries
> (by upstream). Not sure what was original meaning of blocks you're
> referring to, either way - there is no rule stating ">= N uses of
> feature X in scope Y in time frame T is considered abuse"

Blocks are supposed to be an absolute last resort, not something you
throw around willy-nilly to try to get Portage to do what you're after.

> - that being said, I'm surprised you're looking for cheap excuse for
> providing no working block auto-resolution mechanism (or maybe there
> is some I'm not aware of) - it does not need to be in any Gentoo
> specification after all - just to make things easier for users.

Bah. I'm looking for a way of doing this properly, as I was before Zac
went and broke blockers in Portage. Such a way would:

* work by explaining the reason for the blocker, rather than sort-of
  stating the expected resolution.

* provide mechanisms for explaining the block in detail to the user,
  along with instructions on how to resolve it.

* be based around tree requirements, not some side effects of some code
  someone happened to write without considering the implications.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: [gentoo-dev] blocking mixed versions of split QT libraries

[Patrick Lauer]

On Monday 18 May 2009 20:19:24 Ciaran McCreesh wrote:
> On Mon, 18 May 2009 20:05:51 +0200
>
> Maciej Mrozowski <reavertm@poczta.fm> wrote:
> > > That's not in the least bit well defined, and it's also extremely
> > > dangerous.
> >
> > Please elaborate on that.
>
> With Portage's soft blocks, there's no guarantee that your blocks will
> do anything at all. Soft blocks are ignored if "they'll be fixed
> later", but then there's no guarantee that later will be reached.

In terms of the on-disk result it's invariant, the result is what you'd 
expect. There are intermediate stages that are "inconsistent" / "unclean", but 
as these are known and temporary they are an acceptable solution.

> > Everything else like things installed temporarily, no longer pulled
> > packages, are subject of 'depclean'. I don't see why pruning those
> > you consider extremely dangerous - especially when there are
> > parameters like --pretend or --ask.
>
> It's unrealistic to assume that depclean's going to be accurate at
> every given moment, especially given Portage's massively overoptimistic
> treatment of slots. It's also a very bad idea to remove packages
> without the user explicitly giving permission to do so.
Which either means that the deps are wrong/incomplete or that portage has 
algorithmic flaws which should be corrected. 
I'd expect you to at least point to the relevant bugs and not just state some 
emotional mumbo-jumbo.

Plus the packages that are removed were not installed explicitly, and to 
satisfy the needs of the user they are removed. This reflects the demands of 
the user ("Give me this app!" or "Update this pile of apps!") quite well.

> Blocks are supposed to be an absolute last resort, not something you
> throw around willy-nilly to try to get Portage to do what you're after.

No, blocks are what you need to keep the set of installed packages consistent. 
They need to be used as much as the flaws and conflicts of software packaging 
require.
Any emotional statement like "last resort", "obviously", "willy-nilly" or 
"cute" has no place in a discussion about needed technical features.


> > - that being said, I'm surprised you're looking for cheap excuse for
> > providing no working block auto-resolution mechanism (or maybe there
> > is some I'm not aware of) - it does not need to be in any Gentoo
> > specification after all - just to make things easier for users.
>
> Bah. I'm looking for a way of doing this properly, as I was before Zac
> went and broke blockers in Portage.
s/broke/fixed/

> Such a way would:
>
> * work by explaining the reason for the blocker, rather than sort-of
>   stating the expected resolution.
That's dumb ;) (Sorry, emotional statement)
I mean, it does not help to solve the issue and requires user interaction 
where an automated solution has been working reliably for quite some time.

Providing extra information would be nice, but causes more work for the 
package maintainer and the user for little benefit.

> * provide mechanisms for explaining the block in detail to the user,
>   along with instructions on how to resolve it.
User interaction where automated resolution works sounds like a step backwards 
to me. Maybe refining the rules for automated resolution would be a more 
appropriate solution?

> * be based around tree requirements, not some side effects of some code
>   someone happened to write without considering the implications.
What is a tree requirement? (Nice buzzword btw)

And as many devs and users benefit quite a lot from the portage solution, 
which zmedico did not dream up without thinking about the impact on users, I 
find it very rude and condescending of you to disrespect the solution without 
offering an alternative.

As you seem to understand the problem domain I'd expect a coherent unambiguous 
proposal instead of vague accusations and emotional terms that do not help in 
any way to improve the situation.

Re: [gentoo-dev] blocking mixed versions of split QT libraries

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 3599 bytes --]

On Mon, 18 May 2009 21:08:25 +0200
Patrick Lauer <patrick@gentoo.org> wrote:
> In terms of the on-disk result it's invariant, the result is what
> you'd expect. There are intermediate stages that are "inconsistent" /
> "unclean", but as these are known and temporary they are an
> acceptable solution.

No, they're temporary except when things go wrong, at which point
there's no indication that they'll be fixed.

> > It's unrealistic to assume that depclean's going to be accurate at
> > every given moment, especially given Portage's massively
> > overoptimistic treatment of slots. It's also a very bad idea to
> > remove packages without the user explicitly giving permission to do
> > so.
> Which either means that the deps are wrong/incomplete or that portage
> has algorithmic flaws which should be corrected. 
> I'd expect you to at least point to the relevant bugs and not just
> state some emotional mumbo-jumbo.

Look at the new slot deps in EAPI 3.

> > Bah. I'm looking for a way of doing this properly, as I was before
> > Zac went and broke blockers in Portage.
> s/broke/fixed/

No, broke. What he did was in direct violation of PMS and in direct
violation of assumptions made by many packages.

> > * work by explaining the reason for the blocker, rather than sort-of
> >   stating the expected resolution.
> That's dumb ;) (Sorry, emotional statement)
> I mean, it does not help to solve the issue and requires user
> interaction where an automated solution has been working reliably for
> quite some time.

Uhm. Knowing the reason for the block lets the package manager solve
the problem in the most intelligent available way. Merely being sort-of
told the expected resolution does not.

> > * provide mechanisms for explaining the block in detail to the user,
> >   along with instructions on how to resolve it.
> User interaction where automated resolution works sounds like a step
> backwards to me. Maybe refining the rules for automated resolution
> would be a more appropriate solution?

Automated resolution is not always possible or a good idea. Also,
having more information available for the user and being able to
suggest an automated resolution are not in the least bit contradictory.

> > * be based around tree requirements, not some side effects of some
> > code someone happened to write without considering the implications.
>
> What is a tree requirement? (Nice buzzword btw)

A tree requirement is one that comes about as a result of studying what
ebuilds do now and what they'd like to do in the future, rather than
one that comes around based upon what someone happens to code.

> And as many devs and users benefit quite a lot from the portage
> solution, which zmedico did not dream up without thinking about the
> impact on users, I find it very rude and condescending of you to
> disrespect the solution without offering an alternative.

...except for the things that it broke, which necessitated shoving !!
blocks in at the last minute. And I'll remind you that there were
discussions about a proper blocker solution before Zac went and did his
thing, and the overwhelming view was that a solution based around the
things I've been discussing was what was wanted.

> As you seem to understand the problem domain I'd expect a coherent
> unambiguous proposal instead of vague accusations and emotional terms
> that do not help in any way to improve the situation.

See the discussions we had back when the only kind of blocker was a
hard, single ! block.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: [gentoo-dev] blocking mixed versions of split QT libraries

[Patrick Lauer]

On Monday 18 May 2009 21:20:10 Ciaran McCreesh wrote:
> On Mon, 18 May 2009 21:08:25 +0200
>
> Patrick Lauer <patrick@gentoo.org> wrote:
> > In terms of the on-disk result it's invariant, the result is what
> > you'd expect. There are intermediate stages that are "inconsistent" /
> > "unclean", but as these are known and temporary they are an
> > acceptable solution.
>
> No, they're temporary except when things go wrong, at which point
> there's no indication that they'll be fixed.
>
When things go wrong they go wrong. Indeed.
At the moment I can't think of an obvious failure mode, so I guess I'll have 
to wait for you to refresh my memory. Until then I'll just be happy that KDE, 
poppler, e2fsprogs and a few other apps refused to fail and saved me lots of 
time and trouble (and some failure modes that are really frustrating) by just 
magically upgrading in the right sequence, avoiding error-prone user 
interaction (e2fsprogs had one funny bug that a few hundred users found) and 
allowing me to be a lazy cat.

> > > It's unrealistic to assume that depclean's going to be accurate at
> > > every given moment, especially given Portage's massively
> > > overoptimistic treatment of slots. It's also a very bad idea to
> > > remove packages without the user explicitly giving permission to do
> > > so.
> >
> > Which either means that the deps are wrong/incomplete or that portage
> > has algorithmic flaws which should be corrected.
> > I'd expect you to at least point to the relevant bugs and not just
> > state some emotional mumbo-jumbo.
> Look at the new slot deps in EAPI 3.
So the deps were not precise enough, and now we improve that. Which means that 
portage will only get more precise in the future. Awesome!

> No, broke. What he did was in direct violation of PMS and in direct
> violation of assumptions made by many packages.
So PMS did once again not reflect reality, and there were some buggy packages. 
Maybe we should spend more time on making PMS a standard that documents 
current and past behavior instead of omitting things (mtime, bash 3.1) or 
keeping it ambiguous so that two implementations can be compliant while 
delivering incompatible results?
And then ... well ... bugs are there to be fixed.

> > > * work by explaining the reason for the blocker, rather than sort-of
> > >   stating the expected resolution.
> >
> > That's dumb ;) (Sorry, emotional statement)
> > I mean, it does not help to solve the issue and requires user
> > interaction where an automated solution has been working reliably for
> > quite some time.
>
> Uhm. Knowing the reason for the block lets the package manager solve
> the problem in the most intelligent available way. Merely being sort-of
> told the expected resolution does not.
You're contradicting yourself there - until now you only mentioned user-
visible messages, which now suddenly become hints for the package manager. 

Don't try to confuse issues like that.

Now I do wonder what you can "explain" about a block - "some files collide" ? 
How does that help the package manager decide what to do? What can it do, 
except unmerging one package or refusing to merge another one? How does that 
differ from the portage solution to the blocker situation?

> Automated resolution is not always possible 
Indeed, in such cases you can let the package manager abort

> or a good idea. 
Again a subjective thing. This bacon here, buying that was a good idea. I 
should give you some, it's totally awesome!


> Also,
> having more information available for the user and being able to
> suggest an automated resolution are not in the least bit contradictory.
... which means that we can let the package manager handle all the "obvious" 
cases and try to give the user more information in the rare cases where it 
cannot find a valid solution on its own. Cool.

> > > * be based around tree requirements, not some side effects of some
> > > code someone happened to write without considering the implications.
> >
> > What is a tree requirement? (Nice buzzword btw)
>
> A tree requirement is one that comes about as a result of studying what
> ebuilds do now and what they'd like to do in the future, rather than
> one that comes around based upon what someone happens to code.
I am confused. I did not think that ebuilds have desires, so I have no idea 
what they'd like to do in the future. And it does in no way tell me what a 
tree requirement is (unless you mean happy photons, which are emitted by free-
range ebuilds when you try to source them in the kitchen)

Incidentally most of our ebuilds are based upon what someone happened to code. 
Personally I think that many upstream devs didn't have a great plan (or at 
least they didn't like to express it in the code ;) ) and still we have an 
ebuild based upon their coding, uhm, accidents. But we adapt to it, and the 
result is quite impressive.

Somewhere a while ago I read a funny idea - the more rules a contract needs 
the less trust there is. If you trust someone you agree, shake hands, deal 
complete. Only if there is not enough trust do you need lawyers, notaries and 
politicians. A long time ago we used to be able to just agree how things work 
... now we have a ton of legalese (PMS) and things get more confusing every 
day. I don't think this evolution towards self-sustaining bureaucracy and 
incessant lawyering is useful for the end users.

> > And as many devs and users benefit quite a lot from the portage
> > solution, which zmedico did not dream up without thinking about the
> > impact on users, I find it very rude and condescending of you to
> > disrespect the solution without offering an alternative.
>
> ...except for the things that it broke, which necessitated shoving !!
> blocks in at the last minute. And I'll remind you that there were
> discussions about a proper blocker solution before Zac went and did his
> thing,
bikeshedding and circular reasoning in large amounts, stalling any progress 
for a long time ... until someone provided a working implementation that 
solved a large enough amount of issues to gain the support of the majority of 
devs (and big cheers from so many users!). 
(If it had been as bad as you suggest council would have banninated it within 
the shortest amount of time...)

> and the overwhelming view
a minority view, but let's not get distracted by such subjective matters.

> was that a solution based around the
> things I've been discussing was what was wanted.
So your point of view is that the solution proposed by you is the best. 
Intriguing. Circular, subjective, but not really convincing.

> > As you seem to understand the problem domain I'd expect a coherent
> > unambiguous proposal instead of vague accusations and emotional terms
> > that do not help in any way to improve the situation.
>
> See the discussions we had back when the only kind of blocker was a
> hard, single ! block.
I'm not aware of a complete proposal in those discussions. Mind giving me a 
link to that one so I don't have to spend lots of time trying to track it down 
in the almost 70000 mails archived by gmane?

Re: [gentoo-dev] blocking mixed versions of split QT libraries

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 5030 bytes --]

On Mon, 18 May 2009 22:47:30 +0200
Patrick Lauer <patrick@gentoo.org> wrote:
> > No, they're temporary except when things go wrong, at which point
> > there's no indication that they'll be fixed.
> >
> When things go wrong they go wrong. Indeed.

When things go wrong, they go wrong beyond the scope of the failure in
question.

> > > Which either means that the deps are wrong/incomplete or that
> > > portage has algorithmic flaws which should be corrected.
> > > I'd expect you to at least point to the relevant bugs and not just
> > > state some emotional mumbo-jumbo.
> > Look at the new slot deps in EAPI 3.
> So the deps were not precise enough, and now we improve that. Which
> means that portage will only get more precise in the future. Awesome!

...and until they're in wide use, Portage's assumptions are dangerous.

> > No, broke. What he did was in direct violation of PMS and in direct
> > violation of assumptions made by many packages.
> So PMS did once again not reflect reality, and there were some buggy
> packages.

Uhm. No. PMS reflected how Portage used to work, and packages relied
upon how Portage used to work.

> Maybe we should spend more time on making PMS a standard
> that documents current and past behavior instead of omitting things
> (mtime, bash 3.1) or keeping it ambiguous so that two implementations
> can be compliant while delivering incompatible results?

Uhm. PMS correctly reflects current and past behaviour on those issues.

> > > > * work by explaining the reason for the blocker, rather than
> > > > sort-of stating the expected resolution.
> > >
> > > That's dumb ;) (Sorry, emotional statement)
> > > I mean, it does not help to solve the issue and requires user
> > > interaction where an automated solution has been working reliably
> > > for quite some time.
> >
> > Uhm. Knowing the reason for the block lets the package manager solve
> > the problem in the most intelligent available way. Merely being
> > sort-of told the expected resolution does not.
> You're contradicting yourself there - until now you only mentioned
> user- visible messages, which now suddenly become hints for the
> package manager. 

Re-read what I said, alongside the original discussion on replacing
blockers. "Explaining" is not exclusively limited to the user.

> Now I do wonder what you can "explain" about a block - "some files
> collide" ? How does that help the package manager decide what to do?
> What can it do, except unmerging one package or refusing to merge
> another one? How does that differ from the portage solution to the
> blocker situation?

Yes, that's one explanation you'd give to the package manager. As
you'll recall from your reading of the previous discussion, there are
four different ways in which blockers are commonly used, and the
best response from the package manager to each situation is different.

> > A tree requirement is one that comes about as a result of studying
> > what ebuilds do now and what they'd like to do in the future,
> > rather than one that comes around based upon what someone happens
> > to code.
> I am confused. I did not think that ebuilds have desires, so I have
> no idea what they'd like to do in the future. And it does in no way
> tell me what a tree requirement is (unless you mean happy photons,
> which are emitted by free- range ebuilds when you try to source them
> in the kitchen)

Then I suggest you take some basic English classes.

> Incidentally most of our ebuilds are based upon what someone happened
> to code.

But the ebuild design (or at least the clean parts of it...) is not.

> > ...except for the things that it broke, which necessitated
> > shoving !! blocks in at the last minute. And I'll remind you that
> > there were discussions about a proper blocker solution before Zac
> > went and did his thing,
> bikeshedding and circular reasoning in large amounts, stalling any
> progress for a long time ... until someone provided a working
> implementation that solved a large enough amount of issues to gain
> the support of the majority of devs (and big cheers from so many
> users!). 

Funnily enough, the original discussion was extremely productive and
didn't involve any of the nonsense you're coming up with now.

> (If it had been as bad as you suggest council would have
> banninated it within the shortest amount of time...)

That was back in the bad old days before the Council agreed to step in
when Portage did that kind of thing. In fact, the blocker changes were
one of the things that lead to the Council saying "in future, we'll
package.mask Portage if it does that kind of behaviour change again".

> > and the overwhelming view
>
> a minority view, but let's not get distracted by such subjective
> matters.

Please point me to people involved in the discussion who did not agree
with the views presented. Provide a list of message IDs to support your
claim.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 197 bytes --]