* [gentoo-dev] The meaning of || ( a:= b:= ) dependencies
@ 2014-08-03 22:44 Michał Górny
2014-08-04 0:41 ` Ian Stakenvicius
2014-08-04 6:06 ` [gentoo-dev] " Ulrich Mueller
0 siblings, 2 replies; 7+ messages in thread
From: Michał Górny @ 2014-08-03 22:44 UTC (permalink / raw
To: gentoo-dev; +Cc: pms-bugs, dev-portage
[-- Attachment #1: Type: text/plain, Size: 5516 bytes --]
Hello, everyone.
I would like to hear your opinion on what should be the meaning and use
of '|| ( A:= B:= )' dependencies.
By the PMS-y definition, any-of dependency can be satisfied by either
branch of it, and the provider can be safely switched at runtime. That
is:
|| ( A B )
means that either a or b has to be installed. If you built the package
against A, you can install B, uninstall A and everything is supposed to
work without rebuilding. That doesn't really happen when linking is
involved.
With help of subslots and virtuals we were able to partially solve
the issue. For example, look at virtual/libudev. It binds subslot of
the virtual to matching subslots of provider libraries. This way, you
can safely switch providers as long as they have matching ABI; and if
you want to upgrade the provider to another ABI version, you need to
upgrade the virtual as well, and therefore rebuild the revdeps.
Sadly, virtuals like this can only work when you can expect providers
to have matching ABIs. This won't happen e.g. in krb5 providers
(the two have incompatible ABIs) or libav* providers (ABIs of some of
the libraries differ from version to version).
At the moment, some developers already started mixing subslot
and any-of operator syntax:
|| ( A:= B:= )
However, this breeds a really weird behavior in Portage.
With static dependency model, it's partially understandable. ':=' atoms
are expanded into specific subslots when matching package is installed,
otherwise left unspecified. 'Unspecified' here means that any subslot
satisfies the dependency -- like it was plain 'A' or 'B'.
So, if during the build only A was installed, further upgrades to A can
cause subslot rebuilds. If only B was installed, rebuilds are caused by
B likewise.
However, if afterwards the other package is installed, it satisfies
the other branch of the dependency without subslot, so package doesn't
get rebuild on any upgrades of A or B (since the unspecific dep always
matches). This happens until the package is manually rebuild and gets
the other subslot written.
Even more curious behavior is caused if both A and B are installed at
build time. In this case, subslots for both packages are expanded.
And since || means that either of the branches must match, the subslots
of both packages must change for the package to trigger subslot rebuild.
In other words, || ( A:= B:= ) means that subslot rebuilds happen only
if you consistently use a single provider. Provider switching or having
both providers installed break it.
Dynamic deps partially fix it. Since the current := support code is
very dumb, it doesn't notice the '||' and respects all expanded
subslots found in vardb.
The main difference is that installing the other dependency doesn't
prevent subslot rebuilds from the first one from happening. For
example, if you built the package against A and install B afterwards,
upgrade of A will still force rebuild of the package (because
dynamic-deps code accidentally moves the A:0/1= dep out of || ()).
The code also makes the behavior with both providers installed saner.
Since both subslots are expanded, both are copied and rebuild of either
would cause the rebuild of package. However, in practice it usually
causes emerge to fail with slot conflict :).
It should be also noted that the dyndeps behavior makes it impossible
to uninstall either A or B when both were installed at the reverse
dependency build time (since both are added to depgraph).
The question would be -- which behavior is desired? I'm pretty sure
Ciaran will say that the static dep behavior is correct per definitions
but I don't think it's really useful to have slot operator dependencies
which work only randomly. Instead, we may decide to redefine it into
something useful in a future EAPI.
In particular, I was thinking we could reuse this syntax:
|| ( A:= B:= )
to express any-of dependencies that do not support runtime switching of
providers -- since that is pretty much what := does to slots. This
would save us from creating a new syntax like '||= ()' [1].
[1]:https://bugs.gentoo.org/show_bug.cgi?id=489458
If we go this way, we also need to decide whether the order in such
block would matter or not. In other words, whether the application can
be expected to link to the first installed package in the list, or can
link to any of them.
If the order would matter, the package would need to be rebuilt when:
1. first satisfied dependency changes subslot,
2. [optionally] package preceding the first currently satisfied
dependency is installed,
3. first currently satisfied dependency is uninstalled (but another is
installed).
If the order wouldn't matter, the package would need to be rebuilt when:
1. any of satisfied dependencies changes subslot (since we don't know
which one package links to),
2. [optionally] any of the remaining packages is installed,
3. any of satisfied dependencies is uninstalled.
The first option seems more refined, and causes less rebuilds. However,
it diverges further from the basic || () definition. The second tries
to fit || () and := with minimal changes.
Remaining issues:
a. behavior of || ( A:= B:= C ) -- should C cause complete provider
switching rebuilds?
b. do we need ||= ( A B C ) -- i.e. provider switching rebuilds
without subslot rebuilds?
What do you think?
--
Best regards,
Michał Górny
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [gentoo-dev] The meaning of || ( a:= b:= ) dependencies
2014-08-03 22:44 [gentoo-dev] The meaning of || ( a:= b:= ) dependencies Michał Górny
@ 2014-08-04 0:41 ` Ian Stakenvicius
2014-08-04 7:19 ` Michał Górny
2014-08-04 6:06 ` [gentoo-dev] " Ulrich Mueller
1 sibling, 1 reply; 7+ messages in thread
From: Ian Stakenvicius @ 2014-08-04 0:41 UTC (permalink / raw
To: gentoo-dev
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
On 03/08/14 06:44 PM, Michał Górny wrote:
> Hello, everyone.
>
> I would like to hear your opinion on what should be the meaning and
> use of '|| ( A:= B:= )' dependencies. [ Snip! ]
I mentioned this on irc on Friday; as I understand it the following
should occur.
Preface:
A-1.0's SLOT="0/1"
A-2.0's SLOT="0/2"
B-1.0's SLOT="4"
B-2.0's SLOT="5"
When the package is built and merged, it's built against SOMETHING
that satisfies ||( A:= B:= ); this means A or B or both are installed.
In the Portage implementation the slot of dependencies are recorded
in VDB, and so whatever is installed on the system at compile/merge
time is what gets recorded to VDB. It's not possible for Portage to
know which atom in the ||() list, so VDB needs to match the current state.
Case 1 - both A and B are installed - if A-1.0 upgrades to A-2.0, or
B-1.0 upgrades to B-2.0, then rebuilds need to be triggered. This
should be detectable because A:0/1 and B:4 are recorded in VDB. The
fact that they are in a ||() block SHOULD NOT allow the not-upgrading
B:4 to keep the dep satisfied if A-1.0 upgrades to A-2.0. Likewise,
if either A or B are removed, then the package needs to be rebuilt
(once again, it is not known which dep might affect how the package is
linked).
Case 2 - only A is installed - if A-1.0 upgrades to A-2.0 when B isn't
installed at all, this should trigger a rebuild. Hopefully this is
what is happening now in all cases. If A-1.0 stays around and B is
installed, nothing needs to happen because the package is still linked
against A-1.0. However, If A is then removed (say it's dropped from
@world and then --depclean'ed), THEN a rebuild needs to be triggered
so that a proper (slot) dependency is recorded against B. If A is
upgraded after B is installed, then a rebuild needs to be triggered
and we end up with Case#1.
One thing that is a bit unique with case#2 is that, due to the fact
that the subslot is recorded in VDB at merge time, we actually know
which atom is satisfying the ||() dep, and so the changes and/or state
of the other atom doesn't need to have any affect on this package
until the atom that we built against is adjusted (upgraded, removed).
>
> By the PMS-y definition, any-of dependency can be satisfied by
> either branch of it, and the provider can be safely switched at
> runtime. That is:
>
> || ( A B )
>
> means that either a or b has to be installed. If you built the
> package against A, you can install B, uninstall A and everything is
> supposed to work without rebuilding. That doesn't really happen
> when linking is involved.
Hmm.. that "safely switched at runtime" language technicality might
be something we should just honour, since with that in mind " || ( A:=
[anything] ) " isn't a valid syntax, if it can only be used for
runtime-switchable providers. PMS-wise it may be pertinent to use
another operator than || (ie ||= as suggested) to specify a
run-or-compile-time-switchable set of atoms.
Of course, that means we need to wait for a new EAPI and then rewrite
all inappropriate uses of || in the tree, since i believe most ||'s
are in fact runtime-or-compiletime-switchable, rather than -just-
runtime-switchable.
> [ Snip! ] Remaining issues:
>
> a. behavior of || ( A:= B:= C ) -- should C cause complete
> provider switching rebuilds?
As per my cases above, no I don't think C should have any effect when
it's installed. However, if A or B are removed then the removal
should trigger a rebuild.
> b. do we need ||= ( A B C ) -- i.e. provider switching rebuilds
> without subslot rebuilds?
Technically, no I don't think we would need
provider-switching-rebuilds without subslot-rebuilds, but that only
works if everything in the tree migrates to EAPI5 and implements
subslots... Since that's unlikely, though, it might be worth
considering..
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
iF4EAREIAAYFAlPe1sYACgkQ2ugaI38ACPAWiQD/TAilNp7CE5hCaoDikX5ZlSrc
GBpx29M6zvmICsS2dqsBALce6lWlMlBFkRjNJ29XykevaRJIjVSHpsExnJiT5c8R
=UDaD
-----END PGP SIGNATURE-----
^ permalink raw reply [flat|nested] 7+ messages in thread
* [gentoo-dev] Re: The meaning of || ( a:= b:= ) dependencies
2014-08-03 22:44 [gentoo-dev] The meaning of || ( a:= b:= ) dependencies Michał Górny
2014-08-04 0:41 ` Ian Stakenvicius
@ 2014-08-04 6:06 ` Ulrich Mueller
2014-08-04 7:26 ` Michał Górny
1 sibling, 1 reply; 7+ messages in thread
From: Ulrich Mueller @ 2014-08-04 6:06 UTC (permalink / raw
To: Michał Górny; +Cc: gentoo-dev, pms-bugs, dev-portage
[-- Attachment #1: Type: text/plain, Size: 698 bytes --]
>>>>> On Mon, 4 Aug 2014, Michał Górny wrote:
> In particular, I was thinking we could reuse this syntax:
> || ( A:= B:= )
> to express any-of dependencies that do not support runtime switching
> of providers -- since that is pretty much what := does to slots.
> This would save us from creating a new syntax like '||= ()' [1].
Please don't, because it makes things pretty much unreadable. If you
want an operator like || ( ) but without runtime switching, then
define one (e.g., <<= or ||= as suggested in [1]), but don't try
to inherit properties from its children.
An EAPI bump will be required in any case.
Ulrich
> [1]:https://bugs.gentoo.org/show_bug.cgi?id=489458
[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [gentoo-dev] The meaning of || ( a:= b:= ) dependencies
2014-08-04 0:41 ` Ian Stakenvicius
@ 2014-08-04 7:19 ` Michał Górny
0 siblings, 0 replies; 7+ messages in thread
From: Michał Górny @ 2014-08-04 7:19 UTC (permalink / raw
To: Ian Stakenvicius; +Cc: gentoo-dev
[-- Attachment #1: Type: text/plain, Size: 623 bytes --]
Dnia 2014-08-03, o godz. 20:41:42
Ian Stakenvicius <axs@gentoo.org> napisał(a):
> > b. do we need ||= ( A B C ) -- i.e. provider switching rebuilds
> > without subslot rebuilds?
>
> Technically, no I don't think we would need
> provider-switching-rebuilds without subslot-rebuilds, but that only
> works if everything in the tree migrates to EAPI5 and implements
> subslots... Since that's unlikely, though, it might be worth
> considering..
This is not really true. You can safely use := against pre-EAPI5
packages. Slot changes are supposed to trigger rebuilds too.
--
Best regards,
Michał Górny
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [gentoo-dev] Re: The meaning of || ( a:= b:= ) dependencies
2014-08-04 6:06 ` [gentoo-dev] " Ulrich Mueller
@ 2014-08-04 7:26 ` Michał Górny
2014-08-04 7:53 ` Ulrich Mueller
[not found] ` <7cbf12bc6e5646adb74392b7b7e192a1@mail10.futurewins.com>
0 siblings, 2 replies; 7+ messages in thread
From: Michał Górny @ 2014-08-04 7:26 UTC (permalink / raw
To: Ulrich Mueller; +Cc: gentoo-dev, pms-bugs, dev-portage
[-- Attachment #1: Type: text/plain, Size: 1645 bytes --]
Dnia 2014-08-04, o godz. 08:06:42
Ulrich Mueller <ulm@gentoo.org> napisał(a):
> >>>>> On Mon, 4 Aug 2014, Michał Górny wrote:
> > In particular, I was thinking we could reuse this syntax:
>
> > || ( A:= B:= )
>
> > to express any-of dependencies that do not support runtime switching
> > of providers -- since that is pretty much what := does to slots.
> > This would save us from creating a new syntax like '||= ()' [1].
>
> Please don't, because it makes things pretty much unreadable. If you
> want an operator like || ( ) but without runtime switching, then
> define one (e.g., <<= or ||= as suggested in [1]), but don't try
> to inherit properties from its children.
Reasonable. However, as I see it, we'll end up having up to four
different operators:
- || that is deprecated yet everyone will still use it (like they don't
use :* right now),
- ||* that will be used scarcely,
- <<= that would be the preferred variant for compile-time switches yet
many people will not use it because it has different characters than
'||' [we could try maybe '||<' so that people will still see it as
replacement for '||'],
- ||= that most people would use forgetting about '<<=' [or '||<'].
So, banning '|| ( A:= B:= )' in a future EAPI sounds reasonable.
However, there's still the matter of setting current Portage behavior
because I don't we should keep the non-predictable magic.
What should be the current behavior then? Should we assume that all
'||' are not well-defined and need to be compile-switchable? Or try to
invent heuristic like I suggested?
--
Best regards,
Michał Górny
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [gentoo-dev] Re: The meaning of || ( a:= b:= ) dependencies
2014-08-04 7:26 ` Michał Górny
@ 2014-08-04 7:53 ` Ulrich Mueller
[not found] ` <7cbf12bc6e5646adb74392b7b7e192a1@mail10.futurewins.com>
1 sibling, 0 replies; 7+ messages in thread
From: Ulrich Mueller @ 2014-08-04 7:53 UTC (permalink / raw
To: Michał Górny; +Cc: gentoo-dev, pms-bugs, dev-portage
[-- Attachment #1: Type: text/plain, Size: 1948 bytes --]
>>>>> On Mon, 4 Aug 2014, Michał Górny wrote:
> Reasonable. However, as I see it, we'll end up having up to four
> different operators:
> - || that is deprecated yet everyone will still use it (like they don't
> use :* right now),
> - ||* that will be used scarcely,
> - <<= that would be the preferred variant for compile-time switches yet
> many people will not use it because it has different characters than
> '||' [we could try maybe '||<' so that people will still see it as
> replacement for '||'],
> - ||= that most people would use forgetting about '<<=' [or '||<'].
No, we will have only two distinct operators, namely ||* and ||=.
Alternatively, or in addition, || could be kept but would be identical
to ||*.
> So, banning '|| ( A:= B:= )' in a future EAPI sounds reasonable.
What prevents us from banning it now, by adding a repoman check?
The || ( A:= B:= ) construct would mean that you can switch from
provider A to provider B and back to A. Nothing would prevent you from
choosing a different slot in the second step, which renders the :=
operator meaningless.
> However, there's still the matter of setting current Portage behavior
> because I don't we should keep the non-predictable magic.
> What should be the current behavior then? Should we assume that all
> '||' are not well-defined and need to be compile-switchable? Or try to
> invent heuristic like I suggested?
The devmanual [1] is very clear about it. || ( ) is only allowed if
the implementation can be switched at runtime:
# * fnord is merged on a system which has foo and not bar installed.
# foo is then unmerged, and bar is installed. fnord must continue to
# work correctly.
# * A binary package of fnord made on a system with foo and not bar
# can be taken and installed on a system with bar and not foo.
Ulrich
[1] http://devmanual.gentoo.org/general-concepts/dependencies/index.html
[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [gentoo-dev] Re: The meaning of || ( a:= b:= ) dependencies
[not found] ` <7cbf12bc6e5646adb74392b7b7e192a1@mail10.futurewins.com>
@ 2014-08-04 9:55 ` Rich Freeman
0 siblings, 0 replies; 7+ messages in thread
From: Rich Freeman @ 2014-08-04 9:55 UTC (permalink / raw
To: gentoo-dev; +Cc: Michał Górny, pms-bugs, dev-portage
On Mon, Aug 4, 2014 at 3:53 AM, Ulrich Mueller <ulm@gentoo.org> wrote:
>
> The devmanual is very clear about it. || ( ) is only allowed if
> the implementation can be switched at runtime:
>
I do appreciate the issue here, but keep in mind that we have no way
to express an either-or dependency which ISN'T switchable at runtime
right now. So, if we disable the portage auto-rebuild functionality
to make it better align with PMS, we're going to end up doing more
manual rebuilds.
That isn't the end of the world due to @preserved-rebuild I suppose.
One thing that hasn't come up here is the situation where some of the
options are runtime-switchable, and some are not. :) Rather than a
new syntax it is probably simpler to just use ||= and have more
rebuilds than necessary (assuming the dependency even expresses a
subslot). This should also be a rather rare situation.
Rich
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2014-08-04 9:55 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-08-03 22:44 [gentoo-dev] The meaning of || ( a:= b:= ) dependencies Michał Górny
2014-08-04 0:41 ` Ian Stakenvicius
2014-08-04 7:19 ` Michał Górny
2014-08-04 6:06 ` [gentoo-dev] " Ulrich Mueller
2014-08-04 7:26 ` Michał Górny
2014-08-04 7:53 ` Ulrich Mueller
[not found] ` <7cbf12bc6e5646adb74392b7b7e192a1@mail10.futurewins.com>
2014-08-04 9:55 ` Rich Freeman
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox