[gentoo-dev] Dependencies that're available at pkg_*inst

[gentoo-dev] Dependencies that're available at pkg_*inst

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 674 bytes --]

I'm rewording the PMS sections on dependencies to avoid permitting
overly lax circular dependency resolution. Which of these wordings is
accurate, given that usable means "has its RDEPENDs installed and
usable"?

1. During pkg_preinst and pkg_postinst, any package dependency that is
in both DEPEND and RDEPEND must be installed and usable.

2. During pkg_preinst and pkg_postinst, at least one of the following
conditions must be met:
  a. every package dependency in DEPEND must be installed and usable
  b. every package dependency in RDEPEND must be installed and usable

Do not attempt to write on both sides of the paper at once.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Dependencies that're available at pkg_*inst

[Donnie Berkholz]

On 05:31 Sat 19 Apr     , Ciaran McCreesh wrote:
> I'm rewording the PMS sections on dependencies to avoid permitting
> overly lax circular dependency resolution. Which of these wordings is
> accurate, given that usable means "has its RDEPENDs installed and
> usable"?
> 
> 1. During pkg_preinst and pkg_postinst, any package dependency that is
> in both DEPEND and RDEPEND must be installed and usable.
> 
> 2. During pkg_preinst and pkg_postinst, at least one of the following
> conditions must be met:
>   a. every package dependency in DEPEND must be installed and usable
>   b. every package dependency in RDEPEND must be installed and usable

I'd go with RDEPEND only. Any other interpretation results in installing 
build-time-only packages along with a binpkg, which doesn't seem to make 
sense.

Thanks,
Donnie
-- 
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] Dependencies that're available at pkg_*inst

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 653 bytes --]

On Fri, 18 Apr 2008 21:45:13 -0700
Donnie Berkholz <dberkholz@gentoo.org> wrote:
> I'd go with RDEPEND only. Any other interpretation results in
> installing build-time-only packages along with a binpkg, which
> doesn't seem to make sense.

That's definitely not what we want. Only a package's DEPENDs have to be
installed and usable when that package is built. Its RDEPENDs don't
have to be installed until that package is treated as usable.

For why this matters:

cat/a-1: RDEPEND cat/b
cat/b-1: RDEPEND cat/a

This is solvable. If package managers can't solve this, they can't
install Gnome off a stage 3...

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Dependencies that're available at pkg_*inst

[Donnie Berkholz]

On 05:54 Sat 19 Apr     , Ciaran McCreesh wrote:
> On Fri, 18 Apr 2008 21:45:13 -0700
> Donnie Berkholz <dberkholz@gentoo.org> wrote:
> > I'd go with RDEPEND only. Any other interpretation results in
> > installing build-time-only packages along with a binpkg, which
> > doesn't seem to make sense.
> 
> That's definitely not what we want. Only a package's DEPENDs have to be
> installed and usable when that package is built. Its RDEPENDs don't
> have to be installed until that package is treated as usable.

I previously failed to clarify the situation I preferred because either 
1 or 2b qualify as requiring RDEPEND to be installed.

My interpretation is pkg_* counts as runtime (I can imagine a package 
wanting to run itself at this point), so packages in RDEPEND should be 
usable at that point. Really, it seems to be an additional type of 
dependency that neither DEPEND or RDEPEND fully describe, and this 
DEPEND+RDEPEND idea isn't quite capturing it either. I say this because 
I wouldn't want everything in DEPEND installed with a binpkg so it can 
run pkg_*, and I also can see how some people wouldn't consider a 
package in a runnable state until pkg_* have finished (so thus RDEPEND 
shouldn't be required).

> For why this matters:
> 
> cat/a-1: RDEPEND cat/b
> cat/b-1: RDEPEND cat/a
> 
> This is solvable. If package managers can't solve this, they can't
> install Gnome off a stage 3...

Dealing with this under my interpretation is a bit weird. I think it 
might need some sort of staging area. That's one reason I mentioned the 
additional dep type.

Thanks,
Donnie
-- 
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] Dependencies that're available at pkg_*inst

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 622 bytes --]

On Fri, 18 Apr 2008 22:27:21 -0700
Donnie Berkholz <dberkholz@gentoo.org> wrote:
> My interpretation is pkg_* counts as runtime (I can imagine a package 
> wanting to run itself at this point), so packages in RDEPEND should
> be usable at that point.

Which would be fine, except it makes the tree unusable.

> Really, it seems to be an additional type of dependency that neither
> DEPEND or RDEPEND fully describe, and this DEPEND+RDEPEND idea isn't
> quite capturing it either.

Yup, and for future EAPIs labels can fix this. But we have to have a
sound solution for current EAPIs.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Dependencies that're available at pkg_*inst

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 2247 bytes --]

On Sat, 2008-04-19 at 06:33 +0100, Ciaran McCreesh wrote:
> On Fri, 18 Apr 2008 22:27:21 -0700
> Donnie Berkholz <dberkholz@gentoo.org> wrote:
> > My interpretation is pkg_* counts as runtime (I can imagine a package 
> > wanting to run itself at this point), so packages in RDEPEND should
> > be usable at that point.
> 
> Which would be fine, except it makes the tree unusable.
> 
> > Really, it seems to be an additional type of dependency that neither
> > DEPEND or RDEPEND fully describe, and this DEPEND+RDEPEND idea isn't
> > quite capturing it either.
> 
> Yup, and for future EAPIs labels can fix this. But we have to have a
> sound solution for current EAPIs.

I would agree that RDEPEND should likely be installed prior to
pkg_preinst to satisfy the dependency.  After all, PDEPEND is "good
enough" for doing packages that aren't required at
pkg_preinst/pkg_postinst.

We definitely don't want to install DEPEND at the pkg_* stages, so I'd
say the requirement there, if you're asking, is prior to src_*, if that
matters.

I'd love to have some kind of functionality to allow some kind of
"optional" dependencies.  The only real way that I could see this
working is if we tracked what was installed as an optional dependency,
and not reinstall it if it has been removed the next time the depending
package is merged.

Simple example:

ODEPEND="video_cards_nvidia? ( x11-drivers/nvidia-drivers)" would
install x11-drivers/nvidia-drivers the first time it's ran with
VIDEO_CARDS="nvidia", but if I removed x11-drivers/nvidia-drivers, it
wouldn't get reinstalled.  This would probably need some kind of
"--newuse" like capability to allow for installing only *new* optional
dependencies, but I think that the tracking would already allow that.
The idea here would be to allow for installing "recommended" software
along with others.  We could even have --ask ask for the dependencies,
since they are optional, after all.

This way, we could "ship" a more robust configuration/setup for many
popular applications without forcing software on people.

It's an idea, anyway, and I hope that I didn't hijack the thread.

-- 
Chris Gianelloni
Release Engineering Strategic Lead
Games Developer

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: [gentoo-dev] Dependencies that're available at pkg_*inst

[Marijn Schouten (hkBst)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ciaran McCreesh wrote:
| I'm rewording the PMS sections on dependencies to avoid permitting
| overly lax circular dependency resolution. Which of these wordings is
| accurate, given that usable means "has its RDEPENDs installed and
| usable"?
|
| 1. During pkg_preinst and pkg_postinst, any package dependency that is
| in both DEPEND and RDEPEND must be installed and usable.
|
| 2. During pkg_preinst and pkg_postinst, at least one of the following
| conditions must be met:
|   a. every package dependency in DEPEND must be installed and usable
|   b. every package dependency in RDEPEND must be installed and usable
|
| Do not attempt to write on both sides of the paper at once.

Every package dependency in DEPEND is installed and usable before src_unpack starts,
right? So is the question here whether or not they can be uninstalled right before
pkg_{pre,post}inst starts?

I don't know what the general use of pkg_preinst is, but in pkg_postinst the package
itself should be runnable, so its RDEPENDS should be installed and usable at this point.
So perhaps we should define that "usable" means "each of its RDEPENDs is installed and has
had its pkg_postinst function run". The recursion of that definition then comes from the
requirement that RDEPENDs should be usable before pkg_postinst starts running.

| For why this matters:
|
| cat/a-1: RDEPEND cat/b
| cat/b-1: RDEPEND cat/a
|
| This is solvable. If package managers can't solve this, they can't
| install Gnome off a stage 3...

If only one of those packages has a pkg_postinst then it is still solvable.
If they both have a pkg_postinst then one of those is probably not essential for the
actual usability of the package and should be removed. A final possibility is that the
pkg_postinsts are both necessary for a fully functional package but not for the
functionality used in the other pkg_postinst. If this is the case, then perhaps we should
specify deps according to which ebuild phase they are necessary for?

cat/a:

SRC_UNPACK_DEP="app-arch/unzip"
SRC_COMPILE_DEP="dev-scheme/bigloo"
SRC_INSTALL_DEP=""

PKG_PREINST_DEP=""
PKG_POSTINST_DEP="cat/b"
RDEPEND="cat/b"

and then cat/b would say:

PKG_PREINST_DEP=""
PKG_POSTINST_DEP=""
RDEPEND="cat/a"


Marijn

- --
Marijn Schouten (hkBst), Gentoo Lisp project, Gentoo ML
<http://www.gentoo.org/proj/en/lisp/>, #gentoo-{lisp,ml} on FreeNode
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkgKH+4ACgkQp/VmCx0OL2xJOwCfdEO5IhWbjPvFRidzgdyFanEd
0v4An26a2XJ9Y4hwDz/bpqeUWeDMXAuk
=v/UL
-----END PGP SIGNATURE-----
-- 
gentoo-dev@lists.gentoo.org mailing list

[gentoo-dev] Re: Dependencies that're available at pkg_*inst

[Duncan]

Ciaran McCreesh <ciaran.mccreesh@googlemail.com> posted
20080419063300.6d2a2525@snowcone, excerpted below, on  Sat, 19 Apr 2008
06:33:00 +0100:

> On Fri, 18 Apr 2008 22:27:21 -0700
> Donnie Berkholz <dberkholz@gentoo.org> wrote:
>> My interpretation is pkg_* counts as runtime (I can imagine a package
>> wanting to run itself at this point), so packages in RDEPEND should be
>> usable at that point.
> 
> Which would be fine, except it makes the tree unusable.
> 
>> Really, it seems to be an additional type of dependency that neither
>> DEPEND or RDEPEND fully describe, and this DEPEND+RDEPEND idea isn't
>> quite capturing it either.
> 
> Yup, and for future EAPIs labels can fix this. But we have to have a
> sound solution for current EAPIs.

It seems to me that at least for current EAPIs, RDEPEND simply cannot be 
depended upon during pkg_*inst without breaking things.  I can't see a 
way around that.

About the least-bad of multiple bad solutions I can see for Donnie's 
conceivable run scenario would be to print a message in pkg_postinst 
telling the user to run emerge --config before running the program 
normally, maybe even going to the point of renaming the runtime and 
installing a fake that reminds folks to run emerge --config first, if 
it's critical enough.  (pkg_config would then kill the fake and rename 
the runtime back to its proper name.)

Now consider binary packages.  DEPEND can't be used as-is, which in the 
OR case would then mandate RDEPEND and again result in broken behavior 
due to circular dependencies, so that simply doesn't work.  That leaves 
the intersection of both DEPEND and RDEPEND sets as the only possible 
logically consistent resolution...

UNLESS we either (1) accept that binary package behavior simply can't be 
correctly defined under current EAPIs and declare it an indeterminate 
legacy exception, or (2) declare binary packages an exception that works 
by different rules, and then define them (somehow).  Either alternative 
would then leave somewhat more flexibility for the ordinary build case, 
presumably enough to reasonably accurately describe current behavior 
deterministically.  (I'll freely admit to not knowing enough about 
current tree behavior to pick the right option there.)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] Re: Dependencies that're available at pkg_*inst

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 477 bytes --]

On Sat, 19 Apr 2008 18:53:27 +0000 (UTC)
Duncan <1i5t5.duncan@cox.net> wrote:
> It seems to me that at least for current EAPIs, RDEPEND simply cannot
> be depended upon during pkg_*inst without breaking things.  I can't
> see a way around that.

But DEPEND can't either.

The point is, one of the two wordings in the original email is enough.
In fact, both are, but they have different implications, and selecting
the right one is important.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Dependencies that're available at pkg_*inst

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1693 bytes --]

On Sat, 19 Apr 2008 18:38:06 +0200
"Marijn Schouten (hkBst)" <hkBst@gentoo.org> wrote:
> Every package dependency in DEPEND is installed and usable before
> src_unpack starts, right? So is the question here whether or not they
> can be uninstalled right before pkg_{pre,post}inst starts?

If we're using binaries, DEPEND is usually ignored.

> I don't know what the general use of pkg_preinst is, but in
> pkg_postinst the package itself should be runnable, so its RDEPENDS
> should be installed and usable at this point. So perhaps we should
> define that "usable" means "each of its RDEPENDs is installed and has
> had its pkg_postinst function run". The recursion of that definition
> then comes from the requirement that RDEPENDs should be usable before
> pkg_postinst starts running.

No good. That prevents RDEPEND <-> RDEPEND cycles from being solved,
and the package manager has to be able to solve that.

> If only one of those packages has a pkg_postinst then it is still
> solvable. If they both have a pkg_postinst then one of those is
> probably not essential for the actual usability of the package and
> should be removed. A final possibility is that the pkg_postinsts are
> both necessary for a fully functional package but not for the
> functionality used in the other pkg_postinst. If this is the case,
> then perhaps we should specify deps according to which ebuild phase
> they are necessary for?

Not with current EAPIs we can't.

> SRC_UNPACK_DEP="app-arch/unzip"
> SRC_COMPILE_DEP="dev-scheme/bigloo"
> SRC_INSTALL_DEP=""

Labels are a cleaner solution to this. But again, we're discussing
current EAPIs here.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Dependencies that're available at pkg_*inst

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 1462 bytes --]

On Sun, Apr 20, 2008 at 12:57:28AM +0100, Ciaran McCreesh wrote:
> On Sat, 19 Apr 2008 18:38:06 +0200
> "Marijn Schouten (hkBst)" <hkBst@gentoo.org> wrote:
> > I don't know what the general use of pkg_preinst is, but in
> > pkg_postinst the package itself should be runnable, so its RDEPENDS
> > should be installed and usable at this point. So perhaps we should
> > define that "usable" means "each of its RDEPENDs is installed and has
> > had its pkg_postinst function run". The recursion of that definition
> > then comes from the requirement that RDEPENDs should be usable before
> > pkg_postinst starts running.
> 
> No good. That prevents RDEPEND <-> RDEPEND cycles from being solved,
> and the package manager has to be able to solve that.

Not all cycles are solvable w/in the domain of execution which is an 
unfortunate fact.  This loops back into why jstubbs was after use dep 
cycle breaking (fair bit more fine grained of an issue), a discussion 
that thus far has been ignored, and the usage of the build flag.


> > SRC_UNPACK_DEP="app-arch/unzip"
> > SRC_COMPILE_DEP="dev-scheme/bigloo"
> > SRC_INSTALL_DEP=""
> 
> Labels are a cleaner solution to this. But again, we're discussing
> current EAPIs here.

Stop name dropping labels until you tell folk about what labels are.  
I know, but I'd rather not have the notion "labels solves all" pushed 
forth w/ out people knowing what it is, please.

thanks,
~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Dependencies that're available at pkg_*inst

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1200 bytes --]

On Sat, 19 Apr 2008 18:29:10 -0700
Brian Harring <ferringb@gmail.com> wrote:
> Not all cycles are solvable w/in the domain of execution which is an 
> unfortunate fact.  This loops back into why jstubbs was after use dep 
> cycle breaking (fair bit more fine grained of an issue), a discussion 
> that thus far has been ignored, and the usage of the build flag.

Which, again, is a future thing, since it requires additional per-ebuild
information about how use flags are used to be implementable.

> Stop name dropping labels until you tell folk about what labels are.  
> I know, but I'd rather not have the notion "labels solves all" pushed 
> forth w/ out people knowing what it is, please.

Labels are already documented and discussable in the appropriate place.
This is not supposed to be a discussion about future direction. It's
supposed to be about one specific issue regarding how we word things for
current implementations. The two wordings in the original email do not
have equivalent implications, and selecting the correct one affects
whether certain problems are solvable. Let's discuss that, not what
we're going to do six years from now.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Dependencies that're available at pkg_*inst

[Santiago M. Mola]

On Sun, Apr 20, 2008 at 10:36 AM, Ciaran McCreesh
<ciaran.mccreesh@googlemail.com> wrote:
> On Sat, 19 Apr 2008 18:29:10 -0700
>  Brian Harring <ferringb@gmail.com> wrote:
>
>  > Stop name dropping labels until you tell folk about what labels are.
>  > I know, but I'd rather not have the notion "labels solves all" pushed
>  > forth w/ out people knowing what it is, please.
>
>  Labels are already documented and discussable in the appropriate place.
>  This is not supposed to be a discussion about future direction. It's
>  supposed to be about one specific issue regarding how we word things for
>  current implementations. The two wordings in the original email do not
>  have equivalent implications, and selecting the correct one affects
>  whether certain problems are solvable. Let's discuss that, not what
>  we're going to do six years from now.
>

http://bugs.gentoo.org/show_bug.cgi?id=201499

-- 
Santiago M. Mola
Jabber ID: cooldwind@gmail.com
-- 
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] Dependencies that're available at pkg_*inst

[Donnie Berkholz]

On 06:33 Sat 19 Apr     , Ciaran McCreesh wrote:
> On Fri, 18 Apr 2008 22:27:21 -0700
> Donnie Berkholz <dberkholz@gentoo.org> wrote:
> > My interpretation is pkg_* counts as runtime (I can imagine a package 
> > wanting to run itself at this point), so packages in RDEPEND should
> > be usable at that point.
> 
> Which would be fine, except it makes the tree unusable.

Isn't this part of the first situation offered (RDEPEND+DEPEND, rather 
than RDEPEND|DEPEND)?

I don't think I understand the difference between the effects of these 
two options. Could you describe a scenario where one would make things 
possible and the other wouldn't? Here's my attempt from an ebuild dev's 
POV, not from an ebuild dependency POV:

The way I'm thinking, the RDEPEND|DEPEND case means an ebuild developer 
would have no idea which list would be used, so you'd have something 
like COMMON_DEPEND to describe the pkg_*-used dependency, which would be 
in both RDEPEND and DEPEND. This would allow both RDEPEND and DEPEND to 
have unique dependencies, which seems like it could be beneficial. It 
would probably require fixing the tree for packages that currently 
assume RDEPEND will be available, although I don't expect that many 
packages will have unique RDEPEND settings (mainly binaries).

I guess the RDEPEND+DEPEND case would save an ebuild dev the work of 
specifying the COMMON_DEPEND list, but other than that, I can't think of 
any benefits. It would force both RDEPEND and DEPEND to be installed for 
binpkgs, which sucks.

Thanks,
Donnie
-- 
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] Dependencies that're available at pkg_*inst

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 493 bytes --]

On Sun, 20 Apr 2008 22:17:27 -0700
Donnie Berkholz <dberkholz@gentoo.org> wrote:
> I don't think I understand the difference between the effects of
> these two options.

cat/a-1 is installed and has RDEPEND cat/b
cat/a-2 is to be installed and has DEPEND cat/b and RDEPEND =cat/b-2
cat/b-1 is installed and has RDEPEND cat/a
cat/b-2 is to be installed and has DEPEND cat/a and RDEPEND =cat/a-2

Solve this and enlightenment shall be yours!

Or a headache.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Dependencies that're available at pkg_*inst

[Marijn Schouten (hkBst)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ciaran McCreesh wrote:
| On Sun, 20 Apr 2008 22:17:27 -0700
| Donnie Berkholz <dberkholz@gentoo.org> wrote:
|> I don't think I understand the difference between the effects of
|> these two options.
|
| cat/a-1 is installed and has RDEPEND cat/b
| cat/a-2 is to be installed and has DEPEND cat/b and RDEPEND =cat/b-2
| cat/b-1 is installed and has RDEPEND cat/a
| cat/b-2 is to be installed and has DEPEND cat/a and RDEPEND =cat/a-2
|
| Solve this and enlightenment shall be yours!
|
| Or a headache.
|

This problem has the two obvious solutions: either install a-2 and then b-2 or the other
way around. But to be relevant to the current discussion you need to specify whether or
not there are any pkg_{pre,post}inst functions. If there are too many then it becomes
unsolvable and is probably a bug, as I already explained:

|> If only one of those packages has a pkg_postinst then it is still
|> solvable. If they both have a pkg_postinst then one of those is
|> probably not essential for the actual usability of the package and
|> should be removed. A final possibility is that the pkg_postinsts are
|> both necessary for a fully functional package but not for the
|> functionality used in the other pkg_postinst. If this is the case,
|> then perhaps we should specify deps according to which ebuild phase
|> they are necessary for?
|
| Not with current EAPIs we can't.
|
|> SRC_UNPACK_DEP="app-arch/unzip"
|> SRC_COMPILE_DEP="dev-scheme/bigloo"
|> SRC_INSTALL_DEP=""
|
| Labels are a cleaner solution to this. But again, we're discussing
| current EAPIs here.

Labels seems to be another syntax for providing the same information as I proposed AIUI,
i.e. finer-grained deps.

Marijn

- --
Marijn Schouten (hkBst), Gentoo Lisp project, Gentoo ML
<http://www.gentoo.org/proj/en/lisp/>, #gentoo-{lisp,ml} on FreeNode
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkgMVekACgkQp/VmCx0OL2waMgCglvtOPnu1xBIpUn0EbG7jDNsf
xLQAoLfQR4s8hAvzhgfx5JuY4sj9gp7+
=Creb
-----END PGP SIGNATURE-----
-- 
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] Dependencies that're available at pkg_*inst

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1613 bytes --]

On Mon, 21 Apr 2008 10:52:57 +0200
"Marijn Schouten (hkBst)" <hkBst@gentoo.org> wrote:
> | cat/a-1 is installed and has RDEPEND cat/b
> | cat/a-2 is to be installed and has DEPEND cat/b and RDEPEND =cat/b-2
> | cat/b-1 is installed and has RDEPEND cat/a
> | cat/b-2 is to be installed and has DEPEND cat/a and RDEPEND =cat/a-2
> |
> | Solve this and enlightenment shall be yours!
> |
> | Or a headache.
> 
> This problem has the two obvious solutions: either install a-2 and
> then b-2 or the other way around.

Bzzt, wrong! Once you've installed a-2, you can't install b-2 since it
DEPENDs upon cat/a, but cat/a's run dependencies aren't satisfied, so
the dependency isn't met. And likewise for the other way around.

This problem is nowhere near as simple as you think it is.

> But to be relevant to the current discussion you need to specify
> whether or not there are any pkg_{pre,post}inst functions. If there
> are too many then it becomes unsolvable and is probably a bug, as I
> already explained:

The package manager can't sanely know whether such functions exist. (It
could, theoretically, insanely know, but forcing package managers to
be able to work that out really isn't something we want to do.)

> | Labels are a cleaner solution to this. But again, we're discussing
> | current EAPIs here.
> 
> Labels seems to be another syntax for providing the same information
> as I proposed AIUI, i.e. finer-grained deps.

Labels do that and a lot more, and without the explosion in number of
metadata keys. But they're a different discussion.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Dependencies that're available at pkg_*inst

[Luca Barbato]

Ciaran McCreesh wrote:
> cat/a-1: RDEPEND cat/b
> cat/b-1: RDEPEND cat/a
> 
> This is solvable. If package managers can't solve this, they can't
> install Gnome off a stage 3...

Which are the packages involved in such cycle?

lu

-- 

Luca Barbato
Gentoo Council Member
Gentoo/linux Gentoo/PPC
http://dev.gentoo.org/~lu_zero

-- 
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] Dependencies that're available at pkg_*inst

[Luca Barbato]

Ciaran McCreesh wrote:
>> Really, it seems to be an additional type of dependency that neither
>> DEPEND or RDEPEND fully describe, and this DEPEND+RDEPEND idea isn't
>> quite capturing it either.
> 
> Yup, and for future EAPIs labels can fix this. But we have to have a
> sound solution for current EAPIs.

Usually I rather see the specific problem before looking for solutions.
If packages intertwine in strange ways _maybe_ we could work with 
upstream to fix the insanity at the source instead host it ourselves.

lu


-- 

Luca Barbato
Gentoo Council Member
Gentoo/linux Gentoo/PPC
http://dev.gentoo.org/~lu_zero

-- 
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] Dependencies that're available at pkg_*inst

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 881 bytes --]

On Mon, 21 Apr 2008 12:10:53 +0200
Luca Barbato <lu_zero@gentoo.org> wrote:
> Ciaran McCreesh wrote:
> >> Really, it seems to be an additional type of dependency that
> >> neither DEPEND or RDEPEND fully describe, and this DEPEND+RDEPEND
> >> idea isn't quite capturing it either.
> > 
> > Yup, and for future EAPIs labels can fix this. But we have to have a
> > sound solution for current EAPIs.
> 
> Usually I rather see the specific problem before looking for
> solutions.

The specific problem is that ebuilds currently rely upon the package
manager providing circular dependency resolution that works, so we need
a good definition of just what's allowed to resolve cycles. But we
can't take "what Portage does" as that definition, because Portage's
behaviour is "usually get it right by fluke, except when things go
horribly wrong".

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Dependencies that're available at pkg_*inst

[Arfrever Frehtes Taifersar Arahesis]

2008-04-21 12:05 Luca Barbato <lu_zero@gentoo.org> napisał(a):
> Ciaran McCreesh wrote:
>
> > cat/a-1: RDEPEND cat/b
> > cat/b-1: RDEPEND cat/a
> >
> > This is solvable. If package managers can't solve this, they can't
> > install Gnome off a stage 3...
> >
>
>  Which are the packages involved in such cycle?

Maybe app-admin/gamin and >=dev-libs/glib-2.16[fam].

[gentoo-dev] Re: Dependencies that're available at pkg_*inst

[Duncan]

Donnie Berkholz <dberkholz@gentoo.org> posted
20080421051727.GA10765@comet, excerpted below, on  Sun, 20 Apr 2008
22:17:27 -0700:

> I guess the RDEPEND+DEPEND case would save an ebuild dev the work of
> specifying the COMMON_DEPEND list, but other than that, I can't think of
> any benefits. It would force both RDEPEND and DEPEND to be installed for
> binpkgs, which sucks.

If I read the original proposal correctly, it's not proposing a simple +, 
that BOTH RDEPEND and DEPEND be guaranteed installed at pkg_*inst, IOW by 
set theory, not the UNION of the two sets, but the INTERSECTION of the 
two sets, that is, packages that appear in both lists at once, not those 
appearing in one XOR the other.

Thus a COMMON_DEPEND would still be useful as it would be the list 
appearing in both (thus effectively the list necessary for pkg_*inst, 
same as the OR case).  Both lists could still exclusively include 
packages, and packages not listed in DEPEND only would not have to be 
installed for binpkgs.

So it's not OR vs AND, but OR vs INTERSECTION.

As I stated in my other post, RDEPEND alone can't be used without 
breaking things.  That applies to binary package installation as well, 
where DEPEND along can't be used either as that would require 
installation of unwanted packages.  Thus, the OR case doesn't seem to 
work for binary installation at all, since neither RDEPEND nor DEPEND can 
be relied upon alone, and the OR case proposes requiring at least one 
complete set of the two be installed.

Thus, for current EAPIs, the INTERSECTION alternative is the only 
possibly working alternative if we are not to break binary package 
support and not force full DEPEND installation on binary targets.  It's 
not ideal as it'll potentially force unwanted and otherwise unnecessary 
package installation on both the build-host and the binary target, due to 
fact that it forces pkg_*inst dependencies into both DEPEND and RDEPEND, 
but IMO it's better than forcing the whole set of DEPENDs to be installed 
on binary targets, which would be the only working alternative in the OR 
case above.

As others have said, this is certainly a good candidate for future EAPI 
change, but it's not future EAPIs under current discussion, so that fact 
doesn't help the current discussion.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] Dependencies that're available at pkg_*inst

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1140 bytes --]

On Sat, 19 Apr 2008 00:43:08 -0700
Chris Gianelloni <wolf31o2@gentoo.org> wrote:
> I would agree that RDEPEND should likely be installed prior to
> pkg_preinst to satisfy the dependency.  After all, PDEPEND is "good
> enough" for doing packages that aren't required at
> pkg_preinst/pkg_postinst.

It's likely to, but not guaranteed to be.

> We definitely don't want to install DEPEND at the pkg_* stages, so I'd
> say the requirement there, if you're asking, is prior to src_*, if
> that matters.

If the alternatives are not being able to install from a binary at all
due to circular dependencies, or being able to install from a binary
using DEPEND to satisfy circular dependencies, which would you take?

> I'd love to have some kind of functionality to allow some kind of
> "optional" dependencies.  The only real way that I could see this
> working is if we tracked what was installed as an optional dependency,
> and not reinstall it if it has been removed the next time the
> depending package is merged.

Sort of like what kdebuild has for suggested dependencies, but less
strong?

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Dependencies that're available at pkg_*inst

[Chris Gianelloni]

On Tue, 2008-04-22 at 08:09 +0100, Ciaran McCreesh wrote:
> > We definitely don't want to install DEPEND at the pkg_* stages, so I'd
> > say the requirement there, if you're asking, is prior to src_*, if
> > that matters.
> 
> If the alternatives are not being able to install from a binary at all
> due to circular dependencies, or being able to install from a binary
> using DEPEND to satisfy circular dependencies, which would you take?

Given the trouble that we have every release with trying to cram
everything our users want into a limited space, I'd rather the damned
thing not install than pull in a bunch of packages we don't need, just
to satisfy a dependency that isn't even used during execution of the
package.

> > I'd love to have some kind of functionality to allow some kind of
> > "optional" dependencies.  The only real way that I could see this
> > working is if we tracked what was installed as an optional dependency,
> > and not reinstall it if it has been removed the next time the
> > depending package is merged.
> 
> Sort of like what kdebuild has for suggested dependencies, but less
> strong?

Pretty much, yeah.  The main difference that I would see from the
current *DEPEND variables, besides what was said above, would be that a
lack of visibility wouldn't stop the package merge.  If sys-apps/foo had
ODEPEND="dev-libs/bar" and dev-libs/bar was masked, it simply wouldn't
be installed.

-- 
Chris Gianelloni
Release Engineering Strategic Lead
Games Developer
-- 
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] Dependencies that're available at pkg_*inst

[Doug Goldstein]

Ciaran McCreesh wrote:
> On Sat, 19 Apr 2008 18:38:06 +0200
> "Marijn Schouten (hkBst)" <hkBst@gentoo.org> wrote:
>   
>> Every package dependency in DEPEND is installed and usable before
>> src_unpack starts, right? So is the question here whether or not they
>> can be uninstalled right before pkg_{pre,post}inst starts?
>>     
>
> If we're using binaries, DEPEND is usually ignored.
>   
But if we're using binaries then src_unpack isn't called so this is a 
moot statement and the O.P.'s statement is correct.

>   
>> I don't know what the general use of pkg_preinst is, but in
>> pkg_postinst the package itself should be runnable, so its RDEPENDS
>> should be installed and usable at this point. So perhaps we should
>> define that "usable" means "each of its RDEPENDs is installed and has
>> had its pkg_postinst function run". The recursion of that definition
>> then comes from the requirement that RDEPENDs should be usable before
>> pkg_postinst starts running.
>>     
>
> No good. That prevents RDEPEND <-> RDEPEND cycles from being solved,
> and the package manager has to be able to solve that.
>   

Can you give a concrete example? Not a foo and bar or a and b example. 
But a real package example. Because there are a few packages in the tree 
which call themselves in pkg_postinst

-- 
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] Dependencies that're available at pkg_*inst

[Doug Goldstein]

Ciaran McCreesh wrote:
> On Fri, 18 Apr 2008 21:45:13 -0700
> Donnie Berkholz <dberkholz@gentoo.org> wrote:
>   
>> I'd go with RDEPEND only. Any other interpretation results in
>> installing build-time-only packages along with a binpkg, which
>> doesn't seem to make sense.
>>     
>
> That's definitely not what we want. Only a package's DEPENDs have to be
> installed and usable when that package is built. Its RDEPENDs don't
> have to be installed until that package is treated as usable.
>
> For why this matters:
>
> cat/a-1: RDEPEND cat/b
> cat/b-1: RDEPEND cat/a
>
> This is solvable. If package managers can't solve this, they can't
> install Gnome off a stage 3...
>
>   
I think Donnie (and I) are both looking for concrete examples here. 
You're claiming GNOME can't be installed so please give us an example 
with in-tree packages where this breaks.
-- 
gentoo-dev@lists.gentoo.org mailing list

[gentoo-dev] Re: Dependencies that're available at pkg_*inst

[Steve Long]

Ciaran McCreesh wrote:

> On Sat, 19 Apr 2008 18:38:06 +0200
> "Marijn Schouten (hkBst)" <hkBst@gentoo.org> wrote:
>> I don't know what the general use of pkg_preinst is, but in
>> pkg_postinst the package itself should be runnable, so its RDEPENDS
>> should be installed and usable at this point. So perhaps we should
>> define that "usable" means "each of its RDEPENDs is installed and has
>> had its pkg_postinst function run". The recursion of that definition
>> then comes from the requirement that RDEPENDs should be usable before
>> pkg_postinst starts running.
> 
> No good. That prevents RDEPEND <-> RDEPEND cycles from being solved,
> and the package manager has to be able to solve that.
>
Use PDEPEND.

>> SRC_UNPACK_DEP="app-arch/unzip"
>> SRC_COMPILE_DEP="dev-scheme/bigloo"
>> SRC_INSTALL_DEP=""
> 
> Labels are a cleaner solution to this. But again, we're discussing
> current EAPIs here.
> 
While I like labels they need to be discussed more on-list as well as on
bugzilla (it's not reasonable for you simply to advertise them and then
close down discussion.) For instance, there is no reason everything has to
be loaded into just one extant metadatum, not do they preclude new metadata
(such as a SRC_DEP here.)


-- 
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] Re: Dependencies that're available at pkg_*inst

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1181 bytes --]

On Sun, 27 Apr 2008 10:41:57 +0100
Steve Long <slong@rathaus.eclipse.co.uk> wrote:
> Use PDEPEND.

PDEPEND has a different meaning, and isn't suitable for runtime
dependencies.

> While I like labels they need to be discussed more on-list as well as
> on bugzilla (it's not reasonable for you simply to advertise them and
> then close down discussion.) For instance, there is no reason
> everything has to be loaded into just one extant metadatum, not do
> they preclude new metadata (such as a SRC_DEP here.)

Labels can be discussed on-list whenever there's a chance in hell of
Portage implementing any non-trivial new features.

Anyway, I'm going with the second wording in the original email. It
seems fairly clear that most people aren't understanding the issue, and
are jumping in and offering opinions without having looked at the tree
(and no, I'm not going to give examples, because that'll just
degenerate into "oh, so we can change this one particular case to do
$blah", whilst missing the bigger point). Of everything suggested, only
the two original wordings are correct, and of those two, the second is
better defined.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

[gentoo-dev] Re: Re: Dependencies that're available at pkg_*inst

[Steve Long]

Ciaran McCreesh wrote:

> On Sun, 27 Apr 2008 10:41:57 +0100
> Steve Long <slong@rathaus.eclipse.co.uk> wrote:
>> Use PDEPEND.
> 
> PDEPEND has a different meaning, and isn't suitable for runtime
> dependencies.
>
"PDEPEND should be avoided in favour of RDEPEND except where this will
create circular dependency chains."[1]
Sounds very much like it is used for runtime deps, and breaking RDEPEND
cycles has often been given as its purpose in #-portage and #-dev-help, as
well as in the devmanual.

>> While I like labels they need to be discussed more on-list as well as
>> on bugzilla (it's not reasonable for you simply to advertise them and
>> then close down discussion.) For instance, there is no reason
>> everything has to be loaded into just one extant metadatum, not do
>> they preclude new metadata (such as a SRC_DEP here.)
> 
> Labels can be discussed on-list whenever there's a chance in hell of
> Portage implementing any non-trivial new features.
>
That's not exactly in the spirit of collaboration (nor are your continuous
snipes at portage.) New features should be discussed with a wider audience
than bugzilla, not just used to advertise one impl and slipped in via an
overlay. Further, having a consensus would allow pkgcore to move ahead with
a more solid spec, and that /is/ conducive to quicker implementation in
portage, since those two teams do know how to collaborate effectively.

> Anyway, I'm going with the second wording in the original email. 
<snip more insults>
> Of everything suggested, only 
> the two original wordings are correct, and of those two, the second is
> better defined.
> 
2b) seemed better. With use of PDEPEND in the manner outlined, it simply
means pkg_{pre,post}inst can't rely on the PDEPEND'ed pkgs, only those in
RDEPEND.

Build-time dependencies wouldn't appear to cover the use-cases brought up,
nor are they relevant for binary installs. I can see how it would be easier
for the PM to be able to go for one or the other, but it doesn't give an
ebuild author a consistent base. The intersection does but doesn't allow a
package to call itself (one of the use case brought up.) PDEPENDs are used
at ebuild authors' discretion aiui, and in collaboration between the two
maintainers: that judgement would seem to be useful to decide which
interdependent package can call the other, which is very much dependent on
the context. (A classic case of something that can't be solved
automatically.)

[1] http://devmanual.gentoo.org/general-concepts/dependencies/index.html


-- 
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] Re: Re: Dependencies that're available at pkg_*inst

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 2553 bytes --]

On Mon, 28 Apr 2008 05:57:04 +0100
Steve Long <slong@rathaus.eclipse.co.uk> wrote:
> Ciaran McCreesh wrote:
> > On Sun, 27 Apr 2008 10:41:57 +0100
> > Steve Long <slong@rathaus.eclipse.co.uk> wrote:
> >> Use PDEPEND.
> > 
> > PDEPEND has a different meaning, and isn't suitable for runtime
> > dependencies.
> >
> "PDEPEND should be avoided in favour of RDEPEND except where this will
> create circular dependency chains."[1]
> Sounds very much like it is used for runtime deps, and breaking
> RDEPEND cycles has often been given as its purpose in #-portage and
> #-dev-help, as well as in the devmanual.

Yup, but it can't break all circular dependency chains.

> >> While I like labels they need to be discussed more on-list as well
> >> as on bugzilla (it's not reasonable for you simply to advertise
> >> them and then close down discussion.) For instance, there is no
> >> reason everything has to be loaded into just one extant metadatum,
> >> not do they preclude new metadata (such as a SRC_DEP here.)
> > 
> > Labels can be discussed on-list whenever there's a chance in hell of
> > Portage implementing any non-trivial new features.
> >
> That's not exactly in the spirit of collaboration (nor are your
> continuous snipes at portage.) New features should be discussed with
> a wider audience than bugzilla, not just used to advertise one impl
> and slipped in via an overlay. Further, having a consensus would
> allow pkgcore to move ahead with a more solid spec, and that /is/
> conducive to quicker implementation in portage, since those two teams
> do know how to collaborate effectively.

And if there's any chance that labels will ever be usable in the main
tree, that discussion will happen.

> 2b) seemed better. With use of PDEPEND in the manner outlined, it
> simply means pkg_{pre,post}inst can't rely on the PDEPEND'ed pkgs,
> only those in RDEPEND.

2b) isn't an option, since it's wrong. 2) is an option.

> Build-time dependencies wouldn't appear to cover the use-cases
> brought up, nor are they relevant for binary installs.

Which means in some cases binary packages are unusable where source
packages wouldn't be.

> I can see how it would be easier for the PM to be able to go for one
> or the other, but it doesn't give an ebuild author a consistent base.
> The intersection does but doesn't allow a package to call itself (one
> of the use case brought up.)

No, it means ebuilds have to be careful with dependencies if calling
themselves.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]