[gentoo-pms] Rephrasing *DEPEND

[gentoo-pms] Rephrasing *DEPEND

[Michał Górny]

[-- Attachment #1.1: Type: text/plain, Size: 231 bytes --]

After last discussion on #gentoo-portage, I'm submitting a proposal
on rephrasing *DEPEND descriptions in the PMS. The idea is to state
better when and whether these are actually installed.

-- 
Best regards,
Michał Górny

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: 0001-Rephrase-DEPEND-to-state-when-and-whether-they-are-i.patch --]
[-- Type: text/x-patch, Size: 1643 bytes --]

From 9ccce508cb853fa11808b803ef3910fe4cf50f56 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= <mgorny@gentoo.org>
Date: Sat, 11 Jun 2011 09:32:03 +0200
Subject: [PATCH] Rephrase *DEPEND to state when and whether they are
 installed.

---
 dependencies.tex |   13 ++++++++-----
 1 files changed, 8 insertions(+), 5 deletions(-)

diff --git a/dependencies.tex b/dependencies.tex
index f6318d8..ccf8282 100644
--- a/dependencies.tex
+++ b/dependencies.tex
@@ -6,11 +6,14 @@
 There are three classes of dependencies supported by ebuilds:
 
 \begin{compactitem}
-\item Build dependencies (\t{DEPEND}). These must be installed and usable before the ebuild
-    is installed.
-\item Runtime dependencies (\t{RDEPEND}). These must be installed and usable before the ebuild
-    is treated as usable.
-\item Post dependencies (\t{PDEPEND}). These must be installed at some point.
+\item Build dependencies (\t{DEPEND}). These must be installed and usable before any of
+	the ebuild \t{src\_*} phase functions is executed. They may not be installed at all
+	if a binary package is merged.
+\item Runtime dependencies (\t{RDEPEND}). These must be installed and usable before
+	any of the files installed by the ebuild is treated as usable. They must be installed
+	before any of the ebuild \t{pkg\_*} functions are called.
+\item Post dependencies (\t{PDEPEND}). These must be installed at some point before
+	the system is rebooted after installing the ebuild.
 \end{compactitem}
 
 In addition, \t{SRC\_URI}, \t{HOMEPAGE}, \t{PROVIDE}, \t{RESTRICT}, \t{PROPERTIES}, \t{LICENSE} and
-- 
1.7.5.3


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-pms] Rephrasing *DEPEND

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 649 bytes --]

On Sat, 11 Jun 2011 09:32:45 +0200
Michał Górny <mgorny@gentoo.org> wrote:
> After last discussion on #gentoo-portage, I'm submitting a proposal
> on rephrasing *DEPEND descriptions in the PMS. The idea is to state
> better when and whether these are actually installed.

This isn't right. There's no guarantee at all for pkg_pretend, for
example.

It's also wrong to go by "files installed by the ebuild". Think, for
example, a package which installs nothing but that creates a user.

I also dislike the "rebooted" phrasing for PDEPEND. "Before the package
manager finishes the group of installs" is better.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-pms] Rephrasing *DEPEND

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 995 bytes --]

On Sat, 11 Jun 2011 12:53:41 +0100
Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:

> On Sat, 11 Jun 2011 09:32:45 +0200
> Michał Górny <mgorny@gentoo.org> wrote:
> > After last discussion on #gentoo-portage, I'm submitting a proposal
> > on rephrasing *DEPEND descriptions in the PMS. The idea is to state
> > better when and whether these are actually installed.
> 
> This isn't right. There's no guarantee at all for pkg_pretend, for
> example.

True. How about pkg_setup()? Shall we assume RDEPEND are there or rely
on @system only?

> It's also wrong to go by "files installed by the ebuild". Think, for
> example, a package which installs nothing but that creates a user.

I'm not sure how RDEPEND could be useful for such a package.

> I also dislike the "rebooted" phrasing for PDEPEND. "Before the
> package manager finishes the group of installs" is better.

Ok, I dislike it either. Maybe 'a batch of installs'?

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-pms] Rephrasing *DEPEND

[Ulrich Mueller]

>>>>> On Sun, 12 Jun 2011, Michał Górny wrote:

> True. How about pkg_setup()? Shall we assume RDEPEND are there or
> rely on @system only?

IIUC, with Portage's breaking of dependency cycles there's no absolute
guarantee that packages in RDEPEND will be available in pkg_*.

Ulrich

Re: [gentoo-pms] Rephrasing *DEPEND

[Zac Medico]

On 06/12/2011 01:18 AM, Ulrich Mueller wrote:
>>>>>> On Sun, 12 Jun 2011, Michał Górny wrote:
> 
>> True. How about pkg_setup()? Shall we assume RDEPEND are there or
>> rely on @system only?
> 
> IIUC, with Portage's breaking of dependency cycles there's no absolute
> guarantee that packages in RDEPEND will be available in pkg_*.

It would be more accurate to say that it's guaranteed except for cases
in which circular dependencies make it impossible to guarantee.
-- 
Thanks,
Zac

Re: [gentoo-pms] Rephrasing *DEPEND

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 881 bytes --]

On Sun, 12 Jun 2011 02:13:35 -0700
Zac Medico <zmedico@gentoo.org> wrote:

> On 06/12/2011 01:18 AM, Ulrich Mueller wrote:
> >>>>>> On Sun, 12 Jun 2011, Michał Górny wrote:
> > 
> >> True. How about pkg_setup()? Shall we assume RDEPEND are there or
> >> rely on @system only?
> > 
> > IIUC, with Portage's breaking of dependency cycles there's no
> > absolute guarantee that packages in RDEPEND will be available in
> > pkg_*.
> 
> It would be more accurate to say that it's guaranteed except for cases
> in which circular dependencies make it impossible to guarantee.

This or other words, this makes this an unreliable feature. So, right
now, users can't assume anything specific about dependencies being
installed in pkg_*? Doesn't this make it actually impossible to call
the installed application in pkg_postinst()?

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-pms] Rephrasing *DEPEND

[Ulrich Mueller]

>>>>> On Sun, 12 Jun 2011, Michał Górny wrote:

>> It would be more accurate to say that it's guaranteed except for
>> cases in which circular dependencies make it impossible to
>> guarantee.

> This or other words, this makes this an unreliable feature. So,
> right now, users can't assume anything specific about dependencies
> being installed in pkg_*?

Currently, PMS only says: "These must be installed and usable before
the ebuild is treated as usable." It doesn't say anything about phases.

> Doesn't this make it actually impossible to call the installed
> application in pkg_postinst()?

I've always added such packages that I needed in pkg_postinst() to
both DEPEND and RDEPEND.

IIRC, a specific merge order in Portage is only guaranteed for DEPEND
type dependencies, but not for RDEPEND and PDEPEND.

Ulrich

Re: [gentoo-pms] Rephrasing *DEPEND

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1190 bytes --]

On Sun, 12 Jun 2011 16:49:19 +0200
Ulrich Mueller <ulm@gentoo.org> wrote:
> >> It would be more accurate to say that it's guaranteed except for
> >> cases in which circular dependencies make it impossible to
> >> guarantee.
> 
> > This or other words, this makes this an unreliable feature. So,
> > right now, users can't assume anything specific about dependencies
> > being installed in pkg_*?
> 
> Currently, PMS only says: "These must be installed and usable before
> the ebuild is treated as usable." It doesn't say anything about
> phases.

The intention with the "usable" stuff is this that purely RDEPEND
cycles are resolvable, but any such cycles must be resolved before any
package which has a DEPEND upon anything in the cycle is resolved. So
if you've got this:

    first <-- rdepend --- second <-- depend --- third
          --- rdepend -->

Then (first, second, third) and (second, first, third) are the only
legal orderings. But if either RDEPEND became a DEPEND (and if we're
not dealing with binary packages) then there would be no legal ordering.

Whether or not what PMS says matches the intention is another matter...

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-pms] Rephrasing *DEPEND

[Zac Medico]

On 06/12/2011 06:16 AM, Michał Górny wrote:
> On Sun, 12 Jun 2011 02:13:35 -0700
> Zac Medico <zmedico@gentoo.org> wrote:
> 
>> On 06/12/2011 01:18 AM, Ulrich Mueller wrote:
>>>>>>>> On Sun, 12 Jun 2011, Michał Górny wrote:
>>>
>>>> True. How about pkg_setup()? Shall we assume RDEPEND are there or
>>>> rely on @system only?
>>>
>>> IIUC, with Portage's breaking of dependency cycles there's no
>>> absolute guarantee that packages in RDEPEND will be available in
>>> pkg_*.
>>
>> It would be more accurate to say that it's guaranteed except for cases
>> in which circular dependencies make it impossible to guarantee.
> 
> This or other words, this makes this an unreliable feature. So, right
> now, users can't assume anything specific about dependencies being
> installed in pkg_*?

In practice it's very reliable, although the specific behavior varies
depending on whether or not there are circular dependencies. So, it
would probably make sense to specify it like this:

(A) If there are no circular RDEPEND dependencies, then RDEPEND is
guaranteed to be satisfied.

(B) If there are circular RDEPEND dependencies, then RDEPEND is not
guaranteed to be satisfied.

> Doesn't this make it actually impossible to call
> the installed application in pkg_postinst()?

Only when case (B) applies. It may be possible to improve a situation
like this by making the circular dependency PDEPEND in one direction and
RDEPEND in the other:

  first RDEPENDs on second
  second PDEPENDs on first

This make it possible to guarantee case (A) for the first package, while
there is no such guarantee for the second package.
-- 
Thanks,
Zac

Re: [gentoo-pms] Rephrasing *DEPEND

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 332 bytes --]

On Sun, 12 Jun 2011 11:03:17 -0700
Zac Medico <zmedico@gentoo.org> wrote:
> (B) If there are circular RDEPEND dependencies, then RDEPEND is not
> guaranteed to be satisfied.

This should give an error unless the user explicitly says "I really
want you to cheat here, even though it will break stuff".

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-pms] Rephrasing *DEPEND

[Brian Harring]

On Sun, Jun 12, 2011 at 02:13:35AM -0700, Zac Medico wrote:
> On 06/12/2011 01:18 AM, Ulrich Mueller wrote:
> >>>>>> On Sun, 12 Jun 2011, Michał Górny wrote:
> > 
> >> True. How about pkg_setup()? Shall we assume RDEPEND are there or
> >> rely on @system only?
> > 
> > IIUC, with Portage's breaking of dependency cycles there's no absolute
> > guarantee that packages in RDEPEND will be available in pkg_*.
> 
> It would be more accurate to say that it's guaranteed except for cases
> in which circular dependencies make it impossible to guarantee.

Those instances shouldn't be just dropped by the manager; a 
--force-break-ebuild-rules option for those cases is one thing, same 
for attempting multiple merge/replace to break a use cycle.

But if it just says "meh" to a cycle... that's wrong.

Under what scenarios will it pull that?
~brian

Re: [gentoo-pms] Rephrasing *DEPEND

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 415 bytes --]

On Sun, 12 Jun 2011 10:18:57 +0200
Ulrich Mueller <ulm@gentoo.org> wrote:
> >>>>> On Sun, 12 Jun 2011, Michał Górny wrote:
> > True. How about pkg_setup()? Shall we assume RDEPEND are there or
> > rely on @system only?
> 
> IIUC, with Portage's breaking of dependency cycles there's no absolute
> guarantee that packages in RDEPEND will be available in pkg_*.

Isn't that a bug?

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-pms] Rephrasing *DEPEND

[Michał Górny]

[-- Attachment #1.1: Type: text/plain, Size: 251 bytes --]

And here goes another proposal.

This time I introduced a table to clearly state phase function <->
dependency class relationships but I guess someone with a taste should
look at it and make it look good :P.

-- 
Best regards,
Michał Górny

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: 0001-Rephrase-DEPEND-to-state-when-and-whether-they-are-i.patch --]
[-- Type: text/x-patch, Size: 2607 bytes --]

From e1bce9a316494f14d059b5afc0e1912832622334 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= <mgorny@gentoo.org>
Date: Sat, 11 Jun 2011 09:32:03 +0200
Subject: [PATCH] Rephrase *DEPEND to state when and whether they are
 installed.

---
 dependencies.tex |   27 ++++++++++++++++++++++-----
 1 files changed, 22 insertions(+), 5 deletions(-)

diff --git a/dependencies.tex b/dependencies.tex
index f6318d8..b218579 100644
--- a/dependencies.tex
+++ b/dependencies.tex
@@ -6,13 +6,30 @@
 There are three classes of dependencies supported by ebuilds:
 
 \begin{compactitem}
-\item Build dependencies (\t{DEPEND}). These must be installed and usable before the ebuild
-    is installed.
-\item Runtime dependencies (\t{RDEPEND}). These must be installed and usable before the ebuild
-    is treated as usable.
-\item Post dependencies (\t{PDEPEND}). These must be installed at some point.
+\item Build dependencies (\t{DEPEND}). These must be installed and usable before any of
+	the ebuild \t{src\_*} phase functions is executed. These may not be installed at all
+	if a binary package is being merged.
+\item Runtime dependencies (\t{RDEPEND}). These must be installed and usable before
+	the results of an ebuild merging are treated as usable.
+\item Post dependencies (\t{PDEPEND}). These must be installed at some point before
+	the package manager finishes the batch of installs.
 \end{compactitem}
 
+\begin{centertable}{Dependency classes required to be satisfied for a particular phase function} \label{tab:phase-function-dependency-classes}
+    \begin{tabular}{ p{0.2\textwidth} p{0.7\textwidth} }
+        \toprule
+        \multicolumn{1}{c}{\textbf{Phase function}} &
+        \multicolumn{1}{c}{\textbf{Satisfied dependency classes}} \\
+        \midrule
+	\t{pkg\_pretend}, \t{pkg\_setup}, \t{pkg\_info}, \t{pkg\_nofetch} & None (ebuilds can rely only on the packages in the system set) \\
+    \t{src\_unpack}, \t{src\_prepare}, \t{src\_configure}, \t{src\_compile}, \t{src\_test}, \t{src\_install} & \t{DEPEND} \\
+    \t{pkg\_preinst}, \t{pkg\_postinst}, \t{pkg\_prerm}, \t{pkg\_postrm}, \t{pkg\_config} & \t{RDEPEND} \\
+    \bottomrule
+    \end{tabular}
+\end{centertable}
+
+Table~\ref{tab:phase-function-dependency-classes} lists dependencies which must be satisfied before a particular phase function is executed.
+
 In addition, \t{SRC\_URI}, \t{HOMEPAGE}, \t{PROVIDE}, \t{RESTRICT}, \t{PROPERTIES}, \t{LICENSE} and
 \t{REQUIRED\_USE} use dependency-style specifications to specify their values.
 
-- 
1.7.5.3


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-pms] Rephrasing *DEPEND

[Michał Górny]

[-- Attachment #1.1: Type: text/plain, Size: 336 bytes --]

And the third version.

I added a note about the possible circular RDEPEND issue. I wanted to
do that through a footnote but wasn't able to get it working so it's
a standard bracket now).

I separated pkg_config (as the circular RDEP issue shouldn't apply
there) and added PDEPEND as well.

-- 
Best regards,
Michał Górny

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: 0001-Rephrase-DEPEND-to-state-when-and-whether-they-are-i.patch --]
[-- Type: text/x-patch, Size: 2786 bytes --]

From 62fd21802e8c2f93b78e7993dd552444f3751af2 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= <mgorny@gentoo.org>
Date: Sat, 11 Jun 2011 09:32:03 +0200
Subject: [PATCH] Rephrase *DEPEND to state when and whether they are
 installed.

---
 dependencies.tex |   28 +++++++++++++++++++++++-----
 1 files changed, 23 insertions(+), 5 deletions(-)

diff --git a/dependencies.tex b/dependencies.tex
index f6318d8..b2e107d 100644
--- a/dependencies.tex
+++ b/dependencies.tex
@@ -3,16 +3,34 @@
 
 \section{Dependency Classes}
 
+\begin{centertable}{Dependency classes required to be satisfied for a particular phase function} \label{tab:phase-function-dependency-classes}
+    \begin{tabular}{ p{0.2\textwidth} p{0.7\textwidth} }
+        \toprule
+        \multicolumn{1}{c}{\textbf{Phase function}} &
+        \multicolumn{1}{c}{\textbf{Satisfied dependency classes}} \\
+        \midrule
+	\t{pkg\_pretend}, \t{pkg\_setup}, \t{pkg\_info}, \t{pkg\_nofetch} & None (ebuilds can rely only on the packages in the system set) \\
+    \t{src\_unpack}, \t{src\_prepare}, \t{src\_configure}, \t{src\_compile}, \t{src\_test}, \t{src\_install} & \t{DEPEND} \\
+    \t{pkg\_preinst}, \t{pkg\_postinst}, \t{pkg\_prerm}, \t{pkg\_postrm} & \t{RDEPEND} (unless the particular dependency results in a circular dependency, in which case it may be installed later) \\
+	\t{pkg\_config} & \t{RDEPEND}, \t{PDEPEND} \\
+    \bottomrule
+    \end{tabular}
+\end{centertable}
+
 There are three classes of dependencies supported by ebuilds:
 
 \begin{compactitem}
-\item Build dependencies (\t{DEPEND}). These must be installed and usable before the ebuild
-    is installed.
-\item Runtime dependencies (\t{RDEPEND}). These must be installed and usable before the ebuild
-    is treated as usable.
-\item Post dependencies (\t{PDEPEND}). These must be installed at some point.
+\item Build dependencies (\t{DEPEND}). These must be installed and usable before any of
+	the ebuild \t{src\_*} phase functions is executed. These may not be installed at all
+	if a binary package is being merged.
+\item Runtime dependencies (\t{RDEPEND}). These must be installed and usable before
+	the results of an ebuild merging are treated as usable.
+\item Post dependencies (\t{PDEPEND}). These must be installed at some point before
+	the package manager finishes the batch of installs.
 \end{compactitem}
 
+Table~\ref{tab:phase-function-dependency-classes} lists dependencies which must be satisfied before a particular phase function is executed.
+
 In addition, \t{SRC\_URI}, \t{HOMEPAGE}, \t{PROVIDE}, \t{RESTRICT}, \t{PROPERTIES}, \t{LICENSE} and
 \t{REQUIRED\_USE} use dependency-style specifications to specify their values.
 
-- 
1.7.5.3


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-pms] Rephrasing *DEPEND

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 596 bytes --]

On Sun, 19 Jun 2011 23:21:02 +0200
Michał Górny <mgorny@gentoo.org> wrote:
> And the third version.
> 
> I added a note about the possible circular RDEPEND issue. I wanted to
> do that through a footnote but wasn't able to get it working so it's
> a standard bracket now).

I still don't think we should be specifying "RDEPEND is PDEPEND if the
package manager feels like it". That's something for the package
mangler to provide as a horrible --ignore-dependencies-to-break-cycles
option.

Also, isn't pkg_setup the same as pkg_preinst, availability-wise?

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-pms] Rephrasing *DEPEND

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1265 bytes --]

On Sun, 19 Jun 2011 22:36:04 +0100
Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:

> On Sun, 19 Jun 2011 23:21:02 +0200
> Michał Górny <mgorny@gentoo.org> wrote:
> > And the third version.
> > 
> > I added a note about the possible circular RDEPEND issue. I wanted
> > to do that through a footnote but wasn't able to get it working so
> > it's a standard bracket now).
> 
> I still don't think we should be specifying "RDEPEND is PDEPEND if the
> package manager feels like it". That's something for the package
> mangler to provide as a horrible --ignore-dependencies-to-break-cycles
> option.

We either have to mention that or assume the long-outstanding portage
behavior is not PMS compliant. And even if we choose the latter option,
I think we should have a footnote warning about that there.

> Also, isn't pkg_setup the same as pkg_preinst, availability-wise?

IMO not necessarily. During later pkg_* phases, RDEPEND needs to be
satisfied in order to make it possible to call the installed program. I
don't really see a reason to make similar assumptions in pkg_setup, and
I'd really like to avoid saying 'RDEPEND is for program's runtime
dependencies and pkg_setup dependencies'.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-pms] Rephrasing *DEPEND

[Ulrich Mueller]

>>>>> On Sun, 19 Jun 2011, Ciaran McCreesh wrote:

> On Sun, 19 Jun 2011 23:21:02 +0200 Michał Górny <mgorny@gentoo.org> wrote:
>> I added a note about the possible circular RDEPEND issue.

> I still don't think we should be specifying "RDEPEND is PDEPEND if
> the package manager feels like it". That's something for the package
> mangler to provide as a horrible --ignore-dependencies-to-break-cycles
> option.

It's _not_ saying that RDEPEND is like PDEPEND in some cases. The
assertion from your previous message in this thread will always hold:

| The intention with the "usable" stuff is this that purely RDEPEND
| cycles are resolvable, but any such cycles must be resolved before
| any package which has a DEPEND upon anything in the cycle is
| resolved. So if you've got this:
|
|     first <-- rdepend --- second <-- depend --- third
|           --- rdepend -->
|
| Then (first, second, third) and (second, first, third) are the only
| legal orderings. But if either RDEPEND became a DEPEND (and if we're
| not dealing with binary packages) then there would be no legal
| ordering.

This is long-standing Portage behaviour (introduced in 2006 with the
patches attached to bug 147766, I believe).

The footnote would only clarify that in your example neither "first"
nor "second" can rely on their rdepend being available in pkg_*inst.

Ulrich