[gentoo-dev] New PDEPEND behaviour.

[gentoo-dev] New PDEPEND behaviour.

[Piotr Jaroszyński]

Hello,

As a result of bug #180045 PDEPENDs can be now merged even before the package 
that pulls them. Zmedico says that's intended behaviour and PDEPEND is really 
a RDEPEND, but with a ability to resolve circular deps:
circular DEPEND <-> RDEPEND can't be resolved while circular DEPEND <-> 
PDEPEND can.
Random behaviour occurs when there is a circular RDEPEND <-> PDEPEND, e.g. bug 
#186517.

We need to update docs or harass zmedico to force PDEPEND to be pulled as soon 
as possible but not before the pkg that pulls it.

-- 
Best Regards,
Piotr Jaroszyński
--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] New PDEPEND behaviour.

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 1761 bytes --]

On Wed, Jul 25, 2007 at 02:08:39PM +0200, Piotr Jaroszy??ski wrote:
> Hello,
> 
> As a result of bug #180045 PDEPENDs can be now merged even before the package 
> that pulls them. Zmedico says that's intended behaviour and PDEPEND is really 
> a RDEPEND, but with a ability to resolve circular deps:
> circular DEPEND <-> RDEPEND can't be resolved while circular DEPEND <-> 
> PDEPEND can.
> Random behaviour occurs when there is a circular RDEPEND <-> PDEPEND, e.g. bug 
> #186517.
> 
> We need to update docs or harass zmedico to force PDEPEND to be pulled as soon 
> as possible but not before the pkg that pulls it.

PDEPEND (just like RDEPEND), can, and always has been *able* to be 
satisfied prior to the node that requires it- the name may suck, but 
it's better then BREAK_RDEPEND_CYCLES, thus PDEPEND; it's never been 
viewed as a literal "it must be post" however.  Semi curious when the 
ebuild manpage picked up that description also- would expect its just 
a bad choice of words.

If in doubt, suggest you do some experiments with earlier portage 
versions, explicitly trying to force a node that is PDEPEND'd upon to 
come prior- ought to occur fine.  Basically, you're arguing based upon 
*most* PDEPEND'd nodes dep'ing on the original PDEPENDer (a cycle, 
thus with PDEPEND breaking it, the PDEPEND target coming first due to 
resolution rules) - not on rules of PDEPEND.

Either way, proposing that PDEPEND (a cycle breaking RDEPEND), be 
literal post is likely going trigger some fun fallout with the 
existing consumers of it.  Suggest you investigate those before 
pushing this idea further.

On the offchance there isn't nasty fallout from your proposal, still 
view it as -1 for the change.

~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] New PDEPEND behaviour.

[Doug Goldstein]

Piotr Jaroszyński wrote:
> Hello,
> 
> As a result of bug #180045 PDEPENDs can be now merged even before the package 
> that pulls them. Zmedico says that's intended behaviour and PDEPEND is really 
> a RDEPEND, but with a ability to resolve circular deps:
> circular DEPEND <-> RDEPEND can't be resolved while circular DEPEND <-> 
> PDEPEND can.
> Random behaviour occurs when there is a circular RDEPEND <-> PDEPEND, e.g. bug 
> #186517.
> 
> We need to update docs or harass zmedico to force PDEPEND to be pulled as soon 
> as possible but not before the pkg that pulls it.
> 

Now how in the world do you pull a depend that needs to be merged AFTER 
the original package?

i.e. Package ABC and Package XYZ... Package XYZ is a series of config 
files and data for Package ABC. The files of XYZ need to be owned by 
user abc which gets created by Package ABC. Those files also need to be 
installed in /usr/share/abc, which gets created by Package ABC.

-- 
Doug Goldstein <cardoe@gentoo.org>
http://dev.gentoo.org/~cardoe/

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] New PDEPEND behaviour.

[Carsten Lohrke]

[-- Attachment #1: Type: text/plain, Size: 782 bytes --]

On Mittwoch, 25. Juli 2007, Piotr Jaroszyński wrote:
> As a result of bug #180045 PDEPENDs can be now merged even before the
> package that pulls them. Zmedico says that's intended behaviour

Well, then what our Portage devs think the intended behaviour should be is a 
bug. E.g. in the case the bug refers to, we rely on Portage building  
kdnssd-avahi after kdelibs (otherwise it simply would fail), but before any 
other ebuild that might need kdnssd-avahi. And I'm pretty sure nearly 
everyone using PDEPEND in his ebuilds relies on Portage building the PDEPEND 
not before the ebuild, which lists it.

> We need to update docs or harass zmedico to force PDEPEND to be pulled as
> soon as possible but not before the pkg that pulls it.

The latter.


Carsten

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] New PDEPEND behaviour.

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 267 bytes --]

On Wed, 25 Jul 2007 08:46:43 -0400
Doug Goldstein <cardoe@gentoo.org> wrote:
> Now how in the world do you pull a depend that needs to be merged
> AFTER the original package?

You make the after package DEPEND upon the before package.

-- 
Ciaran McCreesh


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] New PDEPEND behaviour.

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 364 bytes --]

On Wed, 25 Jul 2007 14:51:55 +0200
Carsten Lohrke <carlo@gentoo.org> wrote:
> And I'm pretty sure nearly everyone using PDEPEND in his ebuilds
> relies on Portage building the PDEPEND not before the ebuild, which
> lists it.

And I'm pretty sure they don't, since they have the post package
DEPENDing upon the pre package anyway.

-- 
Ciaran McCreesh


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] New PDEPEND behaviour.

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 457 bytes --]

On Wed, 25 Jul 2007 14:08:39 +0200
Piotr Jaroszyński <peper@gentoo.org> wrote:
> We need to update docs or harass zmedico to force PDEPEND to be
> pulled as soon as possible but not before the pkg that pulls it.

Give one example of a legitimate use of dependencies that will break by
this change. In your answer, consider that someone might install the
post package as a target without having its dependencies installed.

-- 
Ciaran McCreesh


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] New PDEPEND behaviour.

[Doug Goldstein]

Piotr Jaroszyński wrote:
> Hello,
>
> As a result of bug #180045 PDEPENDs can be now merged even before the package 
> that pulls them. Zmedico says that's intended behaviour and PDEPEND is really 
> a RDEPEND, but with a ability to resolve circular deps:
> circular DEPEND <-> RDEPEND can't be resolved while circular DEPEND <-> 
> PDEPEND can.
> Random behaviour occurs when there is a circular RDEPEND <-> PDEPEND, e.g. bug 
> #186517.
>
> We need to update docs or harass zmedico to force PDEPEND to be pulled as soon 
> as possible but not before the pkg that pulls it.
>
>   
Actually in a follow up...

I've read the bugs you've referenced and it just sounds like a bug in 
the Paludis ebuilds not having the proper depends.

This e-mail was just some fear mongering on behalf of Paludis devs.
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] New PDEPEND behaviour.

[Doug Goldstein]

Ciaran McCreesh wrote:
> On Wed, 25 Jul 2007 08:46:43 -0400
> Doug Goldstein <cardoe@gentoo.org> wrote:
>   
>> Now how in the world do you pull a depend that needs to be merged
>> AFTER the original package?
>>     
>
> You make the after package DEPEND upon the before package.
>
>   
except the user only has to emerge ABC, and as such XYZ needs to be 
pulled in but AFTER ABC gets emerged. Your answer does not handle that 
situation. It's OBVIOUS (and in all cases in the tree) XYZ already 
DEPENDs on ABC.
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] New PDEPEND behaviour.

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 2734 bytes --]

On Wed, Jul 25, 2007 at 02:51:55PM +0200, Carsten Lohrke wrote:
> On Mittwoch, 25. Juli 2007, Piotr Jaroszyński wrote:
> > As a result of bug #180045 PDEPENDs can be now merged even before the
> > package that pulls them. Zmedico says that's intended behaviour
> 
> Well, then what our Portage devs think the intended behaviour should be is a 
> bug. E.g. in the case the bug refers to, we rely on Portage building  
> kdnssd-avahi after kdelibs (otherwise it simply would fail), but before any 
> other ebuild that might need kdnssd-avahi.

I suggest you in the future check out what actually was changed, and 
do some testing- both the original poster, and yourself are missing 
what is occuring here (if it's any consolation, I missed the real 
cause of 186517 initially too :).  The portage change is to shift the 
placement of PDEPENDed nodes as early as possible- specifically to 
fix cases where deps are a bit screwed up (like 180045, 
kdnssd-avahi/kdelibs).

Note I said 'shift'.  It tries to place it earlier in the graph, while 
*still* maintaining the constraints of kdnssd-avahi- namely the 
kdelibs dependency.

Via that dep, kdnssd-avahi *requires* kdelibs to be installed first, 
and portage honors that- it just now tries to get kdnssd-avahi merged 
as soon as possible after kdelibs due to their PDEPEND relationship 
(try it if in doubt, it lineralizes it properly).

The cases where it doesn't, are when the constraints are already 
satisfied- kdelibs already is merged, basically.  There is a change in 
placement there, but considering the data involved, wouldn't label it 
a regression- same issue can, and does occur in multiple other ways.


> And I'm pretty sure nearly 
> everyone using PDEPEND in his ebuilds relies on Portage building the PDEPEND 
> not before the ebuild, which lists it.

No one has relied on that in my experience.  They've relied on PDEPEND 
being satisfied, but not required to be satisfied by the time the 
PDEPENDer is considered 'satisfied' buildplan wise.

I strongly suspect folks echoing that sentiment are missing that a 
PDEPEND'ed nodes' DEPEND/RDEPEND *must* be satisfied prior to it being 
merged, and are missing what PDEPEND means to the resolver.


> > We need to update docs or harass zmedico to force PDEPEND to be pulled as
> > soon as possible but not before the pkg that pulls it.
> 
> The latter.

Former.  The ebuild manpage is a bit loose in it's description of what 
PDEPEND does.

As cardoe commented, the flaw that folks are hitting in 186517 is a 
data flaw (the data is bad); it's not a flaw in the resolver 
behaviour.

Feed it bad data, it's going to do stupid things basically :)

~harring


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] New PDEPEND behaviour.

[Doug Goldstein]

Ciaran McCreesh wrote:
> On Wed, 25 Jul 2007 14:51:55 +0200
> Carsten Lohrke <carlo@gentoo.org> wrote:
>   
>> And I'm pretty sure nearly everyone using PDEPEND in his ebuilds
>> relies on Portage building the PDEPEND not before the ebuild, which
>> lists it.
>>     
>
> And I'm pretty sure they don't, since they have the post package
> DEPENDing upon the pre package anyway.
>
>   
They do. You didn't understand the question at all. Examples off the top 
of my head..

ipw2100, ipw2200, ivtv, mythtv.

I want ivtv support...

emerge ivtv

It should pull in pvr-firmware immediately after since that pkg contains 
the firmware for the driver.
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] New PDEPEND behaviour.

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 291 bytes --]

On Wed, 25 Jul 2007 10:18:06 -0400
Doug Goldstein <cardoe@gentoo.org> wrote:
> This e-mail was just some fear mongering on behalf of Paludis devs.

No no. It's on behalf of Piotr, who is a Gentoo developer who happens
to not understand dependency resolution.

-- 
Ciaran McCreesh


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] New PDEPEND behaviour.

[Joe Peterson]

Piotr Jaroszyński wrote:
> We need to update docs or harass zmedico to force PDEPEND to be pulled as soon 
> as possible but not before the pkg that pulls it.

There is another problem with PDEPEND that I've run into: if you are
doing an emerge that fails some time after the package containing the
PDEPEND builds but *before* the depended-on package builds successfully,
the dependency will not be "remembered" as needed on the next emerge
(unless you do a resume, of course).  So, basically, the dependency gets
forgotten.  The new behavior discussed here makes this scenario less
likely but does not eliminate it.

						-Joe


-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] New PDEPEND behaviour.

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 736 bytes --]

On Wed, 25 Jul 2007 10:24:14 -0400
Doug Goldstein <cardoe@gentoo.org> wrote:
> Ciaran McCreesh wrote:
> > On Wed, 25 Jul 2007 08:46:43 -0400
> > Doug Goldstein <cardoe@gentoo.org> wrote:
> >   
> >> Now how in the world do you pull a depend that needs to be merged
> >> AFTER the original package?
> >
> > You make the after package DEPEND upon the before package.
> 
> except the user only has to emerge ABC, and as such XYZ needs to be 
> pulled in but AFTER ABC gets emerged. Your answer does not handle
> that situation. It's OBVIOUS (and in all cases in the tree) XYZ
> already DEPENDs on ABC.

ABC: PDEPEND="XYX"
XYZ: DEPEND="ABC"

That's all that's needed to force correct ordering.

-- 
Ciaran McCreesh


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] New PDEPEND behaviour.

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 458 bytes --]

On Wed, 25 Jul 2007 10:28:27 -0400
Doug Goldstein <cardoe@gentoo.org> wrote:
> They do. You didn't understand the question at all. Examples off the
> top of my head..
> 
> ipw2100, ipw2200, ivtv, mythtv.
> 
> I want ivtv support...
> 
> emerge ivtv
> 
> It should pull in pvr-firmware immediately after since that pkg
> contains the firmware for the driver.

ivtv: PDEPEND="pvr-firmware"
pvr-firmware: DEPEND="ivtv"

-- 
Ciaran McCreesh


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] New PDEPEND behaviour.

[Piotr Jaroszyński]

On Wednesday 25 of July 2007 16:18:06 Doug Goldstein wrote:
> I've read the bugs you've referenced and it just sounds like a bug in
> the Paludis ebuilds not having the proper depends.
>
> This e-mail was just some fear mongering on behalf of Paludis devs.

I think you misread my e-mail completly and btw. Paludis is following the same 
rules:
A: PDEPEND="B"
B: RDEPEND="A"

Both portage and paludis can end up installing B before A and it seems there 
is nothing wrong with it other than that I think it doesn't match the docs.

-- 
Best Regards,
Piotr Jaroszyński
--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] New PDEPEND behaviour.

[Carsten Lohrke]

[-- Attachment #1: Type: text/plain, Size: 1589 bytes --]

On Mittwoch, 25. Juli 2007, Brian Harring wrote:
> I suggest you in the future check out what actually was changed, and
> do some testing- both the original poster, and yourself are missing
> what is occuring here

Uh, thanks, I never was fond of reading the code of Portage, so I took Piotr's 
point as given.

> Note I said 'shift'.  It tries to place it earlier in the graph, while
> *still* maintaining the constraints of kdnssd-avahi- namely the
> kdelibs dependency.
>
> Via that dep, kdnssd-avahi *requires* kdelibs to be installed first,
> and portage honors that- it just now tries to get kdnssd-avahi merged
> as soon as possible after kdelibs due to their PDEPEND relationship
> (try it if in doubt, it lineralizes it properly).
>
> The cases where it doesn't, are when the constraints are already
> satisfied- kdelibs already is merged, basically.  There is a change in
> placement there, but considering the data involved, wouldn't label it
> a regression- same issue can, and does occur in multiple other ways.

That's fine.

> > The latter.
>
> Former.  The ebuild manpage is a bit loose in it's description of what
> PDEPEND does.

Well, I should point out where I come from. There is no need to install a pure 
runtime dependency before the ebuild pulling it in. If pure runtime 
dependencies would be handled this way, there would be no need for PDEPEND at 
all. I consider the current way Portage handles pure runtime dependencies 
(causing the need for the artificial PDEPEND in the first place) as 
conceptually broken.


Carsten

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] New PDEPEND behaviour.

[Piotr Jaroszyński]

On Wednesday 25 of July 2007 16:18:04 Ciaran McCreesh wrote:
> Give one example of a legitimate use of dependencies that will break by
> this change. In your answer, consider that someone might install the
> post package as a target without having its dependencies installed.

I am not saying it's breaking something, I just wouldn't except such a 
behaviour after reading the docs.

-- 
Best Regards,
Piotr Jaroszyński
--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] New PDEPEND behaviour.

[Vlastimil Babka]

Carsten Lohrke wrote:
> Well, I should point out where I come from. There is no need to install a pure 
> runtime dependency before the ebuild pulling it in. If pure runtime 
> dependencies would be handled this way, there would be no need for PDEPEND at 
> all. I consider the current way Portage handles pure runtime dependencies 
> (causing the need for the artificial PDEPEND in the first place) as 
> conceptually broken.

There are uses for it:

A: RDEPEND="B"
B: RDEPEND="A"

Here you really don't need PDEPEND because in current portage, pure 
runtime dependency indeed doesn't have to be satisfied before the ebuild 
pulling it. But consider this:

A: PDEPEND="B"
B: DEPEND="A"

Here you can't use RDEPEND instead of PDEPEND, because DEPEND="A" says 
the package must be merged in a working state, thus *with all its 
RDEPENDs* and thus it would cause circular deps. PDEPEND is a way to 
relax this for such cases.

If this is what you call RDEPEND conceptually broken, then sorry for 
useles try to explain it :) Maybe package manager could be smart enough 
and relax the RDEPEND in such cases itself, maybe it's better to say 
that via PDEPEND explicitly...

-- 
Vlastimil Babka (Caster)
Gentoo/Java
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] New PDEPEND behaviour.

[Carsten Lohrke]

[-- Attachment #1: Type: text/plain, Size: 643 bytes --]

On Mittwoch, 25. Juli 2007, Vlastimil Babka wrote:
> A: PDEPEND="B"
> B: DEPEND="A"
>
> If this is what you call RDEPEND conceptually broken, then sorry for
> useles try to explain it :) Maybe package manager could be smart enough
> and relax the RDEPEND in such cases itself, maybe it's better to say
> that via PDEPEND explicitly...

Of course a valid example - and yes, that's something the package manager 
should care about. Admitted, "conceptually broken" was a bit harsh, still 
both, that a pure runtime dependency gets built before the ebuild needing it 
by default and the need for PDEPEND seem ugly to me.


Carsten

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]