Dependent conditional dependencies, ( was Re: [gentoo-dev] Future EAPI feature Request/RFC: ^^( ) for [RP]?DEPEND )

Dependent conditional dependencies, ( was Re: [gentoo-dev] Future EAPI feature Request/RFC: ^^( ) for [RP]?DEPEND )

[Kent Fredric]

On 3 July 2012 20:24, Michał Górny <mgorny@gentoo.org> wrote:
>
> --depclean?

eix Module-Metadata
[I] perl-core/Module-Metadata
     Available versions:  ~1.0.3 ~1.0.4 ~1.0.5 1.0.6 ~1.0.9    <---
not unmasked by --autounmask
     Installed versions:  1.0.6(15:59:00 06/26/12)
     Homepage:            http://search.cpan.org/dist/Module-Metadata/
     Description:         Gather package and POD information from perl
module files

[I] virtual/perl-Module-Metadata
     Available versions:  ~1.0.3 ~1.0.4-r2 ~1.0.5 1.0.6 (~)1.0.9-r1
<----- Unmasked by --autounmask
     Installed versions:  1.0.9-r1(09:37:51 07/02/12)
     Description:         Virtual for Module-Metadata

perl-Module-Metadata-1.0.9.ebuild

   RDEPEND="|| ( =dev-lang/perl-5.16* ~perl-core/${PN#perl-}-${PV} )"

It appears yes, --depclean *will* reap perl-core/* in this scenario  (
portage 2.2.0_alpha114 )

Just I don't tend to use --depclean an awful lot. Usually, I don't
expect to have to use --depclean to avoid a somewhat "broken" system.

>
> Doesn't perl-cleaner handle perl upgrades for this? And the tested
> ABI_SLOTs should help with that too.

ABI_SLOT may be able to help, but the problem is that installing
perl-core/foo-1.0 on perl-5.10 which ships foo-2.0 , is 100% fine.

It will just shadow the 2.0 version with the 1.0 version, and assume
"that is what you want", while the virtual is trying to convey "That
is not what we want".

And perl-cleaner doesn't have any code to handle this scenario last I
checked.  You can't even work out what is the "right" installation
scenario without first knowing "what do installed packages want". If
"installed packages want version foo to be smaller than 2.0" then
perl-core/foo-1.0 being installed on perl-5.10 is "Fine" . For this,
they would depend on "<virtual/perl-foo-2" .

In essence, you can't tell if the right perl-core/* is installed
without first looking that what virtual is installed.


>
> This is a really fragile approach, and is mostly a workaround
> to the real issue. You want to say «I need *only* one of my
> dependencies satisfied» while you actually get «if I'm installed, then
> let every my dependency in those blocks actually block each other».
>
> That's just impossible to achieve. Think of ^^ ( foo bar ). When
> the package gets installed, foo is installed and bar is not. Then you
> want to emerge bar. What should happen?
>
> a) you want portage to refuse to do that. Why would it? AFAIU this
> would no longer be a problem actually.

Given

C = " ^^( a b )"  and you had "A and C" in your world, and you wanted
to install B, portage would tell you that to do that, you would have
to remove either A or C.  ( Yay, the communicative property of XOR :D
)

> b) you want portage to do that. But you just forced it unmerge it? It
> will install the previously made binary package and it's back...

I can't parse this statement. Sorry :/

> c) you want portage to unmerge foo because the dep will now be
> satisfied by bar. Wait... unmerge perl?

No, perl would never be removed, not unless the ^^( )  was simply

  "^^(  perl  foo )"

In practice, it would be "^^( =dev-lang/perl-5.16*  =foo-5.0 )" which
would  mean

a) remove foo
b) downgrade/upgrade dev-lang/perl

of course, I have noticed a fly in my ointment, in that this logic
would mean this blocker could be avoided by down/upgrading foo, which
is precisely what we want to avoid. So its back to the drawing board.


If we were to discard what we currently know about dependencies for a
moment, a dep spec of


perl-Foo-5.0:

   if ( =dev-lang/perl-5.10.0 ) {
         block versions of perl-core/Foo that are not 5.0
         # Because if somebody wants to install perl-core/Foo-5.0 in
perl-5.10 , thats fine, its pointless, but its fine
         # because perl-core/Foo-5.0 'provides'  Foo-5.0, as does
perl-5.10, so this is satisfactory behaviour
   } else {
        install perl-core/Foo-5.0 and only perl-core/Foo-5.0
        # Because something has stated that it "wants" Foo-5.0 for some reason,
        # So we must deliver it at all costs. If its not shipped in
perl, then we provide it.
   }

It would be nice to be able to just say  "Fine, how about we just
always install from perl-core/".

Which hits the road block as soon as upstream release dev-lang/perl
with Foo-5.01 , and Foo-5.01 is not available on CPAN. ( Sometimes its
a 'dev' release, other times its not ) .

Better approaches welcome.

I have thought of scrapping the virtuals entirely and handling it so
that things depend on perl-core/* instead, and perl-core can just
dynamically decide at install time whether or not it needs to no-op (
and sometimes perl-core/* will need to hard depend on perl and just
install nothing ).

This seems a simpler approach until you consider the problem of "How
do we determine dependencies for this ebuild".

Messy. :/

With API_SLOTS I guess we can (maybe) have api-slot conditional
SRC_URI and DEPEND values,

# Not real code, just pesudocode
SRC_URI=" ! API-Perl-5.10 ? ( ..... ) "

DEPEND=" ! API-Perl-5.10 ? ( ..... ) "

Or some shit like that.

If we're really really keen, we might be able to throw it together
with some USE_EXPAND , it might be tidier, it might not be.


-- 
Kent

perl -e  "print substr( \"edrgmaM  SPA NOcomil.ic\\@tfrken\", \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

http://kent-fredric.fox.geek.nz

Re: Dependent conditional dependencies, ( was Re: [gentoo-dev] Future EAPI feature Request/RFC: ^^( ) for [RP]?DEPEND )

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 4950 bytes --]

On Tue, 3 Jul 2012 21:05:46 +1200
Kent Fredric <kentfredric@gmail.com> wrote:

> On 3 July 2012 20:24, Michał Górny <mgorny@gentoo.org> wrote:
> >
> > --depclean?
> 
> eix Module-Metadata
> [I] perl-core/Module-Metadata
>      Available versions:  ~1.0.3 ~1.0.4 ~1.0.5 1.0.6 ~1.0.9    <---
> not unmasked by --autounmask
>      Installed versions:  1.0.6(15:59:00 06/26/12)
>      Homepage:            http://search.cpan.org/dist/Module-Metadata/
>      Description:         Gather package and POD information from perl
> module files
> 
> [I] virtual/perl-Module-Metadata
>      Available versions:  ~1.0.3 ~1.0.4-r2 ~1.0.5 1.0.6 (~)1.0.9-r1
> <----- Unmasked by --autounmask
>      Installed versions:  1.0.9-r1(09:37:51 07/02/12)
>      Description:         Virtual for Module-Metadata
> 
> perl-Module-Metadata-1.0.9.ebuild
> 
>    RDEPEND="|| ( =dev-lang/perl-5.16* ~perl-core/${PN#perl-}-${PV} )"
> 
> It appears yes, --depclean *will* reap perl-core/* in this scenario  (
> portage 2.2.0_alpha114 )
> 
> Just I don't tend to use --depclean an awful lot. Usually, I don't
> expect to have to use --depclean to avoid a somewhat "broken" system.

Yes. Which simply means that something is broken with dependencies.
And by that I mean that either:

a) || ( a b ) should be || ( b a ), to actually state what perl does,
b) perl should be modified to work like our deps specify.

> > Doesn't perl-cleaner handle perl upgrades for this? And the tested
> > ABI_SLOTs should help with that too.
> 
> ABI_SLOT may be able to help, but the problem is that installing
> perl-core/foo-1.0 on perl-5.10 which ships foo-2.0 , is 100% fine.
> 
> It will just shadow the 2.0 version with the 1.0 version, and assume
> "that is what you want", while the virtual is trying to convey "That
> is not what we want".

If user intentionally installs an older version, he should be aware
that the result will be having the older version rather than the newer
one. I think we shouldn't prevent that.

> > This is a really fragile approach, and is mostly a workaround
> > to the real issue. You want to say «I need *only* one of my
> > dependencies satisfied» while you actually get «if I'm installed,
> > then let every my dependency in those blocks actually block each
> > other».
> >
> > That's just impossible to achieve. Think of ^^ ( foo bar ). When
> > the package gets installed, foo is installed and bar is not. Then
> > you want to emerge bar. What should happen?
> >
> > a) you want portage to refuse to do that. Why would it? AFAIU this
> > would no longer be a problem actually.
> 
> Given
> 
> C = " ^^( a b )"  and you had "A and C" in your world, and you wanted
> to install B, portage would tell you that to do that, you would have
> to remove either A or C.  ( Yay, the communicative property of XOR :D
> )
> 
> > b) you want portage to do that. But you just forced it unmerge it?
> > It will install the previously made binary package and it's back...
> 
> I can't parse this statement. Sorry :/

Nevermind it. It was an assumption that a & b could be fine together
later on but I'm not sure if it was really on the topic.

> > c) you want portage to unmerge foo because the dep will now be
> > satisfied by bar. Wait... unmerge perl?
> 
> No, perl would never be removed, not unless the ^^( )  was simply
> 
>   "^^(  perl  foo )"
> 
> In practice, it would be "^^( =dev-lang/perl-5.16*  =foo-5.0 )" which
> would  mean
> 
> a) remove foo
> b) downgrade/upgrade dev-lang/perl
> 
> of course, I have noticed a fly in my ointment, in that this logic
> would mean this blocker could be avoided by down/upgrading foo, which
> is precisely what we want to avoid. So its back to the drawing board.

I don't really think we allow blockers to enforce upgrades/downgrades.

> I have thought of scrapping the virtuals entirely and handling it so
> that things depend on perl-core/* instead, and perl-core can just
> dynamically decide at install time whether or not it needs to no-op (
> and sometimes perl-core/* will need to hard depend on perl and just
> install nothing ).
> 
> This seems a simpler approach until you consider the problem of "How
> do we determine dependencies for this ebuild".

Do you actually need to do that? All those ebuilds will depend on perl
with minimal version number necessary for the package to build.

If perl is older, the package will be built normally. If perl is newer,
the package will install a no-op. It's fine unless you consider
downgrading perl. But thinking about it... any upgrade or downgrade of
perl breaks all the modules anyway.

About other package dependencies, they're probably fine in both cases.
If a newer perl provides the particular module, all its dependencies
have to be satisfied in perl anyway. So it will just pull more
'virtuals'.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: Dependent conditional dependencies, ( was Re: [gentoo-dev] Future EAPI feature Request/RFC: ^^( ) for [RP]?DEPEND )

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 03/07/12 05:05 AM, Kent Fredric wrote:
> On 3 July 2012 20:24, Michał Górny <mgorny@gentoo.org> wrote:
>> 
>> --depclean?
> 
> eix Module-Metadata [I] perl-core/Module-Metadata Available
> versions:  ~1.0.3 ~1.0.4 ~1.0.5 1.0.6 ~1.0.9    <--- not unmasked
> by --autounmask Installed versions:  1.0.6(15:59:00 06/26/12) 
> Homepage:            http://search.cpan.org/dist/Module-Metadata/ 
> Description:         Gather package and POD information from perl 
> module files
> 
> [I] virtual/perl-Module-Metadata Available versions:  ~1.0.3
> ~1.0.4-r2 ~1.0.5 1.0.6 (~)1.0.9-r1 <----- Unmasked by --autounmask 
> Installed versions:  1.0.9-r1(09:37:51 07/02/12) Description:
> Virtual for Module-Metadata
> 
> perl-Module-Metadata-1.0.9.ebuild
> 
> RDEPEND="|| ( =dev-lang/perl-5.16* ~perl-core/${PN#perl-}-${PV} )"
> 
> It appears yes, --depclean *will* reap perl-core/* in this scenario
> ( portage 2.2.0_alpha114 )
> 
> Just I don't tend to use --depclean an awful lot. Usually, I don't 
> expect to have to use --depclean to avoid a somewhat "broken"
> system.
> 
>> 
>> Doesn't perl-cleaner handle perl upgrades for this? And the
>> tested ABI_SLOTs should help with that too.
> 


Just to add a note in here, since the perl stuff is one of the first
things I converted to 4-slot-abi, unfortunately, no it doesn't seem
that ABI slots are going to help with this.  Sub-slots will trigger
rebuilds of anything built against an older perl when it's upgraded,
but it seems to have no effect on any of the virtual/perl-* stuff.
The only effect it may have is giving the ability to specify perl
versions via slots instead of wildcard-versions (ie:
=dev-lang/perl-5.16* would become dev-lang/perl:0/5.16 )

Some of these perl virtuals have rather convoluted perl versions
(like, all 5.16, all 5.14, only 2 versions of 5.12, and a particular
version of 5.10), and so I don't think sub-slot deps will help the
situation here as i don't think we could align the sub-slot bumps by
feature set, at least not right now..

One thing that sub-slots ARE doing, though, is they are (as far as I
can tell) not triggering rebuilds for any installed packages that are
superseded (and therefore their virtual is now satisfied) by the newer
perl.  I realize this didn't occur before either, but I don't know if
perl-cleaner handled their removal or not (or if perl-cleaner actually
rebuilt them)?


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iF4EAREIAAYFAk/zClIACgkQ2ugaI38ACPDIPwEAkd6hlmoMQ4SRhvkttoyP11ih
EKoR+tUNaMEqeb87T34A/irG+h8tPolXKsJQtEoRKr5xIGvWDYT83LlmV4fbOwui
=Vf92
-----END PGP SIGNATURE-----

Re: Dependent conditional dependencies, ( was Re: [gentoo-dev] Future EAPI feature Request/RFC: ^^( ) for [RP]?DEPEND )

[Kent Fredric]

On 4 July 2012 02:16, Michał Górny <mgorny@gentoo.org> wrote:
>> I have thought of scrapping the virtuals entirely and handling it so
>> that things depend on perl-core/* instead, and perl-core can just
>> dynamically decide at install time whether or not it needs to no-op (
>> and sometimes perl-core/* will need to hard depend on perl and just
>> install nothing ).
>>
>> This seems a simpler approach until you consider the problem of "How
>> do we determine dependencies for this ebuild".
>
> Do you actually need to do that? All those ebuilds will depend on perl
> with minimal version number necessary for the package to build.
>
> If perl is older, the package will be built normally. If perl is newer,
> the package will install a no-op. It's fine unless you consider
> downgrading perl. But thinking about it... any upgrade or downgrade of
> perl breaks all the modules anyway.

The problem occurs in a few edge cases, which, fortunately don't
happen often. Usually its when packages appear as new additions to
dev-lang/perl.

ie: When AAAAAAA is added to perl, on perls older than 5.something
have to install all of AAAAAA's deps . ( Which are ussually provided
by perl anyway, so its not a problem ). But if *2* things are added
between perl releases and one depends on the other, ie: AAA and BBB
are added to perl 20,  installing them on perl 15 will require you to
install BBB before installing AAA.

Granted I can't think of any modules that do exactly this off the top
of my head, but its something to think about.  ( Perhaps some things
like CPANPLUS and other things that are miles ahead of the perl
verision and have external deps ), but fair to say, I think a module
that can

a) sometimes install code from CPAN

but

b) has no DEPENDS on other perl modules ( because you want to have no
depends if it is going to just no-op )

Is a recipe for disaster.

And the more I think about doing it with USE_EXPAND/USE flags with
1-flag-perl-perl-version[1] the more It feels like it would be a
bigger nightmare than what we're currently doing. What we're doing
works atm, just it has the annoying quirks that come up from time to
time, notably more around perl  releases, and these quirks slowdown
stabilizations.

*1 : ( though they've tended away from changing versions of  bundled
libs with maintenance releases these days so its not so much a visible
problem if we stick to the Majors, it can still happen that the
version of a module can change between releases, and somebody could
plausibly depend on the more recent of the 2 versions )

> If a newer perl provides the particular module, all its dependencies
> have to be satisfied in perl anyway. So it will just pull more
> 'virtuals'.
>
> --
> Best regards,
> Michał Górny



-- 
Kent

perl -e  "print substr( \"edrgmaM  SPA NOcomil.ic\\@tfrken\", \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

http://kent-fredric.fox.geek.nz

Re: Dependent conditional dependencies, ( was Re: [gentoo-dev] Future EAPI feature Request/RFC: ^^( ) for [RP]?DEPEND )

[Kent Fredric]

On 4 July 2012 02:16, Michał Górny <mgorny@gentoo.org> wrote:
> a) || ( a b ) should be || ( b a ), to actually state what perl does,

I don't really see how that would help much, if anything, I get the
impression that would

1) needlessly install "b" even when it could be satisfied by a instead
( ie: before both a & b  are installed )
2) be more inclined to keep a than it currently does
3) Probably not help in the situation I had above with

=perl-core/Module-Metadata-1.0.6      ( stable , 1.0.9 is masked by ~ still )
=virtual/perl-Module-Metadata-1.0.9    ( --autounmasked from ~ )
=dev-lang/perl-5.16.0                          ( --autounmasked from ~ )


As after installing perl-5.16.0, the || (   )  is again satisfied,
rending the system somewhat broken, but having to wait for --depclean
before it comes right.

But I'm possibly wrong somewhere, I may have fundamentally
misunderstood how || (   ) works.

On 4 July 2012 02:16, Michał Górny <mgorny@gentoo.org> wrote:
> b) perl should be modified to work like our deps specify.

That's probably the most difficult task of all, I think you'd have to
shred modules out of perl and hand package them into perl-core/
packages for that, and then never install them as part of perl,
despite the fact some of perl won't even build without some of those
modules in its tree ( so you have to remove them during install ), and
thats the start of all sorts of fun bootstrapping things I wont touch
dipped head-to-toe in antiseptic.




-- 
Kent

perl -e  "print substr( \"edrgmaM  SPA NOcomil.ic\\@tfrken\", \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

http://kent-fredric.fox.geek.nz

Re: Dependent conditional dependencies, ( was Re: [gentoo-dev] Future EAPI feature Request/RFC: ^^( ) for [RP]?DEPEND )

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 3332 bytes --]

On Wed, 4 Jul 2012 05:49:08 +1200
Kent Fredric <kentfredric@gmail.com> wrote:

> On 4 July 2012 02:16, Michał Górny <mgorny@gentoo.org> wrote:
> >> I have thought of scrapping the virtuals entirely and handling it
> >> so that things depend on perl-core/* instead, and perl-core can
> >> just dynamically decide at install time whether or not it needs to
> >> no-op ( and sometimes perl-core/* will need to hard depend on perl
> >> and just install nothing ).
> >>
> >> This seems a simpler approach until you consider the problem of
> >> "How do we determine dependencies for this ebuild".
> >
> > Do you actually need to do that? All those ebuilds will depend on
> > perl with minimal version number necessary for the package to build.
> >
> > If perl is older, the package will be built normally. If perl is
> > newer, the package will install a no-op. It's fine unless you
> > consider downgrading perl. But thinking about it... any upgrade or
> > downgrade of perl breaks all the modules anyway.
> 
> The problem occurs in a few edge cases, which, fortunately don't
> happen often. Usually its when packages appear as new additions to
> dev-lang/perl.
> 
> ie: When AAAAAAA is added to perl, on perls older than 5.something
> have to install all of AAAAAA's deps . ( Which are ussually provided
> by perl anyway, so its not a problem ). But if *2* things are added
> between perl releases and one depends on the other, ie: AAA and BBB
> are added to perl 20,  installing them on perl 15 will require you to
> install BBB before installing AAA.
> 
> Granted I can't think of any modules that do exactly this off the top
> of my head, but its something to think about.  ( Perhaps some things
> like CPANPLUS and other things that are miles ahead of the perl
> verision and have external deps ), but fair to say, I think a module
> that can
> 
> a) sometimes install code from CPAN
> 
> but
> 
> b) has no DEPENDS on other perl modules ( because you want to have no
> depends if it is going to just no-op )
> 
> Is a recipe for disaster.

I was saying the other way -- always specify the complete deps. If you
install it on older perl, it will work. If you install it on newer
perl, it shouldn't involve installing anything which is not necessary
(or already integrated into perl) anyway.

> And the more I think about doing it with USE_EXPAND/USE flags with
> 1-flag-perl-perl-version[1] the more It feels like it would be a
> bigger nightmare than what we're currently doing. What we're doing
> works atm, just it has the annoying quirks that come up from time to
> time, notably more around perl  releases, and these quirks slowdown
> stabilizations.
> 
> *1 : ( though they've tended away from changing versions of  bundled
> libs with maintenance releases these days so its not so much a visible
> problem if we stick to the Majors, it can still happen that the
> version of a module can change between releases, and somebody could
> plausibly depend on the more recent of the 2 versions )
> 
> > If a newer perl provides the particular module, all its dependencies
> > have to be satisfied in perl anyway. So it will just pull more
> > 'virtuals'.
> >
> > --
> > Best regards,
> > Michał Górny
> 
> 
> 



-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]