[gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Michał Górny]

[-- Attachment #1.1: Type: text/plain, Size: 253 bytes --]

Hello,

A simple solution to a program long-unsolved. In GLEP form.

Both attached and published as a gist:

https://gist.github.com/2945569

(please note that github doesn't render GLEP headers correctly)

-- 
Best regards,
Michał Górny

[-- Attachment #1.2: iuse-runtime.txt.rst --]
[-- Type: text/plain, Size: 4558 bytes --]

GLEP: XXX
Title: Optional runtime dependencies via runtime-switchable USE flags
Version: $Revision:$
Last-Modified: $Date:$
Author: Michał Górny <mgorny@gentoo.org>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 17 Jun 2012
Post-History:


Abstract
========

This GLEP addresses the issue of referencing optional runtime
dependencies in Gentoo packages and ebuilds. It does introduce
a concept of runtime-switchable USE flags to achieve that goal.


Motivation
==========

Optional runtime dependencies are often found in packages installing
various scripts (shell, python, perl). These are not strictly required
for the particular package to work but installing them enables
additional functionality.

Unlike in compiled programs, enabling or disabling those features
(dependencies) does not affect the files installed by the package.
They can be installed and uninstalled independently of the package,
resulting in changes of functionality without a need to rebuild
the package.

Currently such dependencies are usually expressed only through
``pkg_postinst()`` messages. This forces user to manually install
the necessary dependencies, and uninstall them when they are no longer
necessary.

Another solution is using regular USE flags. Those flags do not strictly
follow the principles of USE flags because they do not affect files
installed by the package and are not entirely effective to the package
(a disabled feature will still be available if necessary dependency is
installed). Additionally, it requires unnecessary rebuilds
of the package in order to change the dependencies.


Specification
=============

The ebuilds aiming to provide features enabled through optional runtime
dependencies should:

1. create regular USE flags for all those features, following
   appropriate specifications for Gentoo ebuilds, and including
   the flags in the ``IUSE`` variable;
2. introduce additional ``IUSE_RUNTIME`` variable listing names of USE
   flags related to optional runtime dependencies (without prefixes
   related to IUSE defaults).

Additionally, the ebuilds must obey the following rules:

1. all flags listed in ``IUSE_RUNTIME`` have to be listed in ``IUSE``,
2. flags listed in ``IUSE_RUNTIME`` can be referred in ``RDEPEND``,
   ``PDEPEND`` and ``REQUIRED_USE`` variables,
3. flags listed in ``IUSE_RUNTIME`` must not be referred in phase
   functions, ``DEPEND`` or ``SRC_URI``,
4. flags listed in ``IUSE_RUNTIME`` may be referred through USE
   dependencies by other packages' ``DEPEND``, ``RDEPEND``
   and ``PDEPEND`` variables.

The package manager should treat flags listed in ``IUSE_RUNTIME``
as regular USE flags, except for the following:

1. the state of the flags must be re-evaluated each time the package
   dependency graph is considered,
2. enabling or disabling any of the flags must not involve rebuilding
   the package,
3. the flags may be listed in the visual output in a distinct way
   to inform the user that they affect runtime dependencies only.


Rationale
=========

The proposed solution tries to solve the issue of handling runtime
dependencies while reusing the existing infrastructure. Most
importantly, users will be able to reuse the existing tools
and configuration files to enable and disable optional runtime
and build-time dependencies alike.

The remaining reused features include:

- dependency syntax,
- ability to use ``REQUIRED_USE``, USE dependencies,
- ability to describe flags in `metadata.xml`,
- global flag names (and descriptions).

Alternative proposed solution involved creating additional ``SDEPEND``
variable. That proposition had the following disadvantages:

- being package-oriented rather than feature-oriented,
- lack of ability to express multiple packages required by a single
  feature,
- lack of ability to express cross-feature dependencies,
- lack of ability to describe features provided by enabled packages,
- necessity of implementing a new user interface parts to control
  the dependencies,
- lack of backwards compatibility.


Backwards compatibility
=======================

Package managers not implementing this GLEP will consider
the ``IUSE_RUNTIME`` variable as an irrelevant bash variable and treat
runtime-switchable USE flags as regular USE flags. The dependency tree
will still be consistent yet packages may be rebuilt unnecessarily.


Copyright
=========

This document has been placed in the public domain.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 302 bytes --]

On Sun, 17 Jun 2012 22:31:59 +0200
Michał Górny <mgorny@gentoo.org> wrote:
> A simple solution to a program long-unsolved. In GLEP form.
> 
> Both attached and published as a gist:
> 
> https://gist.github.com/2945569

Do you have an implementation we can play with?

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 478 bytes --]

On Sun, 17 Jun 2012 21:38:50 +0100
Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:

> On Sun, 17 Jun 2012 22:31:59 +0200
> Michał Górny <mgorny@gentoo.org> wrote:
> > A simple solution to a program long-unsolved. In GLEP form.
> > 
> > Both attached and published as a gist:
> > 
> > https://gist.github.com/2945569
> 
> Do you have an implementation we can play with?

Not yet. But expect one in the next few days.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Brian Harring]

On Sun, Jun 17, 2012 at 10:31:59PM +0200, Micha?? G??rny wrote:
> Hello,
> 
> A simple solution to a program long-unsolved. In GLEP form.
> 
> Both attached and published as a gist:
> 
> https://gist.github.com/2945569
> 
> (please note that github doesn't render GLEP headers correctly)
> 
> -- 
> Best regards,
> Micha?? G??rny

> GLEP: XXX
> Title: Optional runtime dependencies via runtime-switchable USE flags
> Version: $Revision:$
> Last-Modified: $Date:$
> Author: Micha?? G??rny <mgorny@gentoo.org>
> Status: Draft
> Type: Standards Track
> Content-Type: text/x-rst
> Created: 17 Jun 2012
> Post-History:
> 
> 
> Abstract
> ========
> 
> This GLEP addresses the issue of referencing optional runtime
> dependencies in Gentoo packages and ebuilds. It does introduce
> a concept of runtime-switchable USE flags to achieve that goal.
> 
> 
> Motivation
> ==========
> 
> Optional runtime dependencies are often found in packages installing
> various scripts (shell, python, perl). These are not strictly required
> for the particular package to work but installing them enables
> additional functionality.
> 
> Unlike in compiled programs, enabling or disabling those features
> (dependencies) does not affect the files installed by the package.
> They can be installed and uninstalled independently of the package,
> resulting in changes of functionality without a need to rebuild
> the package.
> 
> Currently such dependencies are usually expressed only through
> ``pkg_postinst()`` messages. This forces user to manually install
> the necessary dependencies, and uninstall them when they are no longer
> necessary.
> 
> Another solution is using regular USE flags. Those flags do not strictly
> follow the principles of USE flags because they do not affect files
> installed by the package and are not entirely effective to the package
> (a disabled feature will still be available if necessary dependency is
> installed). Additionally, it requires unnecessary rebuilds
> of the package in order to change the dependencies.
> 
> 
> Specification
> =============
> 
> The ebuilds aiming to provide features enabled through optional runtime
> dependencies should:
> 
> 1. create regular USE flags for all those features, following
>    appropriate specifications for Gentoo ebuilds, and including
>    the flags in the ``IUSE`` variable;
> 2. introduce additional ``IUSE_RUNTIME`` variable listing names of USE
>    flags related to optional runtime dependencies (without prefixes
>    related to IUSE defaults).
> 
> Additionally, the ebuilds must obey the following rules:
> 
> 1. all flags listed in ``IUSE_RUNTIME`` have to be listed in ``IUSE``,
> 2. flags listed in ``IUSE_RUNTIME`` can be referred in ``RDEPEND``,
>    ``PDEPEND`` and ``REQUIRED_USE`` variables,
> 3. flags listed in ``IUSE_RUNTIME`` must not be referred in phase
>    functions, ``DEPEND`` or ``SRC_URI``,
> 4. flags listed in ``IUSE_RUNTIME`` may be referred through USE
>    dependencies by other packages' ``DEPEND``, ``RDEPEND``
>    and ``PDEPEND`` variables.

Unless I'm on crack, you're stating that essentially an optional use 
flag (one you label 'runtime'), is able to be used transitively during 
DEPEND.  That's not particularly "here's some suggested pkgs to 
install"- that's "rebuild the fucker for this changed DEPEND", which 
is the existing situation.


> The package manager should treat flags listed in ``IUSE_RUNTIME``
> as regular USE flags, except for the following:
> 
> 1. the state of the flags must be re-evaluated each time the package
>    dependency graph is considered,
> 2. enabling or disabling any of the flags must not involve rebuilding
>    the package,
> 3. the flags may be listed in the visual output in a distinct way
>    to inform the user that they affect runtime dependencies only.
> 
> 
> Rationale
> =========
> 
> The proposed solution tries to solve the issue of handling runtime
> dependencies while reusing the existing infrastructure. Most
> importantly, users will be able to reuse the existing tools
> and configuration files to enable and disable optional runtime
> and build-time dependencies alike.
> 
> The remaining reused features include:
> 
> - dependency syntax,

If you invent new syntax, I'm going to be unhappy.  KISS as it were.

> - ability to use ``REQUIRED_USE``, USE dependencies,
> - ability to describe flags in `metadata.xml`,
> - global flag names (and descriptions).
> 
> Alternative proposed solution involved creating additional ``SDEPEND``
> variable. That proposition had the following disadvantages:
> 
> - being package-oriented rather than feature-oriented,

No; use flags are our configuration space, and they turn on/off 
sections of the given pkgs graph.  Your proposal relies on the same 
concept; bluntly, what you're proposing is just as 'package oriented'.

Effectively, you can't dismiss SDEPEND/ODEPEND via changing the rules 
between your proposal and ODEPEND's proposal.  Nice try though. :)


> - lack of ability to express multiple packages required by a single
>   feature,

Eh?  SDEPEND="my_feature? ( pkg1 pkg 2 )"


> - lack of ability to express cross-feature dependencies,

Clarify.  This is a wee bit too vague for responding to ;)



> - lack of ability to describe features provided by enabled packages,

metadata.xml's local use description already covers that; in your 
proposal above you lock in on use flags for the controlling of it 
(which, presumably you'll abuse to get descriptions of) yet claim it's 
not possible for ODEPEND (better name than SDEPEND :P).


> - necessity of implementing a new user interface parts to control
>   the dependencies,

Um... This is bullshit.  Your proposal above is going to require a lot 
more implementation, documentation of how this all is supposed to 
work, and user ededucation for the weird scenarios where things don't 
behave as expected, or they don't understand why changing use flag X, 
results in the PM pulling in new packages (acting akin to -N), while 
changing flag Y doesn't, and only does so/rebuilds via -N.

That's not one of those things we can easily summarize in a UI; not 
unless we want to extraordinarily verbose.


> - lack of backwards compatibility.

This is a bit of a stretch; to be clear, you're proposing that the 
depgraph changes in subtle/nasty ways on the fly under your scheme, 
and that a PM supporting IUSE_RUNTIME vs one that doesn't, won't find 
new and subtle ways to blow up the packages involved (specifically to 
expose differing behaviour).


As ciaran said, I'd like to see an actual implementation if you're 
dead set on this route; ODEPEND/SDEPEND, not so much; it's 
straightforward.  Your scheme basically requires the manager to 
continually recalculate large swaths of resolution subgraphs (every 
time), doesn't account for the fact that most PMs actually write a 
finalized form of the deps to the DB (for efficiency reasons, and 
since the USE state is locked), requires that optimization to be 
*reversed* basically everywhere.

Mind you, I'm generally a bit of a harsh reviewer; I appreciate the 
notion/effort/emphasis, I just think bluntly that this is overly 
complex, and you're dismissing the saner/simpler solution w/out 
sufficiently strong reasons (especially in light of the complexity of 
the alternative, and consequences on PM behaviour/performance/user 
confusion).  I could be wrong- feel free to blast back.

~harring

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 4182 bytes --]

On Mon, 18 Jun 2012 20:04:48 -0700
Brian Harring <ferringb@gmail.com> wrote:

> On Sun, Jun 17, 2012 at 10:31:59PM +0200, Micha?? G??rny wrote:
> > 4. flags listed in ``IUSE_RUNTIME`` may be referred through USE
> >    dependencies by other packages' ``DEPEND``, ``RDEPEND``
> >    and ``PDEPEND`` variables.
> 
> Unless I'm on crack, you're stating that essentially an optional use 
> flag (one you label 'runtime'), is able to be used transitively
> during DEPEND.  That's not particularly "here's some suggested pkgs
> to install"- that's "rebuild the fucker for this changed DEPEND",
> which is the existing situation.

It could be used by another package. Let's say dev-util/foo
is a documentation generator in Python. It has optional runtime
dependencies for HTML output enabled via USE=html.

If dev-libs/bar uses foo for HTML output during the build, it can
DEPEND on dev-util/foo[html].

> > The remaining reused features include:
> > 
> > - dependency syntax,
> 
> If you invent new syntax, I'm going to be unhappy.  KISS as it were.
> 
> > - ability to use ``REQUIRED_USE``, USE dependencies,
> > - ability to describe flags in `metadata.xml`,
> > - global flag names (and descriptions).
> > 
> > Alternative proposed solution involved creating additional
> > ``SDEPEND`` variable. That proposition had the following
> > disadvantages:
> > 
> > - being package-oriented rather than feature-oriented,
> 
> No; use flags are our configuration space, and they turn on/off 
> sections of the given pkgs graph.  Your proposal relies on the same 
> concept; bluntly, what you're proposing is just as 'package oriented'.
> 
> Effectively, you can't dismiss SDEPEND/ODEPEND via changing the rules 
> between your proposal and ODEPEND's proposal.  Nice try though. :)

USE flags can describe features, like USE=ssl, USE=html, USE=whatever.
The exherbo suggested dependencies just list the relevant packages.

In other words, here you enable USE=html to get HTML output. With
SDEPEND, you would --take dev-python/somerandomhtmllibrary.

> > - lack of ability to express multiple packages required by a single
> >   feature,
> 
> Eh?  SDEPEND="my_feature? ( pkg1 pkg 2 )"

SDEPEND didn't involve USE flags. If it did, we're back to square one.

> > - lack of ability to express cross-feature dependencies,
> 
> Clarify.  This is a wee bit too vague for responding to ;)

REQUIRED_USE. You don't have USE flags, you can't do that.

> > - lack of ability to describe features provided by enabled packages,
> 
> metadata.xml's local use description already covers that; in your 
> proposal above you lock in on use flags for the controlling of it 
> (which, presumably you'll abuse to get descriptions of) yet claim
> it's not possible for ODEPEND (better name than SDEPEND :P).

It didn't use USE flags.

> > - necessity of implementing a new user interface parts to control
> >   the dependencies,
> 
> Um... This is bullshit.  Your proposal above is going to require a
> lot more implementation, documentation of how this all is supposed to 
> work, and user ededucation for the weird scenarios where things don't 
> behave as expected, or they don't understand why changing use flag X, 
> results in the PM pulling in new packages (acting akin to -N), while 
> changing flag Y doesn't, and only does so/rebuilds via -N.
> 
> That's not one of those things we can easily summarize in a UI; not 
> unless we want to extraordinarily verbose.

While ODEPEND requires additional --take option, a way to list all
possible packages which could be taken, teaching users why they are
listed yet not pulled, how to pull and unpull them.

> > - lack of backwards compatibility.
> 
> This is a bit of a stretch; to be clear, you're proposing that the 
> depgraph changes in subtle/nasty ways on the fly under your scheme, 
> and that a PM supporting IUSE_RUNTIME vs one that doesn't, won't find 
> new and subtle ways to blow up the packages involved (specifically to 
> expose differing behaviour).

It does that already. See 'man emerge', --dynamic-deps.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1090 bytes --]

On Tue, 19 Jun 2012 10:43:47 +0200
Michał Górny <mgorny@gentoo.org> wrote:
> > > - being package-oriented rather than feature-oriented,
> > 
> > No; use flags are our configuration space, and they turn on/off 
> > sections of the given pkgs graph.  Your proposal relies on the same 
> > concept; bluntly, what you're proposing is just as 'package
> > oriented'.
> > 
> > Effectively, you can't dismiss SDEPEND/ODEPEND via changing the
> > rules between your proposal and ODEPEND's proposal.  Nice try
> > though. :)
> 
> USE flags can describe features, like USE=ssl, USE=html, USE=whatever.
> The exherbo suggested dependencies just list the relevant packages.
> 
> In other words, here you enable USE=html to get HTML output. With
> SDEPEND, you would --take dev-python/somerandomhtmllibrary.

Incorrect. Exherbo allows suggestions to be grouped, described and
taken by feature. It's done via annotations (the same mechanism used to
provide decent handling of blockers etc). Search for "group-name" in
exheres-for-smarties for an example.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[hasufell]

On 06/17/2012 10:31 PM, Michał Górny wrote:
> Hello,
> 
> A simple solution to a program long-unsolved. In GLEP form.
> 
> Both attached and published as a gist:
> 
> https://gist.github.com/2945569
> 
> (please note that github doesn't render GLEP headers correctly)
> 

This looks very nice, imo.

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[hasufell]

On 06/17/2012 10:31 PM, Michał Górny wrote:
> Hello,
> 
> A simple solution to a program long-unsolved. In GLEP form.
> 
> Both attached and published as a gist:
> 
> https://gist.github.com/2945569
> 
> (please note that github doesn't render GLEP headers correctly)
> 

As already stated I like this idea, because I already got some optional
dep bloat in x11-misc/spacefm and media-sound/gmusicbrowser.

However I have a few objections:

1. Optional deps are SUGGESTIONS from the dev which he considered
nice/good/sane at the time of writing the ebuild. Other people might
totally disagree with those suggestions.
As useflags in IUSE_RUNTIME can pick from global useflags as well or
even set "+foo" the user might have a hard time to turn off things he
does not want without turning them off for regular IUSE as well.

Means: "foo" pulls in an optional dependency for package suckbar/gaybar,
but it also pulls in build-time deps for nerdbar/geekbar

The user has to figure out now what the useflag does for each package
and micromanage useflags to maybe avoid undesired optional deps.

FEATURES="optional-deps" would be one way to overcome this, so I can
globally turn useflags in IUSE_RUNTIME off without those in regular IUSE.

But that may cause problems with REQUIRED_USE then maybe, not sure.


2. Afais useflags that are already in IUSE and used for build-time stuff
must not be used for IUSE_RUNTIME too.
This is a random rule IMO. I don't have many cases in mind where this
would be annoying (could think of "debug" enabling some in-source
switches and adding optional debug tools in RDEPEND. Having one flag
here would make it cleaner and tighter for the user to interact with
useflags.).

However... this is not a logical rule, rather a technical issue. If
there is a way to avoid this restriction that would be nice.

(There was one proposal about expanding useflags in IUSE_RUNTIME, but I
have not thought far in that direction.)


3. There are no unconditional optional deps possible.
ssuominen had an example:
"virtualx.eclass could have suggestion/recommendation to enable USE=xvfb
in x11-base/xorg-server"


and some things I forgot...

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Marien Zwart]

On di, 2012-06-19 at 18:53 +0200, hasufell wrote:
> On 06/17/2012 10:31 PM, Michał Górny wrote:
> > Hello,
> > 
> > A simple solution to a program long-unsolved. In GLEP form.
> > 
> > Both attached and published as a gist:
> > 
> > https://gist.github.com/2945569
> > 
> > (please note that github doesn't render GLEP headers correctly)
> > 
> 
> As already stated I like this idea, because I already got some optional
> dep bloat in x11-misc/spacefm and media-sound/gmusicbrowser.
> 
> However I have a few objections:
> 
> 1. Optional deps are SUGGESTIONS from the dev which he considered
> nice/good/sane at the time of writing the ebuild. Other people might
> totally disagree with those suggestions.
> As useflags in IUSE_RUNTIME can pick from global useflags as well or
> even set "+foo" the user might have a hard time to turn off things he
> does not want without turning them off for regular IUSE as well.

I think we're not all agreeing on which problem is being solved here. I
see IUSE_RUNTIME as an improvement for packages that have a runtime
dependency on another package to provide a certain feature, but no way
of turning this feature off at build time. Examples of this kind of
dependency are executables or python imports, where the executable or
library being absent is dealt with at runtime. For such packages putting
the dependency behind a USE flag makes sense, and for other packages to
depend on the package with that USE flag set also makes sense. This
already works today. However, if you originally built the package with
the USE flag off, and now try to install something that depends on the
package with the USE flag on, you force a rebuild. That's not so nice,
as there's no change to the installed package at all: the rebuild is a
waste of time. IUSE_RUNTIME allows the package manager to skip that
rebuild, as it now knows it is unnecessary.

What you are describing is more like "suggested" dependencies. Those may
also be useful, but do not solve quite the same problem. I do not think
they quite replace the USE flag-based approach described above, as it
makes sense for other ebuilds to depend on "foo with support for feature
blah" through the "blah" USE flag without all those other packages
having to know which specific dependencies this adds to foo, or whether
foo needs a rebuild to enable the feature or not.

Also, even in packages in which IUSE_RUNTIME is used for something like
"suggested" dependencies it is not terribly hard for the user to turn
the USE flag off (package.use) and install the interesting bits by hand.

> 2. Afais useflags that are already in IUSE and used for build-time stuff
> must not be used for IUSE_RUNTIME too.
> This is a random rule IMO. I don't have many cases in mind where this
> would be annoying (could think of "debug" enabling some in-source
> switches and adding optional debug tools in RDEPEND. Having one flag
> here would make it cleaner and tighter for the user to interact with
> useflags.).
> However... this is not a logical rule, rather a technical issue. If
> there is a way to avoid this restriction that would be nice.

I do not see where you are going with this. If it makes sense to turn on
the build-time support for a feature without installing all the
dependencies then the extra dependencies should go behind a separate USE
flag (and that separate USE flag may depend on the USE flag controlling
the build-time support using REQUIRED_USE). Or perhaps the additional
dependencies should be in some new kind of "suggested" depend.

> 3. There are no unconditional optional deps possible.
> ssuominen had an example:
> "virtualx.eclass could have suggestion/recommendation to enable USE=xvfb
> in x11-base/xorg-server"

I do not see where you are going with this. Can you refer me to where
this suggestion was made? virtualx.eclass adds a hard dependency if the
test USE flag (or some other USE flag) is set, and no dependency if this
USE flag is not set. When would virtualx.eclass add an optional
dependency?

-- 
Marien Zwart

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[hasufell]

On 06/20/2012 05:05 PM, Marien Zwart wrote:
> On di, 2012-06-19 at 18:53 +0200, hasufell wrote:
>>
>> 1. Optional deps are SUGGESTIONS from the dev which he considered
>> nice/good/sane at the time of writing the ebuild. Other people might
>> totally disagree with those suggestions.
>> As useflags in IUSE_RUNTIME can pick from global useflags as well or
>> even set "+foo" the user might have a hard time to turn off things he
>> does not want without turning them off for regular IUSE as well.
> 
> I think we're not all agreeing on which problem is being solved here. I
> see IUSE_RUNTIME as an improvement for packages that have a runtime
> dependency on another package to provide a certain feature, but no way
> of turning this feature off at build time. Examples of this kind of
> dependency are executables or python imports, where the executable or
> library being absent is dealt with at runtime. For such packages putting
> the dependency behind a USE flag makes sense, and for other packages to
> depend on the package with that USE flag set also makes sense.

Makes sense to you or the developer who wrote the ebuild.
I know the usecases of IUSE_RUNTIME, but you have to realize that people
might _not_ want additional optional runtime dependencies turned on by
useflags that are already in _make.conf_!
IUSE_RUNTIME is technically not a seperate thing, cause they go all into
IUSE and maintaining useflags might become way more complicated for some
users/usecases than it used to be, because of this new feature and a lot
more dependencies that are triggered by your USE="..." variable.

Something like FEATURES="optional-deps" would simply enable people to
have a minimum of dependencies and still be able to use global useflags
_without_ micro-managing all of them per-package, cause some of them are
in IUSE_RUNTIME and others not.


>> 2. Afais useflags that are already in IUSE and used for build-time stuff
>> must not be used for IUSE_RUNTIME too.
>> This is a random rule IMO. I don't have many cases in mind where this
>> would be annoying (could think of "debug" enabling some in-source
>> switches and adding optional debug tools in RDEPEND. Having one flag
>> here would make it cleaner and tighter for the user to interact with
>> useflags.).
>> However... this is not a logical rule, rather a technical issue. If
>> there is a way to avoid this restriction that would be nice.
> 
> I do not see where you are going with this. If it makes sense to turn on
> the build-time support for a feature without installing all the
> dependencies then the extra dependencies should go behind a separate USE
> flag (and that separate USE flag may depend on the USE flag controlling
> the build-time support using REQUIRED_USE). Or perhaps the additional
> dependencies should be in some new kind of "suggested" depend.

I think it is bad to overuse REQUIRED_USE in that way. REQUIRED_USE
blocks the emerge process and should only be used when there is a
technical (not logical) useflag correlation.

Using a seperate USE flag just because the name is blocked means the
user has to look up another useflag and think about what it is for.

But as I said... that is rather minor. I just don't like it either,
cause I feel it might annoy me in the future.

What do you think about useflag expansion and seperating them in
make.conf like yngwin suggested:

USE_RUNTIME="debug" -> will enable "runtime_debug" useflag for all ebuilds
USE="debug" -> will enable "debug" useflag for all ebuilds

This would also solve point #1 somehow, cause you don't have to fear
that your dependency graph will grow just because you didn't examine all
newly introduced IUSE_RUNTIME flags.

For people who want that stuff unconditionally they could do:
USE_RUNTIME="$USE"

and never bother again with it.


> 
>> 3. There are no unconditional optional deps possible.
>> ssuominen had an example:
>> "virtualx.eclass could have suggestion/recommendation to enable USE=xvfb
>> in x11-base/xorg-server"
> 
> I do not see where you are going with this. Can you refer me to where
> this suggestion was made? virtualx.eclass adds a hard dependency if the
> test USE flag (or some other USE flag) is set, and no dependency if this
> USE flag is not set. When would virtualx.eclass add an optional
> dependency?
> 

I hope ssuominen might explain more about this idea as I already requested.

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2256 bytes --]

On Wed, 20 Jun 2012 18:57:19 +0200
hasufell <hasufell@gentoo.org> wrote:

> >> 2. Afais useflags that are already in IUSE and used for build-time
> >> stuff must not be used for IUSE_RUNTIME too.
> >> This is a random rule IMO. I don't have many cases in mind where
> >> this would be annoying (could think of "debug" enabling some
> >> in-source switches and adding optional debug tools in RDEPEND.
> >> Having one flag here would make it cleaner and tighter for the
> >> user to interact with useflags.).
> >> However... this is not a logical rule, rather a technical issue. If
> >> there is a way to avoid this restriction that would be nice.
> > 
> > I do not see where you are going with this. If it makes sense to
> > turn on the build-time support for a feature without installing all
> > the dependencies then the extra dependencies should go behind a
> > separate USE flag (and that separate USE flag may depend on the USE
> > flag controlling the build-time support using REQUIRED_USE). Or
> > perhaps the additional dependencies should be in some new kind of
> > "suggested" depend.
> 
> I think it is bad to overuse REQUIRED_USE in that way. REQUIRED_USE
> blocks the emerge process and should only be used when there is a
> technical (not logical) useflag correlation.
> 
> Using a seperate USE flag just because the name is blocked means the
> user has to look up another useflag and think about what it is for.
> 
> But as I said... that is rather minor. I just don't like it either,
> cause I feel it might annoy me in the future.
> 
> What do you think about useflag expansion and seperating them in
> make.conf like yngwin suggested:
> 
> USE_RUNTIME="debug" -> will enable "runtime_debug" useflag for all
> ebuilds USE="debug" -> will enable "debug" useflag for all ebuilds
> 
> This would also solve point #1 somehow, cause you don't have to fear
> that your dependency graph will grow just because you didn't examine
> all newly introduced IUSE_RUNTIME flags.
> 
> For people who want that stuff unconditionally they could do:
> USE_RUNTIME="$USE"
> 
> and never bother again with it.

Please read the rationale. Again. The whole thing. Three times.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[hasufell]

On 06/20/2012 07:07 PM, Michał Górny wrote:
> Please read the rationale. Again. The whole thing. Three times.
> 

Please read my suggestions. Again. The whole thing. Three times.

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 409 bytes --]

On Wed, 20 Jun 2012 19:11:33 +0200
hasufell <hasufell@gentoo.org> wrote:
> On 06/20/2012 07:07 PM, Michał Górny wrote:
> > Please read the rationale. Again. The whole thing. Three times.
> 
> Please read my suggestions. Again. The whole thing. Three times.

Can we all agree to just stop this and just restrict the arguing to
being between SDEPEND and DEPENDENCIES? Cheers.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Ralph Sennhauser]

[-- Attachment #1: Type: text/plain, Size: 525 bytes --]

On Wed, 20 Jun 2012 18:24:33 +0100
Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:

> Can we all agree to just stop this and just restrict the arguing to
> being between SDEPEND and DEPENDENCIES? Cheers.

I clearly favour going with SDEPEND now as this fits better what people
are used to and the move to DEPENDENCIES is also a chance to clean up
dep-specs after we added all quirks we need(*). Let's name GLEP 54 here
which we hopefully can add to EAPI 6.

(*) or for when we run out of special chars ;)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 594 bytes --]

On Wed, 20 Jun 2012 18:24:33 +0100
Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:

> On Wed, 20 Jun 2012 19:11:33 +0200
> hasufell <hasufell@gentoo.org> wrote:
> > On 06/20/2012 07:07 PM, Michał Górny wrote:
> > > Please read the rationale. Again. The whole thing. Three times.
> > 
> > Please read my suggestions. Again. The whole thing. Three times.
> 
> Can we all agree to just stop this and just restrict the arguing to
> being between SDEPEND and DEPENDENCIES? Cheers.

You just volunteered to write portage patches. Cheers.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1025 bytes --]

On Thu, 21 Jun 2012 09:29:49 +0200
Michał Górny <mgorny@gentoo.org> wrote:
> On Wed, 20 Jun 2012 18:24:33 +0100
> Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
> > On Wed, 20 Jun 2012 19:11:33 +0200
> > hasufell <hasufell@gentoo.org> wrote:
> > > On 06/20/2012 07:07 PM, Michał Górny wrote:
> > > > Please read the rationale. Again. The whole thing. Three times.
> > > 
> > > Please read my suggestions. Again. The whole thing. Three times.
> > 
> > Can we all agree to just stop this and just restrict the arguing to
> > being between SDEPEND and DEPENDENCIES? Cheers.
> 
> You just volunteered to write portage patches. Cheers.

Both were already implemented in Paludis, if you're looking for a
reference implementation to try it out. There are also examples of
use of SDEPEND in the old kdebuilds, and of DEPENDENCIES in Exherbo. I
can give you a small patch to turn SDEPEND on for an EAPI if you like
(it's just a one line addition to the EAPI definition file).

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 943 bytes --]

On Thu, 21 Jun 2012 09:42:36 +0200
Michał Górny <mgorny@gentoo.org> wrote:
> > > You just volunteered to write portage patches. Cheers.
> > 
> > Both were already implemented in Paludis, if you're looking for a
> > reference implementation to try it out. There are also examples of
> > use of SDEPEND in the old kdebuilds, and of DEPENDENCIES in
> > Exherbo. I can give you a small patch to turn SDEPEND on for an
> > EAPI if you like (it's just a one line addition to the EAPI
> > definition file).
> 
> Wait, did I just write to exherbo ml? No, don't think so. 'Implemented
> in Paludis' doesn't work here. We're discussing Gentoo features,
> and official package manager in Gentoo is portage. If you don't
> believe me, check out the docs.

And since when was "Implemented in Portage" a requirement for an EAPI
feature?

The "implementation" requirement is to avoid REQUIRED_USE-like screwups.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1425 bytes --]

On Thu, 21 Jun 2012 08:30:24 +0100
Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:

> On Thu, 21 Jun 2012 09:29:49 +0200
> Michał Górny <mgorny@gentoo.org> wrote:
> > On Wed, 20 Jun 2012 18:24:33 +0100
> > Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
> > > On Wed, 20 Jun 2012 19:11:33 +0200
> > > hasufell <hasufell@gentoo.org> wrote:
> > > > On 06/20/2012 07:07 PM, Michał Górny wrote:
> > > > > Please read the rationale. Again. The whole thing. Three
> > > > > times.
> > > > 
> > > > Please read my suggestions. Again. The whole thing. Three times.
> > > 
> > > Can we all agree to just stop this and just restrict the arguing
> > > to being between SDEPEND and DEPENDENCIES? Cheers.
> > 
> > You just volunteered to write portage patches. Cheers.
> 
> Both were already implemented in Paludis, if you're looking for a
> reference implementation to try it out. There are also examples of
> use of SDEPEND in the old kdebuilds, and of DEPENDENCIES in Exherbo. I
> can give you a small patch to turn SDEPEND on for an EAPI if you like
> (it's just a one line addition to the EAPI definition file).

Wait, did I just write to exherbo ml? No, don't think so. 'Implemented
in Paludis' doesn't work here. We're discussing Gentoo features,
and official package manager in Gentoo is portage. If you don't believe
me, check out the docs.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1095 bytes --]

On Thu, 21 Jun 2012 08:41:23 +0100
Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:

> On Thu, 21 Jun 2012 09:42:36 +0200
> Michał Górny <mgorny@gentoo.org> wrote:
> > > > You just volunteered to write portage patches. Cheers.
> > > 
> > > Both were already implemented in Paludis, if you're looking for a
> > > reference implementation to try it out. There are also examples of
> > > use of SDEPEND in the old kdebuilds, and of DEPENDENCIES in
> > > Exherbo. I can give you a small patch to turn SDEPEND on for an
> > > EAPI if you like (it's just a one line addition to the EAPI
> > > definition file).
> > 
> > Wait, did I just write to exherbo ml? No, don't think so.
> > 'Implemented in Paludis' doesn't work here. We're discussing Gentoo
> > features, and official package manager in Gentoo is portage. If you
> > don't believe me, check out the docs.
> 
> And since when was "Implemented in Portage" a requirement for an EAPI
> feature?

Remember EAPI4 and features which had reference implementation not
in portage?

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 562 bytes --]

On Thu, 21 Jun 2012 10:54:19 +0200
Michał Górny <mgorny@gentoo.org> wrote:
> > And since when was "Implemented in Portage" a requirement for an
> > EAPI feature?
> 
> Remember EAPI4 and features which had reference implementation not
> in portage?

Actually, yes, since that was "most of them". Nearly all of them got
implemented quickly. Our policy on this has always been "ask Zac
whether he thinks they're reasonably quick to implement".

But you know this, so kindly keep your disruption to places where
you're right.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[David Leverton]

Michał Górny wrote:
> Hello,
>
> A simple solution to a program long-unsolved. In GLEP form.

Just a couple of minor points/nitpicks:

1) If an installed package has both IUSE_RUNTIME and REQUIRED_USE, 
should REQUIRED_USE be re-verified:

a) for every dep resolution
b) when the package is involved in the resolution for some other reason 
(not necessarily being reinstalled, just when the resolver has some 
reason to look at it)
c) something else?

I think b) should be sufficient (and probably easier to implement), but 
is there any reason why it wouldn't be?

2) It's not forbidden for package A to depend on an IUSE_RUNTIME flag of 
package B being disabled, but it's unlikely to be useful.  To make this 
more concrete, a fictional but vaguely plausible example:

app-text/docmangler:

# links to poppler to handle PDFs, and can use Ghostscript for
# PostScript support if available
IUSE="postscript"
IUSE_RUNTIME="postscript"
DEPEND="app-text/poppler"
RDEPEND="${DEPEND}
     postscript? ( app-text/ghostscript )"

app-misc/coolapp:

IUSE="doc"
# if Ghostscript is installed, docmangler uses it for both
# PostScript and PDF files, but Ghostscript misrenders our PDFs
DEPEND="doc? ( app-text/docmangler[-postscript] )"

Here, the [-postscript] dep would force the user to disable that flag, 
but it wouldn't do much good because Ghostscript would still be 
installed.  This doesn't happen with regular USE flags because (if the 
ebuild is written correctly) disabling the flag removes the feature even 
if its dependencies happen to be installed.

Possible solutions:

a) automatically rewrite the dep as
     postscript? ( app-text/ghostscript )
     !postscript? ( !app-text/ghostscript )
b) forbid [-foo]-style deps for IUSE_RUNTIME flags (would also make 
sense in that case to disallow them in !foo-style conditionals in the 
dependencies of the package itself, as that could cause similar paradoxes)
c) don't address it in the spec itself, and require people to manually 
write the dep in the blocker form if it's required
d) something else?

a) is pretty icky IMHO, especially if the flag pulls in multiple 
packages.  I could live with either b) or c), but b) is less flexible 
and c) leaves a potential trap for the unwary.  Any opinions?

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 21/06/12 03:05 PM, David Leverton wrote:
> Michał Górny wrote:
>> Hello,
>> 
>> A simple solution to a program long-unsolved. In GLEP form.
> 
> Just a couple of minor points/nitpicks:
> 
> [ Snip! ]
> 
> 2) It's not forbidden for package A to depend on an IUSE_RUNTIME
> flag of package B being disabled, but it's unlikely to be useful.
> To make this more concrete, a fictional but vaguely plausible
> example:
> 
> app-text/docmangler:
> 
> # links to poppler to handle PDFs, and can use Ghostscript for #
> PostScript support if available IUSE="postscript" 
> IUSE_RUNTIME="postscript" DEPEND="app-text/poppler" 
> RDEPEND="${DEPEND} postscript? ( app-text/ghostscript )"
> 
> app-misc/coolapp:
> 
> IUSE="doc" # if Ghostscript is installed, docmangler uses it for
> both # PostScript and PDF files, but Ghostscript misrenders our
> PDFs DEPEND="doc? ( app-text/docmangler[-postscript] )"
> 
> Here, the [-postscript] dep would force the user to disable that
> flag, but it wouldn't do much good because Ghostscript would still
> be installed.  This doesn't happen with regular USE flags because
> (if the ebuild is written correctly) disabling the flag removes the
> feature even if its dependencies happen to be installed.
> 
> Possible solutions:
> 
> a) automatically rewrite the dep as postscript? (
> app-text/ghostscript ) !postscript? ( !app-text/ghostscript ) b)
> forbid [-foo]-style deps for IUSE_RUNTIME flags (would also make 
> sense in that case to disallow them in !foo-style conditionals in
> the dependencies of the package itself, as that could cause similar
> paradoxes) c) don't address it in the spec itself, and require
> people to manually write the dep in the blocker form if it's
> required d) something else?
> 
> a) is pretty icky IMHO, especially if the flag pulls in multiple 
> packages.  I could live with either b) or c), but b) is less
> flexible and c) leaves a potential trap for the unwary.  Any
> opinions?
> 

I'd say (c) , since IUSE_RUNTIME is not an enforceable state of the
tree and if docmangler is used but fails when ghostscript is installed
anyways, then the DEPEND should specify this regardless of whether the
'postscript' flag (used by IUSE_RUNTIME) is set or whether ghostscript
is in the user's world file.





-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (GNU/Linux)

iF4EAREIAAYFAk/jc+cACgkQ2ugaI38ACPCUOAD+ICKl69MUhUTRj+2HBQ0pNlvo
Bqa5/TmaD0oKIkLi+xgBAIGwynEBXC3dXsG7Amky0OiEUyen41kURybNLR8FIkT2
=jMZ4
-----END PGP SIGNATURE-----

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2889 bytes --]

On Thu, 21 Jun 2012 20:05:46 +0100
David Leverton <levertond@googlemail.com> wrote:

> Michał Górny wrote:
> > Hello,
> >
> > A simple solution to a program long-unsolved. In GLEP form.
> 
> Just a couple of minor points/nitpicks:
> 
> 1) If an installed package has both IUSE_RUNTIME and REQUIRED_USE, 
> should REQUIRED_USE be re-verified:
> 
> a) for every dep resolution
> b) when the package is involved in the resolution for some other
> reason (not necessarily being reinstalled, just when the resolver has
> some reason to look at it)
> c) something else?
> 
> I think b) should be sufficient (and probably easier to implement),
> but is there any reason why it wouldn't be?

Well, I don't understand the difference between a) and b) in your case
but the idea is that REQUIRED_USE should be re-verified if either
enabled USE flag list or REQUIRED_USE changes.

> 2) It's not forbidden for package A to depend on an IUSE_RUNTIME flag
> of package B being disabled, but it's unlikely to be useful.  To make
> this more concrete, a fictional but vaguely plausible example:
> 
> app-text/docmangler:
> 
> # links to poppler to handle PDFs, and can use Ghostscript for
> # PostScript support if available
> IUSE="postscript"
> IUSE_RUNTIME="postscript"
> DEPEND="app-text/poppler"
> RDEPEND="${DEPEND}
>      postscript? ( app-text/ghostscript )"
> 
> app-misc/coolapp:
> 
> IUSE="doc"
> # if Ghostscript is installed, docmangler uses it for both
> # PostScript and PDF files, but Ghostscript misrenders our PDFs
> DEPEND="doc? ( app-text/docmangler[-postscript] )"
> 
> Here, the [-postscript] dep would force the user to disable that
> flag, but it wouldn't do much good because Ghostscript would still be 
> installed.  This doesn't happen with regular USE flags because (if
> the ebuild is written correctly) disabling the flag removes the
> feature even if its dependencies happen to be installed.
> 
> Possible solutions:
> 
> a) automatically rewrite the dep as
>      postscript? ( app-text/ghostscript )
>      !postscript? ( !app-text/ghostscript )
> b) forbid [-foo]-style deps for IUSE_RUNTIME flags (would also make 
> sense in that case to disallow them in !foo-style conditionals in the 
> dependencies of the package itself, as that could cause similar
> paradoxes) c) don't address it in the spec itself, and require people
> to manually write the dep in the blocker form if it's required
> d) something else?
> 
> a) is pretty icky IMHO, especially if the flag pulls in multiple 
> packages.  I could live with either b) or c), but b) is less flexible 
> and c) leaves a potential trap for the unwary.  Any opinions?

Good observation. I think b) is the best solution since forced removal
of random dependencies is a very bad idea (and misuse of blockers).

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[David Leverton]

Michał Górny wrote:
> On Thu, 21 Jun 2012 20:05:46 +0100
> David Leverton <levertond@googlemail.com> wrote:
>> 1) If an installed package has both IUSE_RUNTIME and REQUIRED_USE,
>> should REQUIRED_USE be re-verified:
>>
>> a) for every dep resolution
>> b) when the package is involved in the resolution for some other
>> reason (not necessarily being reinstalled, just when the resolver has
>> some reason to look at it)
>> c) something else?
>
> Well, I don't understand the difference between a) and b) in your case

Suppose kde-base/kde-meta has both IUSE_RUNTIME and REQUIRED_USE, and 
that I have it installed and then modify one of the flags it uses in my 
configuration, but don't run any PM commands.  Then suppose I install a 
Gnome package.  Normally installing a Gnome package wouldn't require the 
PM to go anywhere near kde-meta, but being strict about revalidating 
REQUIRED_USE would require it to look through every installed package, 
just in case any have IUSE_RUNTIME and REQUIRED_USE set, for every 
installation command.  Is that necessary?

 > but the idea is that REQUIRED_USE should be re-verified if either
 > enabled USE flag list or REQUIRED_USE changes.

Would that mean the enabled USE flag list should be updated in the VDB 
every time REQUIRED_USE is checked, even when the package isn't being 
reinstalled?  I think it would probably be easier to just recheck 
REQUIRED_USE, without trying to figure out whether any of the 
IUSE_RUNTIME flags were changed, it's just a question of how eager we 
want to be.

>> 2) It's not forbidden for package A to depend on an IUSE_RUNTIME flag
>> of package B being disabled, but it's unlikely to be useful.  To make
>> this more concrete, a fictional but vaguely plausible example:

>> Possible solutions:
>>
>> a) automatically rewrite the dep as
>>       postscript? ( app-text/ghostscript )
>>       !postscript? ( !app-text/ghostscript )
>> b) forbid [-foo]-style deps for IUSE_RUNTIME flags (would also make
>> sense in that case to disallow them in !foo-style conditionals in the
>> dependencies of the package itself, as that could cause similar
>> paradoxes)
 >> c) don't address it in the spec itself, and require people
>> to manually write the dep in the blocker form if it's required
>> d) something else?

> Good observation. I think b) is the best solution since forced removal
> of random dependencies is a very bad idea (and misuse of blockers).

One case that I forgot to mention before: if some package does something 
like

     if has_version app-text/docmangler[postscript]; then
         # ...
     else
         # ...
     fi

Here, again, the "else" branch can lead to incoherent results as it 
can't assume that the postscript flag being disabled means Ghostscript 
isn't installed, and this one can't be forbidden by restricting the 
allowed dep forms.  I think we'd have to just say "don't do that then"....

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 4095 bytes --]

On Thu, 21 Jun 2012 21:26:26 +0100
David Leverton <levertond@googlemail.com> wrote:

> Michał Górny wrote:
> > On Thu, 21 Jun 2012 20:05:46 +0100
> > David Leverton <levertond@googlemail.com> wrote:
> >> 1) If an installed package has both IUSE_RUNTIME and REQUIRED_USE,
> >> should REQUIRED_USE be re-verified:
> >>
> >> a) for every dep resolution
> >> b) when the package is involved in the resolution for some other
> >> reason (not necessarily being reinstalled, just when the resolver
> >> has some reason to look at it)
> >> c) something else?
> >
> > Well, I don't understand the difference between a) and b) in your
> > case
> 
> Suppose kde-base/kde-meta has both IUSE_RUNTIME and REQUIRED_USE, and 
> that I have it installed and then modify one of the flags it uses in
> my configuration, but don't run any PM commands.  Then suppose I
> install a Gnome package.  Normally installing a Gnome package
> wouldn't require the PM to go anywhere near kde-meta, but being
> strict about revalidating REQUIRED_USE would require it to look
> through every installed package, just in case any have IUSE_RUNTIME
> and REQUIRED_USE set, for every installation command.  Is that
> necessary?

No, of course not. Otherwise, every package manager run would
practically require it to re-validate all packages in the tree
(possibly not only installed ones).

Package manager must ensure the flags are valid when package is
in the graph. I would think of IUSE_RUNTIME as a last-step action where
packages were in the graph for rebuild already but the rebuild is
disabled as being unnecessary.

>  > but the idea is that REQUIRED_USE should be re-verified if either
>  > enabled USE flag list or REQUIRED_USE changes.
> 
> Would that mean the enabled USE flag list should be updated in the
> VDB every time REQUIRED_USE is checked, even when the package isn't
> being reinstalled?  I think it would probably be easier to just
> recheck REQUIRED_USE, without trying to figure out whether any of the 
> IUSE_RUNTIME flags were changed, it's just a question of how eager we 
> want to be.

No, the USE flag list in vdb may be updated every time the package gets
into the graph with changed runtime flags. I don't consider that
necessary, however. Just a nice backwards compatibility feature for
other applications looking at vdb.

> >> 2) It's not forbidden for package A to depend on an IUSE_RUNTIME
> >> flag of package B being disabled, but it's unlikely to be useful.
> >> To make this more concrete, a fictional but vaguely plausible
> >> example:
> 
> >> Possible solutions:
> >>
> >> a) automatically rewrite the dep as
> >>       postscript? ( app-text/ghostscript )
> >>       !postscript? ( !app-text/ghostscript )
> >> b) forbid [-foo]-style deps for IUSE_RUNTIME flags (would also make
> >> sense in that case to disallow them in !foo-style conditionals in
> >> the dependencies of the package itself, as that could cause similar
> >> paradoxes)
>  >> c) don't address it in the spec itself, and require people
> >> to manually write the dep in the blocker form if it's required
> >> d) something else?
> 
> > Good observation. I think b) is the best solution since forced
> > removal of random dependencies is a very bad idea (and misuse of
> > blockers).
> 
> One case that I forgot to mention before: if some package does
> something like
> 
>      if has_version app-text/docmangler[postscript]; then
>          # ...
>      else
>          # ...
>      fi
> 
> Here, again, the "else" branch can lead to incoherent results as it 
> can't assume that the postscript flag being disabled means
> Ghostscript isn't installed, and this one can't be forbidden by
> restricting the allowed dep forms.  I think we'd have to just say
> "don't do that then"....

Well, as I see it restricting is more of a policy than a technical
requirement. But in the current form, the spec doesn't allow passing
IUSE_RUNTIME flags to has_version() so we're on the safe side :P.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[David Leverton]

Michał Górny wrote:
> No, of course not. Otherwise, every package manager run would
> practically require it to re-validate all packages in the tree
> (possibly not only installed ones).
>
> Package manager must ensure the flags are valid when package is
> in the graph. I would think of IUSE_RUNTIME as a last-step action where
> packages were in the graph for rebuild already but the rebuild is
> disabled as being unnecessary.

That's what I thought, was just making sure we're on the same page.

> No, the USE flag list in vdb may be updated every time the package gets
> into the graph with changed runtime flags. I don't consider that
> necessary, however. Just a nice backwards compatibility feature for
> other applications looking at vdb.

'K

> Well, as I see it restricting is more of a policy than a technical
> requirement.

As long as we're clear on which it is, and what restrictions if any the 
PM can/should impose...

> But in the current form, the spec doesn't allow passing
> IUSE_RUNTIME flags to has_version() so we're on the safe side :P.

True.  Do we want to keep it that restrictive?

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Zac Medico]

On 06/21/2012 02:32 PM, David Leverton wrote:
> Michał Górny wrote:
>> But in the current form, the spec doesn't allow passing
>> IUSE_RUNTIME flags to has_version() so we're on the safe side :P.
> 
> True.  Do we want to keep it that restrictive?

Shouldn't has_version allow any atom that would be allowed in *DEPEND?
-- 
Thanks,
Zac

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Doug Goldstein]

On Thu, Jun 21, 2012 at 2:42 AM, Michał Górny <mgorny@gentoo.org> wrote:
> On Thu, 21 Jun 2012 08:30:24 +0100
> Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
>
>> On Thu, 21 Jun 2012 09:29:49 +0200
>> Michał Górny <mgorny@gentoo.org> wrote:
>> > On Wed, 20 Jun 2012 18:24:33 +0100
>> > Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
>> > > On Wed, 20 Jun 2012 19:11:33 +0200
>> > > hasufell <hasufell@gentoo.org> wrote:
>> > > > On 06/20/2012 07:07 PM, Michał Górny wrote:
>> > > > > Please read the rationale. Again. The whole thing. Three
>> > > > > times.
>> > > >
>> > > > Please read my suggestions. Again. The whole thing. Three times.
>> > >
>> > > Can we all agree to just stop this and just restrict the arguing
>> > > to being between SDEPEND and DEPENDENCIES? Cheers.
>> >
>> > You just volunteered to write portage patches. Cheers.
>>
>> Both were already implemented in Paludis, if you're looking for a
>> reference implementation to try it out. There are also examples of
>> use of SDEPEND in the old kdebuilds, and of DEPENDENCIES in Exherbo. I
>> can give you a small patch to turn SDEPEND on for an EAPI if you like
>> (it's just a one line addition to the EAPI definition file).
>
> Wait, did I just write to exherbo ml? No, don't think so. 'Implemented
> in Paludis' doesn't work here. We're discussing Gentoo features,
> and official package manager in Gentoo is portage. If you don't believe
> me, check out the docs.
>
> --
> Best regards,
> Michał Górny

I would recommend the two of you step away from this thread and
discussion for a day or two and come back to it with a fresh look at
the suggestions and the code that's available and then we can move on
getting this into an EAPI from there.

-- 
Doug Goldstein

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1666 bytes --]

On Thu, 21 Jun 2012 22:32:34 +0100
David Leverton <levertond@googlemail.com> wrote:

> Michał Górny wrote:
> > No, of course not. Otherwise, every package manager run would
> > practically require it to re-validate all packages in the tree
> > (possibly not only installed ones).
> >
> > Package manager must ensure the flags are valid when package is
> > in the graph. I would think of IUSE_RUNTIME as a last-step action
> > where packages were in the graph for rebuild already but the
> > rebuild is disabled as being unnecessary.
> 
> That's what I thought, was just making sure we're on the same page.
> 
> > No, the USE flag list in vdb may be updated every time the package
> > gets into the graph with changed runtime flags. I don't consider
> > that necessary, however. Just a nice backwards compatibility
> > feature for other applications looking at vdb.
> 
> 'K
> 
> > Well, as I see it restricting is more of a policy than a technical
> > requirement.
> 
> As long as we're clear on which it is, and what restrictions if any
> the PM can/should impose...
> 
> > But in the current form, the spec doesn't allow passing
> > IUSE_RUNTIME flags to has_version() so we're on the safe side :P.
> 
> True.  Do we want to keep it that restrictive?

I've added that to the spec but I'm not sure if we're not going too far
with this.

Now I'm seriously seeing downside of this solution and starting
thinking about making them auto-enabled when deps are satisfied. It
doesn't prove any practical use except for the above case so it may be
a better idea to just forbid it completely...

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Zac Medico]

On 06/21/2012 11:12 PM, Michał Górny wrote:
> On Thu, 21 Jun 2012 22:32:34 +0100
> David Leverton <levertond@googlemail.com> wrote:
> 
>> Michał Górny wrote:
>>> But in the current form, the spec doesn't allow passing
>>> IUSE_RUNTIME flags to has_version() so we're on the safe side :P.
>>
>> True.  Do we want to keep it that restrictive?
> 
> I've added that to the spec but I'm not sure if we're not going too far
> with this.
> 
> Now I'm seriously seeing downside of this solution and starting
> thinking about making them auto-enabled when deps are satisfied.

The funny part is that this could recurse, if you call has_version
a[a-runtime-flag], which depends on b[b-runtime-flag], which depends on
c[c-runtime-flag] and so on...

I suspect that we'd be better off with a less restrictive spec, and
without this "auto-enabled" thing.
-- 
Thanks,
Zac

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Alec Warner]

On Fri, Jun 22, 2012 at 8:45 AM, Zac Medico <zmedico@gentoo.org> wrote:
> On 06/21/2012 11:12 PM, Michał Górny wrote:
>> On Thu, 21 Jun 2012 22:32:34 +0100
>> David Leverton <levertond@googlemail.com> wrote:
>>
>>> Michał Górny wrote:
>>>> But in the current form, the spec doesn't allow passing
>>>> IUSE_RUNTIME flags to has_version() so we're on the safe side :P.
>>>
>>> True.  Do we want to keep it that restrictive?
>>
>> I've added that to the spec but I'm not sure if we're not going too far
>> with this.
>>
>> Now I'm seriously seeing downside of this solution and starting
>> thinking about making them auto-enabled when deps are satisfied.
>
> The funny part is that this could recurse, if you call has_version
> a[a-runtime-flag], which depends on b[b-runtime-flag], which depends on
> c[c-runtime-flag] and so on...
>
> I suspect that we'd be better off with a less restrictive spec, and
> without this "auto-enabled" thing.

I deleted autouse in like 2006, please don't bring it back ;p

-A

> --
> Thanks,
> Zac
>
>

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 22/06/12 12:48 AM, Zac Medico wrote:
> On 06/21/2012 02:32 PM, David Leverton wrote:
>> Michał Górny wrote:
>>> But in the current form, the spec doesn't allow passing 
>>> IUSE_RUNTIME flags to has_version() so we're on the safe side
>>> :P.
>> 
>> True.  Do we want to keep it that restrictive?
> 
> Shouldn't has_version allow any atom that would be allowed in
> *DEPEND?

Technically it could, but the issue here would be what you are going
to do with a has_version check on an IUSE_RUNTIME dep -- the package
should do filesystem-identical installs no matter what status of
IUSE_RUNTIME flags, so whatever one would do with a has_version check
would have to not change any part of the build or installation.

I could see it being of use within pkg_configure(), though; ie, emerge
- --config pkg should then be able to reliably set up default configs
based on the runtime installs.   However, that's the only place I can
picture it being viable.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (GNU/Linux)

iF4EAREIAAYFAk/kdmIACgkQ2ugaI38ACPBtsQEAs1Ak9JQnDFt4XuG/4ZfYGfH2
u92QpchtCGHhYbERx1wA/3AyghQuEv8WZ2iIfXoW9zWnelutj5fdqF4adSjMwf9x
=0XPN
-----END PGP SIGNATURE-----

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Marien Zwart]

On do, 2012-06-21 at 20:05 +0100, David Leverton wrote:
> 1) If an installed package has both IUSE_RUNTIME and REQUIRED_USE, 
> should REQUIRED_USE be re-verified:
> 
> a) for every dep resolution
> b) when the package is involved in the resolution for some other reason 
> (not necessarily being reinstalled, just when the resolver has some 
> reason to look at it)
> c) something else?
> 
> I think b) should be sufficient (and probably easier to implement), but 
> is there any reason why it wouldn't be?

I would still prefer for the resolver to not treat such packages
specially at all, and to just deal with USE flag changes explicitly the
same way they're dealt with for "normal" packages/flags, just skipping
the actual rebuild and reinstall steps (just updating the vdb). This
sidesteps problems like this one completely. If the package manager
folks think they can safely do something smarter here that might be
nice, but the feature still seems useful (in reducing the number of
silly rebuilds) and far easier to understand without such magic.

This also means things like has_version checks work exactly like they do
right now.

> 2) It's not forbidden for package A to depend on an IUSE_RUNTIME flag of 
> package B being disabled, but it's unlikely to be useful.  To make this 
> more concrete, a fictional but vaguely plausible example:
> 
> app-text/docmangler:
> 
> # links to poppler to handle PDFs, and can use Ghostscript for
> # PostScript support if available
> IUSE="postscript"
> IUSE_RUNTIME="postscript"
> DEPEND="app-text/poppler"
> RDEPEND="${DEPEND}
>      postscript? ( app-text/ghostscript )"
> 
> app-misc/coolapp:
> 
> IUSE="doc"
> # if Ghostscript is installed, docmangler uses it for both
> # PostScript and PDF files, but Ghostscript misrenders our PDFs
> DEPEND="doc? ( app-text/docmangler[-postscript] )"
> 
> Here, the [-postscript] dep would force the user to disable that flag, 
> but it wouldn't do much good because Ghostscript would still be 
> installed.  This doesn't happen with regular USE flags because (if the 
> ebuild is written correctly) disabling the flag removes the feature even 
> if its dependencies happen to be installed.
> 
> Possible solutions:
> 
> a) automatically rewrite the dep as
>      postscript? ( app-text/ghostscript )
>      !postscript? ( !app-text/ghostscript )

There may be more than one version of docmangler, with a postscript flag
with different effects (IUSE_RUNTIME or full IUSE, different
dependencies). That means this kind of rewriting would have to be done
based on the dependencies of the docmangler installed at the time this
package gets built, which seems like entirely too much magic, and...

> b) forbid [-foo]-style deps for IUSE_RUNTIME flags (would also make 
> sense in that case to disallow them in !foo-style conditionals in the 
> dependencies of the package itself, as that could cause similar paradoxes)

this seems generally impossible, as the same USE flag may be
IUSE_RUNTIME in only some versions of docmangler.

> c) don't address it in the spec itself, and require people to manually 
> write the dep in the blocker form if it's required

I would be in favor of leaving this up to the writer of the coolapp
ebuild, especially as if docmangler is currently using a "full" USE flag
for postscript this is already currently broken. It seems coolapp should
not be building the pdfs, or should be deleting them after they're
built, rather than forbidding the user from having a docmangler that can
create pdfs (as presumably not *all* generated pdfs are corrupt).

I really think this GLEP should be purely about reducing silly rebuilds
in packages where it makes sense to USE-depend on them with a flag set,
or where an already popular flag can be used to pull in some obvious
packages. I think something like "suggested" dependencies can be used to
solve a slightly different problem, and perhaps we could have those too,
but let's leave those for later.

-- 
Marien Zwart

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[David Leverton]

Ian Stakenvicius wrote:
> Technically it could, but the issue here would be what you are going
> to do with a has_version check on an IUSE_RUNTIME dep -- the package
> should do filesystem-identical installs no matter what status of
> IUSE_RUNTIME flags, so whatever one would do with a has_version check
> would have to not change any part of the build or installation.

In principle it would be used for more or less the same thing as it 
would in a dependency, i.e. check whether the runtime-only dependencies 
for that feature are satisfied - the difference being that the package 
can specify arbitrary if-yes and if-no behaviours, rather than just 
"fail the dependency resolution" or not.  (Modulo the problem being 
discussed in this subthread, that a "no" answer isn't reliable.)

For example, some tool used during the build might have a "slow" mode 
that always works, and a "fast" mode that requires some other program to 
be installed and that has to be requested explicitly.  So the package 
that uses the tool might want to do something like

src_compile() {
     if has_version dev-util/buildtool[fast]; then
         buildtool --fast
     else
         buildtool
     fi
}

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[David Leverton]

Marien Zwart wrote:
>> Possible solutions:
>>
>> a) automatically rewrite the dep as
>>       postscript? ( app-text/ghostscript )
>>       !postscript? ( !app-text/ghostscript )
>
> There may be more than one version of docmangler, with a postscript flag
> with different effects (IUSE_RUNTIME or full IUSE, different
> dependencies). That means this kind of rewriting would have to be done
> based on the dependencies of the docmangler installed at the time this
> package gets built, which seems like entirely too much magic, and...

To clarify, I meant rewrite the dep in docmangler itself, and not 
necessarily on disk in the VDB or wherever, just in memory when parsing 
it.  But as I said, I don't like that idea anyway, I just mentioned it 
for completeness.

>> b) forbid [-foo]-style deps for IUSE_RUNTIME flags (would also make
>> sense in that case to disallow them in !foo-style conditionals in the
>> dependencies of the package itself, as that could cause similar paradoxes)
>
> this seems generally impossible, as the same USE flag may be
> IUSE_RUNTIME in only some versions of docmangler.

True.  We could declare that in situations like that the developer just 
needs to be more careful, but then it's not that much better than c)....

>> c) don't address it in the spec itself, and require people to manually
>> write the dep in the blocker form if it's required
>
> I would be in favor of leaving this up to the writer of the coolapp
> ebuild, especially as if docmangler is currently using a "full" USE flag
> for postscript this is already currently broken.

Well, in theory if it's a normal USE flag then disabling it should 
actively remove the Ghostscript support from docmangler, even if 
Ghostscript happens to be installed, but maybe it doesn't always work 
out that way.

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1637 bytes --]

On Fri, 22 Jun 2012 19:29:15 +0100
David Leverton <levertond@googlemail.com> wrote:

> Ian Stakenvicius wrote:
> > Technically it could, but the issue here would be what you are going
> > to do with a has_version check on an IUSE_RUNTIME dep -- the package
> > should do filesystem-identical installs no matter what status of
> > IUSE_RUNTIME flags, so whatever one would do with a has_version
> > check would have to not change any part of the build or
> > installation.
> 
> In principle it would be used for more or less the same thing as it 
> would in a dependency, i.e. check whether the runtime-only
> dependencies for that feature are satisfied - the difference being
> that the package can specify arbitrary if-yes and if-no behaviours,
> rather than just "fail the dependency resolution" or not.  (Modulo
> the problem being discussed in this subthread, that a "no" answer
> isn't reliable.)
> 
> For example, some tool used during the build might have a "slow" mode 
> that always works, and a "fast" mode that requires some other program
> to be installed and that has to be requested explicitly.  So the
> package that uses the tool might want to do something like
> 
> src_compile() {
>      if has_version dev-util/buildtool[fast]; then
>          buildtool --fast
>      else
>          buildtool
>      fi
> }

And that particular example should be perfectly valid, to be honest.
There is just a little risk that fast mode wouldn't be used when it
could.

Of course, it's quite unlikely that such an option was actually based
on runtime dep...

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[hasufell]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Is anything holding this back to be reviewed by the council in the
near future?

Do we need an implementation beforehand? Afaik zac said that the
implementation would not be very complicated, so why not vote on this
now/soon?

mgorny?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJQYcgXAAoJEFpvPKfnPDWzFEgH/RD7Lvl8Dp16KK+02F7L0y70
rEJ9Syo6lApIg4u6BjhMglHUd3pAZxGJ4VPtQpCdu5BL7cH7wAV3cZY+JF1j+YNV
Qd+gkVfVGJZf/gklza4pSayZluOTx9iqKr6HzPfWSQi6ugEtPG+46hLD4fGu3yre
z1ndQjZDKy28fbG5p4R4nRDgYWUKkmd981nF8qBLguXuHDw0K3a5Hqx2l+cLZnc9
yynNX02cYDb3qtRH7QoQczfMnAv0nDIfUdWebuHAUFqFuHgAkgws9JrzkscGrAfR
/KFQICl4TAwN0zlW69monOO3oMzZRYMsJlfX79W4RXhEmCRobb+VOLWxu9svCRs=
=Ff4b
-----END PGP SIGNATURE-----

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Ciaran McCreesh]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Tue, 25 Sep 2012 17:04:55 +0200
hasufell <hasufell@gentoo.org> wrote:
> Do we need an implementation beforehand? Afaik zac said that the
> implementation would not be very complicated, so why not vote on this
> now/soon?

Well we can't really compare it to SDEPEND (which is implemented in
Paludis, for kdebuild-1 and exheres-0) without at least a half-arsed
implementation.

Also, speaking as someone who *has* implemented this kind of thing, I
have extreme doubts as to the viability of the proposal. So I'd be
extremely wary of voting in favour of it until we've been able to have
a play with an implementation.

- -- 
Ciaran McCreesh
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iEYEARECAAYFAlBhyYMACgkQ96zL6DUtXhGPsQCeN6muE82mJOPm9aox2zd1r8p0
5scAn3JkHsHGqPkpBJNCH4Nb94zW7b5H
=LMaV
-----END PGP SIGNATURE-----

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[hasufell]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 09/25/2012 05:10 PM, Ciaran McCreesh wrote:
> On Tue, 25 Sep 2012 17:04:55 +0200 hasufell <hasufell@gentoo.org>
> wrote:
>> Do we need an implementation beforehand? Afaik zac said that the 
>> implementation would not be very complicated, so why not vote on
>> this now/soon?
> 
> Well we can't really compare it to SDEPEND (which is implemented
> in Paludis, for kdebuild-1 and exheres-0) without at least a
> half-arsed implementation.
> 
> Also, speaking as someone who *has* implemented this kind of thing,
> I have extreme doubts as to the viability of the proposal. So I'd
> be extremely wary of voting in favour of it until we've been able
> to have a play with an implementation.
> 

sorry?

I don't see an answers to any of my questions.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJQYcrzAAoJEFpvPKfnPDWztIwH/jXV53rVzF4dAImZbOb/zkD1
MU8JfWnYLirERryoix6lhtvMKlSxSCthU+M0wmKYdQ/Rwo5bqkRrgeSgfLg5xQo+
wn9k6YhiU6a0pWNTpG6WUZM9n6vwXYiqBwTX2QAedaPMCw/SbxQeSeF4bq0XNrRC
jmwUXKO3LhhQDREuSFXg55LqBbx9NeFvoPxUlqOf7lPl0jXlbu6B0pR5MpxEqysy
IFqglzXMIrguk9u/UxY/Z3lDfDzA1zxI3zgDGUiZ5sVKMtqEg8iDNbdsr9K0Zg0H
Qe6mbAFYj3V0Nb6lA6n2/54KjWzVKl9dGtzGQsjwyfu4IdbZNX3O5qi+4Y2CYck=
=QaMz
-----END PGP SIGNATURE-----

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Ciaran McCreesh]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Tue, 25 Sep 2012 17:17:07 +0200
hasufell <hasufell@gentoo.org> wrote:
> On 09/25/2012 05:10 PM, Ciaran McCreesh wrote:
> > On Tue, 25 Sep 2012 17:04:55 +0200 hasufell <hasufell@gentoo.org>
> > wrote:
> >> Do we need an implementation beforehand? Afaik zac said that the 
> >> implementation would not be very complicated, so why not vote on
> >> this now/soon?
> > 
> > Well we can't really compare it to SDEPEND (which is implemented
> > in Paludis, for kdebuild-1 and exheres-0) without at least a
> > half-arsed implementation.
> > 
> > Also, speaking as someone who *has* implemented this kind of thing,
> > I have extreme doubts as to the viability of the proposal. So I'd
> > be extremely wary of voting in favour of it until we've been able
> > to have a play with an implementation.
> 
> sorry?
> 
> I don't see an answers to any of my questions.

Really? I thought it was pretty clearly. Yes, you need an
implementation beforehand.

- -- 
Ciaran McCreesh
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iEYEARECAAYFAlBhzH4ACgkQ96zL6DUtXhHT2gCgtDFTdPkQ6X6jCkFzOIaqY+O7
uFQAnRnsznezzmSBha09MrCkRy1/3LZc
=ouSd
-----END PGP SIGNATURE-----

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Alexis Ballier]

On Tue, 25 Sep 2012 17:17:07 +0200
hasufell <hasufell@gentoo.org> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On 09/25/2012 05:10 PM, Ciaran McCreesh wrote:
> > On Tue, 25 Sep 2012 17:04:55 +0200 hasufell <hasufell@gentoo.org>
> > wrote:
> >> Do we need an implementation beforehand? Afaik zac said that the 
> >> implementation would not be very complicated, so why not vote on
> >> this now/soon?
> > 
> > Well we can't really compare it to SDEPEND (which is implemented
> > in Paludis, for kdebuild-1 and exheres-0) without at least a
> > half-arsed implementation.
> > 
> > Also, speaking as someone who *has* implemented this kind of thing,
> > I have extreme doubts as to the viability of the proposal. So I'd
> > be extremely wary of voting in favour of it until we've been able
> > to have a play with an implementation.
> > 
> 
> sorry?
> 
> I don't see an answers to any of my questions.

he wants an implementation beforehand :)

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[hasufell]

On 09/25/2012 05:25 PM, Alexis Ballier wrote:
> On Tue, 25 Sep 2012 17:17:07 +0200
> hasufell <hasufell@gentoo.org> wrote:
> 
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> On 09/25/2012 05:10 PM, Ciaran McCreesh wrote:
>>> On Tue, 25 Sep 2012 17:04:55 +0200 hasufell <hasufell@gentoo.org>
>>> wrote:
>>>> Do we need an implementation beforehand? Afaik zac said that the 
>>>> implementation would not be very complicated, so why not vote on
>>>> this now/soon?
>>>
>>> Well we can't really compare it to SDEPEND (which is implemented
>>> in Paludis, for kdebuild-1 and exheres-0) without at least a
>>> half-arsed implementation.
>>>
>>> Also, speaking as someone who *has* implemented this kind of thing,
>>> I have extreme doubts as to the viability of the proposal. So I'd
>>> be extremely wary of voting in favour of it until we've been able
>>> to have a play with an implementation.
>>>
>>
>> sorry?
>>
>> I don't see an answers to any of my questions.
> 
> he wants an implementation beforehand :)
> 

Is he a council member?

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Alexis Ballier]

On Tue, 25 Sep 2012 17:30:07 +0200
hasufell <hasufell@gentoo.org> wrote:

> On 09/25/2012 05:25 PM, Alexis Ballier wrote:
> > On Tue, 25 Sep 2012 17:17:07 +0200
> > hasufell <hasufell@gentoo.org> wrote:
> > 
> >> -----BEGIN PGP SIGNED MESSAGE-----
> >> Hash: SHA1
> >>
> >> On 09/25/2012 05:10 PM, Ciaran McCreesh wrote:
> >>> On Tue, 25 Sep 2012 17:04:55 +0200 hasufell <hasufell@gentoo.org>
> >>> wrote:
> >>>> Do we need an implementation beforehand? Afaik zac said that the 
> >>>> implementation would not be very complicated, so why not vote on
> >>>> this now/soon?
> >>>
> >>> Well we can't really compare it to SDEPEND (which is implemented
> >>> in Paludis, for kdebuild-1 and exheres-0) without at least a
> >>> half-arsed implementation.
> >>>
> >>> Also, speaking as someone who *has* implemented this kind of
> >>> thing, I have extreme doubts as to the viability of the proposal.
> >>> So I'd be extremely wary of voting in favour of it until we've
> >>> been able to have a play with an implementation.
> >>>
> >>
> >> sorry?
> >>
> >> I don't see an answers to any of my questions.
> > 
> > he wants an implementation beforehand :)
> > 
> 
> Is he a council member?
> 

That doesn't prevent him from talking from past experiences and giving
his opinion. Council is free to ignore his request also.

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Alexis Ballier]

On Tue, 25 Sep 2012 16:10:56 +0100
Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:

> Also, speaking as someone who *has* implemented this kind of thing, I
> have extreme doubts as to the viability of the proposal. So I'd be
> extremely wary of voting in favour of it until we've been able to have
> a play with an implementation.


Could you please elaborate on what kind of problems may arise ? The
proposal seems pretty simple and sane to me: PM only has to switch the
useflags that are IUSE_RUNTIME in his installed packages db after
installing the deps and without triggering a rebuild of said package.

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[hasufell]

On 09/25/2012 05:36 PM, Alexis Ballier wrote:
> On Tue, 25 Sep 2012 17:30:07 +0200
> hasufell <hasufell@gentoo.org> wrote:
> 
>> On 09/25/2012 05:25 PM, Alexis Ballier wrote:
>>> On Tue, 25 Sep 2012 17:17:07 +0200
>>> hasufell <hasufell@gentoo.org> wrote:
>>>
>>>> -----BEGIN PGP SIGNED MESSAGE-----
>>>> Hash: SHA1
>>>>
>>>> On 09/25/2012 05:10 PM, Ciaran McCreesh wrote:
>>>>> On Tue, 25 Sep 2012 17:04:55 +0200 hasufell <hasufell@gentoo.org>
>>>>> wrote:
>>>>>> Do we need an implementation beforehand? Afaik zac said that the 
>>>>>> implementation would not be very complicated, so why not vote on
>>>>>> this now/soon?
>>>>>
>>>>> Well we can't really compare it to SDEPEND (which is implemented
>>>>> in Paludis, for kdebuild-1 and exheres-0) without at least a
>>>>> half-arsed implementation.
>>>>>
>>>>> Also, speaking as someone who *has* implemented this kind of
>>>>> thing, I have extreme doubts as to the viability of the proposal.
>>>>> So I'd be extremely wary of voting in favour of it until we've
>>>>> been able to have a play with an implementation.
>>>>>
>>>>
>>>> sorry?
>>>>
>>>> I don't see an answers to any of my questions.
>>>
>>> he wants an implementation beforehand :)
>>>
>>
>> Is he a council member?
>>
> 
> That doesn't prevent him from talking from past experiences and giving
> his opinion. Council is free to ignore his request also.
> 

Yeah, I thank him for that, but the time for user opinions has passed. I
am asking what is preventing the _council_ from reviewing this or why
the _author_ of that GLEP hasn't requested it for the next council meeting.

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Fabian Groffen]

[-- Attachment #1: Type: text/plain, Size: 629 bytes --]

On 25-09-2012 17:43:46 +0200, hasufell wrote:
> > That doesn't prevent him from talking from past experiences and giving
> > his opinion. Council is free to ignore his request also.
> 
> Yeah, I thank him for that, but the time for user opinions has passed. I
> am asking what is preventing the _council_ from reviewing this or why
> the _author_ of that GLEP hasn't requested it for the next council meeting.

I guess nothing is preventing them from reviewing it.  However, it's
just a waste of time if you're just asking those guys to review the
GLEP, isn't it?


-- 
Fabian Groffen
Gentoo on a different level

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1511 bytes --]

On Tue, 25 Sep 2012 12:43:00 -0300
Alexis Ballier <aballier@gentoo.org> wrote:
> Could you please elaborate on what kind of problems may arise ? The
> proposal seems pretty simple and sane to me: PM only has to switch the
> useflags that are IUSE_RUNTIME in his installed packages db after
> installing the deps and without triggering a rebuild of said package.

a) How do we provide a good user interface for it? It took an awful lot
of experimenting to get the exheres-0 suggestions user interface to be
good, and it requires quite a bit more information from the package
side than this proposal is providing. We want to avoid a REQUIRED_USE
here...

b) How is consistency checking to be done? Related, what happens when a
runtime switch introduces a dependency that then requires a non-runtime
rebuild of the original package?

c) How do we deal with flag? ( cat/dep[foo] ) or flag? ( >=cat/dep-2.1 )
cases where cat/dep[-foo] or =cat/dep-2.0 is installed and flag is off?
From experience, quite a few places where you'd want to use suggestions
will break if their suggested package is installed but doesn't meet
version or use requirements.

However, addressing these probably isn't enough, since this is just
the things we had to think about for SDEPEND-style suggestions... There
are likely to be things I've not thought of specific to this method
that won't crop up until someone tries to deliver a decent
implementation. This isn't a trivial feature.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[hasufell]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 09/25/2012 05:57 PM, Fabian Groffen wrote:
> 
> I guess nothing is preventing them from reviewing it.  However,
> it's just a waste of time if you're just asking those guys to
> review the GLEP, isn't it?
> 
> 

That's what the GLEP workflow states, no? It's currently not accepted.

On 09/25/2012 05:23 PM, Ciaran McCreesh wrote:
> 
> Really? I thought it was pretty clearly. Yes, you need an 
> implementation beforehand.
> 

Maybe you should read:
http://www.gentoo.org/proj/en/glep/glep-0001.html


"The reference implementation must be completed before any GLEP is
given status "Final", but it need not be completed before the GLEP is
accepted."

This is not about some EAPI stuff, this is about getting it _accepted_
not _implemented_.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJQYdWfAAoJEFpvPKfnPDWzjjIH/218/i8cD6jzdTOFskLQr8O8
PxOZAPDsRxKz9sVBLcN5I01RRATnr/xbhcomDa+/rLGP8Zz1ljk7mEwaXhXGg6fN
l/lV0I62cjfWx1OJj9nqgq847TLLw1w+TKM/jfDvu2VXtMwBqiyg1U7+CeN7RWVH
YdFOzKi3lJ1zH5oryT98htr6s+hceFie4JERlveODQn56vGG45c5c9vM0x7xxDce
d+7lRozoEwp4AJA70zNskpEojYrJpBJNES/dt7GYO/Rt+sIqac4S8tUvNV3ro2a9
LV+Lr+qLvughdLtpewt95U63zeQJvfVbGIGwRlAaaUWYEI1IlOED6X25Zv3rQXo=
=yexa
-----END PGP SIGNATURE-----

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Ciaran McCreesh]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Tue, 25 Sep 2012 18:02:39 +0200
hasufell <hasufell@gentoo.org> wrote:
> > Really? I thought it was pretty clearly. Yes, you need an 
> > implementation beforehand.
> 
> Maybe you should read:
> http://www.gentoo.org/proj/en/glep/glep-0001.html
> 
> 
> "The reference implementation must be completed before any GLEP is
> given status "Final", but it need not be completed before the GLEP is
> accepted."
> 
> This is not about some EAPI stuff, this is about getting it _accepted_
> not _implemented_.

"Need not" does not mean "should not", and "completed" is not the same
as "exists". In this case, any reasonable observer will conclude that
for this particular problem, having a reasonable reference
implementation and a bunch of ebuilds to play with before we spend any
more time on discussion would be extremely helpful.

- -- 
Ciaran McCreesh
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iEYEARECAAYFAlBh2SwACgkQ96zL6DUtXhGw5ACg06dBcpHZ3Zfq5bQL7I8x7WPT
+WAAn1xpLtjISzwU1onKiZgG6jLpXV7J
=j+q2
-----END PGP SIGNATURE-----

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 25/09/12 12:00 PM, Ciaran McCreesh wrote:
> On Tue, 25 Sep 2012 12:43:00 -0300 Alexis Ballier
> <aballier@gentoo.org> wrote:
>> Could you please elaborate on what kind of problems may arise ?
>> The proposal seems pretty simple and sane to me: PM only has to
>> switch the useflags that are IUSE_RUNTIME in his installed
>> packages db after installing the deps and without triggering a
>> rebuild of said package.
> 
> a) How do we provide a good user interface for it? It took an awful
> lot of experimenting to get the exheres-0 suggestions user
> interface to be good, and it requires quite a bit more information
> from the package side than this proposal is providing. We want to
> avoid a REQUIRED_USE here...

Standard USE flag interface.  This doesn't need anything special.  Why
will a user care if the flag doesn't trigger a package rebuild?

> 
> b) How is consistency checking to be done? Related, what happens
> when a runtime switch introduces a dependency that then requires a
> non-runtime rebuild of the original package?

flag needs to be dropped from IUSE_RUNTIME, so the rebuild would occur.


> c) How do we deal with flag? ( cat/dep[foo] ) or flag? (
> >=cat/dep-2.1 ) cases where cat/dep[-foo] or =cat/dep-2.0 is
> installed and flag is off? From experience, quite a few places
> where you'd want to use suggestions will break if their suggested
> package is installed but doesn't meet version or use requirements.

Use flag deps are dealt with identically to the way they are now.  the
only difference , again, is that the package doesn't get re-emerged.
The VDB would still update imo as if the package did get re-emerged
(ie:  USE and RDEPEND would update), to handle the use flag change
info in metadata but from what I can tell nothing else would need to
be touched.



> 
> However, addressing these probably isn't enough, since this is
> just the things we had to think about for SDEPEND-style
> suggestions... There are likely to be things I've not thought of
> specific to this method that won't crop up until someone tries to
> deliver a decent implementation. This isn't a trivial feature.
> 

..it really is.  It piggy backs entirely on the current USE
implementation, and only skips triggering rebuilds because the
files-on-disk for a package don't need to change.


Now, I do realize that the potential for abuse here is large, and it
will be up to dev's to ensure that these use flags (or groups of use
flag conditions) added to IUSE_RUNTIME will not result in any
files-changed-on-disk.  However that to me doesn't seem to be a good
enough reason to exclude it from a future EAPI.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iF4EAREIAAYFAlBh2YkACgkQ2ugaI38ACPCrzwD/YhavLfXOjjpivCDZ5gbRFI9V
/LBObF/haI2tMZ2CN4cA+wYBxFH1S2Az6zpSLLfAxWnDtPTe22wHb4nMUZ43uIV3
=r8ld
-----END PGP SIGNATURE-----

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1714 bytes --]

On Tue, 25 Sep 2012 17:00:07 +0100
Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:

> On Tue, 25 Sep 2012 12:43:00 -0300
> Alexis Ballier <aballier@gentoo.org> wrote:
> > Could you please elaborate on what kind of problems may arise ? The
> > proposal seems pretty simple and sane to me: PM only has to switch the
> > useflags that are IUSE_RUNTIME in his installed packages db after
> > installing the deps and without triggering a rebuild of said package.
> 
> a) How do we provide a good user interface for it? It took an awful lot
> of experimenting to get the exheres-0 suggestions user interface to be
> good, and it requires quite a bit more information from the package
> side than this proposal is providing. We want to avoid a REQUIRED_USE
> here...

Who is we? I believe REQUIRED_USE is one of the features which will be
available thanks to staying compatible with USE flags instead of
reinventing the wheel.

> b) How is consistency checking to be done? Related, what happens when a
> runtime switch introduces a dependency that then requires a non-runtime
> rebuild of the original package?

Then the package is rebuilt. Where's the problem? Handling of
REQUIRED_USE is not perfectly user friendly but it works.

> c) How do we deal with flag? ( cat/dep[foo] ) or flag? ( >=cat/dep-2.1 )
> cases where cat/dep[-foo] or =cat/dep-2.0 is installed and flag is off?
> From experience, quite a few places where you'd want to use suggestions
> will break if their suggested package is installed but doesn't meet
> version or use requirements.

Er, you mean how to deal with an optional dependency which is not
enabled at all?

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Ciaran McCreesh]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Tue, 25 Sep 2012 12:19:21 -0400
Ian Stakenvicius <axs@gentoo.org> wrote:
> > a) How do we provide a good user interface for it? It took an awful
> > lot of experimenting to get the exheres-0 suggestions user
> > interface to be good, and it requires quite a bit more information
> > from the package side than this proposal is providing. We want to
> > avoid a REQUIRED_USE here...
> 
> Standard USE flag interface.  This doesn't need anything special.  Why
> will a user care if the flag doesn't trigger a package rebuild?

One of the big selling points of suggestions is displaying them to the
user in a useful way (i.e. not via a bunch of einfo messages). If
you're not planning to allow for that, then you're losing a primary
benefit.

> > b) How is consistency checking to be done? Related, what happens
> > when a runtime switch introduces a dependency that then requires a
> > non-runtime rebuild of the original package?
> 
> flag needs to be dropped from IUSE_RUNTIME, so the rebuild would
> occur.

Uh, you're requiring ebuilds to ensure consistency of every
possible configuration of the entire tree?

> > c) How do we deal with flag? ( cat/dep[foo] ) or flag? (
> > >=cat/dep-2.1 ) cases where cat/dep[-foo] or =cat/dep-2.0 is
> > installed and flag is off? From experience, quite a few places
> > where you'd want to use suggestions will break if their suggested
> > package is installed but doesn't meet version or use requirements.
> 
> Use flag deps are dealt with identically to the way they are now.  the
> only difference , again, is that the package doesn't get re-emerged.
> The VDB would still update imo as if the package did get re-emerged
> (ie:  USE and RDEPEND would update), to handle the use flag change
> info in metadata but from what I can tell nothing else would need to
> be touched.

So such packages would just break at runtime?

> > However, addressing these probably isn't enough, since this is
> > just the things we had to think about for SDEPEND-style
> > suggestions... There are likely to be things I've not thought of
> > specific to this method that won't crop up until someone tries to
> > deliver a decent implementation. This isn't a trivial feature.
> 
> ..it really is.  It piggy backs entirely on the current USE
> implementation, and only skips triggering rebuilds because the
> files-on-disk for a package don't need to change.

It's only trivial if you don't try to do anything with it...

- -- 
Ciaran McCreesh
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iEYEARECAAYFAlBh2xgACgkQ96zL6DUtXhGyFwCfYnK+RGbE+bR1Y53t/X3P7UKb
OW4An3fjTeXsXaksDTSAwf/yENunCGpC
=bWvu
-----END PGP SIGNATURE-----

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1387 bytes --]

On Tue, 25 Sep 2012 18:20:06 +0200
Michał Górny <mgorny@gentoo.org> wrote:
> Who is we? I believe REQUIRED_USE is one of the features which will be
> available thanks to staying compatible with USE flags instead of
> reinventing the wheel.

Yes, but the REQUIRED_USE wheel is square, and gives a *very* bumpy
ride to users. It also isn't particularly easy for developers.

> > b) How is consistency checking to be done? Related, what happens
> > when a runtime switch introduces a dependency that then requires a
> > non-runtime rebuild of the original package?
> 
> Then the package is rebuilt. Where's the problem?

The problem is in implementing that correctly... It's certainly doable,
but it's not entirely trivial, depending upon how you're doing
resolution.

> Handling of
> REQUIRED_USE is not perfectly user friendly but it works.

Like a square wheel, yes.

> > c) How do we deal with flag? ( cat/dep[foo] ) or flag?
> > ( >=cat/dep-2.1 ) cases where cat/dep[-foo] or =cat/dep-2.0 is
> > installed and flag is off? From experience, quite a few places
> > where you'd want to use suggestions will break if their suggested
> > package is installed but doesn't meet version or use requirements.
> 
> Er, you mean how to deal with an optional dependency which is not
> enabled at all?

I mean the *entire* thing I wrote.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 25/09/12 12:25 PM, Ciaran McCreesh wrote:
> On Tue, 25 Sep 2012 12:19:21 -0400 Ian Stakenvicius
> <axs@gentoo.org> wrote:
>>> a) How do we provide a good user interface for it? It took an
>>> awful lot of experimenting to get the exheres-0 suggestions
>>> user interface to be good, and it requires quite a bit more
>>> information from the package side than this proposal is
>>> providing. We want to avoid a REQUIRED_USE here...
> 
>> Standard USE flag interface.  This doesn't need anything special.
>> Why will a user care if the flag doesn't trigger a package
>> rebuild?
> 
> One of the big selling points of suggestions is displaying them to
> the user in a useful way (i.e. not via a bunch of einfo messages).
> If you're not planning to allow for that, then you're losing a
> primary benefit.
> 

Must've missed that.  I don't much care about showing things about
optional program interation to users on emerge.  In fact I see that as
being pretty well useless.  Use flag descriptions via metadata.xml ,
though, are *MUCH* more useful and imo suited entirely to this.

(so again, standard use flag interface :)


>>> b) How is consistency checking to be done? Related, what
>>> happens when a runtime switch introduces a dependency that then
>>> requires a non-runtime rebuild of the original package?
> 
>> flag needs to be dropped from IUSE_RUNTIME, so the rebuild would 
>> occur.
> 
> Uh, you're requiring ebuilds to ensure consistency of every 
> possible configuration of the entire tree?

No, only on a per-atom basis.  Maybe I didn't understand what it is
you're referring to here.  Could you elaborate the issue you are
forseeing with a verbose example?



>>> c) How do we deal with flag? ( cat/dep[foo] ) or flag? (
>>>> =cat/dep-2.1 ) cases where cat/dep[-foo] or =cat/dep-2.0 is
>>> installed and flag is off? From experience, quite a few places 
>>> where you'd want to use suggestions will break if their
>>> suggested package is installed but doesn't meet version or use
>>> requirements.
> 
>> Use flag deps are dealt with identically to the way they are now.
>> the only difference , again, is that the package doesn't get
>> re-emerged. The VDB would still update imo as if the package did
>> get re-emerged (ie:  USE and RDEPEND would update), to handle the
>> use flag change info in metadata but from what I can tell nothing
>> else would need to be touched.
> 
> So such packages would just break at runtime?
> 

Again, you lost me.  The package is still added to the emerge list
same as always, and its dependencies (based on USE) are evaluated same
as always.  The package just doesn't REBUILD, because a rebuild would
not result in any change-on-disk.

If there are conflicts in the emerge list then these would be reported
just like if IUSE_RUNTIME wasn't used at all...??


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iF4EAREIAAYFAlBh3nIACgkQ2ugaI38ACPCilwD9HbOgxa99t0pRPI/wt4f6zvFT
Lsjc140u+i15NIcatM8A/1tTC6LLIIFTBma13I0au9rdFRC9C5+oqTPI3bGpf3bx
=0ebw
-----END PGP SIGNATURE-----

[gentoo-dev] Addressing GLEP-62 itself

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Since hasufell brought it up, and as I believe he's going to ask Council
to approve it before moving forward with this proposal towards including
it in an EAPI, I wanted to clarify some of the points mentioned:

- --- Quote, GLEP-62 ---
> Specifications, paragraph 3: The package manager should treat flags
> listed in IUSE_RUNTIME as regular USE flags, except for the 
> following:
> 
> 1. enabling or disabling any of the flags must not involve 
> rebuilding the package,
> 
> 2. it should be possible for a package manager to change those 
> flags on a installed package without using the original ebuild,
> 
> 3. when queried on a installed package, the package manager must 
> consider a particular flag enabled only if its dependencies are 
> satisfied already,
> 
> 4. the flags may be listed in the visual output in a distinct way 
> to inform the user that they affect runtime dependencies only.


#2 -- this would, if I'm understanding it properly, mean that the IUSE
list and the IUSE_RUNTIME list in the 'original ebuild' (ie in vdb)
would be ignored on an emerged package in favour of the ebuild(s) in
the tree, right?  I'm not so sure this is a good idea.

IE, if IUSE and IUSE_RUNTIME have changed in the in-tree ebuild and
one of those use flags that changed have been triggered or
de-triggered I expect that the package should be rebuilt, to keep it
consistent with current practices.

IE2, shouldn't the original ebuild be what's used to trigger the
skip-rebuild functionality, rather than the in-tree ebuild?


#3 -- this seems to imply to me, that the state of a package's
effective USE could be modified solely on the basis of a dependency
existing or not and have nothing to do with what the flag was set to
at emerge time.  IE, *not* the state of USE in the vdb.  I think this
would also be a problem.

In order to properly handle dependency resolution (which IMO we should
do, because these are still USE flags) I think all use flag settings
should still be honoured by the PM and related metadata in the vdb be
updated for IUSE_RUNTIME flags identically to how it would be done if
IUSE_RUNTIME wasn't set.


Thoughts?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iF4EAREIAAYFAlBh8fgACgkQ2ugaI38ACPAENwD/VUmHAIQxuMF8p6FHYLtPk7J+
Xmar9LA0pNPTme27BiEA/3mKSC4VVlUMAT8IWxBcGCFcIQAx8pfirqhx7tfc3TZl
=4oUH
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Addressing GLEP-62 itself

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 25/09/12 02:03 PM, Ian Stakenvicius wrote:
> Since hasufell brought it up, and as I believe he's going to ask
> Council to approve it before moving forward with this proposal
> towards including it in an EAPI, I wanted to clarify some of the
> points mentioned:
> 
> --- Quote, GLEP-62 ---
>> Specifications, paragraph 3: The package manager should treat
>> flags listed in IUSE_RUNTIME as regular USE flags, except for the
>>  following:
> 
>> 1. enabling or disabling any of the flags must not involve 
>> rebuilding the package,
> 
>> 2. it should be possible for a package manager to change those 
>> flags on a installed package without using the original ebuild,
> 
>> 3. when queried on a installed package, the package manager must
>>  consider a particular flag enabled only if its dependencies are
>>  satisfied already,
> 
>> 4. the flags may be listed in the visual output in a distinct way
>>  to inform the user that they affect runtime dependencies only.
> 
> 
> #2 -- this would, if I'm understanding it properly, mean that the
> IUSE list and the IUSE_RUNTIME list in the 'original ebuild' (ie in
> vdb) would be ignored on an emerged package in favour of the
> ebuild(s) in the tree, right?  I'm not so sure this is a good
> idea.
> 
> IE, if IUSE and IUSE_RUNTIME have changed in the in-tree ebuild
> and one of those use flags that changed have been triggered or 
> de-triggered I expect that the package should be rebuilt, to keep
> it consistent with current practices.
> 
> IE2, shouldn't the original ebuild be what's used to trigger the 
> skip-rebuild functionality, rather than the in-tree ebuild?
> 
> 
> #3 -- this seems to imply to me, that the state of a package's 
> effective USE could be modified solely on the basis of a
> dependency existing or not and have nothing to do with what the
> flag was set to at emerge time.  IE, *not* the state of USE in the
> vdb.  I think this would also be a problem.
> 
> In order to properly handle dependency resolution (which IMO we
> should do, because these are still USE flags) I think all use flag
> settings should still be honoured by the PM and related metadata in
> the vdb be updated for IUSE_RUNTIME flags identically to how it
> would be done if IUSE_RUNTIME wasn't set.
> 
> 
> Thoughts?
> 


Based on the above I do expect the reference implementation would also
need to change.  I expect, for instance, that the PM's
metadata-handling would need to occur as normal even though none of
the package's phase functions would run, that is, *DEPEND
(realistically RDEPEND as that should be the only one affected here,
maybe PDEPEND too) and USE/PKGUSE would get updated.  Since portage
would not be re-emerging the package from the tree the original ebuild
would remain.

I expect, as a corollary to this, that a rebuild would be necessary if
(on-disk-IUSE_RUNTIME xor in-ebuild-IUSE_RUNTIME) was non-empty
(--newuse) or resulted in any flags that are in USE
(--reinstall=changed-use).  IMO this would be necessary to ensure the
local ebuild copy and all related metadata for it gets updated in vdb.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iF4EAREIAAYFAlBh/EUACgkQ2ugaI38ACPAFfAD/UsYVQg6ZkwlsaWIafuFr0sqC
7IuvqIgroxNWJ/5XRS8BAJ+5awXZanZftOmFWRmDUAxOvPc8+J073dAn78N0CPdB
=zKzg
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Addressing GLEP-62 itself

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2538 bytes --]

On Tue, 25 Sep 2012 14:03:36 -0400
Ian Stakenvicius <axs@gentoo.org> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
> 
> Since hasufell brought it up, and as I believe he's going to ask Council
> to approve it before moving forward with this proposal towards including
> it in an EAPI, I wanted to clarify some of the points mentioned:
> 
> - --- Quote, GLEP-62 ---
> > Specifications, paragraph 3: The package manager should treat flags
> > listed in IUSE_RUNTIME as regular USE flags, except for the 
> > following:
> > 
> > 1. enabling or disabling any of the flags must not involve 
> > rebuilding the package,
> > 
> > 2. it should be possible for a package manager to change those 
> > flags on a installed package without using the original ebuild,
> > 
> > 3. when queried on a installed package, the package manager must 
> > consider a particular flag enabled only if its dependencies are 
> > satisfied already,
> > 
> > 4. the flags may be listed in the visual output in a distinct way 
> > to inform the user that they affect runtime dependencies only.
> 
> 
> #2 -- this would, if I'm understanding it properly, mean that the IUSE
> list and the IUSE_RUNTIME list in the 'original ebuild' (ie in vdb)
> would be ignored on an emerged package in favour of the ebuild(s) in
> the tree, right?  I'm not so sure this is a good idea.

The exact opposite. The PM should use vdb whenever no non-IUSE_RUNTIME
flags have changed in vdb and no other reasons have caused it to use
the ebuild repository.

> IE, if IUSE and IUSE_RUNTIME have changed in the in-tree ebuild and
> one of those use flags that changed have been triggered or
> de-triggered I expect that the package should be rebuilt, to keep it
> consistent with current practices.

If --newuse triggers that, the ebuild will be used.

> IE2, shouldn't the original ebuild be what's used to trigger the
> skip-rebuild functionality, rather than the in-tree ebuild?

I already said that :).

> #3 -- this seems to imply to me, that the state of a package's
> effective USE could be modified solely on the basis of a dependency
> existing or not and have nothing to do with what the flag was set to
> at emerge time.  IE, *not* the state of USE in the vdb.  I think this
> would also be a problem.

No, you're overestimating it. It just means that the package manager
should first install the dependencies, and then update USE in vdb,
and not the other way around.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] Addressing GLEP-62 itself

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1164 bytes --]

On Tue, 25 Sep 2012 14:47:33 -0400
Ian Stakenvicius <axs@gentoo.org> wrote:

> Based on the above I do expect the reference implementation would also
> need to change.  I expect, for instance, that the PM's
> metadata-handling would need to occur as normal even though none of
> the package's phase functions would run, that is, *DEPEND
> (realistically RDEPEND as that should be the only one affected here,
> maybe PDEPEND too) and USE/PKGUSE would get updated.  Since portage
> would not be re-emerging the package from the tree the original ebuild
> would remain.

Yes, unless I'm missing something that's the intent. I will re-read
and update the GLEP a bit sometime this week.

> I expect, as a corollary to this, that a rebuild would be necessary if
> (on-disk-IUSE_RUNTIME xor in-ebuild-IUSE_RUNTIME) was non-empty
> (--newuse) or resulted in any flags that are in USE
> (--reinstall=changed-use).  IMO this would be necessary to ensure the
> local ebuild copy and all related metadata for it gets updated in vdb.

I think that's a common package manager logic and it's out of scope
of the GLEP.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] Addressing GLEP-62 itself

[Brian Harring]

On Tue, Sep 25, 2012 at 08:58:07PM +0200, Micha?? G??rny wrote:
> On Tue, 25 Sep 2012 14:47:33 -0400
> Ian Stakenvicius <axs@gentoo.org> wrote:
> 
> > Based on the above I do expect the reference implementation would also
> > need to change.  I expect, for instance, that the PM's
> > metadata-handling would need to occur as normal even though none of
> > the package's phase functions would run, that is, *DEPEND
> > (realistically RDEPEND as that should be the only one affected here,
> > maybe PDEPEND too) and USE/PKGUSE would get updated.  Since portage
> > would not be re-emerging the package from the tree the original ebuild
> > would remain.
> 
> Yes, unless I'm missing something that's the intent. I will re-read
> and update the GLEP a bit sometime this week.

There's a fairly strong user interaction component here, along w/ 
potential nastyness for ebuilds (the proposal assume that a flag will 
be toggable in all cases within an ebuild if IUSE_RUNTIME specified; I 
guarantee instances where that fails can be found in the tree if a 
basic audit was done).  Additionally, this *is* useless if it's done 
in a form the UI an't display/handle; Ciaran may bitch about 
REQUIRED_USE's UI (which I knew going in was going to be 
problematic, just to be clear), but he's right on that front.

Additionally, this needs to be thought out how it'll interact with 
eclasses; what stacking, etc.  It doesn't cover the basics there to 
say the least.

Pretty much, this needs an implementation, partial conversion of the 
tree to demonstrate it.

Just to prove that fricking point; if you had tried implementing this, 
a full investigation of what's involved alone, you'd have spotted that 
the core of the proposal is based on a wrong assumption.

Portage doesn't write unfinalized DEPEND/RDEPEND/PDEPEND to the VDB.  

Literally, USE=-x RDEPEND="x? ( dev-util/diffball )", the RDEPEND in 
the vdb is going to be "".

The retort is "meh, that's minor"; the fact this was missed doesn't 
inspire confidence; further, aside from the fact that writing 
finalized deps to the VDB ie, an accurate form of the deps- they're 
locked at that point, a smart PM can take advantage of that to avoid 
reading USE/IUSE till it's needed.  This has measurable speed impact; 
if in doubt, rebuild a tree without it- that little trick I came up 
with first in pkgcore and it works on two fronts; 1) parsing is 
quicker, less nodes/smaller tree, 2) a smart PM can lazy load the 
USE/IUSE for performance gain.  This is why portage/pkgcore were 
intentionally modified to do this; paludis may do it, haven't looked.

My views, if the council is sane they'll slap it back down stating 
"prove it's a viable solution"; regardless of my views of the proposal 
(that it's the wrong way to solve it), it's not ready for an actual 
vote since approving it without actually verifying it could work makes 
things fucktons worse.


> > I expect, as a corollary to this, that a rebuild would be necessary if
> > (on-disk-IUSE_RUNTIME xor in-ebuild-IUSE_RUNTIME) was non-empty
> > (--newuse) or resulted in any flags that are in USE
> > (--reinstall=changed-use).  IMO this would be necessary to ensure the
> > local ebuild copy and all related metadata for it gets updated in vdb.
> 
> I think that's a common package manager logic and it's out of scope
> of the GLEP.

Portage doesn't do physical updates last I saw- if it did, well, 
that's fairly dangerous.  Only thing stored in the VDB is the ebuild 
and the environment dump, and the environment dump is what's ran from.

You cannot sanely pick part the dump and try to intermix current 
ebuilds; rephrasing, I'll kick in the head anyone who tries that.

We used to do that shit; it broke, frequently.  Env saving took a long 
time to get landed across the board- doing what you're proposing the 
PM should do breaks those guarantess and is a step back on that front 
alone.  Basically, not happening, if it is, a series of bugs need 
filing.

What portage does is that if it sees an ebuild in the tree that 
matches, it uses the deps from the tree; that's wholy different from 
what you think is happening in the vdb.

Additionally, portage is the only PM that does this; the rest of the 
PMs treat the vdb ebuild as a constant- without extreme care, or some 
form of insane deptree comparison, you can't just swap in what a new 
ebuild has and expect it to work in all cases.

~harring

[gentoo-dev] Re: [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Duncan]

Ciaran McCreesh posted on Tue, 25 Sep 2012 17:30:19 +0100 as excerpted:

> On Tue, 25 Sep 2012 18:20:06 +0200 Michał Górny <mgorny@gentoo.org>
> wrote:
>> Who is we? I believe REQUIRED_USE is one of the features which will be
>> available thanks to staying compatible with USE flags instead of
>> reinventing the wheel.

Umm... if I read the preceding posts correctly, you missed his intent 
there.  He's not saying it'll be unavailable to be used in the 
implementation, he's saying we don't want to repeat the experience of the 
"bumpy ride" (see below) that required use is, for the user, in a new 
implementation.

> Yes, but the REQUIRED_USE wheel is square, and gives a *very* bumpy ride
> to users. It also isn't particularly easy for developers.

Umm... perhaps pentagonal, the physics of a square wheel... <shudder>.

But yes, as a user who has had to resolve REQUIRED_USE related problems a 
number of times recently, a *very* bumpy ride for the user, it often is!  
Definitely agreed there.

And also agreed with the implication, that we want to avoid a similarly 
bumpy-ride implementation here.

(Implimentation-wise, IMO the problem with REQUIRED_USE is often how the 
ebuild used the REQUIRED_USE, something I'm hopeful it'll improve over 
time as devs get a bit more used to how REQUIRED_USE works and design 
ebuild functionality around it, not triggering the double-uses where they 
can be avoided without seriously impairing the choices exposed by USE 
flags in the first place, and a better REQUIRED_USE implementation is 
certainly a challenge, but it's equally certainly an extremely bumpy ride 
for the user, as it is today.)

>> > b) How is consistency checking to be done? Related, what happens when
>> > a runtime switch introduces a dependency that then requires a
>> > non-runtime rebuild of the original package?
>> 
>> Then the package is rebuilt. Where's the problem?
> 
> The problem is in implementing that correctly... It's certainly doable,
> but it's not entirely trivial, depending upon how you're doing
> resolution.
> 
>> Handling of REQUIRED_USE is not perfectly user friendly but it works.
> 
> Like a square wheel, yes.

Pentagonal (or at least rounded corners on the square... tho of course 
then there's patent issues!), but agreed.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Addressing GLEP-62 itself

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 3276 bytes --]

On Tue, 25 Sep 2012 12:54:39 -0700
Brian Harring <ferringb@gmail.com> wrote:

> On Tue, Sep 25, 2012 at 08:58:07PM +0200, Micha?? G??rny wrote:
> > On Tue, 25 Sep 2012 14:47:33 -0400
> > Ian Stakenvicius <axs@gentoo.org> wrote:
> > 
> > > Based on the above I do expect the reference implementation would also
> > > need to change.  I expect, for instance, that the PM's
> > > metadata-handling would need to occur as normal even though none of
> > > the package's phase functions would run, that is, *DEPEND
> > > (realistically RDEPEND as that should be the only one affected here,
> > > maybe PDEPEND too) and USE/PKGUSE would get updated.  Since portage
> > > would not be re-emerging the package from the tree the original ebuild
> > > would remain.
> > 
> > Yes, unless I'm missing something that's the intent. I will re-read
> > and update the GLEP a bit sometime this week.
> 
> There's a fairly strong user interaction component here, along w/ 
> potential nastyness for ebuilds (the proposal assume that a flag will 
> be toggable in all cases within an ebuild if IUSE_RUNTIME specified; I 
> guarantee instances where that fails can be found in the tree if a 
> basic audit was done).  Additionally, this *is* useless if it's done 
> in a form the UI an't display/handle; Ciaran may bitch about 
> REQUIRED_USE's UI (which I knew going in was going to be 
> problematic, just to be clear), but he's right on that front.
> 
> Additionally, this needs to be thought out how it'll interact with 
> eclasses; what stacking, etc.  It doesn't cover the basics there to 
> say the least.

The proposal didn't cover eclasses at all. Is there a need to do so or
are we chasing some kind of perfection based on filling all unused
slots?

> Pretty much, this needs an implementation, partial conversion of the 
> tree to demonstrate it.
> 
> Just to prove that fricking point; if you had tried implementing this, 
> a full investigation of what's involved alone, you'd have spotted that 
> the core of the proposal is based on a wrong assumption.
> 
> Portage doesn't write unfinalized DEPEND/RDEPEND/PDEPEND to the VDB.

There's a footnote there, saying:

  The package manager has to ensure that all relevant information is
  stored in the installed package metadata.

> > > I expect, as a corollary to this, that a rebuild would be necessary if
> > > (on-disk-IUSE_RUNTIME xor in-ebuild-IUSE_RUNTIME) was non-empty
> > > (--newuse) or resulted in any flags that are in USE
> > > (--reinstall=changed-use).  IMO this would be necessary to ensure the
> > > local ebuild copy and all related metadata for it gets updated in vdb.
> > 
> > I think that's a common package manager logic and it's out of scope
> > of the GLEP.
> 
> Portage doesn't do physical updates last I saw- if it did, well, 
> that's fairly dangerous.  Only thing stored in the VDB is the ebuild 
> and the environment dump, and the environment dump is what's ran from.
> 
> You cannot sanely pick part the dump and try to intermix current 
> ebuilds; rephrasing, I'll kick in the head anyone who tries that.

What are you talking about? As far as I can see, we are talking here
about *full rebuild*.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] Addressing GLEP-62 itself

[Brian Harring]

On Wed, Sep 26, 2012 at 08:02:44AM +0200, Micha?? G??rny wrote:
> On Tue, 25 Sep 2012 12:54:39 -0700
> Brian Harring <ferringb@gmail.com> wrote:
> 
> > On Tue, Sep 25, 2012 at 08:58:07PM +0200, Micha?? G??rny wrote:
> > > On Tue, 25 Sep 2012 14:47:33 -0400
> > > Ian Stakenvicius <axs@gentoo.org> wrote:
> > > 
> > > > Based on the above I do expect the reference implementation would also
> > > > need to change.  I expect, for instance, that the PM's
> > > > metadata-handling would need to occur as normal even though none of
> > > > the package's phase functions would run, that is, *DEPEND
> > > > (realistically RDEPEND as that should be the only one affected here,
> > > > maybe PDEPEND too) and USE/PKGUSE would get updated.  Since portage
> > > > would not be re-emerging the package from the tree the original ebuild
> > > > would remain.
> > > 
> > > Yes, unless I'm missing something that's the intent. I will re-read
> > > and update the GLEP a bit sometime this week.
> > 
> > There's a fairly strong user interaction component here, along w/ 
> > potential nastyness for ebuilds (the proposal assume that a flag will 
> > be toggable in all cases within an ebuild if IUSE_RUNTIME specified; I 
> > guarantee instances where that fails can be found in the tree if a 
> > basic audit was done).  Additionally, this *is* useless if it's done 
> > in a form the UI an't display/handle; Ciaran may bitch about 
> > REQUIRED_USE's UI (which I knew going in was going to be 
> > problematic, just to be clear), but he's right on that front.

^^^ This point still needs addressing.


> > Additionally, this needs to be thought out how it'll interact with 
> > eclasses; what stacking, etc.  It doesn't cover the basics there to 
> > say the least.
> 
> The proposal didn't cover eclasses at all. Is there a need to do so or
> are we chasing some kind of perfection based on filling all unused
> slots?

Eclass stacking here matters; if it's stacked, it means ebuilds have 
to use out of bound (ie, other vars) to tell the eclass when it 
shouldn't mark a flag as runtime toggable.  If it's not stacked by 
the pm, then they have to manually stack; that differs from the norm 
and makes it easier to screwup; however; does allow for them to 
filter, albeit a slight pain in the ass doing so.

There's a choice there, and the answer matters, so yes, you should 
actually have a complete glep before trying to shove it up to the 
council and extract a vote out of them.  Lest the intention is to just 
have them kick it back to the curb...


> > Pretty much, this needs an implementation, partial conversion of the 
> > tree to demonstrate it.
> > 
> > Just to prove that fricking point; if you had tried implementing this, 
> > a full investigation of what's involved alone, you'd have spotted that 
> > the core of the proposal is based on a wrong assumption.
> > 
> > Portage doesn't write unfinalized DEPEND/RDEPEND/PDEPEND to the VDB.
> 
> There's a footnote there, saying:
> 
>   The package manager has to ensure that all relevant information is
>   stored in the installed package metadata.

Frankly I don't fully buy that you were aware of this issue from the 
start of the proposal; the wording partially covers it however.  
Ddoesn't call it out, but via tha req it dumps it on the package 
manager developers heads to sort it- which already is the case. 
Binpkgs minimally weren't addressed which is why I still don't think 
this was actually spotted up front.

Were it, the performance impact should've been mentioned, and 
quantified; con's are part of a normal glep.

Meaning.. wait for it... writing a prototype is required to get those 
stats. ;)


> > > > Based on the above I do expect the reference implementation 
> > > > would alsoneed to change.  I expect, for instance, that the 
> > > > PM's metadata-handling would need to occur as normal even 
> > > > though none of the package's phase functions would run, that 
> > > > is, *DEPEND (realistically RDEPEND as that should be the only 
> > > > one affected here, maybe PDEPEND too) and USE/PKGUSE would get 
> > > > updated.  Since portage would not be re-emerging the package 
> > > > from the tree the original ebuild
> > > > would remain.
> > > > 
> > > > I expect, as a corollary to this, that a rebuild would be necessary if
> > > > (on-disk-IUSE_RUNTIME xor in-ebuild-IUSE_RUNTIME) was non-empty
> > > > (--newuse) or resulted in any flags that are in USE
> > > > (--reinstall=changed-use).  IMO this would be necessary to ensure the
> > > > local ebuild copy and all related metadata for it gets updated in vdb.
> > > 
> > > I think that's a common package manager logic and it's out of scope
> > > of the GLEP.
> > 
> > Portage doesn't do physical updates last I saw- if it did, well, 
> > that's fairly dangerous.  Only thing stored in the VDB is the ebuild 
> > and the environment dump, and the environment dump is what's ran from.
> > 
> > You cannot sanely pick part the dump and try to intermix current 
> > ebuilds; rephrasing, I'll kick in the head anyone who tries that.
> 
> What are you talking about? As far as I can see, we are talking here
> about *full rebuild*.

Added the full quote above.  Read prior to the corollary; that's 
proposing transferance of *depend from live tree to vdb; that's not 
common- portage alone does that, and it's got faults that make it 
unlikely you'll convince others to replicate that behaviour.

~harring

Re: [gentoo-dev] Addressing GLEP-62 itself

[Alexis Ballier]

On Wed, 26 Sep 2012 03:29:17 -0700
Brian Harring <ferringb@gmail.com> wrote:

> On Wed, Sep 26, 2012 at 08:02:44AM +0200, Micha?? G??rny wrote:
> > On Tue, 25 Sep 2012 12:54:39 -0700
> > Brian Harring <ferringb@gmail.com> wrote:
> > 
> > > On Tue, Sep 25, 2012 at 08:58:07PM +0200, Micha?? G??rny wrote:
> > > > On Tue, 25 Sep 2012 14:47:33 -0400
> > > > Ian Stakenvicius <axs@gentoo.org> wrote:
> > > > 
> > > > > Based on the above I do expect the reference implementation
> > > > > would also need to change.  I expect, for instance, that the
> > > > > PM's metadata-handling would need to occur as normal even
> > > > > though none of the package's phase functions would run, that
> > > > > is, *DEPEND (realistically RDEPEND as that should be the only
> > > > > one affected here, maybe PDEPEND too) and USE/PKGUSE would
> > > > > get updated.  Since portage would not be re-emerging the
> > > > > package from the tree the original ebuild would remain.
> > > > 
> > > > Yes, unless I'm missing something that's the intent. I will
> > > > re-read and update the GLEP a bit sometime this week.
> > > 
> > > There's a fairly strong user interaction component here, along w/ 
> > > potential nastyness for ebuilds (the proposal assume that a flag
> > > will be toggable in all cases within an ebuild if IUSE_RUNTIME
> > > specified; I guarantee instances where that fails can be found in
> > > the tree if a basic audit was done).  Additionally, this *is*
> > > useless if it's done in a form the UI an't display/handle; Ciaran
> > > may bitch about REQUIRED_USE's UI (which I knew going in was
> > > going to be problematic, just to be clear), but he's right on
> > > that front.
> 
> ^^^ This point still needs addressing.

IUSE_RUNTIME is optional for PMs, why does the UI matter at all ?
Also, the proposal doesn't assume flags are toggable at will, it assumes
they are useflags and obey the same rules.

> > > Additionally, this needs to be thought out how it'll interact
> > > with eclasses; what stacking, etc.  It doesn't cover the basics
> > > there to say the least.
> > 
> > The proposal didn't cover eclasses at all. Is there a need to do so
> > or are we chasing some kind of perfection based on filling all
> > unused slots?
> 
> Eclass stacking here matters; if it's stacked, it means ebuilds have 
> to use out of bound (ie, other vars) to tell the eclass when it 
> shouldn't mark a flag as runtime toggable.  If it's not stacked by 
> the pm, then they have to manually stack; that differs from the norm 
> and makes it easier to screwup; however; does allow for them to 
> filter, albeit a slight pain in the ass doing so.
> 
> There's a choice there, and the answer matters, so yes, you should 
> actually have a complete glep before trying to shove it up to the 
> council and extract a vote out of them.  Lest the intention is to
> just have them kick it back to the curb...

It can't be stacked and it's not wise to do so; it is a simple bash
variable like tons of others in eclasses:
"""
Package managers not implementing this GLEP will consider
the ``IUSE_RUNTIME`` variable as an irrelevant bash variable
"""
"""
2. introduce additional ``IUSE_RUNTIME`` variable listing names of USE
   flags related to optional runtime dependencies (without prefixes
   related to IUSE defaults).
"""

Treating bash variables as bash variables is rather the norm,
stacking is the exception. As I understand it, your only objection here
is that you want to see written 'IUSE_RUNTIME gets no special treatment'
in the proposal ?

A.

Re: [gentoo-dev] Addressing GLEP-62 itself

[Brian Harring]

On Wed, Sep 26, 2012 at 02:38:02PM -0300, Alexis Ballier wrote:
> On Wed, 26 Sep 2012 03:29:17 -0700
> Brian Harring <ferringb@gmail.com> wrote:
> 
> > On Wed, Sep 26, 2012 at 08:02:44AM +0200, Micha?? G??rny wrote:
> > > On Tue, 25 Sep 2012 12:54:39 -0700
> > > Brian Harring <ferringb@gmail.com> wrote:
> > > 
> > > > On Tue, Sep 25, 2012 at 08:58:07PM +0200, Micha?? G??rny wrote:
> > > > > On Tue, 25 Sep 2012 14:47:33 -0400
> > > > > Ian Stakenvicius <axs@gentoo.org> wrote:
> > > > > 
> > > > > > Based on the above I do expect the reference implementation
> > > > > > would also need to change.  I expect, for instance, that the
> > > > > > PM's metadata-handling would need to occur as normal even
> > > > > > though none of the package's phase functions would run, that
> > > > > > is, *DEPEND (realistically RDEPEND as that should be the only
> > > > > > one affected here, maybe PDEPEND too) and USE/PKGUSE would
> > > > > > get updated.  Since portage would not be re-emerging the
> > > > > > package from the tree the original ebuild would remain.
> > > > > 
> > > > > Yes, unless I'm missing something that's the intent. I will
> > > > > re-read and update the GLEP a bit sometime this week.
> > > > 
> > > > There's a fairly strong user interaction component here, along w/ 
> > > > potential nastyness for ebuilds (the proposal assume that a flag
> > > > will be toggable in all cases within an ebuild if IUSE_RUNTIME
> > > > specified; I guarantee instances where that fails can be found in
> > > > the tree if a basic audit was done).  Additionally, this *is*
> > > > useless if it's done in a form the UI an't display/handle; Ciaran
> > > > may bitch about REQUIRED_USE's UI (which I knew going in was
> > > > going to be problematic, just to be clear), but he's right on
> > > > that front.
> > 
> > ^^^ This point still needs addressing.
> 
> IUSE_RUNTIME is optional for PMs, why does the UI matter at all ?
> Also, the proposal doesn't assume flags are toggable at will, it assumes
> they are useflags and obey the same rules.

I truly hate claims like this; "it's optional, so who cares if it's 
whacky".  Think through the proposal; for it to work reliably, it's 
not optional.  Same issue I've been y'all over the head with, 
rendered/finalized vs raw/unfinalized deps being stored in the VDB.  

All managers have to write unfinalized if that proposal goes through, 
even if they don't support the optional toggling after the fact.  

As for the UI... arguing "but it's optional!" doesn't give a blank 
check for the UI angle.  What the plan, more colorization and a new 
char for emerge -vp?  Because we're kind of running out of chars 
there...

It's a simple enough request; one that wouldn't even need to be made 
if there was code backing this proposal; on a related note, hell yes 
I'm wary of having this dumped on manager authors heads and having to 
be told "sort out the mess/make it so".  So I'm asking y'all to at 
least put in an equivalent time investment doing a quick prototype.

This isn't an unreasonable request, and has been the norm for most 
gleps for a long while.


> > > > Additionally, this needs to be thought out how it'll interact
> > > > with eclasses; what stacking, etc.  It doesn't cover the basics
> > > > there to say the least.
> > > 
> > > The proposal didn't cover eclasses at all. Is there a need to do so
> > > or are we chasing some kind of perfection based on filling all
> > > unused slots?
> > 
> > Eclass stacking here matters; if it's stacked, it means ebuilds have 
> > to use out of bound (ie, other vars) to tell the eclass when it 
> > shouldn't mark a flag as runtime toggable.  If it's not stacked by 
> > the pm, then they have to manually stack; that differs from the norm 
> > and makes it easier to screwup; however; does allow for them to 
> > filter, albeit a slight pain in the ass doing so.
> > 
> > There's a choice there, and the answer matters, so yes, you should 
> > actually have a complete glep before trying to shove it up to the 
> > council and extract a vote out of them.  Lest the intention is to
> > just have them kick it back to the curb...
> 
> It can't be stacked and it's not wise to do so; it is a simple bash
> variable like tons of others in eclasses:

It cannot be stacked because y'all are trying to shove this in as an 
optional; unlike it's brother IUSE, which stacks.

As for "ons of others that don't stack"; very few actually influence 
the package manager; ~14 roughly, minimally 5 of those stack (those 
that don't, basically aren't lists).


> """
> Package managers not implementing this GLEP will consider
> the ``IUSE_RUNTIME`` variable as an irrelevant bash variable
> """
> """
> 2. introduce additional ``IUSE_RUNTIME`` variable listing names of USE
>    flags related to optional runtime dependencies (without prefixes
>    related to IUSE defaults).
> """
> 
> Treating bash variables as bash variables is rather the norm,
> stacking is the exception. As I understand it, your only objection here
> is that you want to see written 'IUSE_RUNTIME gets no special treatment'
> in the proposal ?

My objection is punting it to the council till it's actually nailed 
down/sane; having them mark it accepted means we're stuck w/ the 
results if it turns out to be shit at the implementation/UI level as a 
lot of us think it will be.

So yes, I want it actually finalized.  Bluntly; there's zero point 
having the council comment if it ain't finalized.

~harring

Re: [gentoo-dev] Addressing GLEP-62 itself

[hasufell]

On 09/26/2012 11:02 PM, Brian Harring wrote:
> 
> So yes, I want it actually finalized.  Bluntly; there's zero point 
> having the council comment if it ain't finalized.
> 

I understand. We probably have to wait for it then.

Meanwhile... is there an alternative approach?

Re: [gentoo-dev] Addressing GLEP-62 itself

[Alexis Ballier]

On Wed, 26 Sep 2012 14:02:57 -0700
Brian Harring <ferringb@gmail.com> wrote:

> On Wed, Sep 26, 2012 at 02:38:02PM -0300, Alexis Ballier wrote:
> > On Wed, 26 Sep 2012 03:29:17 -0700
> > Brian Harring <ferringb@gmail.com> wrote:
> > 
> > > On Wed, Sep 26, 2012 at 08:02:44AM +0200, Micha?? G??rny wrote:
> > > > On Tue, 25 Sep 2012 12:54:39 -0700
> > > > Brian Harring <ferringb@gmail.com> wrote:
> > > > 
> > > > > On Tue, Sep 25, 2012 at 08:58:07PM +0200, Micha?? G??rny
> > > > > wrote:
> > > > > > On Tue, 25 Sep 2012 14:47:33 -0400
> > > > > > Ian Stakenvicius <axs@gentoo.org> wrote:
> > > > > > 
> > > > > > > Based on the above I do expect the reference
> > > > > > > implementation would also need to change.  I expect, for
> > > > > > > instance, that the PM's metadata-handling would need to
> > > > > > > occur as normal even though none of the package's phase
> > > > > > > functions would run, that is, *DEPEND (realistically
> > > > > > > RDEPEND as that should be the only one affected here,
> > > > > > > maybe PDEPEND too) and USE/PKGUSE would get updated.
> > > > > > > Since portage would not be re-emerging the package from
> > > > > > > the tree the original ebuild would remain.
> > > > > > 
> > > > > > Yes, unless I'm missing something that's the intent. I will
> > > > > > re-read and update the GLEP a bit sometime this week.
> > > > > 
> > > > > There's a fairly strong user interaction component here,
> > > > > along w/ potential nastyness for ebuilds (the proposal assume
> > > > > that a flag will be toggable in all cases within an ebuild if
> > > > > IUSE_RUNTIME specified; I guarantee instances where that
> > > > > fails can be found in the tree if a basic audit was done).
> > > > > Additionally, this *is* useless if it's done in a form the UI
> > > > > an't display/handle; Ciaran may bitch about REQUIRED_USE's UI
> > > > > (which I knew going in was going to be problematic, just to
> > > > > be clear), but he's right on that front.
> > > 
> > > ^^^ This point still needs addressing.
> > 
> > IUSE_RUNTIME is optional for PMs, why does the UI matter at all ?
> > Also, the proposal doesn't assume flags are toggable at will, it
> > assumes they are useflags and obey the same rules.
> 
> I truly hate claims like this; "it's optional, so who cares if it's 
> whacky".  Think through the proposal; for it to work reliably, it's 
> not optional.  Same issue I've been y'all over the head with, 
> rendered/finalized vs raw/unfinalized deps being stored in the VDB.  
> 
> All managers have to write unfinalized if that proposal goes through, 
> even if they don't support the optional toggling after the fact.

Why? _Current_ PMs will rebuild the package. The point of this is just
to give them a hint that they do not need to rebuild it. We already
have an implementation actually: one that ignores the hint :)

> As for the UI... arguing "but it's optional!" doesn't give a blank 
> check for the UI angle.  What the plan, more colorization and a new 
> char for emerge -vp?  Because we're kind of running out of chars 
> there...

How is this relevant ?

> It's a simple enough request; one that wouldn't even need to be made 
> if there was code backing this proposal; on a related note, hell yes 
> I'm wary of having this dumped on manager authors heads and having to 
> be told "sort out the mess/make it so".  So I'm asking y'all to at 
> least put in an equivalent time investment doing a quick prototype.
> 
> This isn't an unreasonable request, and has been the norm for most 
> gleps for a long while.

I guess people do not want to invest time in writing code for something
doomed. The original request was just to have it 'accepted' so that an
implementation can start. If the implementation is good then make it
final, otherwise amend or reject the glep. This isn't unreasonable
either.

> > > > > Additionally, this needs to be thought out how it'll interact
> > > > > with eclasses; what stacking, etc.  It doesn't cover the
> > > > > basics there to say the least.
> > > > 
> > > > The proposal didn't cover eclasses at all. Is there a need to
> > > > do so or are we chasing some kind of perfection based on
> > > > filling all unused slots?
> > > 
> > > Eclass stacking here matters; if it's stacked, it means ebuilds
> > > have to use out of bound (ie, other vars) to tell the eclass when
> > > it shouldn't mark a flag as runtime toggable.  If it's not
> > > stacked by the pm, then they have to manually stack; that differs
> > > from the norm and makes it easier to screwup; however; does allow
> > > for them to filter, albeit a slight pain in the ass doing so.
> > > 
> > > There's a choice there, and the answer matters, so yes, you
> > > should actually have a complete glep before trying to shove it up
> > > to the council and extract a vote out of them.  Lest the
> > > intention is to just have them kick it back to the curb...
> > 
> > It can't be stacked and it's not wise to do so; it is a simple bash
> > variable like tons of others in eclasses:
> 
> It cannot be stacked because y'all are trying to shove this in as an 
> optional; unlike it's brother IUSE, which stacks.
> 
> As for "ons of others that don't stack"; very few actually influence 
> the package manager; ~14 roughly, minimally 5 of those stack (those 
> that don't, basically aren't lists).

So it's not stacked, nothing else to discuss here :)

> > """
> > Package managers not implementing this GLEP will consider
> > the ``IUSE_RUNTIME`` variable as an irrelevant bash variable
> > """
> > """
> > 2. introduce additional ``IUSE_RUNTIME`` variable listing names of
> > USE flags related to optional runtime dependencies (without prefixes
> >    related to IUSE defaults).
> > """
> > 
> > Treating bash variables as bash variables is rather the norm,
> > stacking is the exception. As I understand it, your only objection
> > here is that you want to see written 'IUSE_RUNTIME gets no special
> > treatment' in the proposal ?
> 
> My objection is punting it to the council till it's actually nailed 
> down/sane; having them mark it accepted means we're stuck w/ the 
> results if it turns out to be shit at the implementation/UI level as
> a lot of us think it will be.
> 
> So yes, I want it actually finalized.  Bluntly; there's zero point 
> having the council comment if it ain't finalized.

Then the proposal itself should be discussed, not the implementation
details: chars for emerge -pv are irrelevant; you pointed the raw dep
issue in the VDB, that's good, but never said anything as of why they
can't additionally be stored, making this a non-issue for the
_proposal_ acceptation.

Anyway, without implementation I expect the council to just give a vote
of opinion, showing support or non-support to the proposal; the
proposal will be final only once the council will be happy with
portage's implementation. And I agree, the council doesn't need to be
involved to start the implementation, but knowing this proposal has
supporters will motivate people to do the implementation, vs. not even
being sure the idea itself will get some support.

Would you support the proposal if you are happy with its
implementation ?

A.

Re: [gentoo-dev] Addressing GLEP-62 itself

[Brian Harring]

On Wed, Sep 26, 2012 at 07:25:11PM -0300, Alexis Ballier wrote:
> On Wed, 26 Sep 2012 14:02:57 -0700
> Brian Harring <ferringb@gmail.com> wrote:
> > On Wed, Sep 26, 2012 at 02:38:02PM -0300, Alexis Ballier wrote:
> > > IUSE_RUNTIME is optional for PMs, why does the UI matter at all ?
> > > Also, the proposal doesn't assume flags are toggable at will, it
> > > assumes they are useflags and obey the same rules.
> > 
> > I truly hate claims like this; "it's optional, so who cares if it's 
> > whacky".  Think through the proposal; for it to work reliably, it's 
> > not optional.  Same issue I've been y'all over the head with, 
> > rendered/finalized vs raw/unfinalized deps being stored in the VDB.  
> > 
> > All managers have to write unfinalized if that proposal goes through, 
> > even if they don't support the optional toggling after the fact.
> 
> Why? _Current_ PMs will rebuild the package. The point of this is just
> to give them a hint that they do not need to rebuild it. We already
> have an implementation actually: one that ignores the hint :)

Bullshit.  This is optional in the sense of embrace/extend 'optional'; 
if one PM takes up the new functionality, all have to switch to 
writing unfinalized deps to the VDB, and all have to switch to 
transfering that IUSE_RUNTIME crap to the VDB.

If they don't, whatever sole/crappy PM that runs w/ this proposal will 
wind up forcing rebuilds on any packages merged by the PM's that don't 
do this "optional" glep.

Thus rendering it nonoptional since it implicitly is reliant on all 
switching to the degrade *DEPEND writing that this proposal is reliant 
on.  The blame game becomes "well, you shouldn't use the PMs that 
don't do this *optional* thing".  There in is the implicit lie of that 
'optional' crap.

Claiming other wise is ignoring reality; I called it embrace/extend 
because this is exactly how that shit goes- sure it's optional, 'cept 
you're forced to support it (even partially) else the whole degrades 
and that PM winds up getting blackballed or fragmentation occuring.  
As far as I'm concerned, any PMS intended proposal must not pull the 
'should' or 'optional' crap; it has no place in a spec (spec's are 
supposed to be assertions after all).


> > As for the UI... arguing "but it's optional!" doesn't give a blank 
> > check for the UI angle.  What the plan, more colorization and a new 
> > char for emerge -vp?  Because we're kind of running out of chars 
> > there...
> 
> How is this relevant ?

Um... dude... This proposal is about adding suggested/optional deps so 
people can inspect/select/enable them per package.

You're asking "how is the UI relevant" in light of that.

Just saying; it's kind of core to this whole damn thing, else we're 
just trying to add an optimization hack; either one runs a strong risk 
of my next response including a joke about elderberries and hamsters. 
;)


> > It's a simple enough request; one that wouldn't even need to be made 
> > if there was code backing this proposal; on a related note, hell yes 
> > I'm wary of having this dumped on manager authors heads and having to 
> > be told "sort out the mess/make it so".  So I'm asking y'all to at 
> > least put in an equivalent time investment doing a quick prototype.
> > 
> > This isn't an unreasonable request, and has been the norm for most 
> > gleps for a long while.
> 
> I guess people do not want to invest time in writing code for something
> doomed.

This is one of the cases where "tough fucking luck" really/truly fits.

If it's doomed, consider why it's doomed.  I'm not requiring a 
prototype just because I'm a dick; I'm requiring a prototype because 
I fully expect since y'all won't listen to what people are telling 
you, trying to write the code will educate y'all to what we've been 
saying.   This is ignoring that prototypes are bsaically the norm for 
proposals of this sort (both PMS and gleps)... meaning it's the 
standard, and y'all are trying to get this proposal special cased.

Does it suck you can't just get what you want via writing a quick doc 
and arguing on an ml?  At times, yes.  If you believe it's worth it, 
you do the legwork.

If the folks backing this can't be bothered to write a freaking patch, 
well, I think that's a pretty strong vote of no-confidence on the 
backers part.


> The original request was just to have it 'accepted' so that an
> implementation can start. If the implementation is good then make it
> final, otherwise amend or reject the glep. This isn't unreasonable
> either.

Also known as rubber stamp it.  And if it sucks, of course it's easy 
to roll that bad idea back?  Right?

If the idea was universally accepted and lacked issues, that may fly; 
that's not been the case.


> > It cannot be stacked because y'all are trying to shove this in as an 
> > optional; unlike it's brother IUSE, which stacks.
> > 
> > As for "ons of others that don't stack"; very few actually influence 
> > the package manager; ~14 roughly, minimally 5 of those stack (those 
> > that don't, basically aren't lists).
> 
> So it's not stacked, nothing else to discuss here :)

Grr.  You're being daft if you think I'll back down just because of 
some word play and ignoring my points.

It *should* be stacked is my view; that matches it's sibling IUSE, and 
general behaviour for metadata lists.

However, that cannot be done if it's treated as 'optional'- that 
'optional' crap was attempted as a way to glue this onto existing 
EAPIs.  I'm pointing at the metadata issue since 1) that optional 
requirement results in the metadata key being ill fitting in 
comparison to IUSE, 2) it's easier to beat on that point then to have 
to argue w/ y'all as to the fact y'all are ignoring the implicit 
requirement of this proposal that IUSE_RUNTIME get added into the 
mainline caches (without it, more PM hacks would be required; 
additionally, the metadata cache angle is a further example of this 
being non-optional).


> > > """
> > > Package managers not implementing this GLEP will consider
> > > the ``IUSE_RUNTIME`` variable as an irrelevant bash variable
> > > """
> > > """
> > > 2. introduce additional ``IUSE_RUNTIME`` variable listing names of
> > > USE flags related to optional runtime dependencies (without prefixes
> > >    related to IUSE defaults).
> > > """
> > > 
> > > Treating bash variables as bash variables is rather the norm,
> > > stacking is the exception. As I understand it, your only objection
> > > here is that you want to see written 'IUSE_RUNTIME gets no special
> > > treatment' in the proposal ?
> > 
> > My objection is punting it to the council till it's actually nailed 
> > down/sane; having them mark it accepted means we're stuck w/ the 
> > results if it turns out to be shit at the implementation/UI level as
> > a lot of us think it will be.
> > 
> > So yes, I want it actually finalized.  Bluntly; there's zero point 
> > having the council comment if it ain't finalized.
> 
> Then the proposal itself should be discussed, not the implementation
> details: chars for emerge -pv are irrelevant; you pointed the raw dep
> issue in the VDB, that's good, but never said anything as of why they
> can't additionally be stored, making this a non-issue for the
> _proposal_ acceptation.
> 
> Anyway, without implementation I expect the council to just give a vote
> of opinion, showing support or non-support to the proposal; the
> proposal will be final only once the council will be happy with
> portage's implementation. And I agree, the council doesn't need to be
> involved to start the implementation, but knowing this proposal has
> supporters will motivate people to do the implementation, vs. not even
> being sure the idea itself will get some support.

I reiterate; if a proposal needs the council to motivate people, that 
proposal is fucked- period.  If the backing author(s) can't be 
bothered to do the implementation, it's doubly so fucked.  Trace 
gentoo's history before arguing that one- there is a lot of examples 
already.

You can phrase it, argue it, whatever, all you want, but that's a fact 
of reality.  We've had lots of things get pushed up to the council and 
get 'approved', ie rubber stamped- and a lot of them went jack-all 
nowhere because approval != reality, nor does it even mean "of course 
someone will step up to do the work you refuse to do".  If you can't 
do the work yourself (or won't allocate the time to do so), or can't 
convince someone to do it on your behalf, the proposal is 
effectively dead already.

The Heinlein "There ain't no such thing as a free lunch" quote is dead 
on in this regard; you want it, do the work.  Involving the council 
just wastes their time, and our time via this argument continuing.


> Would you support the proposal if you are happy with its
> implementation ?

The implementation frankly isn't for me; y'all might manage something 
unexpected, but I've been laying out- much more so than the people 
pushing this- exactly what is going to be involved here, and the 
problems involved and why I've been -1'ing this bugger from day one..  
As said, y'all may surprise me, but I expect the implementation to
prove my points.

And... now the argument will be "well why should we waste our time 
doing it?".

A rather valid question I'd ask in that case is "if you respect my 
views enough that you'd *skip* doing the implementation if I said it 
was fucked, why were you freaking arguing and trying to railroad 
it through the council?"... just saying.

The reason to do the implementation is that if y'all think everyone 
else is wrong on this, do the legwork to prove them wrong, prove the 
idea works.

Arguing on the ML, needling people about "well this would've solved 
it" (see ciaran's labels behaviour from the last N years) aren't
productive.  Produce code, or shut up, basically; that's roughly the 
productive courses of action at this point.

I do want suggested/optional depends; I just don't want this mess 
jammed in since it doesn't solve it particularly well (and that's 
being generous in my word choice), and the associated cost isn't 
worth the gains in my view.  That simple.

Either way, if you'd like to keep trading blows, try pushing it to 
the council despite people bitching... have at it, albeit by yourself 
.  I've made my points, done with this thread (sparing people more 
emails).

~harring

Re: [gentoo-dev] Addressing GLEP-62 itself

[Zac Medico]

On 09/27/2012 12:53 PM, Brian Harring wrote:
> Bullshit.  This is optional in the sense of embrace/extend 'optional'; 
> if one PM takes up the new functionality, all have to switch to 
> writing unfinalized deps to the VDB, and all have to switch to 
> transfering that IUSE_RUNTIME crap to the VDB.

I think the proposal suddenly becomes a lot saner if it's done as an
EAPI extension, the optional runtime deps and IUSE_RUNTIME conditionals
are isolated in a new separate variable (perhaps SRDEPEND), and
IUSE_RUNTIME is not allowed to intersect with IUSE. Using a separate
SRDEPEND variable means that the package manager only has to preserve
USE conditionals in the vdb for that one variable.
-- 
Thanks,
Zac

Re: [gentoo-dev] Addressing GLEP-62 itself

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 27/09/12 04:13 PM, Zac Medico wrote:
> On 09/27/2012 12:53 PM, Brian Harring wrote:
>> Bullshit.  This is optional in the sense of embrace/extend
>> 'optional'; if one PM takes up the new functionality, all have to
>> switch to writing unfinalized deps to the VDB, and all have to
>> switch to transfering that IUSE_RUNTIME crap to the VDB.
> 
> I think the proposal suddenly becomes a lot saner if it's done as
> an EAPI extension, the optional runtime deps and IUSE_RUNTIME
> conditionals are isolated in a new separate variable (perhaps
> SRDEPEND), and IUSE_RUNTIME is not allowed to intersect with IUSE.
> Using a separate SRDEPEND variable means that the package manager
> only has to preserve USE conditionals in the vdb for that one
> variable.


Saner, perhaps, but that would also mean the feature would be more or
less independent of the current USE handling within the PM.

Mind you if it's easier to deal with in the PM then I guess
piggy-backing on the current USE implementation isn't an advantage.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iF4EAREIAAYFAlBkt0oACgkQ2ugaI38ACPAKtQEAgkIJJfyBV20VsKVL8/dPlKF9
B4+SnJGlA+daYTCjXvgA/jq7aNzN8Cuj/sE+S+VWCK5U50vtHX3CqhoeOitgf9Zl
=G5Tc
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Addressing GLEP-62 itself

[Zac Medico]

On 09/27/2012 01:30 PM, Ian Stakenvicius wrote:
> On 27/09/12 04:13 PM, Zac Medico wrote:
>> On 09/27/2012 12:53 PM, Brian Harring wrote:
>>> Bullshit.  This is optional in the sense of embrace/extend
>>> 'optional'; if one PM takes up the new functionality, all have to
>>> switch to writing unfinalized deps to the VDB, and all have to
>>> switch to transfering that IUSE_RUNTIME crap to the VDB.
> 
>> I think the proposal suddenly becomes a lot saner if it's done as
>> an EAPI extension, the optional runtime deps and IUSE_RUNTIME
>> conditionals are isolated in a new separate variable (perhaps
>> SRDEPEND), and IUSE_RUNTIME is not allowed to intersect with IUSE.
>> Using a separate SRDEPEND variable means that the package manager
>> only has to preserve USE conditionals in the vdb for that one
>> variable.
> 
> 
> Saner, perhaps, but that would also mean the feature would be more or
> less independent of the current USE handling within the PM.

Well, the package manager could still treat IUSE_RUNTIME flags as valid
flags for things like USE deps in _other_ packages. So, they don't have
to be entirely independent. The idea is just to limit the scope where
those flags are allowed the package that declares the flags as runtime
flags, so that those runtime flags are only allowed in SRDEPEND.

> Mind you if it's easier to deal with in the PM then I guess
> piggy-backing on the current USE implementation isn't an advantage.

Part of my concern is not just the implementation details, but also
being able to explain/document for communication purposes. If it's such
a mess that it's difficult to communicate, then it sucks for everyone
involved.
-- 
Thanks,
Zac

Re: [gentoo-dev] Addressing GLEP-62 itself

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 4512 bytes --]

On Wed, 26 Sep 2012 03:29:17 -0700
Brian Harring <ferringb@gmail.com> wrote:

> On Wed, Sep 26, 2012 at 08:02:44AM +0200, Micha?? G??rny wrote:
> > On Tue, 25 Sep 2012 12:54:39 -0700
> > Brian Harring <ferringb@gmail.com> wrote:
> > 
> > > On Tue, Sep 25, 2012 at 08:58:07PM +0200, Micha?? G??rny wrote:
> > > > On Tue, 25 Sep 2012 14:47:33 -0400
> > > > Ian Stakenvicius <axs@gentoo.org> wrote:
> > > > 
> > > > > Based on the above I do expect the reference implementation would also
> > > > > need to change.  I expect, for instance, that the PM's
> > > > > metadata-handling would need to occur as normal even though none of
> > > > > the package's phase functions would run, that is, *DEPEND
> > > > > (realistically RDEPEND as that should be the only one affected here,
> > > > > maybe PDEPEND too) and USE/PKGUSE would get updated.  Since portage
> > > > > would not be re-emerging the package from the tree the original ebuild
> > > > > would remain.
> > > > 
> > > > Yes, unless I'm missing something that's the intent. I will re-read
> > > > and update the GLEP a bit sometime this week.
> > > 
> > > There's a fairly strong user interaction component here, along w/ 
> > > potential nastyness for ebuilds (the proposal assume that a flag will 
> > > be toggable in all cases within an ebuild if IUSE_RUNTIME specified; I 
> > > guarantee instances where that fails can be found in the tree if a 
> > > basic audit was done).  Additionally, this *is* useless if it's done 
> > > in a form the UI an't display/handle; Ciaran may bitch about 
> > > REQUIRED_USE's UI (which I knew going in was going to be 
> > > problematic, just to be clear), but he's right on that front.
> 
> ^^^ This point still needs addressing.

What kind of addressing? The user interaction works like usual --
portage lists a bunch of flags, some of them may have additional
hammers or sickles to mean that they will not trigger the rebuild.
Nothing more is required.

What is even more important, there is nothing new to learn
for the user. In fact, he doesn't need anything new from UI. He will
just set the flag like he did 10 years ago.

> > > Additionally, this needs to be thought out how it'll interact with 
> > > eclasses; what stacking, etc.  It doesn't cover the basics there to 
> > > say the least.
> > 
> > The proposal didn't cover eclasses at all. Is there a need to do so or
> > are we chasing some kind of perfection based on filling all unused
> > slots?
> 
> Eclass stacking here matters; if it's stacked, it means ebuilds have 
> to use out of bound (ie, other vars) to tell the eclass when it 
> shouldn't mark a flag as runtime toggable.  If it's not stacked by 
> the pm, then they have to manually stack; that differs from the norm 
> and makes it easier to screwup; however; does allow for them to 
> filter, albeit a slight pain in the ass doing so.
> 
> There's a choice there, and the answer matters, so yes, you should 
> actually have a complete glep before trying to shove it up to the 
> council and extract a vote out of them.  Lest the intention is to just 
> have them kick it back to the curb...

As others have said, we're not stacking it. Using it in eclasses
is whacky and should be avoided. End of the story.

> > > Pretty much, this needs an implementation, partial conversion of the 
> > > tree to demonstrate it.
> > > 
> > > Just to prove that fricking point; if you had tried implementing this, 
> > > a full investigation of what's involved alone, you'd have spotted that 
> > > the core of the proposal is based on a wrong assumption.
> > > 
> > > Portage doesn't write unfinalized DEPEND/RDEPEND/PDEPEND to the VDB.
> > 
> > There's a footnote there, saying:
> > 
> >   The package manager has to ensure that all relevant information is
> >   stored in the installed package metadata.
> 
> Frankly I don't fully buy that you were aware of this issue from the 
> start of the proposal; the wording partially covers it however.  
> Ddoesn't call it out, but via tha req it dumps it on the package 
> manager developers heads to sort it- which already is the case. 
> Binpkgs minimally weren't addressed which is why I still don't think 
> this was actually spotted up front.

We talked about it, don't you remember? That's why I have updated
the spec and put the whole implementation details thing with special
note what needs to be stored in metadata.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] Addressing GLEP-62 itself

[Brian Harring]

On Sat, Sep 29, 2012 at 11:55:22AM +0200, Micha?? G??rny wrote:
> On Wed, 26 Sep 2012 03:29:17 -0700
> Brian Harring <ferringb@gmail.com> wrote:
> 
> > On Wed, Sep 26, 2012 at 08:02:44AM +0200, Micha?? G??rny wrote:
> > > On Tue, 25 Sep 2012 12:54:39 -0700
> > > Brian Harring <ferringb@gmail.com> wrote:
> > > 
> > > > On Tue, Sep 25, 2012 at 08:58:07PM +0200, Micha?? G??rny wrote:
> > > > > On Tue, 25 Sep 2012 14:47:33 -0400
> > > > > Ian Stakenvicius <axs@gentoo.org> wrote:
> > > > > 
> > > > > > Based on the above I do expect the reference implementation would also
> > > > > > need to change.  I expect, for instance, that the PM's
> > > > > > metadata-handling would need to occur as normal even though none of
> > > > > > the package's phase functions would run, that is, *DEPEND
> > > > > > (realistically RDEPEND as that should be the only one affected here,
> > > > > > maybe PDEPEND too) and USE/PKGUSE would get updated.  Since portage
> > > > > > would not be re-emerging the package from the tree the original ebuild
> > > > > > would remain.
> > > > > 
> > > > > Yes, unless I'm missing something that's the intent. I will re-read
> > > > > and update the GLEP a bit sometime this week.
> > > > 
> > > > There's a fairly strong user interaction component here, along w/ 
> > > > potential nastyness for ebuilds (the proposal assume that a flag will 
> > > > be toggable in all cases within an ebuild if IUSE_RUNTIME specified; I 
> > > > guarantee instances where that fails can be found in the tree if a 
> > > > basic audit was done).  Additionally, this *is* useless if it's done 
> > > > in a form the UI an't display/handle; Ciaran may bitch about 
> > > > REQUIRED_USE's UI (which I knew going in was going to be 
> > > > problematic, just to be clear), but he's right on that front.
> > 
> > ^^^ This point still needs addressing.
> 
> What kind of addressing? The user interaction works like usual --
> portage lists a bunch of flags, some of them may have additional
> hammers or sickles to mean that they will not trigger the rebuild.
> Nothing more is required.
> 
> What is even more important, there is nothing new to learn
> for the user. In fact, he doesn't need anything new from UI. He will
> just set the flag like he did 10 years ago.

1) REQUIRED_USE interaction.  That's a rats nest, trust me on that 
one.  If your proposal is to just to have people tweak, get yelled at 
by the pm, tweak, etc, well, on behalf of users, thanks a lot.

2) While hard to comment since your 'updated' glep wasn't fully 
updated- now is self inconsistent (minimally, trace backwards 
compatibility; fix it in full next time)... you're not exactly 
covering how this will go; best I can figure, you just want to shove 
yet another coloring (great for color blind people) or syntax markup 
on emerge -pv style output, somehow indicating runtime toggable or 
not; this is getting picked at because that display already is a 
crapfest and overloaded.

3) You're ignoring cycles here; specifically suggested dep based 
cycles that influence the originating node, and how that is 
represented- this isn't counting representing during --tree mode what 
is brought in where/when because of it.

4) Finally, and more damningly, you're ignoring apps like porthole.  
You need to think long/hard about how *exactly* porthole is going to 
indicate to users what optionals are there- more importantly, what 
those optionals induce/require (that's where it truly gets ugly and 
your lack of dep resolver knowledge, and unwillingness to do a patch 
and learn the basics there become infuriating); || () or blockers w/in 
suggested alone are going to make things painful.


> > > > Additionally, this needs to be thought out how it'll interact with 
> > > > eclasses; what stacking, etc.  It doesn't cover the basics there to 
> > > > say the least.
> > > 
> > > The proposal didn't cover eclasses at all. Is there a need to do so or
> > > are we chasing some kind of perfection based on filling all unused
> > > slots?
> > 
> > Eclass stacking here matters; if it's stacked, it means ebuilds have 
> > to use out of bound (ie, other vars) to tell the eclass when it 
> > shouldn't mark a flag as runtime toggable.  If it's not stacked by 
> > the pm, then they have to manually stack; that differs from the norm 
> > and makes it easier to screwup; however; does allow for them to 
> > filter, albeit a slight pain in the ass doing so.
> > 
> > There's a choice there, and the answer matters, so yes, you should 
> > actually have a complete glep before trying to shove it up to the 
> > council and extract a vote out of them.  Lest the intention is to just 
> > have them kick it back to the curb...
> 
> As others have said, we're not stacking it. Using it in eclasses
> is whacky and should be avoided. End of the story.

It's your proposal there boss.  That's a stupid decision, but as said, 
your proposal to run into the ground if you'd like.

However.  I suggest you actually document that in your proposal that 
it breaks from the stacking norm.  Also, drop the backwards 
compatibility claim at the bottom.  

It was bullshit before, the fact I keep having to picking at this 
means I'm going to start doing so in creative ways; thus I'll just 
quote an exchange from 'the carnival'- it's "pure beeship, beeship, 
not true, false, Beeship!" "oh... bullshit" "Oui!".  Quote wasn't a 
perfect fit, but I'm getting tired of having to use the plain 
'bullshit' response for your emails.


> > > > Pretty much, this needs an implementation, partial conversion of the 
> > > > tree to demonstrate it.
> > > > 
> > > > Just to prove that fricking point; if you had tried implementing this, 
> > > > a full investigation of what's involved alone, you'd have spotted that 
> > > > the core of the proposal is based on a wrong assumption.
> > > > 
> > > > Portage doesn't write unfinalized DEPEND/RDEPEND/PDEPEND to the VDB.
> > > 
> > > There's a footnote there, saying:
> > > 
> > >   The package manager has to ensure that all relevant information is
> > >   stored in the installed package metadata.
> > 
> > Frankly I don't fully buy that you were aware of this issue from the 
> > start of the proposal; the wording partially covers it however.  
> > Ddoesn't call it out, but via tha req it dumps it on the package 
> > manager developers heads to sort it- which already is the case. 
> > Binpkgs minimally weren't addressed which is why I still don't think 
> > this was actually spotted up front.
> 
> We talked about it, don't you remember?

Think you're being cracky there; the point I made there was that your 
proposal didn't address the need for unfinalized across all bulit 
deps- there was no discussion about that.


> That's why I have updated
> the spec and put the whole implementation details thing with special
> note what needs to be stored in metadata.

Not going to give you an inch on this shit anymore; Gleps are derived 
from PEPs; standard to include the cons of a proposal, in this case 
you actually need to quantify/document the perf hit (portage in 
particular) in there.

As for "updated to cover it"... meh.  I read that section; you're 
requirements of the PM imply we can always extract out the relevant 
bits from the original source trees.

Simple example,
encode? ( || ( x? ( blah ) y:=* ) )

with x being an IUSE_RUNTIME toggle.  Try extracting that.  It's a 
PITFA; partial rendering of all non IUSE_RUNTIME for the full tree, 
storing that tree in full, is what's required- that example 
demonstrates why thinking "just extract the parts" is cracked out 
(bluntly: moreso naive, as ciaran said, if you dick around w/ this 
level of the managers code, you're going to find that sort of thing 
intractable because of the rules of the syntax).


Finally, going to fold in a snippet from ciaran/you in a separate 
branch of this thread:

On Sat, Sep 29, 2012 at 08:43:14PM +0200, Micha?? G??rny wrote:
> On Sat, 29 Sep 2012 17:13:14 +0100
> Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
> > You've also still not provided any kind of reference implementation,
> > and your "reference implementation" section is still written with a
> > complete lack of awareness of how dependency resolution is actually
> > done.
>
> I'm sorry I haven't got time yet to write a package manager. I promise
> I will do it as soon as I have time to do so. Thank you for your
> patience.

Good news!

You don't have to write a PM from scratch- just modify portage to 
prototype it.  We realize how annoying it is to have your time wasted, 
thus a patch is enough.  Enjoy!


1) Don't be a tool.  Also, if you want to write a PM, I suggest you do 
so- the timbre of your proposals would likely shift towards sanity (or 
at least things would be quieter while you're off having at it with a 
windmill).
2) PMS requires portage support exist before a patch goes in; you've 
got to do that anyways if you actually think this is going anywhere.
3) The things the PM authors are beating you over the head with all 
map back to realities at the implementation level; if you'd get off 
your ass and do it rather than arguing, you'd be getting further with 
your proposal (or find it nonviable, and do a different proposal).


Honestly, your proposal doesn't feel like "optional deps"; it feels 
like you're just trying to shove in an efficiency hack to avoid 
rebuilds in certain cases, rather than designing a system for exposing 
suggested deps to people.  Efficiency hack doesn't involve a heavy 
UI angle, optiional/suggested deps do.

Either way, I'm done w/ this proposal; you come up w/ a prototype, 
I'll look, till then this is a waste of time.

Should the council/folk care, consider that a strong -1 on my part.
~harring