[gentoo-portage-dev] Dynamic USE dependencies

[gentoo-portage-dev] Dynamic USE dependencies

[Rich Freeman]

Out of curiosity, what is keeping us from having USE flag dependencies
handled dynamically, in the same way that package dependencies are?
If portage can figure out that I need libxml2 installed even if I
don't put it in /var/lib/portage/world, why can't it figure out that I
need it built with USE=icu even if I don't put that in
/etc/portage/package.use?

I was concerned that it might be more work in calculating the
dependencies, but then I was thinking that portage probably already
does all this work just to validate that the current configuration is
still consistent.

I fully appreciate that there could be USE blocks, just as there can
be package blocks, and that resolving these could require hints such
as adding some USE settings to config files, or doing oneshot installs
(perhaps the dynamic configuration would be set up to preserve
whatever is installed unless it conflicts - just as is done with
virtuals today).

-- 
Rich

Re: [gentoo-portage-dev] Dynamic USE dependencies

[Kent Fredric]

[-- Attachment #1: Type: text/plain, Size: 1604 bytes --]

On 3 April 2015 at 05:32, Rich Freeman <rich0@gentoo.org> wrote:

> Out of curiosity, what is keeping us from having USE flag dependencies
> handled dynamically, in the same way that package dependencies are?
> If portage can figure out that I need libxml2 installed even if I
> don't put it in /var/lib/portage/world, why can't it figure out that I
> need it built with USE=icu even if I don't put that in
> /etc/portage/package.use?
>


I'd say its more a concept issue than an application issue.

USE flags often signify a need for code to be recompiled to grant the
feature.

How do you disambiguate between USE flags that *do* need a recompile to
enable their power, and those that *dont* need a recompile to enable their
power.

Or even clarify to portage that, "The older version of X that didn't have
IUSE=foo, actually had feature foo, but just didn't have the use flag" vs
"The older version of X that didn't have IUSE=foo, didnt have feature foo
or the IUSE".

Splitting this logic into an explicit bump kinda avoids the need for some
of these questions.

That last one however I'd like to see improved, because I often see new USE
flags turn up on packages, and I have no idea of knowing "Is this useflag
adding a feature, or exposing an existing one"

"Is this new useflag defaulted on because it already existed, or is it
defaulted on because its a new feature and its awesome"

"Is this new useflag defaulted off because it didnt already exist, or did
it exist and were disabling the feature because its bad"

.... Ad Infinitum.




-- 
Kent

*KENTNL* - https://metacpan.org/author/KENTNL

[-- Attachment #2: Type: text/html, Size: 2737 bytes --]

Re: [gentoo-portage-dev] Dynamic USE dependencies

[Rich Freeman]

On Thu, Apr 2, 2015 at 12:56 PM, Kent Fredric <kentfredric@gmail.com> wrote:
>
> On 3 April 2015 at 05:32, Rich Freeman <rich0@gentoo.org> wrote:
>>
>> Out of curiosity, what is keeping us from having USE flag dependencies
>> handled dynamically, in the same way that package dependencies are?
>> If portage can figure out that I need libxml2 installed even if I
>> don't put it in /var/lib/portage/world, why can't it figure out that I
>> need it built with USE=icu even if I don't put that in
>> /etc/portage/package.use?
>
> I'd say its more a concept issue than an application issue.
>
> USE flags often signify a need for code to be recompiled to grant the
> feature.
>
> How do you disambiguate between USE flags that *do* need a recompile to
> enable their power, and those that *dont* need a recompile to enable their
> power.

Why is this necessary?  If a USE flag changes, just rebuild the application.

>
> Or even clarify to portage that, "The older version of X that didn't have
> IUSE=foo, actually had feature foo, but just didn't have the use flag" vs
> "The older version of X that didn't have IUSE=foo, didnt have feature foo or
> the IUSE".

Do what --newuse does.  If a flag is added/removed, then rebuild the
application.

>
> "Is this new useflag defaulted on because it already existed, or is it
> defaulted on because its a new feature and its awesome"
>
> "Is this new useflag defaulted off because it didnt already exist, or did it
> exist and were disabling the feature because its bad"
>

I'd suggest that the algorithm is:

1.  When installing a new or updated package:
1a. set all the flags in accordance with configuration (explicit
set/unset in profiles, make.conf, package.use)
1b. if these explicitly conflict with package dependencies then fail
with an immediate error (that is, config says USE=foo, and package dep
says atom[-foo] and so on
1c. additionally set flags as needed to satisfy package dependencies
when they don't conflict with explicit configuration

2.  When evaluating the dep tree to see if a package needs to be
rebuilt, just keep the existing USE configuration unless it conflicts
with explicit configuration or a package dependency.

So, the goal would be to avoid rebuilding stuff just because we can.
However, we would rebuild things if a user wants a flag to be set
differently, or if there is now a dependency problem.


-- 
Rich

Re: [gentoo-portage-dev] Dynamic USE dependencies

[Kent Fredric]

[-- Attachment #1: Type: text/plain, Size: 1584 bytes --]

On 3 April 2015 at 06:32, Rich Freeman <rich0@gentoo.org> wrote:

> Why is this necessary?  If a USE flag changes, just rebuild the
> application.
>


Isn't the nature of your proposal,( that is, dynamic deps for USE flags )
inherently "Use flags change, _dont_ rebuild the application" ? :)

It may help to think in terms of : Any ebuild without IUSE="foo" , once
installed, actually has a property of USE_foo=undef

Once an ebuild has IUSE="foo", the installed package then can subsequently
get

USE_foo=y
USE_foo=n


So under that logic, adding a new IUSE implies "Use flag changes" ( either
from state undef to y , or state undef to n ).

We have mechanisms for other ebuilds to declare what USE_foo=undef means in
dependency defaults:

    >=X[foo(+)]   # Assume USE_foo = undef to mean USE_foo = y
    >=X[foo(-)]   # Assume USE_foo = undef to mean USE_foo = n

There's just no mechanism similar to that for an ebuild with CVS revision
02 to declare what USE_foo = undef meant on ebuild with CVS revision 01


Nor is there a way to express what USE_foo=undef meant on  X<$PV



( And subsequently, there is no user visible way /in portage/ for portage
to express the meaning of a new USE flag becoming visible )

So I'm basically having trouble with groking the logic you're proposing of
"add a new use flag" -> "implied change of useflag" -> "rebuild when
useflags change" -> "but don't rebuild for this useflag change using some
kind of magic"

( Mostly due to the implied-change via addition I can't seem to ignore )


-- 
Kent

*KENTNL* - https://metacpan.org/author/KENTNL

[-- Attachment #2: Type: text/html, Size: 3015 bytes --]

Re: [gentoo-portage-dev] Dynamic USE dependencies

[Rich Freeman]

On Thu, Apr 2, 2015 at 2:03 PM, Kent Fredric <kentfredric@gmail.com> wrote:
>
> So I'm basically having trouble with groking the logic you're proposing of
> "add a new use flag" -> "implied change of useflag" -> "rebuild when
> useflags change" -> "but don't rebuild for this useflag change using some
> kind of magic"
>

I'm fine with rebuilding a package anytime the IUSE changes.  I just
don't want to see a package built with +a -b rebuilt as -a +b because
either configuration works, so it gets rebuilt every time you run
emerge.  If multiple configurations work, just use the one that is
already installed.

emerge --newuse is already going to rebuild any package that has IUSE
change anyway.

-- 
Rich

[gentoo-portage-dev] Re: Dynamic USE dependencies

[Duncan]

Rich Freeman posted on Thu, 02 Apr 2015 12:32:41 -0400 as excerpted:

> Out of curiosity, what is keeping us from having USE flag dependencies
> handled dynamically, in the same way that package dependencies are? If
> portage can figure out that I need libxml2 installed even if I don't put
> it in /var/lib/portage/world, why can't it figure out that I need it
> built with USE=icu even if I don't put that in /etc/portage/package.use?

Because USE flags are binary, with "not-yes" implying "no", while world 
set listings are binary, with "not-yes" implying "do it anyway if needed"?

OK, so the USE flag "not-yes" implied "no" /is/ a bit weak, packages omit 
the USE flag if they (or their maintainer) actually require the feature 
and simply hard-require what would otherwise be toggled by the USE flag, 
but by that same token, the very fact that the USE flag exists makes it 
an option for the admin that would toggle the flag, strengthening the USE 
flag's implied-no of the not-yes, and in any case, it's still *far* 
stronger than the "do-it-anyway-if-needed" of simply not listing a 
package in the world set.

Meanwhile, there is of course a way to have "no" for a world listing, put 
it in package.mask.  Similarly, there'a a way to enforce an explicit "no" 
for that implied by a USE "not-yes", by setting USE=-* at the beginning, 
and users who eventually get tired of having to worry about profile 
changes meaning implied USE flag changes, etc, may well eventually set 
it, as I did.  But that doesn't change the basic difference in what "not-
yes" is implied to mean in the two cases.


Changing the implied meaning of "not-yes" to match in both cases could 
certainly be done, but while I'm not opposed if the justification really 
is there, I think there's the potential here for a rather massive change 
in base assumptions, and if that is indeed what's involved, I believe 
it'd call for equally massive justifications.  OTOH, maybe you're 
thinking something a bit more incremental, which would accordingly 
require lower justification.  I'm just a bit worried...


... And I /still/ don't like that --ask, implies that I think it's OK for 
portage to write to my package.* files (even with config-protection), if 
I accidentally hit enter.  When the danger was simply that a package 
merge that would take some time and thus could easily be aborted, that 
was IMO reasonable.  The new situation is IMO far more borderline.  I'd 
be a whole lot more comfortable if some EMERGE_DEFAULTs parameter could 
be set to turn off portage's writing to package.* entirely, while keeping 
the old --ask package-merging /and/ use-flag/mask/keywording SUGGESTION 
behavior.

And I'm worried that the suggestion here is going further down that 
emerge writing its own config path, without (IMO) appropriate safeguards.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-portage-dev] Re: Dynamic USE dependencies

[Rich Freeman]

On Thu, Apr 2, 2015 at 10:10 PM, Duncan <1i5t5.duncan@cox.net> wrote:
> Rich Freeman posted on Thu, 02 Apr 2015 12:32:41 -0400 as excerpted:
>
>> Out of curiosity, what is keeping us from having USE flag dependencies
>> handled dynamically, in the same way that package dependencies are? If
>> portage can figure out that I need libxml2 installed even if I don't put
>> it in /var/lib/portage/world, why can't it figure out that I need it
>> built with USE=icu even if I don't put that in /etc/portage/package.use?
>
> Because USE flags are binary, with "not-yes" implying "no", while world
> set listings are binary, with "not-yes" implying "do it anyway if needed"?

That doesn't need to be true.  Not listing a flag in your config can
just mean "do it anyway if needed."  Package USE dependencies already
work this way - you can depend on a flag being set, on it being unset,
or not mention the flag at all which means the package doesn't care.

> Changing the implied meaning of "not-yes" to match in both cases could
> certainly be done, but while I'm not opposed if the justification really
> is there, I think there's the potential here for a rather massive change
> in base assumptions, and if that is indeed what's involved, I believe
> it'd call for equally massive justifications.

There is no question that it would be a change in behavior.  However,
I wouldn't anticipate a lot of changes.

If you stuck -* in your make.conf then this change would have no
affect at all, since you've explicitly set the configuration of every
use flag.

Anything already in package.use would still stick, since that is also
configuration.  If you wiped your package.use clean then there would
be a possible change, but that is something the user has control over,
and presumably for them it is a change for the better if they're
making it.

Things would still default to their profile/package defaults as they
already do if you don't stick anything in your config files.  However,
future package installs that require a USE change would just make the
change if you didn't explicitly bar that configuration.  It wouldn't
modify your config files.

>
> And I'm worried that the suggestion here is going further down that
> emerge writing its own config path, without (IMO) appropriate safeguards.

I think this is exactly the opposite.  It would get rid of the need to
have portage go modifying package.use at all in many cases.  You'd
still need to modify configuration (perhaps with automatic help) if
you had explicitly set something which conflicts with what you want to
install (such as USE=-* globally).  However, for users who just go
with the profile defaults they wouldn't have to stick nearly as much
stuff in package.use just to change a flag setting they never
expressed a personal preference for one way or the other.

The current configuration forces you to use config files to capture
settings you care about, and also ones you don't actually care about,
and unless you're careful you'll have trouble telling these settings
apart later.  It is like sticking every installed package in your
world file.

-- 
Rich

[gentoo-portage-dev] Re: Dynamic USE dependencies

[Duncan]

Rich Freeman posted on Thu, 02 Apr 2015 22:26:03 -0400 as excerpted:

> If you stuck -* in your make.conf then this change would have no
> affect at all, since you've explicitly set the configuration of every
> use flag.

That (and package.use still sticking) eases my mind considerably.

> The current configuration forces you to use config files to capture
> settings you care about, and also ones you don't actually care about,
> and unless you're careful you'll have trouble telling these settings
> apart later.  It is like sticking every installed package in your
> world file.

That comparison is quite persuasive, indeed. =:^)

Thanks!  I understand your idea much better, now, and (cautiously) agree. 
=:^)


(Tho FWIW, I guess I'm a careful one.  I use -* and put non-global USE 
flags in make.conf too if possible, and review USE flags for all new 
packages and changes, so everything there is cared about for one reason 
or another.  Package.use thus contains only individual package 
exceptions, and I comment those with both a date and why they /are/ 
exceptions to the otherwise global policy, so if the only justification 
is because package X requires it, that's in the comment.  Make.conf's 
USE= setting does still accumulate unannotated stale flags over time, but 
I just went thru and verified all USE flags were still used recently, 
deleting the ones that equery hasuse didn't raise a hit on, and 
justifying either by-name or by equery uses every remaining entry, so 
everything there is verified there for a reason now, too.)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-portage-dev] Re: Dynamic USE dependencies

[Brian Dolbec]

On Fri, 3 Apr 2015 06:38:48 +0000 (UTC)
Duncan <1i5t5.duncan@cox.net> wrote:

> Rich Freeman posted on Thu, 02 Apr 2015 22:26:03 -0400 as excerpted:
> 
> > If you stuck -* in your make.conf then this change would have no
> > affect at all, since you've explicitly set the configuration of
> > every use flag.
> 
> That (and package.use still sticking) eases my mind considerably.
> 
> > The current configuration forces you to use config files to capture
> > settings you care about, and also ones you don't actually care
> > about, and unless you're careful you'll have trouble telling these
> > settings apart later.  It is like sticking every installed package
> > in your world file.
> 
> That comparison is quite persuasive, indeed. =:^)
> 
> Thanks!  I understand your idea much better, now, and (cautiously)
> agree. =:^)
> 
> 
> (Tho FWIW, I guess I'm a careful one.  I use -* and put non-global
> USE flags in make.conf too if possible, and review USE flags for all
> new packages and changes, so everything there is cared about for one
> reason or another.  Package.use thus contains only individual package 
> exceptions, and I comment those with both a date and why they /are/ 
> exceptions to the otherwise global policy, so if the only
> justification is because package X requires it, that's in the
> comment.  Make.conf's USE= setting does still accumulate unannotated
> stale flags over time, but I just went thru and verified all USE
> flags were still used recently, deleting the ones that equery hasuse
> didn't raise a hit on, and justifying either by-name or by equery
> uses every remaining entry, so everything there is verified there for
> a reason now, too.)
> 

This is a reminder to you all...  since enalyze is little known to
users.

enalyze application in gentoolkit I made does installed db analysis.
The analyze submodule creates detailed reports about use flags (and
other stuff) usage.  The rebuild module can create new pkg.* files for
you to maintain the installed state.  Allowing you to easily make
make.conf USE= changes or profile changes and have your pkg.* files
fixed to match.  Of course detailed comments it does not do ;)  

plus it saves the new file to your user space, you can do what you like
with them from there.  Editing your detail commented files to match,
etc..  stright swap,...



-- 
Brian Dolbec <dolsen>

[gentoo-portage-dev] Re: Dynamic USE dependencies

[Duncan]

Brian Dolbec posted on Thu, 02 Apr 2015 23:59:06 -0700 as excerpted:

> This is a reminder to you all...  since enalyze is little known to
> users.

Thanks for the hint.  I'd never heard of enalyze before but it sounds 
useful.  I'll investigate. =:^)

Please consider writing up a tips-n-tricks section feature for enalyze in 
an upcoming gentoo news, too.  That should bring it rather more exposure 
among the power users at least, who will propagate the knowledge thru the 
lists and forums, etc. I know that's the way I've come across several 
tools I now keep in my virtual toolbox, over the years. =:^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-portage-dev] Re: Dynamic USE dependencies

[Brian Dolbec]

On Fri, 3 Apr 2015 11:52:39 +0000 (UTC)
Duncan <1i5t5.duncan@cox.net> wrote:

> Brian Dolbec posted on Thu, 02 Apr 2015 23:59:06 -0700 as excerpted:
> 
> > This is a reminder to you all...  since enalyze is little known to
> > users.
> 
> Thanks for the hint.  I'd never heard of enalyze before but it sounds 
> useful.  I'll investigate. =:^)
> 
> Please consider writing up a tips-n-tricks section feature for
> enalyze in an upcoming gentoo news, too.  That should bring it rather
> more exposure among the power users at least, who will propagate the
> knowledge thru the lists and forums, etc. I know that's the way I've
> come across several tools I now keep in my virtual toolbox, over the
> years. =:^)
> 


Been there, done that... ;)

https://blogs.gentoo.org/news/2014/03/

And there are several main #gentoo contributors that know about it.
I keep coming across people that don't know about it
still.
-- 
Brian Dolbec <dolsen>

[gentoo-portage-dev] Re: Dynamic USE dependencies

[Duncan]

Brian Dolbec posted on Fri, 03 Apr 2015 06:31:27 -0700 as excerpted:

> On Fri, 3 Apr 2015 11:52:39 +0000 (UTC)
> Duncan <1i5t5.duncan@cox.net> wrote:
> 
>> Brian Dolbec posted on Thu, 02 Apr 2015 23:59:06 -0700 as excerpted:
>> 
>> > enalyze is little known to users.
>> 
>> I'd never heard of enalyze before
>> 
>> Please consider writing up a tips-n-tricks section feature for enalyze
>> in an upcoming gentoo news
> 
> Been there, done that... ;)
> 
> https://blogs.gentoo.org/news/2014/03/

<brown paper bag>

I "read" that issue too.  But I think I was running on about four hours 
sleep in three days (OK, maybe six hours, eight if you count dozing on 
the bus and lunch break... obviously not enough!) when I did...  Given 
the evidence, I guess "read" isn't the appropriate word...

I /slept/ /thru/ that issue???  I /sleep-read/ that issue???

Anyway, thanks. =:^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-portage-dev] Dynamic USE dependencies

[Zac Medico]

On 04/02/2015 09:32 AM, Rich Freeman wrote:
> Out of curiosity, what is keeping us from having USE flag dependencies
> handled dynamically, in the same way that package dependencies are?
> If portage can figure out that I need libxml2 installed even if I
> don't put it in /var/lib/portage/world, why can't it figure out that I
> need it built with USE=icu even if I don't put that in
> /etc/portage/package.use?

It's already able to adjust USE automatically via autounmask support. We
can add support for for it to do this without the need to alter user
config files. We could cache the automatic USE settings somewhere, so
that commands like 'ebuild' would be able to build things with the
correct USE without having to process any dependencies.

> I was concerned that it might be more work in calculating the
> dependencies, but then I was thinking that portage probably already
> does all this work just to validate that the current configuration is
> still consistent.

Considering that autounmask is enabled by default, it's already doing
the work.
-- 
Thanks,
Zac

[gentoo-portage-dev] Re: Dynamic USE dependencies

[Martin Vaeth]

Zac Medico <zmedico@gentoo.org> wrote:
> On 04/02/2015 09:32 AM, Rich Freeman wrote:
>> Out of curiosity, what is keeping us from having USE flag dependencies
>> handled dynamically, in the same way that package dependencies are?
>
> It's already able to adjust USE automatically via autounmask support.

The problem is that once it is in your package.use
(no matter whether added manually or with autounmask),
it will stay there forever if the user does not analyze the
dependencies manually eventually.
And this is different from the dependency handling.

The most obvious example (which presumably also Rich had in mind)
is the case of abi_x86_32:

Assume that you have package foo in your world file which
(in version foo-1) depends on package bar. In version foo-2,
which appears some years later, it no longer depends on package
bar (but e.g. on package bazooka, instead).
In this case, "emerge --depclean" will eventually unmerge bar.

This works nicely.

However, if foo-1 depends on bar[abi_x86_32(-)] and something
else depends on bar[abi_x86_64] then, after upgrading to foo-2,
most users wil keep compiling bar with ABI_X86="32 64" forever,
because "emerge --depclean" (or any other tool) does not tell
them that now they might remove abi_x86_32 from package bar.

In fact, such a tool *cannot* exist, currently:
There is no way to distinguish from package.use what is
there because the user *wanted* this feature and what is there
"only" because of a (possibly temporary) USE-dependency for
another package.

In other words: The difference between "wanted" and "needed"
(reflected usually in the difference between the packages
in "world" and those in "/var/db/pkg") is not visible in
package.use, in principle.

So far, this was never a problem, because the number of
flags which were there because they were needed (and not,
because they were wanted) was rather low.
But with the arrival of ABI_X86="32" the situation has changed;
100-200 "needed" flags are now normal on many systems.

One suggestion around this problem would be to use different
directories for these two types of use-flags, say
"package.use" and "package.use.needed".
Perhaps it would make sense that portage --autounmask
writes only to "package.use.needed", und that there is
a "cleanup" command which removes all data from
package.use.needed except for that strictly necessary
to resolve the dependencies of the running system
(analogously like "emerge --depclean" currently removes
unneeded packages).

Re: [gentoo-portage-dev] Re: Dynamic USE dependencies

[Rich Freeman]

On Sun, Apr 5, 2015 at 11:47 AM, Martin Vaeth <martin@mvath.de> wrote:
> One suggestion around this problem would be to use different
> directories for these two types of use-flags, say
> "package.use" and "package.use.needed".

I still think we need a better long-term solution, but the workaround
is simpler than that.

I try to keep files in directories for package.keywords and
package.use.  One starts with a z and is a dumping ground for
auto-whatever.  The other files are where I try to remember to store
personal preferences.  So, in theory at any time I can wipe the z-file
and run emerge and watch it wrestle with re-creating it.

However, I think a cleaner solution would be better...

-- 
Rich

[gentoo-portage-dev] Re: Dynamic USE dependencies

[Martin Vaeth]

Rich Freeman <rich0@gentoo.org> wrote:
> On Sun, Apr 5, 2015 at 11:47 AM, Martin Vaeth <martin@mvath.de> wrote:
>> One suggestion around this problem would be to use different
>> directories for these two types of use-flags, say
>> "package.use" and "package.use.needed".
>
> I still think we need a better long-term solution, but the workaround
> is simpler than that.
>
> I try to keep files in directories for package.keywords and
> package.use.  One starts with a z and is a dumping ground for
> auto-whatever.

So you use package.use/z instead of package.use.needed
as in my suggestion. (In fact, I do currently a similar
thing, only that I use a file named zzz_autounmask,
because this is even later in the alphabet).

I think it would be more transparent to the user if
really a dedicated directory/file would be used and not
some which "by accident" is the last in the alphabet.

Anyway, this is a minor issue.
The main problem is how to cleanup this directory/file
(no matter how it is called).

Currently, the only way to do this, is to remove this
file and to hope that portage can recreate it.
Unfortunately, quite often portage is not able to do this.
The problem is at least NP hard, so probably any
backtracking value just is too low.

If portage would use the old values as a hint, how
conflicts might possibly be resolved, perhaps this
algorithm might be improved.

> However, I think a cleaner solution would be better...

++

My suggestion is something in between - less invasive
(and, in particular, less time consuming)
than your suggestion to recalculate the USE-settings
with every emerge, but more automatic than the current state.

With my suggestion the USE-settings have to be recalculated
only if a special command is used (or - as currently -
if resolving is not possible without changing USE-flags).

Probably other approaches are possible, too.

Re: [gentoo-portage-dev] Re: Dynamic USE dependencies

[Rich Freeman]

On Mon, Apr 6, 2015 at 8:02 AM, Martin Vaeth <martin@mvath.de> wrote:
> My suggestion is something in between - less invasive
> (and, in particular, less time consuming)
> than your suggestion to recalculate the USE-settings
> with every emerge, but more automatic than the current state.

Keep in mind that keeping track of past decisions made by portage does
not require user-editable config files in /etc.  It just requires a
cache of some kind, much as we do with installed packages/etc.

That said, portage still has to spend time basically re-validating the
consistency of the entire system because we allow the use of overlays
and other situations that don't guarantee that portage will have some
kind of consistent pre-calculated depgraph handed to it.  If we
required all repositories to have some kind of pre-generated cache in
them and ensured that this was always up-to-date and better controlled
the kinds of dependency changes we made, then maybe there might be an
opportunity to offload some of the work to the repository level
instead of doing it on every Gentoo system.  Still, unless we banned
overlays I'm not sure how much even this would buy you, since you'd
have many of these that need to be merged somehow.

From Zac's email and other discussions in the past it seems like we're
basically committed to doing all these calculations all the time
anyway, so we shouldn't be too shy about taking advantage of them.

-- 
Rich

[gentoo-portage-dev] Re: Dynamic USE dependencies

[Martin Vaeth]

Rich Freeman <rich0@gentoo.org> wrote:
>
> Keep in mind that keeping track of past decisions made by portage does
> not require user-editable config files in /etc.

Yes, but you might not always agree with portage's decisions,
and the resolution might be non-unique.
Although the user might always influence it with /etc/portage/package.use
it might be better to keep this transparent to the user - therefore,
I suggested a directory in /etc and not in /var
(Moreover, a cache would not make sense, since the
previous decision could always be trivially calculated from /var/db/pkg,
and the user-selected state of the USE-flag for the corresponding
package. You could not check the validity of such a cache, since
you cannot check whether USE or /etc/portage/package.use has changed)

> That said, portage still has to spend time basically re-validating the
> consistency of the entire system

*Validating* consistency is not time-consuming. Looking for possibilities
to unset use-flags is a much more time-consuming story.
If everything is consistent or can easily be resolved, you hardly need
any backtracking...

Re: [gentoo-portage-dev] Re: Dynamic USE dependencies

[Rich Freeman]

On Wed, Apr 8, 2015 at 3:45 AM, Martin Vaeth <martin@mvath.de> wrote:
> Rich Freeman <rich0@gentoo.org> wrote:
>>
>> Keep in mind that keeping track of past decisions made by portage does
>> not require user-editable config files in /etc.
>
> Yes, but you might not always agree with portage's decisions,
> and the resolution might be non-unique.

How is this different with USE flags vs package installs?  The
satisfaction of virtuals could have many possible solutions.  We make
it deterministic by defaulting to the first option listed for the
first package that happens to get installed.  If you install a series
of packages in various orders from a fresh install, you could get
different packages installed to satisfy virtuals.

The solution we provide for package installs is that the user can just
emerge a dependency manually if they have a preference, and then
portage will stick with it unless there is a conflict.

For dynamic USE flags I've already proposed two mechanisms to give the
user control:
1.  They can STILL populate /etc/portage/package.use and make explicit
choices, which portage will follow.
2.  They could manually install a package with one-time specified USE
flags, and portage would stick with these as long as they don't create
a conflict.

Why do we need another mechanism to control what flags a package gets
installed with other than these two, such as making more detailed
cache data user-editable?

-- 
Rich

[gentoo-portage-dev] Re: Dynamic USE dependencies

[Martin Vaeth]

Rich Freeman <rich0@gentoo.org> wrote:
>
> 1.  They can STILL populate /etc/portage/package.use
> 2.  They could manually install a package with one-time specified USE
>
> Why do we need another mechanism to control what flags a package gets
> installed

We do not *need* it. By why reject it if you can get it for free
by just making the it transparent to the user?
The user should be able to decide e.g. when to clean it up
and select perhaps e.g. only a certain classes which should be
cleaned up (e.g. by removing all/certain cpu_flags_x86 and
letting them regenerate).