[gentoo-dev] Dirt: To shove under the rug or not shove under the rug? (aka another round of USE_EXPAND)

[gentoo-dev] Dirt: To shove under the rug or not shove under the rug? (aka another round of USE_EXPAND)

[Jason Stubbs]

Hello all,

To shove under the rug:

Bug 62001: Have portage QA checks consider USE_EXPAND
Bug 70648: QA warnings about USE_EXPAND-derived use variables
Bug 101998: Portage shouldn't warn on missing IUSE for USE_EXPAND

To not shove under the rug:

Bug 23826: Give more visibility to ebuilds variables (ALSA_CARDS, etc.)
Bug 65012: emerge should display USE_EXPAND flags when using --verbose and 
           --pretend (or --ask)

The above are only when searching for USE_EXPAND in the summary field.


So let's step back a bit and look at the purpose of the QA check first. What 
it does is check whether an ebuild is utilizing a USE flag that it has not 
declared. Why does this matter for QA? It means that users are not informed 
of what options affect the package. It also means the same for portage, which 
will become more and more important as time goes on.

Removing the QA check while the QA problem still exists is, in my mind, just 
plain ludicrous. Removing the QA check once the QA problem is fixed doesn't 
make much sense to me either. However, adjusting the check where necessary to 
match the fix is reasonable.

So what needs to be done to fix it? Well, what is the purpose of USE_EXPAND? 
Put simply, it is to allow the user to select one or more features of a 
package from a list of choices. How is this different to USE flags? The 
choices all pertain to one aspect of the package(s).

Beyond that is where we start to run into the gray murkiness that has been 
created due to having no policy on USE_EXPAND.

1) What to do if nothing is set?
2) What to do if an invalid value is set?

   a) install everything
   b) install nothing
   c) install some predefined default
   d) install some default based on USE flags
   e) die and tell the user to make a choice

3) How to ebuild behave regarding a USE_EXPAND variable?

   a) install everything chosen that is valid
   b) install only the first that is chosen that is valid

Of these, 1) and 2) absolutely must be whittled down to one standard. 
Preferably, 3) should follow one standard as well. Not following one standard 
will simply lead to users thinking, "but that's not what I wanted..!" It will 
also lead portage to do needless recompiles due to the information available 
being limited.

Next, storing the information of what choices are valid. If it can be 
guaranteed that all packages supporting a variable (LINGUAS for example) have 
exactly the same list of choices in all cases, storing the choice list in a 
global file is acceptable. If not, each package absolutely must list what 
choices are available for it. Not doing so means the flow may head into 2) in 
the above list even when the user has set a valid choice for a different 
package. Again, it's against the user's expectations.

Anybody not caring enough to fix this, please don't respond with "wah! work!?" 
You've dug your own hole...

--
Jason Stubbs
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Dirt: To shove under the rug or not shove under the rug? (aka another round of USE_EXPAND)

[Diego 'Flameeyes' Pettenò]

[-- Attachment #1: Type: text/plain, Size: 1415 bytes --]

On Tuesday 27 September 2005 11:23, Jason Stubbs wrote:
> So what needs to be done to fix it? Well, what is the purpose of
> USE_EXPAND? Put simply, it is to allow the user to select one or more
> features of a package from a list of choices. How is this different to USE
> flags? The choices all pertain to one aspect of the package(s).
The way ELIBC, KERNEL, USERLAND are used, is instead something different. They 
don't allow users to select what they want, they allow profiles to declare 
what they are created for.
If some user changes one of these variables, he's *really* screwed up, as they 
change quite a few things in the ebuilds (for example, if kernel is not 
linux, kdelibs doesn't build support for dnotify, gamin for inotify, and a 
few more options in the way).

I think at least these three variables should be hidden from users, as they 
should not mean anything to them.
In alternative, there was the proposal of a use.force file, that would allow 
to force some flags on and use that instead of the use-expanded variables, 
but currently it doesn't seem to be created and the QA notice problem is 
still not solved, those flags should be forced by some profiles and masked by 
others, as they are not intended to be changed by users.

-- 
Diego "Flameeyes" Pettenò
Gentoo Developer - http://dev.gentoo.org/~flameeyes/
(Gentoo/FreeBSD, Video, Gentoo/AMD64, Sound, PAM)

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Dirt: To shove under the rug or not shove under the rug? (aka another round of USE_EXPAND)

[Jason Stubbs]

On Tuesday 27 September 2005 18:38, Diego 'Flameeyes' Pettenò wrote:
> On Tuesday 27 September 2005 11:23, Jason Stubbs wrote:
> > So what needs to be done to fix it? Well, what is the purpose of
> > USE_EXPAND? Put simply, it is to allow the user to select one or more
> > features of a package from a list of choices. How is this different to
> > USE flags? The choices all pertain to one aspect of the package(s).
>
> The way ELIBC, KERNEL, USERLAND are used, is instead something different.
> They don't allow users to select what they want, they allow profiles to
> declare what they are created for.

Which leads me to the one thing I didn't say but feel strongest about.. What 
is the real point of USE_EXPAND? What can/does it do that USE flags do not?

> If some user changes one of these variables, he's *really* screwed up, as
> they change quite a few things in the ebuilds (for example, if kernel is
> not linux, kdelibs doesn't build support for dnotify, gamin for inotify,
> and a few more options in the way).

This doesn't quite apply to cross compiling and such, but in general yeah.

> I think at least these three variables should be hidden from users, as they
> should not mean anything to them.

Similar to "build" and "bootstrap"? Note, these aren't hidden either but if 
the ELIBC and friends should be hidden those should be hidden too.

> In alternative, there was the proposal of a use.force file, that would
> allow to force some flags on and use that instead of the use-expanded
> variables, but currently it doesn't seem to be created and the QA notice
> problem is still not solved, those flags should be forced by some profiles
> and masked by others, as they are not intended to be changed by users.

And we're back to USE flags again... ;)

--
Jason Stubbs

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Dirt: To shove under the rug or not shove under the rug? (aka another round of USE_EXPAND)

[Diego 'Flameeyes' Pettenò]

[-- Attachment #1: Type: text/plain, Size: 2093 bytes --]

On Tuesday 27 September 2005 12:12, Jason Stubbs wrote:
> Which leads me to the one thing I didn't say but feel strongest about..
> What is the real point of USE_EXPAND? What can/does it do that USE flags do
> not?
They are forced by the profile, as we don't want users to go away WITHOUT them 
when they are needed. That's the main part of it.
Then there's the fact that I can test for a generic BSD libc using [[ ${ELIBC} 
== *BSD ]], too.

> This doesn't quite apply to cross compiling and such, but in general yeah.
Cross-compiling with different libc can be possible (glibc->uclibc) but I'm 
not sure on what extent. Currently trying to have cross-compiling between 
different kernels is a no-go.

The proposal of domains or whatever they are called that Brian talked about 
some time ago when he proposed BDEPEND would probably fix this issue, tho.

> Similar to "build" and "bootstrap"? Note, these aren't hidden either but if
> the ELIBC and friends should be hidden those should be hidden too.
But they have sort of meaning to users, for example with newuse, and does not 
screw your system around, or at least not completely (one can build a few 
packages with build useflag active and still have a working system, at the 
end).
Changing the ELIBC/KERNEL/USERLAND is like using a sparc profile on an x86 ...

Also, most ebuilds does not use the above variables in a complete way, they 
usually check for a certain content (GNU userland, FBSD libc, Linux Kernel).
Saying for example that kdelibs uses kernel_linux can make people think that 
kdelibs works ONLY for Linux kernel, while that's not true at all.
They are special features or workarounds that does not concern users at all.

> And we're back to USE flags again... ;)
As they are the only way to change the dependencies, yeah, always USE is what 
we need. For the most of the uses, variables are fine, but for dependency we 
use them use-expanded.

-- 
Diego "Flameeyes" Pettenò
Gentoo Developer - http://dev.gentoo.org/~flameeyes/
(Gentoo/FreeBSD, Video, Gentoo/AMD64, Sound, PAM)

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Dirt: To shove under the rug or not shove under the rug? (aka another round of USE_EXPAND)

[Jason Stubbs]

Before you reply to this.. Can you enlighten me on what the solution to the 
problem is that you are heading toward? I'm having trouble seeing what your 
real point is.

On Tuesday 27 September 2005 19:41, Diego 'Flameeyes' Pettenò wrote:
> On Tuesday 27 September 2005 12:12, Jason Stubbs wrote:
> > Which leads me to the one thing I didn't say but feel strongest about..
> > What is the real point of USE_EXPAND? What can/does it do that USE flags
> > do not?
>
> They are forced by the profile, as we don't want users to go away WITHOUT
> them when they are needed. That's the main part of it.

I wasn't referring specifically to the recent profile additions, but to 
USE_EXPAND in general. However, this is a mis-usage of USE_EXPAND. At least, 
USE_EXPAND was not designed for this purpose.

$ USE="-*" emerge info | grep USE
USE="amd64 userland_GNU kernel_linux elibc_glibc"
$ USE="-*" KERNEL="" ELIBC="" emerge info | grep USE
USE="amd64 userland_GNU"

ARCH and USERLAND are the only ones this can't be done for as portage is 
hardcoded to set them. Thus, they are the only ones you can be guaranteed of.

> Then there's the fact that I can test for a generic BSD libc using [[
> ${ELIBC} == *BSD ]], too.

This would fail in the above case as well. It is not very good for ebuilds to 
test for the values of arbitrary variables. If some information needs to be 
passed to ebuilds, there needs to be an explicit contract with portage.

> > Similar to "build" and "bootstrap"? Note, these aren't hidden either but
> > if the ELIBC and friends should be hidden those should be hidden too.
>
> But they have sort of meaning to users, for example with newuse, and does
> not screw your system around, or at least not completely (one can build a
> few packages with build useflag active and still have a working system, at
> the end).
> Changing the ELIBC/KERNEL/USERLAND is like using a sparc profile on an x86
> ...

bootstrap - !!internal use only!! DO NOT SET THIS FLAG YOURSELF!
build - !!internal use only!! DO NOT SET THIS FLAG YOURSELF!

Saying that setting these flags probably won't do much harm to the user's 
system doesn't mean that they should be exposed.

> Also, most ebuilds does not use the above variables in a complete way, they
> usually check for a certain content (GNU userland, FBSD libc, Linux
> Kernel). Saying for example that kdelibs uses kernel_linux can make people
> think that kdelibs works ONLY for Linux kernel, while that's not true at
> all. They are special features or workarounds that does not concern users
> at all.

So what are you really saying? Just that some USE_EXPAND usages shouldn't be 
exposed to users? That some usages should not be overridable by users? 
Neither USE_EXPAND or regular USE flags offer either of these features at the 
moment, other than all USE_EXPAND variables being non-exposed.

> > And we're back to USE flags again... ;)
>
> As they are the only way to change the dependencies, yeah, always USE is
> what we need. For the most of the uses, variables are fine, but for
> dependency we use them use-expanded.

Variables are _not_ fine. I would think it should be clear to everybody by now 
that ebuilds can not pick random things from the computer they are installing 
on to define how they will build.

# LINGUAS="" emerge kde-i18n | grep '*'
* You must define a LINGUAS environment variable that contains a list
* of the language codes for which languages you would like to install.
* Look at the LANGS variable inside the ebuild to see the list of
* available languages.
* e.g.: LINGUAS="sv de pt"

"Look at the LANGS variable inside the ebuild"!?
This is what this thread is about.

--
Jason Stubbs

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Dirt: To shove under the rug or not shove under the rug? (aka another round of USE_EXPAND)

[Diego 'Flameeyes' Pettenò]

[-- Attachment #1: Type: text/plain, Size: 899 bytes --]

On Tuesday 27 September 2005 14:51, Jason Stubbs wrote:
> Variables are _not_ fine. I would think it should be clear to everybody by
> now that ebuilds can not pick random things from the computer they are
> installing on to define how they will build.
If variables are not fine, so can't be find thigns like profile arch, isn't 
it? But that was a solution chosen for the merge of x86 and amd64...
And the same goes with quite everything defined on profile level, like the 
others.

I still think that to users it shouldn't give a damn about ELIBC, USERLAND and 
KERNEL, as they can just be mislead by thinking that they can change anything 
about that..

How we can test for and condition dependencies with special profiles others 
than this way?

-- 
Diego "Flameeyes" Pettenò - http://dev.gentoo.org/~flameeyes/
Gentoo/ALT lead, Gentoo/FreeBSD, Video, AMD64, Sound, PAM, KDE

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Dirt: To shove under the rug or not shove under the rug? (aka another round of USE_EXPAND)

[Kito]

On Sep 27, 2005, at 8:44 AM, Diego 'Flameeyes' Pettenò wrote:

> On Tuesday 27 September 2005 14:51, Jason Stubbs wrote:
>> Variables are _not_ fine. I would think it should be clear to  
>> everybody by
>> now that ebuilds can not pick random things from the computer they  
>> are
>> installing on to define how they will build.
>
> I still think that to users it shouldn't give a damn about ELIBC,  
> USERLAND and
> KERNEL, as they can just be mislead by thinking that they can  
> change anything
> about that..
>
> How we can test for and condition dependencies with special  
> profiles others
> than this way?

[Portage devs please don't throw rocks at me]

My impression of the userland, elibc, and kernel use expanded vars is  
it was a quick way to sidestep some of the issues with GLEP22... it  
would seem the full keywords have still not been taken advantage of.  
 From the ebuild perspective, if the profile has a keyword of x86- 
fbsd-bsd-fbsd, there is no clean way to just do a conditional based  
on a 'Keyword Fragment' as there are obviously namespace collisions.

Ideal to me would be syntax something like:

   kernel !fbsd && foo
   libc glibc || bar
   userland darwin && boof


Probably not feasible, but seems the cleanest in the context of  
ebuilds IMHO.

--Kito
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Dirt: To shove under the rug or not shove under the rug? (aka another round of USE_EXPAND)

[Stephen Bennett]

Kito wrote:
> My impression of the userland, elibc, and kernel use expanded vars is  
> it was a quick way to sidestep some of the issues with GLEP22...

That was pretty much my thinking when I put them there, yes. It's 
nothing like an ideal solution, but current portage didn't (and to my 
knowledge still doesn't) allow for anything better.
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Dirt: To shove under the rug or not shove under the rug? (aka another round of USE_EXPAND)

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 1021 bytes --]

On Tue, Sep 27, 2005 at 09:07:00AM -0500, Kito wrote:
> [Portage devs please don't throw rocks at me]
All out of rocks :/

> My impression of the userland, elibc, and kernel use expanded vars is  
> it was a quick way to sidestep some of the issues with GLEP22... it  
> would seem the full keywords have still not been taken advantage of.  
> From the ebuild perspective, if the profile has a keyword of x86- 
> fbsd-bsd-fbsd, there is no clean way to just do a conditional based  
> on a 'Keyword Fragment' as there are obviously namespace collisions.
> 
> Ideal to me would be syntax something like:
> 
>   kernel !fbsd && foo
>   libc glibc || bar
>   userland darwin && boof

Bash side of it's pretty easy to implement however needed, the problem 
here (and why imo USE_EXPAND came into existance) is getting those 
conditional nodes into the dep syntax evaluation without trampling 
other use vars.

And *-*-*-fbsd as a conditional node sucks in depends, is ugly, oh so 
ugly.  :)
~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Dirt: To shove under the rug or not shove under the rug? (aka another round of USE_EXPAND)

[Jason Stubbs]

On Tuesday 27 September 2005 22:44, Diego 'Flameeyes' Pettenò wrote:
> On Tuesday 27 September 2005 14:51, Jason Stubbs wrote:
> > Variables are _not_ fine. I would think it should be clear to everybody
> > by now that ebuilds can not pick random things from the computer they are
> > installing on to define how they will build.
>
> If variables are not fine, so can't be find thigns like profile arch, isn't
> it? But that was a solution chosen for the merge of x86 and amd64...
> And the same goes with quite everything defined on profile level, like the
> others.

ARCH has been specifically protected so that whatever the profile sets it what 
the ebuilds see, regardless of the user's environment (except for
/etc/portage/profile/make.defaults)

> I still think that to users it shouldn't give a damn about ELIBC, USERLAND
> and KERNEL, as they can just be mislead by thinking that they can change
> anything about that..

I'm not suggesting that it should be shown. I'm stating that USE_EXPAND'ed 
variables in general should be shown to the users. Whether there's a profile 
control file to hide specific variables is another question.

> How we can test for and condition dependencies with special profiles others
> than this way?

Figure out what you want and then get it supported. Right now, the only 
variables that you're guaranteed to see in an ebuild function are ARCH and 
the variables that you find in the ebuild(5) man page.

--
Jason Stubbs

-- 
gentoo-dev@gentoo.org mailing list

[gentoo-dev] Re: Dirt: To shove under the rug or not shove under the rug? (aka another round of USE_EXPAND)

[Duncan]

Diego 'Flameeyes' Pettenò posted
<200509271242.05438@enterprise.flameeyes.is-a-geek.org>, excerpted below, 
on Tue, 27 Sep 2005 12:41:50 +0200:

> Saying for example that kdelibs uses kernel_linux can make people think that 
> kdelibs works ONLY for Linux kernel, while that's not true at all.

I see your point on most of your post, but this simply doesn't ring true.

What is one of the PRIMARY things that is drilled into the user's heads in
all the documentation, forums, lists, etc, concerning USE flags?  Is it
not that they configure for OPTIONAL factors, that non-optional
dependencies are merged unconditionally, so that USE flags have no effect
on them?

I agree that USERLAND and the like shouldn't be changed by an ordinary
user under ordinary circumstances.  use.force and the like could be very
useful in this sort of situation, particularly since who knows /what/ sort
of vars a user may have set up in his environment, quite apart from
portage use, and these could do /very/ /bad/ things to emerges (which is,
I believe, one of Jason's points, we need to either make users  aware of
these or isolate portage from normal vars that might have unintended
consequences, failure to do so is a QA issue). 

However, were an ebuild to spit out kernel_linux among the USE flags and
the like, anybody that takes it as you suggest they might, that it works
ONLY for the  Linux kernel, can be said to know little enough about
portage and how it works, specifically about how it treats optional vs
non-optional dependencies, that they are a danger to themselves and the
system they are attempting to maintain!  Anybody NOT understanding that
USE flags (and USE_EXPAND if we start displaying that as well) govern
optional, NOT hard dependencies, while trying to work with portage, is
ALREADY a danger to his system, to the point he shouldn't be making
decisions about merges and the like in the /first/ place, because he
doesn't understand enough about the process to do it in a logically
coherent manner, and should REALLY spend a bit more time with the docs (or
away from the booze or whatever may be clouding his judgement), before he
starts emerging stuff.

So... for these sorts of things, I'd say treat them like build and 
boostrap, display them, but document the consequences of messing with them
equally well.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman in
http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html


-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Dirt: To shove under the rug or not shove under the rug? (aka another round of USE_EXPAND)

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 1109 bytes --]

On Tue, 2005-09-27 at 19:12 +0900, Jason Stubbs wrote:
> On Tuesday 27 September 2005 18:38, Diego 'Flameeyes' Pettenò wrote:
> > On Tuesday 27 September 2005 11:23, Jason Stubbs wrote:
> > > So what needs to be done to fix it? Well, what is the purpose of
> > > USE_EXPAND? Put simply, it is to allow the user to select one or more
> > > features of a package from a list of choices. How is this different to
> > > USE flags? The choices all pertain to one aspect of the package(s).
> >
> > The way ELIBC, KERNEL, USERLAND are used, is instead something different.
> > They don't allow users to select what they want, they allow profiles to
> > declare what they are created for.
> 
> Which leads me to the one thing I didn't say but feel strongest about.. What 
> is the real point of USE_EXPAND? What can/does it do that USE flags do not?

As far as I can tell, it was created to keep people from adding a huge
number of USE flags to IUSE on a package.  I can't see any other reason
for it.

-- 
Chris Gianelloni
Release Engineering - Strategic Lead
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Dirt: To shove under the rug or not shove under the rug? (aka another round of USE_EXPAND)

[Donnie Berkholz]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Chris Gianelloni wrote:
| On Tue, 2005-09-27 at 19:12 +0900, Jason Stubbs wrote:
|>Which leads me to the one thing I didn't say but feel strongest
about.. What
|>is the real point of USE_EXPAND? What can/does it do that USE flags do
not?
|
|
| As far as I can tell, it was created to keep people from adding a huge
| number of USE flags to IUSE on a package.  I can't see any other reason
| for it.

See my other email.

Thanks,
Donnie
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (GNU/Linux)

iD8DBQFDOWbhXVaO67S1rtsRAvkMAKDHiibxHRrRRKwFFfY26A0SmjPyzgCfcBB1
xGzhvHzmm0j/AdIQiOheu44=
=Aq2Q
-----END PGP SIGNATURE-----
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Dirt: To shove under the rug or not shove under the rug? (aka another round of USE_EXPAND)

[Thomas de Grenier de Latour]

On Tue, 27 Sep 2005 18:23:25 +0900
Jason Stubbs <jstubbs@gentoo.org> wrote:

> Bug 23826: Give more visibility to ebuilds variables (ALSA_CARDS,
> etc.)

Just to make it clear if it wasn't: although some comments made
it derive toward USE_EXPANDed vars, the above bug was at the
begining about advertising any kind of unusual vars that influence
ebuilds.  Whether they are USE_EXPANDed or not is an internal
technical detail that the user should not really care about.  I
don't even know whether ALSA_CARDS is USE_EXPANDed for instance (at
least, the alsa-driver ebuild doesn't use it as such), but what
matters to me is that i know i have to list sound drivers for my
system there.  As for LIRC_OPTS, i'm sure it is not USE_EXPANDed,
but still it lacks documentation for people who don't read the
ebuilds before emerging them.

So i don't think this bug is really relevant in this discussion.
It has more to do with the idea (which was in the air at some
point iirc) of whitelisting vars that should be allowed to enter the
ebuild environment, and the possibility for ebuilds to declare some
addition to the standard whitelist.

--
TGL.
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Dirt: To shove under the rug or not shove under the rug? (aka another round of USE_EXPAND)

[Jason Stubbs]

On Tuesday 27 September 2005 19:54, Thomas de Grenier de Latour wrote:
> On Tue, 27 Sep 2005 18:23:25 +0900
>
> Jason Stubbs <jstubbs@gentoo.org> wrote:
> > Bug 23826: Give more visibility to ebuilds variables (ALSA_CARDS,
> > etc.)
>
> So i don't think this bug is really relevant in this discussion.
> It has more to do with the idea (which was in the air at some
> point iirc) of whitelisting vars that should be allowed to enter the
> ebuild environment, and the possibility for ebuilds to declare some
> addition to the standard whitelist.

Maybe not exactly about USE_EXPAND but it hits directly on the point of the QA 
check that many are asking be removed.

--
Jason Stubbs
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Dirt: To shove under the rug or not shove under the rug? (aka another round of USE_EXPAND)

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 2482 bytes --]

On Tue, 2005-09-27 at 18:23 +0900, Jason Stubbs wrote:
> 1) What to do if nothing is set?

Install everything.  If everything cannot be installed, due to
incompatibilities, then die.

> 2) What to do if an invalid value is set?
> 
>    a) install everything

Install everything.  If everything cannot be installed, due to
incompatibilities, then die.

> 3) How to ebuild behave regarding a USE_EXPAND variable?
> 
>    a) install everything chosen that is valid

Yes.

>    b) install only the first that is chosen that is valid
> 
> Of these, 1) and 2) absolutely must be whittled down to one standard. 
> Preferably, 3) should follow one standard as well. Not following one standard 
> will simply lead to users thinking, "but that's not what I wanted..!" It will 
> also lead portage to do needless recompiles due to the information available 
> being limited.
> 
> Next, storing the information of what choices are valid. If it can be 
> guaranteed that all packages supporting a variable (LINGUAS for example) have 
> exactly the same list of choices in all cases, storing the choice list in a 
> global file is acceptable. If not, each package absolutely must list what 
> choices are available for it. Not doing so means the flow may head into 2) in 
> the above list even when the user has set a valid choice for a different 
> package. Again, it's against the user's expectations.

Let's take an example of this...  Neverwinter Nights.  Currently, it
installs the language packs via LINGUAS.  If nothing is selected via
LINGUAS, then it installs English, which is considered the default.
Unfortunately, even trying to add -linguas_fr to package.use, still
results in the French language pack being installed over the English.  I
honestly do not know how to correct this.  I see a couple things that
would be needed.  For one, things in USE_EXPAND would need to be
negate-able in package.use.  The problem with NWN is that only one
language pack may be installed at any given time due to them providing
the same files.

You can see where I've been working with this on bug #106789.

> Anybody not caring enough to fix this, please don't respond with "wah! work!?" 
> You've dug your own hole...

I would love to see this fixed.  I'm guessing that this would mean
defining all of the USE_EXPAND capabilities in IUSE, correct?

-- 
Chris Gianelloni
Release Engineering - Strategic Lead
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Dirt: To shove under the rug or not shove under the rug? (aka another round of USE_EXPAND)

[Thomas de Grenier de Latour]

On Tue, 27 Sep 2005 08:35:43 -0400
Chris Gianelloni <wolf31o2@gentoo.org> wrote:

> Unfortunately, even trying to add -linguas_fr to package.use,
> still results in the French language pack being installed over
> the English.  

This reminds me bug #104573: it was the same problem where some
LINGUAS="fr" users didn't want to have manpages-fr installed. The
workaround used there was to add the "nls" USE flag. This way you
separate the "do you want some translations?" question from the
"what translations do you want?" question.

But sure, it's more a workaround than an ideal solution.

> For one, things in USE_EXPAND would need to be negate-able in
> package.use.  

It may be a problem if an ebuild relies on the USE flags being
coherent with the env var itself.  I don't have example of such
ebuilds in mind though, but i guess it could happen that, for
instance, on one hand the LINGUAS var is used by autotools to
choose the locales to install, and on the other hand the linguas_*
flags are used to add some help files or spell dicts: this could
result in some kind of half-translated apps, maybe even broken
apps, i don't know...

--
TGL. 
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Dirt: To shove under the rug or not shove under the rug? (aka another round of USE_EXPAND)

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 1869 bytes --]

On Tue, 2005-09-27 at 15:07 +0200, Thomas de Grenier de Latour wrote:
> On Tue, 27 Sep 2005 08:35:43 -0400
> Chris Gianelloni <wolf31o2@gentoo.org> wrote:
> 
> > Unfortunately, even trying to add -linguas_fr to package.use,
> > still results in the French language pack being installed over
> > the English.  
> 
> This reminds me bug #104573: it was the same problem where some
> LINGUAS="fr" users didn't want to have manpages-fr installed. The
> workaround used there was to add the "nls" USE flag. This way you
> separate the "do you want some translations?" question from the
> "what translations do you want?" question.

Actually, I like this idea.  It isn't perfect, but it allows one to add
-nls for the package in package.use to turn off translations.  It still
doesn't completely solve the problem, however.

The user in question has LINGUAS="en_US fr" but wants the English
translation.  This would be fine for this user, but what if the user had
wanted the French translation and not English?

> But sure, it's more a workaround than an ideal solution.
> 
> > For one, things in USE_EXPAND would need to be negate-able in
> > package.use.  
> 
> It may be a problem if an ebuild relies on the USE flags being
> coherent with the env var itself.  I don't have example of such
> ebuilds in mind though, but i guess it could happen that, for
> instance, on one hand the LINGUAS var is used by autotools to
> choose the locales to install, and on the other hand the linguas_*
> flags are used to add some help files or spell dicts: this could
> result in some kind of half-translated apps, maybe even broken
> apps, i don't know...

I hadn't considered this situation, as I was stuck thinking of binary
apps (nwn and civ-ctp).

-- 
Chris Gianelloni
Release Engineering - Strategic Lead
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Dirt: To shove under the rug or not shove under the rug? (aka another round of USE_EXPAND)

[Jason Stubbs]

On Tuesday 27 September 2005 21:35, Chris Gianelloni wrote:
> On Tue, 2005-09-27 at 18:23 +0900, Jason Stubbs wrote:
> > 1) What to do if nothing is set?
> > 2) What to do if an invalid value is set?
>
> Install everything.  If everything cannot be installed, due to
> incompatibilities, then die.

This causes trouble though as it has in your case below.

> > Of these, 1) and 2) absolutely must be whittled down to one standard.
> > Preferably, 3) should follow one standard as well. Not following one
> > standard will simply lead to users thinking, "but that's not what I
> > wanted..!" It will also lead portage to do needless recompiles due to the
> > information available being limited.
> >
> > Next, storing the information of what choices are valid. If it can be
> > guaranteed that all packages supporting a variable (LINGUAS for example)
> > have exactly the same list of choices in all cases, storing the choice
> > list in a global file is acceptable. If not, each package absolutely must
> > list what choices are available for it. Not doing so means the flow may
> > head into 2) in the above list even when the user has set a valid choice
> > for a different package. Again, it's against the user's expectations.
>
> Let's take an example of this...  Neverwinter Nights.  Currently, it
> installs the language packs via LINGUAS.  If nothing is selected via
> LINGUAS, then it installs English, which is considered the default.
> Unfortunately, even trying to add -linguas_fr to package.use, still
> results in the French language pack being installed over the English.  I
> honestly do not know how to correct this.  I see a couple things that
> would be needed.  For one, things in USE_EXPAND would need to be
> negate-able in package.use.  The problem with NWN is that only one
> language pack may be installed at any given time due to them providing
> the same files.

The current implementation extends the USE flags with those derived from 
USE_EXPAND after everything has been cascaded. It was likely done this way 
out of ease of implementation, but now that USERLAND and such are using it...

> I would love to see this fixed.  I'm guessing that this would mean
> defining all of the USE_EXPAND capabilities in IUSE, correct?

That is one solution that isn't hard to support, can be done quickly(?) on the 
ebuild side and is already implemented (though unreleased) on the portage 
side. However, it really seems to me that USE_EXPAND just doesn't expand well 
(pardon the pun ;). Thinking about it, USE has its own set of issues as well. 
There are number of cases in the tree of "you can have A or B, but not both" 
and "you must pick at least A or B". There's also "if A isn't set then B and 
C aren't used".

Long term solution would seem to be to completely revamp the 
USE/USE_EXPAND/ARCH-USERLAND-* system to clear out at least the existing 
pitfalls and hopefully not fall into new ones (and of course maintain 
backward compatibility). If there's agreement in that, it might not be the 
best bet to put a whole lot of work into getting USE_EXPAND working 
"correctly"; that is to say, getting things following the above rules is 
perhaps not worth the effort.

As it stands, it would seem package.env support, a way to express dependencies 
between USE flags, a way to express USE_EXPAND choices, a way to express 
limits on those choices and a way to protect profile variables (just to begin 
with) are all required to get the current system working a bit more smoothly. 
Seems very much like a bandaid on a bandaid on a ...

Bah.. I'm mumbling. I really don't mind what the solution is in the short term 
as long as the USE_EXPAND choices get delivered to the users and that how 
those choices work is clear. If the solution I've already done a patch for 
isn't acceptable to everyone, find a solution that is and I'll support it 
accordingly. I just don't want it to sit around and gather more dust.

--
Jason Stubbs
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Dirt: To shove under the rug or not shove under the rug? (aka another round of USE_EXPAND)

[Donnie Berkholz]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jason Stubbs wrote:
| Removing the QA check while the QA problem still exists is, in my
mind, just
| plain ludicrous. Removing the QA check once the QA problem is fixed
doesn't
| make much sense to me either. However, adjusting the check where
necessary to
| match the fix is reasonable.
|
| So what needs to be done to fix it? Well, what is the purpose of
USE_EXPAND?
| Put simply, it is to allow the user to select one or more features of a
| package from a list of choices. How is this different to USE flags? The
| choices all pertain to one aspect of the package(s).

What I have done in my ebuilds using USE_EXPAND is add extra IUSE-like
variables, for example:

IUSE_VIDEO_CARDS="radeon sis mga"
IUSE_INPUT_DEVICES="synaptics wacom"

for `use video_cards_sis` etc..

This would allow for possible QA checks. Potentially, portage could look
through and display things at --verbose like:

[ebuild  N    ] x11-base/x11-drm-20050807 +dga -nls VIDEO_CARDS="+sis
+mga -radeon" 538 kB [2]

USE_EXPAND allows grouping a similar set of USE flags together to
indicate their relatedness, and it is particularly useful when there are
both USE_EXPAND and non-USE_EXPAND flags on the same ebuild. It is also
useful when a number of packages make use of the same group of
USE_EXPAND variables.

The main problem with USE_EXPAND right now is that people just don't
realize it exists, so they don't take advantage of it. We need a way to
tell them, such as the --verbose output I suggested and some more
documentation.

Thanks,
Donnie
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (GNU/Linux)

iD8DBQFDOWbZXVaO67S1rtsRAulnAKCkevKbDcd1ib2S9W3m8Ila797KGgCeLWNS
ZY+Fat5UX9sjVA0r+yB159U=
=Kysv
-----END PGP SIGNATURE-----
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Dirt: To shove under the rug or not shove under the rug? (aka another round of USE_EXPAND)

[Jason Stubbs]

On Wednesday 28 September 2005 00:35, Donnie Berkholz wrote:
> What I have done in my ebuilds using USE_EXPAND is add extra IUSE-like
> variables, for example:
>
> IUSE_VIDEO_CARDS="radeon sis mga"
> IUSE_INPUT_DEVICES="synaptics wacom"
>
> for `use video_cards_sis` etc..
>
> This would allow for possible QA checks. Potentially, portage could look
> through and display things at --verbose like:
>
> [ebuild  N    ] x11-base/x11-drm-20050807 +dga -nls VIDEO_CARDS="+sis
> +mga -radeon" 538 kB [2]

This output is exactly what my patch gives, but it currently works by 
inspecting IUSE. However, it should be possible to indirectly pull the 
information from the above flags instead. Adding support for a 
USE_EXPAND_HIDDEN var should cover USERLAND and friends as well (although 
those would still need to be defined in IUSE_USERLAND="...")

> The main problem with USE_EXPAND right now is that people just don't
> realize it exists, so they don't take advantage of it. We need a way to
> tell them, such as the --verbose output I suggested and some more
> documentation.

This still doesn't cover the discrepencies in behaviour between different 
packages though. How to handle that in the short term? einfo and/or ewarn?

--
Jason Stubbs
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Dirt: To shove under the rug or not shove under the rug? (aka another round of USE_EXPAND)

[Jason Stubbs]

[-- Attachment #1: Type: text/plain, Size: 982 bytes --]

On Wednesday 28 September 2005 10:23, Jason Stubbs wrote:
> On Wednesday 28 September 2005 00:35, Donnie Berkholz wrote:
> > What I have done in my ebuilds using USE_EXPAND is add extra IUSE-like
> > variables, for example:
> >
> > IUSE_VIDEO_CARDS="radeon sis mga"
> > IUSE_INPUT_DEVICES="synaptics wacom"
> >
> > for `use video_cards_sis` etc..
> >
> > This would allow for possible QA checks. Potentially, portage could look
> > through and display things at --verbose like:
> >
> > [ebuild  N    ] x11-base/x11-drm-20050807 +dga -nls VIDEO_CARDS="+sis
> > +mga -radeon" 538 kB [2]
>
> This output is exactly what my patch gives, but it currently works by
> inspecting IUSE. However, it should be possible to indirectly pull the
> information from the above flags instead. Adding support for a
> USE_EXPAND_HIDDEN var should cover USERLAND and friends as well (although
> those would still need to be defined in IUSE_USERLAND="...")

<warcraft voice>Job's done!</warcraft voice>

[-- Attachment #2: verbose-USE_EXPAND-support.patch --]
[-- Type: text/x-diff, Size: 6110 bytes --]

diff -uNr 2.0-original/bin/ebuild.sh 2.0/bin/ebuild.sh
--- 2.0-original/bin/ebuild.sh	2005-09-26 11:48:16.000000000 +0900
+++ 2.0/bin/ebuild.sh	2005-09-28 11:32:47.000000000 +0900
@@ -129,7 +129,7 @@
 	local x
 	
 	# Make sure we have this USE flag in IUSE
-	if ! hasq "${u}" ${IUSE} ${E_IUSE} && ! hasq "${u}" ${PORTAGE_ARCHLIST} selinux; then
+	if ! hasq "${u}" ${IUSE} ${E_IUSE} ${IUSE_EXPAND} && ! hasq "${u}" ${PORTAGE_ARCHLIST} selinux; then
 		echo "QA Notice: USE Flag '${u}' not in IUSE for ${CATEGORY}/${PF}" >&2
 	fi
 
@@ -1870,6 +1870,17 @@
 		fi
 		;;
 	depend)
+		# Create IUSE_EXPAND from child variables
+		for EXPAND_VAR in $USE_EXPAND; do
+			EXPAND_NAME="IUSE_${EXPAND_VAR}"
+			EXPAND_VAL="${!EXPAND_NAME}"
+			[ -z "${EXPAND_VAL}" ] && continue
+			LOWER_VAR="`echo ${EXPAND_VAR} | awk '{print tolower($0)}'`"
+			for VAL in ${EXPAND_VAL}; do
+				IUSE_EXPAND="${IUSE_EXPAND} ${LOWER_VAR}_${VAL}"
+			done
+		done
+
 		export SANDBOX_ON="0"
 		set -f
 
@@ -1899,13 +1910,13 @@
 		echo `echo "$PDEPEND"`     >> $dbkey
 		echo `echo "$PROVIDE"`     >> $dbkey
 		echo `echo "${EAPI:-0}"`   >> $dbkey
+		echo `echo "${IUSE_EXPAND}"` >> $dbkey
 		echo `echo "$UNUSED_01"`   >> $dbkey
 		echo `echo "$UNUSED_02"`   >> $dbkey
 		echo `echo "$UNUSED_03"`   >> $dbkey
 		echo `echo "$UNUSED_04"`   >> $dbkey
 		echo `echo "$UNUSED_05"`   >> $dbkey
 		echo `echo "$UNUSED_06"`   >> $dbkey
-		echo `echo "$UNUSED_07"`   >> $dbkey
 		set +f
 		#make sure it is writable by our group:
 		exit 0
diff -uNr 2.0-original/bin/emerge 2.0/bin/emerge
--- 2.0-original/bin/emerge	2005-09-27 13:16:09.000000000 +0900
+++ 2.0/bin/emerge	2005-09-28 12:09:49.000000000 +0900
@@ -1466,6 +1466,31 @@
 
 		if "--verbose" in myopts:
 			overlays = string.split(portage.settings['PORTDIR_OVERLAY'])
+			use_expand = portage.settings["USE_EXPAND"].lower().split()
+			use_expand_hidden = portage.settings["USE_EXPAND_HIDDEN"].lower().split()
+			for exp in use_expand[:]:
+				if exp in use_expand_hidden:
+					use_expand.remove(exp)
+
+			def create_use_string(iuse, cur_use, old_use, masked_use):
+				usestr=""
+				for flag in iuse:
+					usechange=""
+					if old_use:
+						if (flag in old_use and flag not in cur_use) or (flag not in old_use and flag in cur_use):
+							usechange="*"
+
+					if flag in cur_use:
+						if usechange == "*":
+							substr = green("+"+flag)
+						else:
+							substr = red("+"+flag)
+					elif flag in masked_use:
+						substr = blue("(-"+flag+")")
+					else:
+						substr = blue("-"+flag)
+					usestr += substr + usechange + " "
+				return usestr
 
 		if "--tree" in myopts:
 			mylist.reverse()
@@ -1558,18 +1583,29 @@
 					try:
 						if x[0] == "binary":
 							iuse_split = string.split(portage.db["/"]["bintree"].dbapi.aux_get(x[2],["IUSE"])[0])
+							iuse_expand_split = portage.db["/"]["bintree"].dbapi.aux_get(x[2],["IUSE_EXPAND"])[0].split()
 						elif x[0] == "ebuild":
 							iuse_split = string.split(portage.portdb.aux_get(x[2],["IUSE"])[0])
+							iuse_expand_split = portage.portdb.aux_get(x[2],["IUSE_EXPAND"])[0].split()
 						else:
 							iuse_split = []
+							iuse_expand_split = []
 					except SystemExit, e:
 						raise # Needed else can't exit
 					except:
 						portage.writemsg("!!! Error getting IUSE (report this to bugs.gentoo.org)\n")
 						portage.writemsg("!!! %s\n" % x)
 						iuse_split = []
+						iuse_expand_split = []
+
+					iuse_split = portage.unique_array(iuse_split)
 					iuse_split.sort()
-					old_use=None
+					iuse_expand_split = portage.unique_array(iuse_expand_split)
+					iuse_expand_split.sort()
+
+					cur_use = self.applied_useflags[x[2]]
+
+					old_use = []
 					if myoldbest:
 						pkg=myoldbest
 					else:
@@ -1581,24 +1617,25 @@
 							raise # Needed else can't exit
 						except:
 							pass
-					iuse=""
-					now_use=self.applied_useflags[x[2]]
-					for ebuild_iuse in portage_util.unique_array(iuse_split):
-						usechange=""
-						if old_use:
-							if (old_use.count(ebuild_iuse) and not now_use.count(ebuild_iuse)) or (not old_use.count(ebuild_iuse) and now_use.count(ebuild_iuse)):
-								usechange="*"
-
-						if ebuild_iuse in self.applied_useflags[x[2]]:
-							if usechange == "*":
-								iuse=green("+"+ebuild_iuse)
-							else:
-								iuse=red("+"+ebuild_iuse)
-						elif ebuild_iuse in portage.settings.usemask:
-							iuse=blue("(-"+ebuild_iuse+")")
-						else:
-							iuse=blue("-"+ebuild_iuse)
-						verboseadd+=iuse+usechange+" "
+
+					verboseadd += create_use_string(iuse_split, cur_use, old_use, portage.settings.usemask)
+
+					for expvar in use_expand:
+						expiuse = []
+						expcur = []
+						expold = []
+						expmask = []
+						for flag in iuse_expand_split:
+							if flag.startswith(expvar+"_"):
+								expiuse.append(flag[len(expvar)+1:])
+								if flag in cur_use:
+									expcur.append(expiuse[-1])
+								if flag in old_use:
+									expold.append(expiuse[-1])
+								if flag in portage.settings.usemask:
+									expmask.append(expiuse[-1])
+						if expiuse:
+							verboseadd += expvar.upper()+'="'+create_use_string(expiuse,expcur,expold,expmask).strip()+'" '
 
 					# size verbose
 					mysize=0
diff -uNr 2.0-original/pym/portage.py 2.0/pym/portage.py
--- 2.0-original/pym/portage.py	2005-09-26 11:48:15.000000000 +0900
+++ 2.0/pym/portage.py	2005-09-28 12:06:36.000000000 +0900
@@ -1392,7 +1392,7 @@
 				if self.has_key(var):
 					for x in string.split(self[var]):
 						mystr = string.lower(var)+"_"+x
-						if mystr not in usesplit:
+						if mystr not in usesplit and mystr not in self.usemask:
 							usesplit.append(mystr)
 
 		# Pre-Pend ARCH variable to USE settings so '-*' in env doesn't kill arch.
@@ -5112,8 +5112,8 @@
 	'RESTRICT',  'HOMEPAGE',  'LICENSE',   'DESCRIPTION',
 	'KEYWORDS',  'INHERITED', 'IUSE',      'CDEPEND',
 	'PDEPEND',   'PROVIDE', 'EAPI',
-	'UNUSED_01', 'UNUSED_02', 'UNUSED_03', 'UNUSED_04',
-	'UNUSED_05', 'UNUSED_06', 'UNUSED_07',
+	'IUSE_EXPAND', 'UNUSED_01', 'UNUSED_02', 'UNUSED_03',
+	'UNUSED_04', 'UNUSED_05', 'UNUSED_06',
 	]
 auxdbkeylen=len(auxdbkeys)
 

Re: [gentoo-dev] Dirt: To shove under the rug or not shove under the rug? (aka another round of USE_EXPAND)

[Jason Stubbs]

On Wednesday 28 September 2005 00:35, Donnie Berkholz wrote:
> IUSE_VIDEO_CARDS="radeon sis mga"
> IUSE_INPUT_DEVICES="synaptics wacom"

So, my patch (even though it works) puts these flags into an IUSE_EXPAND 
variable and would require an upgrade on the CVS server to get correct cache 
generation for users.

What are the exact reasons for not wanting to put the expanded flags directly 
into IUSE? If it's just a matter of the horrid display existing tools would 
give, the functionality can go in and IUSE updated after the functional 
versions are stabled. Are there any reasons beyond that?

--
Jason Stubbs
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Dirt: To shove under the rug or not shove under the rug? (aka another round of USE_EXPAND)

[Donnie Berkholz]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jason Stubbs wrote:
| What are the exact reasons for not wanting to put the expanded flags
directly
| into IUSE? If it's just a matter of the horrid display existing tools
would
| give, the functionality can go in and IUSE updated after the functional
| versions are stabled. Are there any reasons beyond that?

It's useful to group things logically like this. You don't need to worry
about forgetting to add in variables like:

IUSE_VIDEO_CARDS="foo bar"
IUSE="${IUSE_VIDEO_CARDS} baz bop"

Particularly when you start doing things in eclasses as well as ebuilds,
this gets worrisome. It's also more aesthetically pleasing. And it
creates a logical parallel with the user-set variables in make.conf.

In addition, it saves IUSE from having super-long flags:

IUSE="video_cards_foo video_cards_bar video_cards_baz ... " -- picture
that repeated 10 times. It's just plain ugly and overly long.

Most importantly IMHO, it suggests that users should set
USE="video_cards_foo" instead of VIDEO_CARDS="foo".

Thanks,
Donnie
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (GNU/Linux)

iD8DBQFDOhnaXVaO67S1rtsRAvT2AJoDRphFhWJaPc1G0OuISixe1jE9AgCeIZnQ
YS1fBH92GQOG6tDreeP93sw=
=XK5J
-----END PGP SIGNATURE-----
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Dirt: To shove under the rug or not shove under the rug? (aka another round of USE_EXPAND)

[Jason Stubbs]

On Wednesday 28 September 2005 13:19, Donnie Berkholz wrote:
> Jason Stubbs wrote:
> | What are the exact reasons for not wanting to put the expanded flags 
> | directly into IUSE? If it's just a matter of the horrid display existing 
> | tools would give, the functionality can go in and IUSE updated after the 
> | functional versions are stabled. Are there any reasons beyond that?  
>
> It's useful to group things logically like this. You don't need to worry
> about forgetting to add in variables like:
>
> IUSE_VIDEO_CARDS="foo bar"
> IUSE="${IUSE_VIDEO_CARDS} baz bop"
>
> Particularly when you start doing things in eclasses as well as ebuilds,
> this gets worrisome. It's also more aesthetically pleasing. And it
> creates a logical parallel with the user-set variables in make.conf.

iuse_expand()
{
        VAR="`echo $1 | awk '{print tolower($0)}'`"
        shift
        for FLAG in "$@"; do
                IUSE="${IUSE} ${VAR}_${FLAG}"
        done
        export IUSE
}

IUSE="baz bop"
iuse_expand VIDEO_CARDS foo bar

> In addition, it saves IUSE from having super-long flags:
>
> IUSE="video_cards_foo video_cards_bar video_cards_baz ... " -- picture
> that repeated 10 times. It's just plain ugly and overly long.

It doesn't save anything because internally it needs to be done that way 
regardless. The only difference between the two patches is that one has 
IUSE="..." and the other has IUSE_EXPAND="...".

> Most importantly IMHO, it suggests that users should set
> USE="video_cards_foo" instead of VIDEO_CARDS="foo".

Users don't see it either way, so I can't see how it suggests anything.

--
Jason Stubbs
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Dirt: To shove under the rug or not shove under the rug? (aka another round of USE_EXPAND)

[Donnie Berkholz]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jason Stubbs wrote:
| On Wednesday 28 September 2005 13:19, Donnie Berkholz wrote:
| IUSE="baz bop"
| iuse_expand VIDEO_CARDS foo bar

This has the ring of hack to me, if it's something to be used in ebuilds.

|>In addition, it saves IUSE from having super-long flags:
|>
|>IUSE="video_cards_foo video_cards_bar video_cards_baz ... " -- picture
|>that repeated 10 times. It's just plain ugly and overly long.
|
|
| It doesn't save anything because internally it needs to be done that way
| regardless. The only difference between the two patches is that one has
| IUSE="..." and the other has IUSE_EXPAND="...".

Right on, but portage is a lot more internal than ebuilds.

|>Most importantly IMHO, it suggests that users should set
|>USE="video_cards_foo" instead of VIDEO_CARDS="foo".
|
|
| Users don't see it either way, so I can't see how it suggests anything.

Let me narrow that to users who look at ebuilds to figure out wtf is
going on with USE_EXPAND because they have no idea how to use it.

Thanks,
Donnie
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (GNU/Linux)

iD8DBQFDOjbGXVaO67S1rtsRAobfAJ9MVftU8dklnq6t+MFaMrmHHbFtiQCg5o06
Lh26STJWnP8Ph6gJT5k9sus=
=kv7o
-----END PGP SIGNATURE-----
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Dirt: To shove under the rug or not shove under the rug? (aka another round of USE_EXPAND)

[Jason Stubbs]

On Wednesday 28 September 2005 15:23, Donnie Berkholz wrote:
> Jason Stubbs wrote:
> | On Wednesday 28 September 2005 13:19, Donnie Berkholz wrote:
> | IUSE="baz bop"
> | iuse_expand VIDEO_CARDS foo bar
>
> This has the ring of hack to me, if it's something to be used in ebuilds.

USE_EXPAND was a hack to begin with. It's broken by design. If you don't like 
hacks, don't use USE_EXPAND at all.

Brian just noted another reason for using IUSE instead of IUSE_EXPAND; using 
IUSE_EXPAND would mean the entire cache would be invalidated any time that 
USE_EXPAND changes.

> |>In addition, it saves IUSE from having super-long flags:
> |>
> |>IUSE="video_cards_foo video_cards_bar video_cards_baz ... " -- picture
> |>that repeated 10 times. It's just plain ugly and overly long.
> |
> | It doesn't save anything because internally it needs to be done that way
> | regardless. The only difference between the two patches is that one has
> | IUSE="..." and the other has IUSE_EXPAND="...".
>
> Right on, but portage is a lot more internal than ebuilds.

iuse_expand as implemented above is just as internal as IUSE_VIDEO_CARDS 
(which is an undocumented variable mind you).

> |>Most importantly IMHO, it suggests that users should set
> |>USE="video_cards_foo" instead of VIDEO_CARDS="foo".
> |
> | Users don't see it either way, so I can't see how it suggests anything.
>
> Let me narrow that to users who look at ebuilds to figure out wtf is
> going on with USE_EXPAND because they have no idea how to use it.

Which is exactly what this thread is looking to stop users from needing to do.

--
Jason Stubbs
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Dirt: To shove under the rug or not shove under the rug? (aka another round of USE_EXPAND)

[Jason Stubbs]

[-- Attachment #1: Type: text/plain, Size: 994 bytes --]

On Wednesday 28 September 2005 12:58, Jason Stubbs wrote:
> On Wednesday 28 September 2005 00:35, Donnie Berkholz wrote:
> > IUSE_VIDEO_CARDS="radeon sis mga"
> > IUSE_INPUT_DEVICES="synaptics wacom"
>
> So, my patch (even though it works) puts these flags into an IUSE_EXPAND
> variable and would require an upgrade on the CVS server to get correct
> cache generation for users.
>
> What are the exact reasons for not wanting to put the expanded flags
> directly into IUSE? If it's just a matter of the horrid display existing
> tools would give, the functionality can go in and IUSE updated after the
> functional versions are stabled. Are there any reasons beyond that?

And a patch to do it the IUSE way. Note the size difference below even though 
both patches end up with the same output for the end user. Also note that 
this way doesn't require an upgrade on the CVS server.

$ wc -c verbose-IUSE*
3698 verbose-IUSE-support.patch
6110 verbose-IUSE_EXPAND-support.patch

--
Jason Stubbs

[-- Attachment #2: verbose-IUSE-support.patch --]
[-- Type: text/x-diff, Size: 3698 bytes --]

diff -uNr 2.0-original/bin/emerge 2.0/bin/emerge
--- 2.0-original/bin/emerge	2005-09-28 12:24:10.000000000 +0900
+++ 2.0/bin/emerge	2005-09-28 13:17:00.000000000 +0900
@@ -1466,6 +1466,28 @@
 
 		if "--verbose" in myopts:
 			overlays = string.split(portage.settings['PORTDIR_OVERLAY'])
+			use_expand = portage.settings["USE_EXPAND"].lower().split()
+			use_expand_hidden = portage.settings["USE_EXPAND_HIDDEN"].lower().split()
+
+			def create_use_string(iuse, cur_use, old_use, masked_use):
+				usestr=""
+				for flag in iuse:
+					usechange=""
+					if old_use:
+						if (flag in old_use and flag not in cur_use) or (flag not in old_use and flag in cur_use):
+							usechange="*"
+
+					if flag in cur_use:
+						if usechange == "*":
+							substr = green("+"+flag)
+						else:
+							substr = red("+"+flag)
+					elif flag in masked_use:
+						substr = blue("(-"+flag+")")
+					else:
+						substr = blue("-"+flag)
+					usestr += substr + usechange + " "
+				return usestr
 
 		if "--tree" in myopts:
 			mylist.reverse()
@@ -1568,8 +1590,13 @@
 						portage.writemsg("!!! Error getting IUSE (report this to bugs.gentoo.org)\n")
 						portage.writemsg("!!! %s\n" % x)
 						iuse_split = []
+
+					iuse_split = portage.unique_array(iuse_split)
 					iuse_split.sort()
-					old_use=None
+
+					cur_use = self.applied_useflags[x[2]]
+
+					old_use = []
 					if myoldbest:
 						pkg=myoldbest
 					else:
@@ -1581,24 +1608,38 @@
 							raise # Needed else can't exit
 						except:
 							pass
-					iuse=""
-					now_use=self.applied_useflags[x[2]]
-					for ebuild_iuse in portage_util.unique_array(iuse_split):
-						usechange=""
-						if old_use:
-							if (old_use.count(ebuild_iuse) and not now_use.count(ebuild_iuse)) or (not old_use.count(ebuild_iuse) and now_use.count(ebuild_iuse)):
-								usechange="*"
-
-						if ebuild_iuse in self.applied_useflags[x[2]]:
-							if usechange == "*":
-								iuse=green("+"+ebuild_iuse)
-							else:
-								iuse=red("+"+ebuild_iuse)
-						elif ebuild_iuse in portage.settings.usemask:
-							iuse=blue("(-"+ebuild_iuse+")")
-						else:
-							iuse=blue("-"+ebuild_iuse)
-						verboseadd+=iuse+usechange+" "
+
+					reg_use = []
+					exp_map = {}
+					for flag in iuse_split:
+						found = False
+						for var in use_expand:
+							if flag.startswith(var+"_"):
+								if var in exp_map:
+									exp_map[var]+= [flag[len(var)+1:]]
+								else:
+									exp_map[var] = [flag[len(var)+1:]]
+								found = True
+								break
+						if not found:
+							reg_use.append(flag)
+
+					verboseadd += create_use_string(reg_use, cur_use, old_use, portage.settings.usemask)
+
+					for var in use_expand:
+						if var not in exp_map:
+							continue
+						expcur = []
+						expold = []
+						expmask = []
+						for flag in exp_map[var]:
+							if var+"_"+flag in cur_use:
+								expcur+= [flag]
+							if var+"_"+flag in old_use:
+								expold+= [flag]
+							if var+"_"+flag in portage.settings.usemask:
+								expmask+= [flag]
+						verboseadd += var.upper()+'="'+create_use_string(exp_map[var],expcur,expold,expmask).strip()+'" '
 
 					# size verbose
 					mysize=0
diff -uNr 2.0-original/pym/portage.py 2.0/pym/portage.py
--- 2.0-original/pym/portage.py	2005-09-28 12:24:10.000000000 +0900
+++ 2.0/pym/portage.py	2005-09-28 13:03:40.000000000 +0900
@@ -1392,7 +1392,7 @@
 				if self.has_key(var):
 					for x in string.split(self[var]):
 						mystr = string.lower(var)+"_"+x
-						if mystr not in usesplit:
+						if mystr not in usesplit and mystr not in self.usemask:
 							usesplit.append(mystr)
 
 		# Pre-Pend ARCH variable to USE settings so '-*' in env doesn't kill arch.