[gentoo-dev] per-package environment variables.

[gentoo-dev] per-package environment variables.

[Antst GD]

OK, lets develop discussion about option to choose environment variables 
per-package (compiler, flags and not only).

As some of us know, definition of problem already include 90% solution. 
So, lets start from user-side.
Want we want to see?
1) Possibility to change ENV during building some packages.
2) it should be automatic, no more definition of environment variables 
and manual emerging packages. No more spaming /etc/make.conf with some 
environment variables which correspond only to one package (RXVT_TERM, 
for example)
3) Solution must  be flexible and allow us to have not only 
"per-package" variables set, but also define some general sets for 
inclusion. For example if we want to build soft1,soft2,ans soft3 with 
IFC, we really don't want to have 3 copies of something like:
F77=ifc
FC=ifc
F90=ifc
FFLAGS="-xN -ip"
Moreover, when we will decide to change CPU, we will not need to edit 3 
files and change -xN to -xP. We want to do it once. :)

4) BTW, in such case it also not bad to have option to include something 
like
USE="ifc"
into this environment, and it should be parsed in proper way. I hope 
that this is clear. If we want to build something with IFC and set 
proper variables, we really don't want to go and edit one more file 
(/etc/portage/package.use). And we don't want to edit also two files, 
when we will figure out that we like coming gfortran :)

5) even more. I'll start not with per-package basic, but with 
per-environment basic, as a first iteration.

So, lets define our problem in more details:
1) Assume that default configuration of my system is
GCC 3.4.1
CFLAGS="-O2 -pipe" in /etc/make.conf
LINGUAS="ru" in /etc/make.conf
there is no defined ifc and icc USE flags in /etc/make.conf

2) I have set of packages, that require something "extra"
lam-mpi : want to compile with ICC, IFC
blas-reference : want to compile with g77 and high optimization
lapack-reference : want to compile with g77 and high optimization
blas-atlas: want to compile with IFC, ICC
lapack-atlas: want to compile with IFC, ICC
xine-lib : want to compile with low-optimization for GCC
open office : want to compile with LANGUAGE="RUSS" and GCC 3.3.4 and 
low-optimization
kde-i18n : want to have with LINGUAS="fr" and GCC 3.3.4
rxvt : want to compile with RXVT_TERM=xterm
octave : want to build with high-optimization for GCC


so, what I see?
Something like:

#cat /etc/portage/package.env
lam-mpi: icc ifc
blas-reference: g77high.env
lapack-reference: g77high.env
blas-atlas: ifc.env icc.env
lapack-atlas: ifc.env icc.env
xine-lib: GCC_lowopt.env
openoffice: language_ru.env GCC_334.env GCC_lowopt.env
kde-i18n: linguas_fr.env
rxvt: rxvt.env
octave: GCC_highopt.env


#ls /etc/portage/env
ifc.env
ifc.env
GCC_highopt.env
GCC_lowopt.env
rxvt.env
linguas_fr.env
language_ru.env
GCC_334.env
g77high.env

and
#cat /etc/portage/env/icc.env
CC=icc
CXX=icc
CFLAGS="-O2 -xN -noalign"
CXXFLAGS=$CFLAGS
USE="icc"

#cat /etc/portage/env/GCC_highopt.env
CFLAGS="-O3 -ffast-math -funroll-all-loops -fmove-all-movables"

#cat /etc/portage/env/GCC_lowopt.env
CFLAGS="-O1"

#cat /etc/portage/env/GCC_334.env
CC=/usr/i686-pc-linux-gnu/gcc-bin/3.3/gcc
g77=/usr/i686-pc-linux-gnu/gcc-bin/3.3/g77

And rest it similar.

Of course, it could be not so bad, to have option, to include name of 
file with ENV, or just env, to not produce tens of files with one 
variable, like RXVT_TERM="xterm"... There is seems to be two solutions:
1) just have second file with definitions, where not filenames, but 
defined variables.
2) change syntax of /etc/portage/package.env in such way, that allow to 
include both kind of records, filename and strict definition inline.
something like
openoffice: GCC_334.env GCC_lowopt.env -- LANGUAGE="RUSS"


For the moment both solutions looks ugly for me. At least I can't 
imaging simple and easy-to-understand syntax for package.env, which will 
look nice and will parce without problems.

What do you think about?

-- 
Anton Starikov

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] per-package environment variables.

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 236 bytes --]

On Tuesday 31 August 2004 06:26 pm, Antst GD wrote:
> What do you think about?

already been requested (i like the idea personally, the bashrc hack is just 
that ... a hack ...)
http://bugs.gentoo.org/show_bug.cgi?id=44796
-mike

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 838 bytes --]

Re: [gentoo-dev] per-package environment variables.

[Antst GD]

I've checked /usr/lib/portage/pym/portage.py
With minor changes, this is is extremely easy to add provided syntax to 
current portage. Just about 5 lines of code. :)
I can make a patch, people who interesting can try to use it as a 
testers  :)


Antst GD wrote:
> OK, lets develop discussion about option to choose environment variables 
> per-package (compiler, flags and not only).
> 
> As some of us know, definition of problem already include 90% solution. 
> So, lets start from user-side.
> Want we want to see?
> 1) Possibility to change ENV during building some packages.
> 2) it should be automatic, no more definition of environment variables 
> and manual emerging packages. No more spaming /etc/make.conf with some 
> environment variables which correspond only to one package (RXVT_TERM, 
> for example)
> 3) Solution must  be flexible and allow us to have not only 
> "per-package" variables set, but also define some general sets for 
> inclusion. For example if we want to build soft1,soft2,ans soft3 with 
> IFC, we really don't want to have 3 copies of something like:
> F77=ifc
> FC=ifc
> F90=ifc
> FFLAGS="-xN -ip"
> Moreover, when we will decide to change CPU, we will not need to edit 3 
> files and change -xN to -xP. We want to do it once. :)
> 
> 4) BTW, in such case it also not bad to have option to include something 
> like
> USE="ifc"
> into this environment, and it should be parsed in proper way. I hope 
> that this is clear. If we want to build something with IFC and set 
> proper variables, we really don't want to go and edit one more file 
> (/etc/portage/package.use). And we don't want to edit also two files, 
> when we will figure out that we like coming gfortran :)
> 
> 5) even more. I'll start not with per-package basic, but with 
> per-environment basic, as a first iteration.
> 
> So, lets define our problem in more details:
> 1) Assume that default configuration of my system is
> GCC 3.4.1
> CFLAGS="-O2 -pipe" in /etc/make.conf
> LINGUAS="ru" in /etc/make.conf
> there is no defined ifc and icc USE flags in /etc/make.conf
> 
> 2) I have set of packages, that require something "extra"
> lam-mpi : want to compile with ICC, IFC
> blas-reference : want to compile with g77 and high optimization
> lapack-reference : want to compile with g77 and high optimization
> blas-atlas: want to compile with IFC, ICC
> lapack-atlas: want to compile with IFC, ICC
> xine-lib : want to compile with low-optimization for GCC
> open office : want to compile with LANGUAGE="RUSS" and GCC 3.3.4 and 
> low-optimization
> kde-i18n : want to have with LINGUAS="fr" and GCC 3.3.4
> rxvt : want to compile with RXVT_TERM=xterm
> octave : want to build with high-optimization for GCC
> 
> 
> so, what I see?
> Something like:
> 
> #cat /etc/portage/package.env
> lam-mpi: icc ifc
> blas-reference: g77high.env
> lapack-reference: g77high.env
> blas-atlas: ifc.env icc.env
> lapack-atlas: ifc.env icc.env
> xine-lib: GCC_lowopt.env
> openoffice: language_ru.env GCC_334.env GCC_lowopt.env
> kde-i18n: linguas_fr.env
> rxvt: rxvt.env
> octave: GCC_highopt.env
> 
> 
> #ls /etc/portage/env
> ifc.env
> ifc.env
> GCC_highopt.env
> GCC_lowopt.env
> rxvt.env
> linguas_fr.env
> language_ru.env
> GCC_334.env
> g77high.env
> 
> and
> #cat /etc/portage/env/icc.env
> CC=icc
> CXX=icc
> CFLAGS="-O2 -xN -noalign"
> CXXFLAGS=$CFLAGS
> USE="icc"
> 
> #cat /etc/portage/env/GCC_highopt.env
> CFLAGS="-O3 -ffast-math -funroll-all-loops -fmove-all-movables"
> 
> #cat /etc/portage/env/GCC_lowopt.env
> CFLAGS="-O1"
> 
> #cat /etc/portage/env/GCC_334.env
> CC=/usr/i686-pc-linux-gnu/gcc-bin/3.3/gcc
> g77=/usr/i686-pc-linux-gnu/gcc-bin/3.3/g77
> 
> And rest it similar.
> 
> Of course, it could be not so bad, to have option, to include name of 
> file with ENV, or just env, to not produce tens of files with one 
> variable, like RXVT_TERM="xterm"... There is seems to be two solutions:
> 1) just have second file with definitions, where not filenames, but 
> defined variables.
> 2) change syntax of /etc/portage/package.env in such way, that allow to 
> include both kind of records, filename and strict definition inline.
> something like
> openoffice: GCC_334.env GCC_lowopt.env -- LANGUAGE="RUSS"
> 
> 
> For the moment both solutions looks ugly for me. At least I can't 
> imaging simple and easy-to-understand syntax for package.env, which will 
> look nice and will parce without problems.
> 
> What do you think about?
> 


-- 
Anton Starikov

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] per-package environment variables.

[Ned Ludd]

[-- Attachment #1: Type: text/plain, Size: 486 bytes --]

On Tue, 2004-08-31 at 18:44, Antst GD wrote:
> I've checked /usr/lib/portage/pym/portage.py
> With minor changes, this is is extremely easy to add provided syntax to 
> current portage. Just about 5 lines of code. :)
> I can make a patch, people who interesting can try to use it as a 
> testers  :)

I'm ready to start testing.. Where is this code?



[snip]
...
 
-- 
Ned Ludd <solar@gentoo.org>
Gentoo (hardened,security,infrastructure,embedded,toolchain) Developer

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] per-package environment variables.

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 893 bytes --]

On Tuesday 31 August 2004 06:26 pm, Antst GD wrote:
> For the moment both solutions looks ugly for me. At least I can't
> imaging simple and easy-to-understand syntax for package.env, which will
> look nice and will parce without problems.
>
> What do you think about?

most of what you proposed seems excessive and confusing (and like you say, 
ugly) ... a cleaner/simpler solution can be taken from your work though i 
think ...

we have a directory:
/etc/profile/packages.env/

in this directory we have simple bash (bash because we can just source it and 
be done with it) files which line up in a similar fashion to the portage 
tree ... for example, if i want an env file for bash, i would have:
/etc/profile/packages.env/app-shells/bash

i think this is about all we need ... and like Antst pointed out, this looks 
like it'd be real simple to implement ...
-mike

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 838 bytes --]

Re: [gentoo-dev] per-package environment variables.

[Anton Starikov]

I didn't get it :)
I'm one person, Anton Starikov and Antst :) antst - just my mailbox :))

So, /etc/portage/packages.env/ seems to be good. That sound cleaner, 
right. But, can you imaging, I'm going to edit 10 files next time, when 
to ICC, for example, will be added some nice optimization flag which I'm 
going to use.
So, the basic point is that they are really "per-package" sollution. If 
we want to keep it. I guess it can be extended, to have directories like 
/etc/portage/packages.env/bash/
Where you can put multiple files similar to /etc/env.d/
So, you can make one env file, and make just symbolic links to it for 
different packages.
So, idea to include some kind of sollution which not strictly 
"per-package". So, you can define something, and easy include/exclude it 
for different packages.


Mike Frysinger wrote:
> On Tuesday 31 August 2004 06:26 pm, Antst GD wrote:
> 
>>For the moment both solutions looks ugly for me. At least I can't
>>imaging simple and easy-to-understand syntax for package.env, which will
>>look nice and will parce without problems.
>>
>>What do you think about?
> 
> 
> most of what you proposed seems excessive and confusing (and like you say, 
> ugly) ... a cleaner/simpler solution can be taken from your work though i 
> think ...
> 
> we have a directory:
> /etc/profile/packages.env/
> 
> in this directory we have simple bash (bash because we can just source it and 
> be done with it) files which line up in a similar fashion to the portage 
> tree ... for example, if i want an env file for bash, i would have:
> /etc/profile/packages.env/app-shells/bash
> 
> i think this is about all we need ... and like Antst pointed out, this looks 
> like it'd be real simple to implement ...
> -mike


-- 
Anton Starikov

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] per-package environment variables.

[Anton Starikov]

But with /etc/profile/packages.env/ there is only one thing, which not 
good enough. I really can't make a one look and see what and where I 
define. I mean, if I look into one file, I see something like, let say, 
30 lines, which looks clear and understandable for me.
But with recursive subdirectories...Simple sollution is ls -R, but if I 
have 30 packages with redefinition, and, let say, average two ENV files 
in each directory... That going to be a difficult for simple 
understanding, probably :) Of course directories more logic. But I guess 
  it will be never 300 definitions of some ENV's for 300 packages. 
But...who know..

And about confusing...I just think that mostly this ENV files will be 
used for some kind of typical job. So, you will not have 100 files for 
100 packages. But raser you will have 10 files, no more (there is just 
not so many things to define :) ). And will include some of them to some 
of 100 packages, more than one at the same time, probably.

Anton Starikov aka Antst :)


Mike Frysinger wrote:
> On Tuesday 31 August 2004 06:26 pm, Antst GD wrote:
> 
>>For the moment both solutions looks ugly for me. At least I can't
>>imaging simple and easy-to-understand syntax for package.env, which will
>>look nice and will parce without problems.
>>
>>What do you think about?
> 
> 
> most of what you proposed seems excessive and confusing (and like you say, 
> ugly) ... a cleaner/simpler solution can be taken from your work though i 
> think ...
> 
> we have a directory:
> /etc/profile/packages.env/
> 
> in this directory we have simple bash (bash because we can just source it and 
> be done with it) files which line up in a similar fashion to the portage 
> tree ... for example, if i want an env file for bash, i would have:
> /etc/profile/packages.env/app-shells/bash
> 
> i think this is about all we need ... and like Antst pointed out, this looks 
> like it'd be real simple to implement ...
> -mike


-- 
Anton Starikov

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] per-package environment variables.

[Olivier Fisette]

[-- Attachment #1: Type: text/plain, Size: 2370 bytes --]

Hello all,

I see two problems with the implementations that were suggested up to now for 
per-package environment variables.

1) They do not allow any flexibility with regard to package versions, as 
opposed to other Portage configuration files ("package.mask", "package.use", 
etc.). The file "/etc/portage/packages.env/app-shells/bash" would be sourced 
for all versions of "app-shells/bash", for instance. This may be problematic 
for SLOTable packages, and prevents a user from defining or modifying a 
variable for a particular version of a package (to work around a bug, for 
example, such as "pymol-0.95" will segfault when raytracing if compiled with 
"-fgcse").

2) Having a hierarchy of files and directories miroring the Portage tree, with 
symlinks and such, looks a bit complicated more complicated than is needed. 
I'd rather have to deal with only one or a few files.

My initial suggestion was not flexible enough, since it required to declare 
the variables repeatedly for each package. Here is another idea:

A file named "/etc/portage/package.env" could contain entries such as:

app-office/openoffice lang_ru
>=app-sci/emboss-2.8.0 icc
=app-sci/pymol-0.95* gcc_low_opt

The keywords "lang_ru", "ifc" and "gcc_low_opt" would match files with these 
names, contained in "/etc/portage/env.d". These bash scripts would be sourced 
to provide per-package env vars. "/etc/portage/env.d/icc" would contain 
something along these lines:

CC=icc
CXX=icc
CFLAGS="-O2 -xN -noalign"
CXXFLAGS=$CFLAGS
USE="icc"

If we wanted to make things even easier to configure, we could have something 
like the following in "/etc/portage/package.env":

=app-sci/pymol-0.95* [ CC="gcc" CFLAGS="-O1" ]

... were a keyword is replaced by a delimited short bash script that would be 
sourced to provide per package env vars. This would avoid having to create 
files in "/etc/portage/env.d" which target only one package.

I don't think this approach is elegant or powerful. A better mechanism would 
allow to filter and replace flags (like flag-o-matic), to choose compilers 
and have them merged as dependencies when needed, etc. However, per-package 
env vars would be enough in 99 % of the cases, and would extend farther than 
just compilers and compiler flags.

-- 
Olivier Fisette (ribosome)
Gentoo Linux Developer

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] per-package environment variables.

[Anton Starikov]

Olivier Fisette wrote:

  > A file named "/etc/portage/package.env" could contain entries such as:
> 
> app-office/openoffice lang_ru
> 
>>=app-sci/emboss-2.8.0 icc
> 
> =app-sci/pymol-0.95* gcc_low_opt
Agree, support for version is also important, as for group, because we 
have some packages with the name but in dfifferent groups.

> If we wanted to make things even easier to configure, we could have something 
> like the following in "/etc/portage/package.env":
> 
> =app-sci/pymol-0.95* [ CC="gcc" CFLAGS="-O1" ]
> 
> ... were a keyword is replaced by a delimited short bash script that would be 
> sourced to provide per package env vars. This would avoid having to create 
> files in "/etc/portage/env.d" which target only one package.
Anyway, if we going to include both, files and inlined vars, we have to 
have a bit complicated parser of lines. In such case we even don't need 
[], we can leave as
=app-sci/pymol-0.95* gcc_low_opt CC="gcc"
"=" specify enough that we have inlined expression.

-- 
Anton Starikov

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] per-package environment variables.

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 1346 bytes --]

On Wed, Sep 01, 2004 at 01:08:11PM +0200, Anton Starikov wrote:
> Olivier Fisette wrote:
> >If we wanted to make things even easier to configure, we could have 
> >something like the following in "/etc/portage/package.env":
> >
> >=app-sci/pymol-0.95* [ CC="gcc" CFLAGS="-O1" ]
> >
> >... were a keyword is replaced by a delimited short bash script that would 
> >be sourced to provide per package env vars. This would avoid having to 
> >create files in "/etc/portage/env.d" which target only one package.
> Anyway, if we going to include both, files and inlined vars, we have to 
> have a bit complicated parser of lines. In such case we even don't need 
> [], we can leave as
> =app-sci/pymol-0.95* gcc_low_opt CC="gcc"
> "=" specify enough that we have inlined expression.
So you want to take out those of us that have strange filenames?
I'd say the inlined case shouldn't be used, but the general idea here is
a good one.

The format definetly should be:
atom list_of_files
As that is the only non-ambiguous way of doing it.

See bug #51552 for more on environment variables in portage.

-- 
Robin Hugh Johnson
E-Mail     : robbat2@orbis-terrarum.net
Home Page  : http://www.orbis-terrarum.net/?l=people.robbat2
ICQ#       : 30269588 or 41961639
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: [gentoo-dev] per-package environment variables.

[Olivier Fisette]

[-- Attachment #1: Type: text/plain, Size: 797 bytes --]

On Wednesday, 1 September, 2004 07:08 am, Anton Starikov wrote:
> Anyway, if we going to include both, files and inlined vars, we have to
> have a bit complicated parser of lines. In such case we even don't need
> [], we can leave as
> =app-sci/pymol-0.95* gcc_low_opt CC="gcc"
> "=" specify enough that we have inlined expression.

I don't think so. What if you want to source a bash script containing 
functions to manipulate variables, such as filtering or replacing CFLAGS? 
There is no "=" in such a command. If inline is used, we'll need delimiters. 

However, I don't think inline definitions are that important. A first 
implementation dealing only with "atom [env_file] [...]" in package.env would 
be enough, certainly.

-- 
Olivier Fisette (ribosome)
Gentoo Developer

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] per-package environment variables.

[Anton Starikov]

Olivier Fisette wrote:
>>Anyway, if we going to include both, files and inlined vars, we have to
>>have a bit complicated parser of lines. In such case we even don't need
>>[], we can leave as
>>=app-sci/pymol-0.95* gcc_low_opt CC="gcc"
>>"=" specify enough that we have inlined expression.
> 
> 
> I don't think so. What if you want to source a bash script containing 
> functions to manipulate variables, such as filtering or replacing CFLAGS? 
> There is no "=" in such a command. If inline is used, we'll need delimiters. 
> 
> However, I don't think inline definitions are that important. A first 
> implementation dealing only with "atom [env_file] [...]" in package.env would 
> be enough, certainly.
> 
Even more, I can say that my first idea was idea with delimiters or some 
kind of brackets. But after time I've realized that it's looks a bit 
ugly and not intuitive. Moreover, I really believe that if you want to 
include something really complicated, better to do it via separate file, 
rather via inlining, by this you will decrease probability that file 
will be parsed wrong. Moreover, I also suggest that inline definitions 
not such important, more, I believe that in 99.9% cases you will want to 
inline something like VAR="foo bar". So, I think it not worst to make 
limitation about inlining, only simple variable definitions. Rest goes 
to separate files. In such case we have somehow easy and intuitive (I 
hope) syntax, which not looks ugly (at least for me :)), simple parsing 
and so on.
But of course, implementing of "atom env_file1 env_file2 ..." is 
simplest for now, at least from my ponit of view. (I'm really not great 
specialist in python, and really didn't pass throw all portage scripts, 
but I've found that I can really implement easy this syntax for adding 
about 5-10 lines of code into portage, via using methods that already 
exist there)

-- 
Anton Starikov

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] per-package environment variables.

[Anton Starikov]

Finally, shell I make a patch for testing with first iteration syntax like:

#cat /etc/portage/packages.env
some-cat/someapp env_file1 env_file2 ...

#ls /etc/portage/env
env_file1
env_file2
...

Is somebody interesting? :)


-- 
Anton Starikov

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] per-package environment variables.

[Olivier Fisette]

[-- Attachment #1: Type: text/plain, Size: 494 bytes --]

I am interested. I you go ahead, I hope you will use the "atom" syntax instead 
of the "some-cat/app" syntax, however.

On Thursday, 2 September, 2004 06:49 pm, Anton Starikov wrote:
> Finally, shell I make a patch for testing with first iteration syntax like:
>
> #cat /etc/portage/packages.env
> some-cat/someapp env_file1 env_file2 ...
>
> #ls /etc/portage/env
> env_file1
> env_file2
> ...
>
> Is somebody interesting? :)

-- 
Olivier Fisette (ribosome)
Gentoo Developer

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] per-package environment variables.

[Anton Starikov]

Probably some misunderstanding from my side. What you mean with "atom" ?
just "app" in my terms? If no, can you explain?

Basically, I think best case - is when you can specify one of:

some-cat/app (this MUST be, because we have doubling of names for some pkgs)
=some-cat/app-1.2.3 (this also MUST be, because of SLOTed pkgs, some 
bugs workaround and so on)
app (this is not really necessary I guess, I would suggest to not have 
it at all)

-- 
Anton Starikov


Olivier Fisette wrote:
> I am interested. I you go ahead, I hope you will use the "atom" syntax instead 
> of the "some-cat/app" syntax, however.
> 
> On Thursday, 2 September, 2004 06:49 pm, Anton Starikov wrote:
> 
>>Finally, shell I make a patch for testing with first iteration syntax like:
>>
>>#cat /etc/portage/packages.env
>>some-cat/someapp env_file1 env_file2 ...
>>
>>#ls /etc/portage/env
>>env_file1
>>env_file2
>>...
>>
>>Is somebody interesting? :)
> 
> 



--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] per-package environment variables.

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 475 bytes --]

On Fri, Sep 03, 2004 at 03:36:23AM +0200, Anton Starikov wrote:
> Probably some misunderstanding from my side. What you mean with "atom" ?
> just "app" in my terms? If no, can you explain?
See ebuild(5) for documentation about DEPEND atoms.

-- 
Robin Hugh Johnson
E-Mail     : robbat2@orbis-terrarum.net
Home Page  : http://www.orbis-terrarum.net/?l=people.robbat2
ICQ#       : 30269588 or 41961639
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: [gentoo-dev] per-package environment variables.

[Olivier Fisette]

[-- Attachment #1: Type: text/plain, Size: 929 bytes --]

A "DEPEND atom" is the term used in the portage(5) and ebuild(5) man pages to 
describe an expression of the type: ">=app-sci/emboss-2.9.0" (a category and 
a package, with optional logical operator and version). It is the syntax used 
in the ebuilds' "DEPEND" variable, and in many Portage configuration files, 
such as "package.use".

On Thursday, 2 September, 2004 09:36 pm, Anton Starikov wrote:
> Probably some misunderstanding from my side. What you mean with "atom" ?
> just "app" in my terms? If no, can you explain?
>
> Basically, I think best case - is when you can specify one of:
>
> some-cat/app (this MUST be, because we have doubling of names for some
> pkgs) =some-cat/app-1.2.3 (this also MUST be, because of SLOTed pkgs, some
> bugs workaround and so on)
> app (this is not really necessary I guess, I would suggest to not have
> it at all)

-- 
Olivier Fisette (ribosome)
Gentoo Developer

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] per-package environment variables.

[Anton Starikov]

Olivier Fisette wrote:
 > A "DEPEND atom" is the term used in the portage(5) and ebuild(5) man 
pages to
 > describe an expression of the type: ">=app-sci/emboss-2.9.0" (a 
category and
 > a package, with optional logical operator and version). It is the 
syntax used
 > in the ebuilds' "DEPEND" variable, and in many Portage configuration 
files,
 > such as "package.use".

Thanks :)
So, I was meaning exactly the same :)
Sure, it should be atom.
OK, I'm going to finish (hopefully) work on my current code somewhere at 
   the morning (TZ=GMT+1) and will try to write a patch.


Robin H. Johnson wrote:
 > See ebuild(5) for documentation about DEPEND atoms.

Just didn't know that there is special name for it....
It seems that there is bit of true in:
"Real Programmers don't read manuals. Reliance on a reference is a 
hallmark of the novice and the coward. " :)))

-- 
Anton Starikov

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] per-package environment variables.

[Nicholas Jones]

[-- Attachment #1: Type: text/plain, Size: 6053 bytes --]


Disclaimer:
Don't mind that it starts off ranting. I actually like some
of the implementation ideas presented. Any discussion on my
dillusions of grandjeur or my inability to spell will get
you smited in a very not nice way.

> 1) Possibility to change ENV during building some packages.

I certainly hope you realize the depths to which people go to
make your life miserable when attempting to do support.

Something like this will happen:
	source /etc/profile

Which does something great to the users console, but will
probably have destructive results otherwise. Bugs will now
require you to review not only the settings that portage
reports, but you will also have to check every related script
that the user decides to make part of the build process.
Unless we ignore anyone using these features. Would be funny
in that bated, ironic-reversal kinda way.

If they decide to alias something, it will have an affect
IN THE BUILD. Portage makes a lot of effort to prevent
things like this right now. unsets + unalias, are a quick
example.

> 2) it should be automatic, no more definition of environment variables 
> and manual emerging packages. No more spaming /etc/make.conf with some 
> environment variables which correspond only to one package (RXVT_TERM, 
> for example)

The idea is that it takes effort. If it's not trivial, you have to
LEARN to make it work. Having super-easy things is great, but not
when it happens to make FIXING it super complicated, developers lose.

The single package variables are mostly new to me, thus I have no
opinion on them.

> 3) Solution must  be flexible and allow us to have not only 
> "per-package" variables set, but also define some general sets for 
> inclusion. For example if we want to build soft1,soft2,ans soft3 with 
> IFC, we really don't want to have 3 copies of something like:
> F77=ifc
> FC=ifc
> F90=ifc
> FFLAGS="-xN -ip"
> Moreover, when we will decide to change CPU, we will not need to edit 3 
> files and change -xN to -xP. We want to do it once. :)

Some of this might be nice. If you're going to make this kind of
effort, why not integrate it into the ebuild?

> 4) BTW, in such case it also not bad to have option to include something 
> like
> USE="ifc"
> into this environment, and it should be parsed in proper way. I hope 
> that this is clear. If we want to build something with IFC and set 
> proper variables, we really don't want to go and edit one more file 
> (/etc/portage/package.use). And we don't want to edit also two files, 
> when we will figure out that we like coming gfortran :)

Absolutely not. This violates portage's dep calculations and is
completely wrong. Please do not change any variables that are
READONLY inside of ebuild.sh after the depend phase.

> So, lets define our problem in more details:
> 1) Assume that default configuration of my system is
> GCC 3.4.1
> CFLAGS="-O2 -pipe" in /etc/make.conf
> LINGUAS="ru" in /etc/make.conf
> there is no defined ifc and icc USE flags in /etc/make.conf
> 
> 2) I have set of packages, that require something "extra"
> lam-mpi : want to compile with ICC, IFC
> blas-reference : want to compile with g77 and high optimization
> lapack-reference : want to compile with g77 and high optimization
> blas-atlas: want to compile with IFC, ICC
> lapack-atlas: want to compile with IFC, ICC
> xine-lib : want to compile with low-optimization for GCC
> open office : want to compile with LANGUAGE="RUSS" and GCC 3.3.4 and 
> low-optimization
> kde-i18n : want to have with LINGUAS="fr" and GCC 3.3.4
> rxvt : want to compile with RXVT_TERM=xterm
> octave : want to build with high-optimization for GCC
> 
> 
> so, what I see?
> Something like:
> 
> #cat /etc/portage/package.env
> lam-mpi: icc ifc
> blas-reference: g77high.env
> lapack-reference: g77high.env
> blas-atlas: ifc.env icc.env
> lapack-atlas: ifc.env icc.env
> xine-lib: GCC_lowopt.env
> openoffice: language_ru.env GCC_334.env GCC_lowopt.env
> kde-i18n: linguas_fr.env
> rxvt: rxvt.env
> octave: GCC_highopt.env
> 
> 
> #ls /etc/portage/env
> ifc.env
> ifc.env
> GCC_highopt.env
> GCC_lowopt.env
> rxvt.env
> linguas_fr.env
> language_ru.env
> GCC_334.env
> g77high.env
> 
> and
> #cat /etc/portage/env/icc.env
> CC=icc
> CXX=icc
> CFLAGS="-O2 -xN -noalign"
> CXXFLAGS=$CFLAGS
> USE="icc"
> 
> #cat /etc/portage/env/GCC_highopt.env
> CFLAGS="-O3 -ffast-math -funroll-all-loops -fmove-all-movables"
> 
> #cat /etc/portage/env/GCC_lowopt.env
> CFLAGS="-O1"
> 
> #cat /etc/portage/env/GCC_334.env
> CC=/usr/i686-pc-linux-gnu/gcc-bin/3.3/gcc
> g77=/usr/i686-pc-linux-gnu/gcc-bin/3.3/g77
> 
> And rest it similar.
> 
> Of course, it could be not so bad, to have option, to include name of 
> file with ENV, or just env, to not produce tens of files with one 
> variable, like RXVT_TERM="xterm"... There is seems to be two solutions:
> 1) just have second file with definitions, where not filenames, but 
> defined variables.
> 2) change syntax of /etc/portage/package.env in such way, that allow to 
> include both kind of records, filename and strict definition inline.
> something like
> openoffice: GCC_334.env GCC_lowopt.env -- LANGUAGE="RUSS"
> 
> 
> For the moment both solutions looks ugly for me. At least I can't 
> imaging simple and easy-to-understand syntax for package.env, which will 
> look nice and will parce without problems.
> 
> What do you think about?

Some of the ideas actually look trackable, which I happen
to like. Inline definitions of variables, I am not
particularly fond of.

If portage has a way to accurately track these files and
there is a way to avoid arbitary inclusion, then it might
be a lot more feasable.

Providing code would be nice too. An interface to track and
an interface whereby portage controls the inclusions.

I'm very much against out-of-file references.

Eh. That is all I have to say.
I'm going to go home and take a nap now.

--NJ


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] per-package environment variables.

[Anton Starikov]

Nicholas Jones wrote:
>>1) Possibility to change ENV during building some packages.
> 
> 
> I certainly hope you realize the depths to which people go to
> make your life miserable when attempting to do support.
Yes :) There is something
> The idea is that it takes effort. If it's not trivial, you have to
> LEARN to make it work. Having super-easy things is great, but not
> when it happens to make FIXING it super complicated, developers lose.
Yes, but sollution can be simple...some kind of, probably, limitation 
applied, at first (for example ONLY env variables, nothing more in this 
script, onle A=B syntax). And second, idea to get this environment 
BEFORE something. So, it should be absolutely equal, theoretically, to 
something like this:
CC="bla-bla-bla" CFLAGS="bla-bla-bla" I_WANT_THAT="bla-bla-bla" emerge 
nice_package
I thing that is OK, because anyway users can do it now :)

> Some of this might be nice. If you're going to make this kind of
> effort, why not integrate it into the ebuild?
If I uderstand right, you mean make copy of 10 ebuilds into local 
overlay and edit them twice a week? When new version of ebuild is 
coming? :) And in the middle have troubles, because fresh version will 
be at first automatically updated with standard ENV and at morning I'll 
reallize that my code don't want to be linked against of library XXX, 
which was compiled at night with g77 instead ifc or lf95 or pgf90 or 
something else ? :) That is boring. I've checked already :)


>>4) BTW, in such case it also not bad to have option to include something 
>>like
>>USE="ifc"
>>into this environment, and it should be parsed in proper way. I hope 
>>that this is clear. If we want to build something with IFC and set 
>>proper variables, we really don't want to go and edit one more file 
>>(/etc/portage/package.use). And we don't want to edit also two files, 
>>when we will figure out that we like coming gfortran :)
> 
> 
> Absolutely not. This violates portage's dep calculations and is
> completely wrong. Please do not change any variables that are
> READONLY inside of ebuild.sh after the depend phase.
Again. Idea of this way is to have something equal to
CC="bla-bla-bla" CFLAGS="bla-bla-bla" I_WANT_THAT="bla-bla-bla" 
USE="I_dont_want_that" emerge nice_package
This way is OK and works now. Yes, it probably going to be a bit 
complicated and probably you will need to parse it twice...that is 
subject of dicussion.


> Some of the ideas actually look trackable, which I happen
> to like. Inline definitions of variables, I am not
> particularly fond of.
> 
> If portage has a way to accurately track these files and
> there is a way to avoid arbitary inclusion, then it might
> be a lot more feasable.
Keep in mind, it was a first iteration :) Not really well designed
Just wanted to rise discussion. After which idea could take shape of 
brilliant :)


> Eh. That is all I have to say.
> I'm going to go home and take a nap now.
:)

-- 
Anton Starikov

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] per-package environment variables.

[aye]

well - i've got one more idea. all your proposal have one weakness from
my point of view. i've reported bug (and reopened the second one) for
one reason: i want to have some base system compiled in way, which makes
me possibility to run on diffrent procesors (mainly i'm interested in
pentiums and athlons), while the rest of programs should be compiled
with optimalization for my home cpu.
in your solutions i'd have to had a long file looks like it:

category/package1 low_c_flags*
category/package2 low_c_flags
category/package3 low_c_flags
category/package4 low_c_flags
category/package5 low_c_flags
[...]
category/package100 low_c_flags

*low_c_flags is just example - no matter how it'd be called, if it would
be file name or settings or some kind of keyword - it looks ugly. in the
other hand i don't think that anyboby would need more then 3 or 4 sets
of flag. so what i propose is to have a /etc/portage/package.env with
syntax as below:

CFLAGS="-option1 -option2 -option3"
PACKAGES="category/package1
category/package2
category/package3
category/package4
[...]
category/packageN"

CFLAGS="-option1 -option4 -option4"
PACKAGES="category/packageN+1
category/packageN+2
category/packageN+3
category/packageN+4
[...]
category/packageN+M"

LINGUAS="pl"
PACKAGES="category/package3
category/package5"

and so on. 
there is one disadvantage in this idea. it's pretty certain, that man
want to have diffrent CFLAGS and CXXFLAGS for the same package. in
contruction as above, he'd have to copy packages and have the same
package in 2 places, so i think, we could set these flags together, for
example:

CFLAGS="..."
CXXFLAGS="..."
PACKAGES="..."

CFLAGS="..."
PACKAGES="..." <no CXXFLAGS definition here -> CXXFLAGS are taken from make.conf

now one can say: "hey - why the hell CFLAGS and CXXFLAGS are treated in
diffrent way than the rest?". ok. so let's make:

CFLAGS="..."
CXXFLAGS="..."
MAKEOPTS="..."
PACKAGES="..."

CFLAGS="..."
PACKAGES="..." < again - not declared setting are taken from enviroment here

isn't that clear anought? let's make /etc/portage/package.env a
directory.

ls /etc/portage/package.env
set1.env
set2.env
set3.env

cat /etc/portage/package.env/set1.env
# definitions for base-system
CFLAGS="..."
CXXFLAGS="..."
MAKEOPTS="..."
PACKAGES="..."

cat /etc/portage/package.env/set2.env
# definitions for kde
CFLAGS="..."
LINGUAS="..."
PACKAGES="..."

and so on...
what do you think of that?

-- 
"Life's a bitch and so am I, the world owes me, so fuck you." - Green Day
PoLiSh YoUr EnGlIsH:
 KIOSK RUCHU - LITTLE BUSINESS OF MOVEMENT 
  

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] per-package environment variables.

[Anton Starikov]

aye wrote:
> and so on...
> what do you think of that?

I think that ideal solution somewhere between your and my solutions :)

Anton Starikov


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] per-package environment variables.

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 1092 bytes --]

On Wed, Sep 01, 2004 at 04:02:46PM +0200, aye wrote:
> well - i've got one more idea. all your proposal have one weakness from
> my point of view. i've reported bug (and reopened the second one) for
> one reason: i want to have some base system compiled in way, which makes
> me possibility to run on diffrent procesors (mainly i'm interested in
> pentiums and athlons), while the rest of programs should be compiled
> with optimalization for my home cpu.
> in your solutions i'd have to had a long file looks like it:
[snip]

Your setup here could be better solved with portage recognizing groups
of packages - both predefined groups 'system' and custom groups.

Eg you can replace everything with one line:
package-group/system low_c_flags

And that would compile everything portage considers as system with the
environment from low_c_flags.

-- 
Robin Hugh Johnson
E-Mail     : robbat2@orbis-terrarum.net
Home Page  : http://www.orbis-terrarum.net/?l=people.robbat2
ICQ#       : 30269588 or 41961639
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: [gentoo-dev] per-package environment variables.

[aye]

On Wed, Sep 01, 2004 at 08:59:11AM -0700, Robin H. Johnson wrote:
> Your setup here could be better solved with portage recognizing groups
> of packages - both predefined groups 'system' and custom groups.
> 
> Eg you can replace everything with one line:
> package-group/system low_c_flags
> 
> And that would compile everything portage considers as system with the
> environment from low_c_flags.
> 
yep - it's also good idea, however i'd still have to declare 'system'
class somewhere. the one used in profile isn't good for me (for example
x11 is in profile), but you've ofcourse right - it would look nicer if i
had just package-group/system and profile, which satisfy me.


-- 
"Life's a bitch and so am I, the world owes me, so fuck you." - Green Day
PoLiSh YoUr EnGlIsH:
 ROZWODZIC SIE NAD FAKTAMI - TO DIVORCE THE FACTS 
  

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] per-package environment variables.

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 1708 bytes --]

On Wed, Sep 01, 2004 at 08:42:57PM +0200, aye wrote:
> On Wed, Sep 01, 2004 at 08:59:11AM -0700, Robin H. Johnson wrote:
> > Your setup here could be better solved with portage recognizing groups
> > of packages - both predefined groups 'system' and custom groups.
> > 
> > Eg you can replace everything with one line:
> > package-group/system low_c_flags
> > 
> > And that would compile everything portage considers as system with the
> > environment from low_c_flags.
> > 
> yep - it's also good idea, however i'd still have to declare 'system'
> class somewhere. the one used in profile isn't good for me (for example
> x11 is in profile), but you've ofcourse right - it would look nicer if i
> had just package-group/system and profile, which satisfy me.
Somebody else was ahead of us on it, but I don't know how much progress
was made on it:
http://www.gentoo.org/proj/en/glep/glep-0021.html

Coredumb notes that USE flags for sets could prove a problem, and
unfortunately the same problem would effect this package.env proposal.
Namely, if you have two entries:
virtual-set/foo env1
virtual-set/bar env2
some-cat/foo env3

some-cat/foo is present is both sets.
If some-cat/foo is being merged directly, the settings from the sets
will be totally ignored - even if we could detect them, which one would
we use? There are a couple of other similar problem cases with sets and
any of the portage settings that might be applied per-package :-(.

-- 
Robin Hugh Johnson
E-Mail     : robbat2@orbis-terrarum.net
Home Page  : http://www.orbis-terrarum.net/?l=people.robbat2
ICQ#       : 30269588 or 41961639
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]