[gentoo-dev] RFC: Standard build environment variables

[gentoo-dev] RFC: Standard build environment variables

[Michael Orlitzky]

As many of you probably know, ago@ has been expanding the scope of our
CFLAGS/CC support to include some other common build variables:

  * CC
  * CXX
  * AR
  * CPP
  * NM
  * RANLIB
  * AS
  * LD

Some of those are POSIX standards[0],

  * CC
  * AR

Others are de-facto GNU make standards[1],

  * CXX
  * CPP
  * AS

and a few are de-facto GNU libtool standards[2]:

  * NM
  * RANLIB
  * LD

If we expect them all to work properly in Gentoo, we have to agree on
what they mean, and thus how they should be injected into build systems.
For example, we had a problem with sci-mathematics/pari, whose upstream
is using the LD environment variable for something other than what GNU
libtool uses it for. With LD set to something libtooly in the
environment, the pari build fails. We can solve that by unsetting LD in
the ebuild, but for that to be The Right Thing To Do, we should be
expecting LD to contain something libtooly, and thus something
inappropriate to be passed to the pari build.

To avoid these issues, I suggest creating a list of "Gentoo environment
variables" in the devmanual with descriptions of how they should be used
and pointers to the references (for why we chose that meaning). That way
a user can export LD, for example, and know that it will be used how he
thinks it will be used.


[0] https://pubs.opengroup.org/onlinepubs/009695399/utilities/make.html

[1]
https://www.gnu.org/software/make/manual/html_node/Implicit-Variables.html

[2] https://www.gnu.org/software/libtool/manual/libtool.html

Re: [gentoo-dev] RFC: Standard build environment variables

[Kent Fredric]

[-- Attachment #1: Type: text/plain, Size: 934 bytes --]

On Sun, 28 Jun 2020 08:18:23 -0400
Michael Orlitzky <mjo@gentoo.org> wrote:

> With LD set to something libtooly in the
> environment, the pari build fails. We can solve that by unsetting LD in
> the ebuild, but for that to be The Right Thing To Do, we should be
> expecting LD to contain something libtooly, and thus something
> inappropriate to be passed to the pari build.
> 
> To avoid these issues, I suggest creating a list of "Gentoo environment
> variables" in the devmanual with descriptions of how they should be used
> and pointers to the references (for why we chose that meaning). That way
> a user can export LD, for example, and know that it will be used how he
> thinks it will be used.

That LD problem is not unique to Pari, ftr. Its a problem in *all* Perl
things that involve C parts.

Because Perl code expects "LD" to be a "CCLD", and passes flags a CCLD
would expect, which LD can't handle.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] RFC: Standard build environment variables

[Mike Gilbert]

On Sun, Jun 28, 2020 at 8:18 AM Michael Orlitzky <mjo@gentoo.org> wrote:
>
> As many of you probably know, ago@ has been expanding the scope of our
> CFLAGS/CC support to include some other common build variables:
>
>   * CC
>   * CXX
>   * AR
>   * CPP
>   * NM
>   * RANLIB
>   * AS
>   * LD
>
> Some of those are POSIX standards[0],
>
>   * CC
>   * AR
>
> Others are de-facto GNU make standards[1],
>
>   * CXX
>   * CPP
>   * AS
>
> and a few are de-facto GNU libtool standards[2]:
>
>   * NM
>   * RANLIB
>   * LD
>
> If we expect them all to work properly in Gentoo, we have to agree on
> what they mean, and thus how they should be injected into build systems.
> For example, we had a problem with sci-mathematics/pari, whose upstream
> is using the LD environment variable for something other than what GNU
> libtool uses it for. With LD set to something libtooly in the
> environment, the pari build fails. We can solve that by unsetting LD in
> the ebuild, but for that to be The Right Thing To Do, we should be
> expecting LD to contain something libtooly, and thus something
> inappropriate to be passed to the pari build.
>
> To avoid these issues, I suggest creating a list of "Gentoo environment
> variables" in the devmanual with descriptions of how they should be used
> and pointers to the references (for why we chose that meaning). That way
> a user can export LD, for example, and know that it will be used how he
> thinks it will be used.

Makes sense to me.

[gentoo-dev] Re: RFC: Standard build environment variables

[Agostino Sarubbo]

On domenica 28 giugno 2020 14:18:23 CEST Michael Orlitzky wrote:
> To avoid these issues, I suggest creating a list of "Gentoo environment
> variables" in the devmanual with descriptions of how they should be used
> and pointers to the references (for why we chose that meaning). That way
> a user can export LD, for example, and know that it will be used how he
> thinks it will be used.

+1

Agostino

Re: [gentoo-dev] Re: RFC: Standard build environment variables

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 479 bytes --]

>>>>> On Mon, 29 Jun 2020, Agostino Sarubbo wrote:

> On domenica 28 giugno 2020 14:18:23 CEST Michael Orlitzky wrote:
>> To avoid these issues, I suggest creating a list of "Gentoo environment
>> variables" in the devmanual with descriptions of how they should be used
>> and pointers to the references (for why we chose that meaning). That way
>> a user can export LD, for example, and know that it will be used how he
>> thinks it will be used.

> +1

Patches are welcome. :)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 507 bytes --]

Re: [gentoo-dev] RFC: Standard build environment variables

[Matthew Thode]

[-- Attachment #1: Type: text/plain, Size: 2757 bytes --]

On 20-06-28 08:18:23, Michael Orlitzky wrote:
> As many of you probably know, ago@ has been expanding the scope of our
> CFLAGS/CC support to include some other common build variables:
> 
>   * CC
>   * CXX
>   * AR
>   * CPP
>   * NM
>   * RANLIB
>   * AS
>   * LD
> 
> Some of those are POSIX standards[0],
> 
>   * CC
>   * AR
> 
> Others are de-facto GNU make standards[1],
> 
>   * CXX
>   * CPP
>   * AS
> 
> and a few are de-facto GNU libtool standards[2]:
> 
>   * NM
>   * RANLIB
>   * LD
> 
> If we expect them all to work properly in Gentoo, we have to agree on
> what they mean, and thus how they should be injected into build systems.
> For example, we had a problem with sci-mathematics/pari, whose upstream
> is using the LD environment variable for something other than what GNU
> libtool uses it for. With LD set to something libtooly in the
> environment, the pari build fails. We can solve that by unsetting LD in
> the ebuild, but for that to be The Right Thing To Do, we should be
> expecting LD to contain something libtooly, and thus something
> inappropriate to be passed to the pari build.
> 
> To avoid these issues, I suggest creating a list of "Gentoo environment
> variables" in the devmanual with descriptions of how they should be used
> and pointers to the references (for why we chose that meaning). That way
> a user can export LD, for example, and know that it will be used how he
> thinks it will be used.
> 
> 
> [0] https://pubs.opengroup.org/onlinepubs/009695399/utilities/make.html
> 
> [1]
> https://www.gnu.org/software/make/manual/html_node/Implicit-Variables.html
> 
> [2] https://www.gnu.org/software/libtool/manual/libtool.html
> 

I'd like to suggest allowing only approved variables in the build
environment, having portage unset all variables and setting only what is
needed (or configured).

I'd like to see some variable like this used in make.conf (or maybe one
exists that I don't know about...).

# set the default bash array if GENTOO_EMERGE_ENV is not defined as an array
if ! declare -p GENTOO_EMERGE_ENV  2> /dev/null | grep -q '^declare \-a'; then
    declare -a GENTOO_EMERGE_ENV
    GENTOO_EMERGE_ENV+=("USE=\"-build\"")
    GENTOO_EMERGE_ENV+=("FEATURES=\"binpkg-multi-instance buildpkg parallel-fetch parallel-install\"")
    GENTOO_EMERGE_ENV+=("PKGDIR=\"/tmp/portage-pkgdir\"")
    GENTOO_EMERGE_ENV+=("DISTDIR=\"/tmp/portage-distdir\"")
    GENTOO_EMERGE_ENV+=("PORTDIR=\"/tmp/portage-portdir\"")
    export GENTOO_EMERGE_ENV
fi
# itterate over the array, exporting each 'line'
for (( i=0; i<${#GENTOO_EMERGE_ENV[@]}; i++ )); do
    eval export "${GENTOO_EMERGE_ENV[i]}"
done

-- 
Matthew Thode (prometheanfire)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] RFC: Standard build environment variables

[Michael Orlitzky]

On 2020-06-30 12:22, Matthew Thode wrote:
> 
> I'd like to suggest allowing only approved variables in the build
> environment, having portage unset all variables and setting only what is
> needed (or configured).

I think this is orthogonal to the problem I'm trying to solve. Even if
all environment variables had to be whitelisted, ebuilds would still
need to know how to use them when they happen to be defined.

I basically just want to write down things like "If set, CC is assumed
to contain the name of a compiler driver such as /usr/bin/gcc." That way
ebuilds can be written to pass $CC to the build system in places that
are expecting a compiler driver. Conversely, if LD is documented to
contain a dynamic linker such as /bin/ld, then ebuilds must mangle LD
whenever the upstream build system (e.g. pari, perl) interprets it
otherwise.

These meanings are already enshrined in the tc-getFOO() functions and
the various de-facto standards, but there's no user or developer
documentation promising that the variables will be used in any
particular way.

Re: [gentoo-dev] RFC: Standard build environment variables

[Rich Freeman]

On Wed, Jul 1, 2020 at 9:36 AM Michael Orlitzky <mjo@gentoo.org> wrote:
>
> On 2020-06-30 12:22, Matthew Thode wrote:
> >
> > I'd like to suggest allowing only approved variables in the build
> > environment, having portage unset all variables and setting only what is
> > needed (or configured).
>
> I think this is orthogonal to the problem I'm trying to solve. Even if
> all environment variables had to be whitelisted, ebuilds would still
> need to know how to use them when they happen to be defined.
>

Agree.  I'm not actually certain what that proposal was intended to
convey.  Are we talking about:

1.  Blocking anything that happens to be in the environment when
emerge is run?  (Ie 'CFLAGS="-O2" emerge -1 foo'?)
2.  Blocking any variable at all that isn't whitelisted by an ebuild
or eclass?  (ie CFLAGS in make.conf is ignored unless the ebuild
whitelists it)

I get how environment pollution can cause issues, but #1 is something
we've generally supported for a long time, and it is useful for
troubleshooting/etc or just trying out different things.  Maybe a
FEATURE flag could be used to control it to keep newbs out of trouble,
and you can just as easily pass that in the environment too.

I'm not sure that #2 adds a lot of value.  The default phase functions
probably already don't work well for exotic build systems, and
eclasses can of course take care of remapping for most of the popular
ones.  For one-offs some flag-o-matic or other eclass functions to aid
in remapping variables might be helpful in some cases if there isn't
already something there.

But in any case it isn't essential to what you're proposing.  It does
go along with it to a degree and is worth at least thinking about
(imo)...

-- 
Rich