[gentoo-dev] [PATCH 00/15] scons-utils.eclass: EAPI 6, better docs and cleanup

[gentoo-dev] [PATCH 00/15] scons-utils.eclass: EAPI 6, better docs and cleanup

[Michał Górny]

Hello, everyone.

Here's a bunch of scons-utils.eclass patches for review. This includes
EAPI 6 support with some API changes, MAKEOPTS parsing cleanup
and improved documentation.

Fighting dev-db/mongodb lately, I've decided to explain a few common
issues with SCons in the eclass. This will hopefully help developers
noticing common QA issues like complete disrespect of CC, CXX or PATH,
turning distcc setup into a dumb OOM-killer.

EAPI 6 simplifies the eclass API and makes it more explicit.

The confusing myesconsargs variable is no longer used in EAPI 6 and all
custom arguments need to passed explicitly to each escons call. This
makes it more consistent with emake. And like emake, escons starts truly
respecting nonfatal in EAPI 6.

The use_scons function is completely banned in EAPI 6. Already since
EAPI 5, usex was a cleaner alternative. As a result, USE_SCONS_TRUE
and USE_SCONS_FALSE variables are no longer used (not that anybody used
them anyway).

Finally, the MAKEOPTS parsing and transforming has been improved a bit.
It has been redesigned so that the caching mechanism could actually
work. --jobs lacking argument are now replaced by no-of-cpus+1
(obtained using nproc or python) rather than random, fixed number.

Please review.


Michał Górny (15):
  scons-utils.eclass: Modernize the example not to rely on myescons...
  scons-utils.eclass: Modernize the example to use usex
  scons-utils.eclass: Describe common issues with scons
  scons-utils.eclass: escons doc, 'die' does not respect nonfatal
  scons-utils.eclass: escons, invert EAPI check to cover future EAPIs
  scons-utils.eclass: escons, respect nonfatal in EAPI 6
  scons-utils.eclass: Deprecate myesconsargs, and kill it in EAPI 6
  scons-utils.eclass: Deprecate use_scons, ban it in EAPI 6
  scons-utils.eclass: tests, be more verbose on tests being performed
  scons-utils.eclass: scons_clean_makeopts, mark internal
  scons-utils.eclass: _scons_clean_makeopts, fix result caching
  scons-utils.eclass: _scons_clean_makeopts, stop exporting cache vars
  scons-utils.eclass: _scons_clean_makeopts, clean up and simplify
  scons-utils.eclass: Use nproc when --jobs is used without an argument
  scons-utils.eclass: Enable EAPI 6

 eclass/scons-utils.eclass   | 138 +++++++++++++++++++++++++++++++++-----------
 eclass/tests/scons-utils.sh |  21 ++++---
 2 files changed, 114 insertions(+), 45 deletions(-)

-- 
2.6.4

[gentoo-dev] [PATCH 01/15] scons-utils.eclass: Modernize the example not to rely on myescons...

[Michał Górny]

---
 eclass/scons-utils.eclass | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/eclass/scons-utils.eclass b/eclass/scons-utils.eclass
index a2a6884..e1b3a1b6 100644
--- a/eclass/scons-utils.eclass
+++ b/eclass/scons-utils.eclass
@@ -1,4 +1,4 @@
-# Copyright 1999-2012 Gentoo Foundation
+# Copyright 1999-2015 Gentoo Foundation
 # Distributed under the terms of the GNU General Public License v2
 # $Id$
 
@@ -17,19 +17,19 @@
 # EAPI=4
 #
 # src_configure() {
-# 	myesconsargs=(
+# 	MYSCONS=(
 # 		CC="$(tc-getCC)"
 # 		$(use_scons nls ENABLE_NLS)
 # 	)
 # }
 #
 # src_compile() {
-# 	escons
+# 	escons "${MYSCONS[@]}"
 # }
 #
 # src_install() {
 # 	# note: this can be DESTDIR, INSTALL_ROOT, ... depending on package
-# 	escons DESTDIR="${D}" install
+# 	escons "${MYSCONS[@]}" DESTDIR="${D}" install
 # }
 # @CODE
 
-- 
2.6.4

[gentoo-dev] Re: [PATCH 01/15] scons-utils.eclass: Modernize the example not to rely on myescons...

[Michael Palimaka]

On 02/01/16 03:41, Michał Górny wrote:
> ---
>  eclass/scons-utils.eclass | 8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/eclass/scons-utils.eclass b/eclass/scons-utils.eclass
> index a2a6884..e1b3a1b6 100644
> --- a/eclass/scons-utils.eclass
> +++ b/eclass/scons-utils.eclass
> @@ -1,4 +1,4 @@
> -# Copyright 1999-2012 Gentoo Foundation
> +# Copyright 1999-2015 Gentoo Foundation
>  # Distributed under the terms of the GNU General Public License v2
>  # $Id$
>  
> @@ -17,19 +17,19 @@
>  # EAPI=4
>  #
>  # src_configure() {
> -# 	myesconsargs=(
> +# 	MYSCONS=(
>  # 		CC="$(tc-getCC)"
>  # 		$(use_scons nls ENABLE_NLS)
>  # 	)
>  # }
>  #
>  # src_compile() {
> -# 	escons
> +# 	escons "${MYSCONS[@]}"
>  # }
>  #
>  # src_install() {
>  # 	# note: this can be DESTDIR, INSTALL_ROOT, ... depending on package
> -# 	escons DESTDIR="${D}" install
> +# 	escons "${MYSCONS[@]}" DESTDIR="${D}" install
>  # }
>  # @CODE
>  
> 

What is the benefit of MYSCONS vs myesconsargs? The latter seems to be
more consistent with other eclasses (mycmakeargs, myqmakeargs, ...)

Re: [gentoo-dev] Re: [PATCH 01/15] scons-utils.eclass: Modernize the example not to rely on myescons...

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1920 bytes --]

On Sat, 2 Jan 2016 03:57:59 +1100
Michael Palimaka <kensington@gentoo.org> wrote:

> On 02/01/16 03:41, Michał Górny wrote:
> > ---
> >  eclass/scons-utils.eclass | 8 ++++----
> >  1 file changed, 4 insertions(+), 4 deletions(-)
> > 
> > diff --git a/eclass/scons-utils.eclass b/eclass/scons-utils.eclass
> > index a2a6884..e1b3a1b6 100644
> > --- a/eclass/scons-utils.eclass
> > +++ b/eclass/scons-utils.eclass
> > @@ -1,4 +1,4 @@
> > -# Copyright 1999-2012 Gentoo Foundation
> > +# Copyright 1999-2015 Gentoo Foundation
> >  # Distributed under the terms of the GNU General Public License v2
> >  # $Id$
> >  
> > @@ -17,19 +17,19 @@
> >  # EAPI=4
> >  #
> >  # src_configure() {
> > -# 	myesconsargs=(
> > +# 	MYSCONS=(
> >  # 		CC="$(tc-getCC)"
> >  # 		$(use_scons nls ENABLE_NLS)
> >  # 	)
> >  # }
> >  #
> >  # src_compile() {
> > -# 	escons
> > +# 	escons "${MYSCONS[@]}"
> >  # }
> >  #
> >  # src_install() {
> >  # 	# note: this can be DESTDIR, INSTALL_ROOT, ... depending on package
> > -# 	escons DESTDIR="${D}" install
> > +# 	escons "${MYSCONS[@]}" DESTDIR="${D}" install
> >  # }
> >  # @CODE
> >  
> >   
> 
> What is the benefit of MYSCONS vs myesconsargs? The latter seems to be
> more consistent with other eclasses (mycmakeargs, myqmakeargs, ...)

It's shorter and follows the convention of keeping global variables
uppercase.

Consistency with bad example set by one eclass is not a good argument.
In fact, cmake-utils sets a very bad example of enforcing declarative
model over function-oriented ebuilds which results in really absurd
src_configure() implementations.

  src_configure() {
    local mycmakeargs=(
      ...
    )
    cmake-utils_src_configure
  }

vs

  src_configure() {
    local myconf=(
      ...
    )
    ecmake "${myconf[@]}"
  }

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

[gentoo-dev] Re: [PATCH 01/15] scons-utils.eclass: Modernize the example not to rely on myescons...

[Michael Palimaka]

On 02/01/16 04:15, Michał Górny wrote:
> On Sat, 2 Jan 2016 03:57:59 +1100
> Michael Palimaka <kensington@gentoo.org> wrote:
> 
>> On 02/01/16 03:41, Michał Górny wrote:
>>> ---
>>>  eclass/scons-utils.eclass | 8 ++++----
>>>  1 file changed, 4 insertions(+), 4 deletions(-)
>>>
>>> diff --git a/eclass/scons-utils.eclass b/eclass/scons-utils.eclass
>>> index a2a6884..e1b3a1b6 100644
>>> --- a/eclass/scons-utils.eclass
>>> +++ b/eclass/scons-utils.eclass
>>> @@ -1,4 +1,4 @@
>>> -# Copyright 1999-2012 Gentoo Foundation
>>> +# Copyright 1999-2015 Gentoo Foundation
>>>  # Distributed under the terms of the GNU General Public License v2
>>>  # $Id$
>>>  
>>> @@ -17,19 +17,19 @@
>>>  # EAPI=4
>>>  #
>>>  # src_configure() {
>>> -# 	myesconsargs=(
>>> +# 	MYSCONS=(
>>>  # 		CC="$(tc-getCC)"
>>>  # 		$(use_scons nls ENABLE_NLS)
>>>  # 	)
>>>  # }
>>>  #
>>>  # src_compile() {
>>> -# 	escons
>>> +# 	escons "${MYSCONS[@]}"
>>>  # }
>>>  #
>>>  # src_install() {
>>>  # 	# note: this can be DESTDIR, INSTALL_ROOT, ... depending on package
>>> -# 	escons DESTDIR="${D}" install
>>> +# 	escons "${MYSCONS[@]}" DESTDIR="${D}" install
>>>  # }
>>>  # @CODE
>>>  
>>>   
>>
>> What is the benefit of MYSCONS vs myesconsargs? The latter seems to be
>> more consistent with other eclasses (mycmakeargs, myqmakeargs, ...)
> 
> It's shorter and follows the convention of keeping global variables
> uppercase.

The five-character saving is lost by then requiring "${myconf[@]}" to be
written twice.

> Consistency with bad example set by one eclass is not a good argument.
> In fact, cmake-utils sets a very bad example of enforcing declarative
> model over function-oriented ebuilds which results in really absurd
> src_configure() implementations.
> 
>   src_configure() {
>     local mycmakeargs=(
>       ...
>     )
>     cmake-utils_src_configure
>   }
> 
> vs
> 
>   src_configure() {
>     local myconf=(
>       ...
>     )
>     ecmake "${myconf[@]}"
>   }
> 

I don't see the absurdity.

Re: [gentoo-dev] Re: [PATCH 01/15] scons-utils.eclass: Modernize the example not to rely on myescons...

[Michael Orlitzky]

On 01/01/2016 12:42 PM, Michael Palimaka wrote:
> I don't see the absurdity.

Two formally independent statements that are actually dependent because
of some implicit shared state:

>>
>>   src_configure() {
>>     local mycmakeargs=(
>>       ...
>>     )
>>     cmake-utils_src_configure
>>   }
>>

In this example it's clear that the behavior of ecmake depends on the
value of $myconf:

>>
>>   src_configure() {
>>     local myconf=(
>>       ...
>>     )
>>     ecmake "${myconf[@]}"
>>   }
>>

The latter is a lot more peaceful to maintain. I can hop right into the
code and see that changing $myconf is going to have some effect on the
ecmake line. In the first example, I could change $mycmakeargs 100 lines
away for some unrelated issue. The only way for me to know that it would
effect the cmake-utils_src_configure line is to read the source of that
function proactively (or after it breaks).

[gentoo-dev] [PATCH 02/15] scons-utils.eclass: Modernize the example to use usex

[Michał Górny]

---
 eclass/scons-utils.eclass | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/eclass/scons-utils.eclass b/eclass/scons-utils.eclass
index e1b3a1b6..2c52b59 100644
--- a/eclass/scons-utils.eclass
+++ b/eclass/scons-utils.eclass
@@ -14,12 +14,12 @@
 # @CODE
 # inherit scons-utils toolchain-funcs
 #
-# EAPI=4
+# EAPI=5
 #
 # src_configure() {
 # 	MYSCONS=(
 # 		CC="$(tc-getCC)"
-# 		$(use_scons nls ENABLE_NLS)
+#		ENABLE_NLS=$(usex nls)
 # 	)
 # }
 #
-- 
2.6.4

[gentoo-dev] [PATCH 03/15] scons-utils.eclass: Describe common issues with scons

[Michał Górny]

---
 eclass/scons-utils.eclass | 17 ++++++++++++++++-
 1 file changed, 16 insertions(+), 1 deletion(-)

diff --git a/eclass/scons-utils.eclass b/eclass/scons-utils.eclass
index 2c52b59..4b1fe49 100644
--- a/eclass/scons-utils.eclass
+++ b/eclass/scons-utils.eclass
@@ -9,8 +9,23 @@
 # @DESCRIPTION:
 # This eclass provides a set of function to help developers sanely call
 # dev-util/scons and pass parameters to it.
-# @EXAMPLE:
 #
+# Please note that SCons is more like a 'build system creation kit',
+# and requires a lot of upstream customization to be useful.
+# You will often need to request fixes upstream and/or patch the build
+# system. In particular:
+#
+# 1. There are no 'standard' variables. To respect CC, CXX, CFLAGS,
+# CXXFLAGS, CPPFLAGS, LDFLAGS, upstream needs to define appropriate
+# variables explicitly. In some cases, upstreams respect envvars,
+# in others you need to pass them as options.
+#
+# 2. SCons scrubs out environment by default and replaces it with some
+# pre-defined values. To respect environment variables such as PATH,
+# upstreams need to explicitly get them from os.environ and copy them
+# to the build environment.
+#
+# @EXAMPLE:
 # @CODE
 # inherit scons-utils toolchain-funcs
 #
-- 
2.6.4

[gentoo-dev] [PATCH 04/15] scons-utils.eclass: escons doc, 'die' does not respect nonfatal

[Michał Górny]

---
 eclass/scons-utils.eclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/eclass/scons-utils.eclass b/eclass/scons-utils.eclass
index 4b1fe49..fa82236 100644
--- a/eclass/scons-utils.eclass
+++ b/eclass/scons-utils.eclass
@@ -107,7 +107,7 @@ fi
 # @DESCRIPTION:
 # Call scons, passing the supplied arguments, ${myesconsargs[@]},
 # filtered ${MAKEOPTS}, ${EXTRA_ESCONS}. Similar to emake. Like emake,
-# this function does die on failure in EAPI 4 (unless called nonfatal).
+# this function does die on failure in EAPI 4.
 escons() {
 	local ret
 
-- 
2.6.4

[gentoo-dev] [PATCH 05/15] scons-utils.eclass: escons, invert EAPI check to cover future EAPIs

[Michał Górny]

---
 eclass/scons-utils.eclass | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/eclass/scons-utils.eclass b/eclass/scons-utils.eclass
index fa82236..4defbac 100644
--- a/eclass/scons-utils.eclass
+++ b/eclass/scons-utils.eclass
@@ -120,7 +120,9 @@ escons() {
 	"${@}"
 	ret=${?}
 
-	[[ ${ret} -ne 0 ]] && has "${EAPI:-0}" 4 5 && die "escons failed."
+	if [[ ${ret} -ne 0 ]]; then
+		[[ ${EAPI:-0} != [0123] ]] && die "escons failed."
+	fi
 	return ${ret}
 }
 
-- 
2.6.4

[gentoo-dev] [PATCH 06/15] scons-utils.eclass: escons, respect nonfatal in EAPI 6

[Michał Górny]

---
 eclass/scons-utils.eclass | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/eclass/scons-utils.eclass b/eclass/scons-utils.eclass
index 4defbac..fe9160a0 100644
--- a/eclass/scons-utils.eclass
+++ b/eclass/scons-utils.eclass
@@ -107,7 +107,8 @@ fi
 # @DESCRIPTION:
 # Call scons, passing the supplied arguments, ${myesconsargs[@]},
 # filtered ${MAKEOPTS}, ${EXTRA_ESCONS}. Similar to emake. Like emake,
-# this function does die on failure in EAPI 4.
+# this function does die on failure in EAPI 4. Respects nonfatal
+# in EAPI 6 and newer.
 escons() {
 	local ret
 
@@ -121,7 +122,16 @@ escons() {
 	ret=${?}
 
 	if [[ ${ret} -ne 0 ]]; then
-		[[ ${EAPI:-0} != [0123] ]] && die "escons failed."
+		case "${EAPI:-0}" in
+			0|1|2|3) # nonfatal in EAPIs 0 through 3
+				;;
+			4|5) # 100% fatal in 4 & 5
+				die "escons failed."
+				;;
+			*) # respect nonfatal in 6 onwards
+				die -n "escons failed."
+				;;
+		esac
 	fi
 	return ${ret}
 }
-- 
2.6.4

[gentoo-dev] [PATCH 07/15] scons-utils.eclass: Deprecate myesconsargs, and kill it in EAPI 6

[Michał Górny]

---
 eclass/scons-utils.eclass | 17 +++++++++++------
 1 file changed, 11 insertions(+), 6 deletions(-)

diff --git a/eclass/scons-utils.eclass b/eclass/scons-utils.eclass
index fe9160a0..fdc6af3 100644
--- a/eclass/scons-utils.eclass
+++ b/eclass/scons-utils.eclass
@@ -58,6 +58,8 @@
 # @VARIABLE: myesconsargs
 # @DEFAULT_UNSET
 # @DESCRIPTION:
+# DEPRECATED, EAPI 0..5 ONLY: pass options to escons instead
+#
 # List of package-specific options to pass to all SCons calls. Supposed to be
 # set in src_configure().
 
@@ -103,20 +105,23 @@ fi
 # -- public functions --
 
 # @FUNCTION: escons
-# @USAGE: [scons-arg] ...
+# @USAGE: [<args>...]
 # @DESCRIPTION:
-# Call scons, passing the supplied arguments, ${myesconsargs[@]},
-# filtered ${MAKEOPTS}, ${EXTRA_ESCONS}. Similar to emake. Like emake,
-# this function does die on failure in EAPI 4. Respects nonfatal
-# in EAPI 6 and newer.
+# Call scons, passing the supplied arguments. Like emake, this function
+# does die on failure in EAPI 4. Respects nonfatal in EAPI 6 and newer.
 escons() {
 	local ret
 
 	debug-print-function ${FUNCNAME} "${@}"
 
+	# Use myesconsargs in EAPI 5 and older
+	if [[ ${EAPI} == [012345] ]]; then
+		set -- "${myesconsargs[@]}" "${@}"
+	fi
+
 	# if SCONSOPTS are _unset_, use cleaned MAKEOPTS
 	set -- scons ${SCONSOPTS-$(scons_clean_makeopts)} ${EXTRA_ESCONS} \
-		"${myesconsargs[@]}" "${@}"
+		"${@}"
 	echo "${@}" >&2
 	"${@}"
 	ret=${?}
-- 
2.6.4

[gentoo-dev] [PATCH 08/15] scons-utils.eclass: Deprecate use_scons, ban it in EAPI 6

[Michał Górny]

---
 eclass/scons-utils.eclass | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/eclass/scons-utils.eclass b/eclass/scons-utils.eclass
index fdc6af3..b8051b0 100644
--- a/eclass/scons-utils.eclass
+++ b/eclass/scons-utils.eclass
@@ -79,11 +79,15 @@
 
 # @ECLASS-VARIABLE: USE_SCONS_TRUE
 # @DESCRIPTION:
+# DEPRECATED: use usex instead
+#
 # The default value for truth in scons-use() (1 by default).
 : ${USE_SCONS_TRUE:=1}
 
 # @ECLASS-VARIABLE: USE_SCONS_FALSE
 # @DESCRIPTION:
+# DEPRECATED: use usex instead
+#
 # The default value for false in scons-use() (0 by default).
 : ${USE_SCONS_FALSE:=0}
 
@@ -236,6 +240,8 @@ scons_clean_makeopts() {
 # @FUNCTION: use_scons
 # @USAGE: <use-flag> [var-name] [var-opt-true] [var-opt-false]
 # @DESCRIPTION:
+# DEPRECATED, EAPI 0..5 ONLY: use usex instead
+#
 # Output a SCons parameter with value depending on the USE flag state.
 # If the USE flag is set, output <var-name>=<var-opt-true>; otherwise
 # <var-name>=<var-opt-false>.
@@ -247,6 +253,9 @@ scons_clean_makeopts() {
 # If <var-opt-true> and/or <var-opt-false> are omitted,
 # ${USE_SCONS_TRUE} and/or ${USE_SCONS_FALSE} will be used instead.
 use_scons() {
+	[[ ${EAPI} == [012345] ]] \
+		|| die "${FUNCNAME} is banned in EAPI ${EAPI}, use usex instead"
+
 	local flag=${1}
 	local varname=${2:-${flag/\!/no}}
 	local vartrue=${3:-${USE_SCONS_TRUE}}
-- 
2.6.4

[gentoo-dev] [PATCH 09/15] scons-utils.eclass: tests, be more verbose on tests being performed

[Michał Górny]

---
 eclass/tests/scons-utils.sh | 14 ++++++--------
 1 file changed, 6 insertions(+), 8 deletions(-)

diff --git a/eclass/tests/scons-utils.sh b/eclass/tests/scons-utils.sh
index 5a65fbe..7387135 100755
--- a/eclass/tests/scons-utils.sh
+++ b/eclass/tests/scons-utils.sh
@@ -8,7 +8,9 @@ source tests-common.sh
 inherit scons-utils
 
 test-scons_clean_makeopts() {
-	local sconsopts=$(scons_clean_makeopts ${1})
+	tbegin "scons_clean_makeopts() for ${1}"
+
+	local sconsopts=$(scons_clean_makeopts ${1}) ret=0
 
 	if [[ ${sconsopts} != ${2-${1}} ]]; then
 		eerror "Self-test failed:"
@@ -17,11 +19,11 @@ test-scons_clean_makeopts() {
 		eerror "Expected: ${2-${1}}"
 		eerror "Actual: ${sconsopts}"
 		eoutdent
-		(( ++failed ))
-		return 1
+		ret=1
 	fi
 
-	return 0
+	tend ${ret}
+	return ${ret}
 }
 
 # jobcount expected for non-specified state
@@ -29,8 +31,6 @@ jc=5
 # failed test counter
 failed=0
 
-tbegin "scons_clean_makeopts()"
-
 # sane MAKEOPTS
 test-scons_clean_makeopts '--jobs=14 -k'
 test-scons_clean_makeopts '--jobs=14 -k'
@@ -59,6 +59,4 @@ test-scons_clean_makeopts '--jobs funnystuff -k' "--jobs=${jc} -k"
 test-scons_clean_makeopts '--jobs -l3' "--jobs=${jc}"
 test-scons_clean_makeopts '-j -l3' "-j ${jc}"
 
-tend ${failed}
-
 texit
-- 
2.6.4

[gentoo-dev] [PATCH 10/15] scons-utils.eclass: scons_clean_makeopts, mark internal

[Michał Górny]

---
 eclass/scons-utils.eclass   | 7 ++++---
 eclass/tests/scons-utils.sh | 2 +-
 2 files changed, 5 insertions(+), 4 deletions(-)

diff --git a/eclass/scons-utils.eclass b/eclass/scons-utils.eclass
index b8051b0..e5e309b 100644
--- a/eclass/scons-utils.eclass
+++ b/eclass/scons-utils.eclass
@@ -124,7 +124,7 @@ escons() {
 	fi
 
 	# if SCONSOPTS are _unset_, use cleaned MAKEOPTS
-	set -- scons ${SCONSOPTS-$(scons_clean_makeopts)} ${EXTRA_ESCONS} \
+	set -- scons ${SCONSOPTS-$(_scons_clean_makeopts)} ${EXTRA_ESCONS} \
 		"${@}"
 	echo "${@}" >&2
 	"${@}"
@@ -145,14 +145,15 @@ escons() {
 	return ${ret}
 }
 
-# @FUNCTION: scons_clean_makeopts
+# @FUNCTION: _scons_clean_makeopts
+# @INTERNAL
 # @USAGE: [makeflags] [...]
 # @DESCRIPTION:
 # Strip the supplied makeflags (or ${MAKEOPTS} if called without
 # an argument) of options not supported by SCons and make sure --jobs
 # gets an argument. Output the resulting flag list (suitable
 # for an assignment to SCONSOPTS).
-scons_clean_makeopts() {
+_scons_clean_makeopts() {
 	local new_makeopts
 
 	debug-print-function ${FUNCNAME} "${@}"
diff --git a/eclass/tests/scons-utils.sh b/eclass/tests/scons-utils.sh
index 7387135..df8af7a 100755
--- a/eclass/tests/scons-utils.sh
+++ b/eclass/tests/scons-utils.sh
@@ -10,7 +10,7 @@ inherit scons-utils
 test-scons_clean_makeopts() {
 	tbegin "scons_clean_makeopts() for ${1}"
 
-	local sconsopts=$(scons_clean_makeopts ${1}) ret=0
+	local sconsopts=$(_scons_clean_makeopts ${1}) ret=0
 
 	if [[ ${sconsopts} != ${2-${1}} ]]; then
 		eerror "Self-test failed:"
-- 
2.6.4

[gentoo-dev] [PATCH 11/15] scons-utils.eclass: _scons_clean_makeopts, fix result caching

[Michał Górny]

Stop calling _scons_clean_makeopts in a subshell in order to make
it possible for the cache to be preserved. Pass the result through
SCONSOPTS variable.
---
 eclass/scons-utils.eclass   | 15 +++++++++------
 eclass/tests/scons-utils.sh |  7 ++++---
 2 files changed, 13 insertions(+), 9 deletions(-)

diff --git a/eclass/scons-utils.eclass b/eclass/scons-utils.eclass
index e5e309b..4da2c4a 100644
--- a/eclass/scons-utils.eclass
+++ b/eclass/scons-utils.eclass
@@ -124,8 +124,12 @@ escons() {
 	fi
 
 	# if SCONSOPTS are _unset_, use cleaned MAKEOPTS
-	set -- scons ${SCONSOPTS-$(_scons_clean_makeopts)} ${EXTRA_ESCONS} \
-		"${@}"
+	if [[ ! ${SCONSOPTS+set} ]]; then
+		local SCONSOPTS
+		_scons_clean_makeopts
+	fi
+
+	set -- scons ${SCONSOPTS} ${EXTRA_ESCONS} "${@}"
 	echo "${@}" >&2
 	"${@}"
 	ret=${?}
@@ -169,9 +173,8 @@ _scons_clean_makeopts() {
 	# empty MAKEOPTS give out empty SCONSOPTS
 	# thus, we do need to worry about the initial setup
 	if [[ ${*} = ${_SCONS_CACHE_MAKEOPTS} ]]; then
-		set -- ${_SCONS_CACHE_SCONSOPTS}
-		debug-print "Cache hit: [${*}]"
-		echo ${*}
+		SCONSOPTS=${_SCONS_CACHE_SCONSOPTS}
+		debug-print "Cache hit: [${SCONSOPTS}]"
 		return
 	fi
 	export _SCONS_CACHE_MAKEOPTS=${*}
@@ -235,7 +238,7 @@ _scons_clean_makeopts() {
 	set -- ${new_makeopts}
 	export _SCONS_CACHE_SCONSOPTS=${*}
 	debug-print "New SCONSOPTS: [${*}]"
-	echo ${*}
+	SCONSOPTS=${*}
 }
 
 # @FUNCTION: use_scons
diff --git a/eclass/tests/scons-utils.sh b/eclass/tests/scons-utils.sh
index df8af7a..6355c54 100755
--- a/eclass/tests/scons-utils.sh
+++ b/eclass/tests/scons-utils.sh
@@ -10,14 +10,15 @@ inherit scons-utils
 test-scons_clean_makeopts() {
 	tbegin "scons_clean_makeopts() for ${1}"
 
-	local sconsopts=$(_scons_clean_makeopts ${1}) ret=0
+	local SCONSOPTS ret=0
+	_scons_clean_makeopts ${1}
 
-	if [[ ${sconsopts} != ${2-${1}} ]]; then
+	if [[ ${SCONSOPTS} != ${2-${1}} ]]; then
 		eerror "Self-test failed:"
 		eindent
 		eerror "MAKEOPTS: ${1}"
 		eerror "Expected: ${2-${1}}"
-		eerror "Actual: ${sconsopts}"
+		eerror "Actual: ${SCONSOPTS}"
 		eoutdent
 		ret=1
 	fi
-- 
2.6.4

[gentoo-dev] [PATCH 12/15] scons-utils.eclass: _scons_clean_makeopts, stop exporting cache vars

[Michał Górny]

Cache vars need only to be global to be preserved across calls.
Exporting them is undesired.
---
 eclass/scons-utils.eclass | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/eclass/scons-utils.eclass b/eclass/scons-utils.eclass
index 4da2c4a..82e45e3 100644
--- a/eclass/scons-utils.eclass
+++ b/eclass/scons-utils.eclass
@@ -177,7 +177,7 @@ _scons_clean_makeopts() {
 		debug-print "Cache hit: [${SCONSOPTS}]"
 		return
 	fi
-	export _SCONS_CACHE_MAKEOPTS=${*}
+	_SCONS_CACHE_MAKEOPTS=${*}
 
 	while [[ ${#} -gt 0 ]]; do
 		case ${1} in
@@ -236,7 +236,7 @@ _scons_clean_makeopts() {
 	done
 
 	set -- ${new_makeopts}
-	export _SCONS_CACHE_SCONSOPTS=${*}
+	_SCONS_CACHE_SCONSOPTS=${*}
 	debug-print "New SCONSOPTS: [${*}]"
 	SCONSOPTS=${*}
 }
-- 
2.6.4

[gentoo-dev] [PATCH 13/15] scons-utils.eclass: _scons_clean_makeopts, clean up and simplify

[Michał Górny]

---
 eclass/scons-utils.eclass | 25 ++++++++++++-------------
 1 file changed, 12 insertions(+), 13 deletions(-)

diff --git a/eclass/scons-utils.eclass b/eclass/scons-utils.eclass
index 82e45e3..3185282 100644
--- a/eclass/scons-utils.eclass
+++ b/eclass/scons-utils.eclass
@@ -158,7 +158,7 @@ escons() {
 # gets an argument. Output the resulting flag list (suitable
 # for an assignment to SCONSOPTS).
 _scons_clean_makeopts() {
-	local new_makeopts
+	local new_makeopts=()
 
 	debug-print-function ${FUNCNAME} "${@}"
 
@@ -183,16 +183,16 @@ _scons_clean_makeopts() {
 		case ${1} in
 			# clean, simple to check -- we like that
 			--jobs=*|--keep-going)
-				new_makeopts=${new_makeopts+${new_makeopts} }${1}
+				new_makeopts+=( ${1} )
 				;;
 			# need to take a look at the next arg and guess
 			--jobs)
 				if [[ ${#} -gt 1 && ${2} =~ ^[0-9]+$ ]]; then
-					new_makeopts="${new_makeopts+${new_makeopts} }${1} ${2}"
+					new_makeopts+=( ${1} ${2} )
 					shift
 				else
 					# no value means no limit, let's pass a random int
-					new_makeopts=${new_makeopts+${new_makeopts} }${1}=5
+					new_makeopts+=( ${1}=5 )
 				fi
 				;;
 			# strip other long options
@@ -207,20 +207,20 @@ _scons_clean_makeopts() {
 				while [[ -n ${str} ]]; do
 					case ${str} in
 						k*)
-							new_optstr=${new_optstr}k
+							new_optstr+=k
 							;;
 						# -j needs to come last
 						j)
 							if [[ ${#} -gt 1 && ${2} =~ ^[0-9]+$ ]]; then
-								new_optstr="${new_optstr}j ${2}"
+								new_optstr+="j ${2}"
 								shift
 							else
-								new_optstr="${new_optstr}j 5"
+								new_optstr+="j 5"
 							fi
 							;;
 						# otherwise, everything after -j is treated as an arg
 						j*)
-							new_optstr=${new_optstr}${str}
+							new_optstr+=${str}
 							break
 							;;
 					esac
@@ -228,17 +228,16 @@ _scons_clean_makeopts() {
 				done
 
 				if [[ -n ${new_optstr} ]]; then
-					new_makeopts=${new_makeopts+${new_makeopts} }-${new_optstr}
+					new_makeopts+=( -${new_optstr} )
 				fi
 				;;
 		esac
 		shift
 	done
 
-	set -- ${new_makeopts}
-	_SCONS_CACHE_SCONSOPTS=${*}
-	debug-print "New SCONSOPTS: [${*}]"
-	SCONSOPTS=${*}
+	SCONSOPTS=${new_makeopts[*]}
+	_SCONS_CACHE_SCONSOPTS=${SCONSOPTS}
+	debug-print "New SCONSOPTS: [${SCONSOPTS}]"
 }
 
 # @FUNCTION: use_scons
-- 
2.6.4

[gentoo-dev] [PATCH 14/15] scons-utils.eclass: Use nproc when --jobs is used without an argument

[Michał Górny]

Try to guess the number of processors when --jobs is passed without
an argument. We can't use a high number equivalent to GNU make behavior
(no limit) since SCons does not have an equivalent of --load-avg option.
Still, this is better than assuming some random, fixed number.
---
 eclass/scons-utils.eclass   | 30 ++++++++++++++++++++++++++++--
 eclass/tests/scons-utils.sh |  2 +-
 2 files changed, 29 insertions(+), 3 deletions(-)

diff --git a/eclass/scons-utils.eclass b/eclass/scons-utils.eclass
index 3185282..89618f9 100644
--- a/eclass/scons-utils.eclass
+++ b/eclass/scons-utils.eclass
@@ -149,6 +149,32 @@ escons() {
 	return ${ret}
 }
 
+# @FUNCTION: _scons_get_default_jobs
+# @INTERNAL
+# @DESCRIPTION:
+# Output the default number of jobs, used if -j is used without
+# argument. Tries to figure out the number of logical CPUs, falling
+# back to hardcoded constant.
+_scons_get_default_jobs() {
+	local nproc
+
+	if type -P nproc &>/dev/null; then
+		# GNU
+		nproc=$(nproc)
+	elif type -P python &>/dev/null; then
+		# fallback to python2.6+
+		# note: this may fail (raise NotImplementedError)
+		nproc=$(python -c 'import multiprocessing; print(multiprocessing.cpu_count());' 2>/dev/null)
+	fi
+
+	if [[ ${nproc} ]]; then
+		echo $(( nproc + 1 ))
+	else
+		# random default
+		echo 5
+	fi
+}
+
 # @FUNCTION: _scons_clean_makeopts
 # @INTERNAL
 # @USAGE: [makeflags] [...]
@@ -192,7 +218,7 @@ _scons_clean_makeopts() {
 					shift
 				else
 					# no value means no limit, let's pass a random int
-					new_makeopts+=( ${1}=5 )
+					new_makeopts+=( ${1}=$(_scons_get_default_jobs) )
 				fi
 				;;
 			# strip other long options
@@ -215,7 +241,7 @@ _scons_clean_makeopts() {
 								new_optstr+="j ${2}"
 								shift
 							else
-								new_optstr+="j 5"
+								new_optstr+="j $(_scons_get_default_jobs)"
 							fi
 							;;
 						# otherwise, everything after -j is treated as an arg
diff --git a/eclass/tests/scons-utils.sh b/eclass/tests/scons-utils.sh
index 6355c54..fcb5125 100755
--- a/eclass/tests/scons-utils.sh
+++ b/eclass/tests/scons-utils.sh
@@ -28,7 +28,7 @@ test-scons_clean_makeopts() {
 }
 
 # jobcount expected for non-specified state
-jc=5
+jc=$(_scons_get_default_jobs)
 # failed test counter
 failed=0
 
-- 
2.6.4

[gentoo-dev] [PATCH 15/15] scons-utils.eclass: Enable EAPI 6

[Michał Górny]

---
 eclass/scons-utils.eclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/eclass/scons-utils.eclass b/eclass/scons-utils.eclass
index 89618f9..0493ec1 100644
--- a/eclass/scons-utils.eclass
+++ b/eclass/scons-utils.eclass
@@ -94,7 +94,7 @@
 # -- EAPI support check --
 
 case ${EAPI:-0} in
-	0|1|2|3|4|5) ;;
+	0|1|2|3|4|5|6) ;;
 	*) die "EAPI ${EAPI} unsupported."
 esac
 
-- 
2.6.4

Re: [gentoo-dev] [PATCH 00/15] scons-utils.eclass: EAPI 6, better docs and cleanup

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2713 bytes --]

On Fri,  1 Jan 2016 17:41:06 +0100
Michał Górny <mgorny@gentoo.org> wrote:

> Hello, everyone.
> 
> Here's a bunch of scons-utils.eclass patches for review. This includes
> EAPI 6 support with some API changes, MAKEOPTS parsing cleanup
> and improved documentation.
> 
> Fighting dev-db/mongodb lately, I've decided to explain a few common
> issues with SCons in the eclass. This will hopefully help developers
> noticing common QA issues like complete disrespect of CC, CXX or PATH,
> turning distcc setup into a dumb OOM-killer.
> 
> EAPI 6 simplifies the eclass API and makes it more explicit.
> 
> The confusing myesconsargs variable is no longer used in EAPI 6 and all
> custom arguments need to passed explicitly to each escons call. This
> makes it more consistent with emake. And like emake, escons starts truly
> respecting nonfatal in EAPI 6.
> 
> The use_scons function is completely banned in EAPI 6. Already since
> EAPI 5, usex was a cleaner alternative. As a result, USE_SCONS_TRUE
> and USE_SCONS_FALSE variables are no longer used (not that anybody used
> them anyway).
> 
> Finally, the MAKEOPTS parsing and transforming has been improved a bit.
> It has been redesigned so that the caching mechanism could actually
> work. --jobs lacking argument are now replaced by no-of-cpus+1
> (obtained using nproc or python) rather than random, fixed number.
> 
> Please review.
> 
> 
> Michał Górny (15):
>   scons-utils.eclass: Modernize the example not to rely on myescons...
>   scons-utils.eclass: Modernize the example to use usex
>   scons-utils.eclass: Describe common issues with scons
>   scons-utils.eclass: escons doc, 'die' does not respect nonfatal
>   scons-utils.eclass: escons, invert EAPI check to cover future EAPIs
>   scons-utils.eclass: escons, respect nonfatal in EAPI 6
>   scons-utils.eclass: Deprecate myesconsargs, and kill it in EAPI 6
>   scons-utils.eclass: Deprecate use_scons, ban it in EAPI 6
>   scons-utils.eclass: tests, be more verbose on tests being performed
>   scons-utils.eclass: scons_clean_makeopts, mark internal
>   scons-utils.eclass: _scons_clean_makeopts, fix result caching
>   scons-utils.eclass: _scons_clean_makeopts, stop exporting cache vars
>   scons-utils.eclass: _scons_clean_makeopts, clean up and simplify
>   scons-utils.eclass: Use nproc when --jobs is used without an argument
>   scons-utils.eclass: Enable EAPI 6
> 
>  eclass/scons-utils.eclass   | 138 +++++++++++++++++++++++++++++++++-----------
>  eclass/tests/scons-utils.sh |  21 ++++---
>  2 files changed, 114 insertions(+), 45 deletions(-)

Merged.

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]