[gentoo-dev] [PATCHES] multibuild.eclass: custom phase function helpers

[gentoo-dev] [PATCHES] multibuild.eclass: custom phase function helpers

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1413 bytes --]

Hello,

distutils-r1 (and previously python-distutils-ng) was using custom
phase functions for a while. Recently, hasufell added multilib-minimal
which does the same. Since in both cases the custom functions are
closely related to building multiple variants of the package, I'm
thinking of adding a few helper functions to the multibuild.eclass.

The framework I'm suggesting will make the custom phase functions
behave similarly to regular ones. Most importantly, eclasses will be
provided with a function to 'export' them.

I'm sending the following patches in reply to this thread:

1) introduces two functions to handle phases:

- multibuild_export_phases -- which exports the listed phases for
  current eclass,
- multibuild_get_phase_function -- which finds and prints the best
  handler for named phase.

The export/find logic follows one used for regular phases. If function
named ${phase} is declared, it is used. Otherwise, the function
exported by most recent eclass is used.


2) uses the new functions in distutils-r1,

3-4) general cleanup of multilib-minimal to make it suitable
   for conversion,

5) uses the new functions in multilib-minimal,

6) runs multilib_src_configure() in parallel to save time,

7) makes autotools-multilib reuse multilib-minimal through exporting
sub-phase functions.


Comments? Questions?

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]

[gentoo-dev] [PATCH 1/7] multibuild: introduce a generic framework for custom phase functions.

[Michał Górny]

The framework provides functions to declare, export and obtain custom
phase functions.

Each of the custom phases can be defined by eclasses and ebuilds
in a manner similar to regular phases. The eclasses define
${ECLASS}_${phase} function and run 'multibuild_export_phases' to
register them. The ebuilds define ${phase} function and it automatically
takes precedence over eclass-defined ones.
---
 gx86/eclass/multibuild.eclass | 61 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 61 insertions(+)

diff --git a/gx86/eclass/multibuild.eclass b/gx86/eclass/multibuild.eclass
index bc510e9..ea0500c 100644
--- a/gx86/eclass/multibuild.eclass
+++ b/gx86/eclass/multibuild.eclass
@@ -28,6 +28,8 @@ if [[ ! ${_MULTIBUILD} ]]; then
 
 inherit multiprocessing
 
+DEPEND=">=app-shells/bash-4.2"
+
 # @ECLASS-VARIABLE: MULTIBUILD_VARIANTS
 # @DESCRIPTION:
 # An array specifying all enabled variants which multibuild_foreach*
@@ -245,5 +247,64 @@ run_in_build_dir() {
 	return ${ret}
 }
 
+# @FUNCTION: multibuild_export_phases
+# @USAGE: <phase-name>...
+# @DESCRIPTION:
+# Export the eclass phase functions for named phases. The functions need
+# be named ${ECLASS}_<phase-name>. The exported functions will override
+# any previously exported phases.
+multibuild_export_phases() {
+	debug-print-function ${FUNCNAME} "${@}"
+
+	[[ ${#} -eq 0 ]] && die "Usage: multibuild_export_phases <phase-name>..."
+
+	# This is necessary to allow sourcing of the ebuild with old bash
+	# versions, e.g. for cachegen.
+	if [[ $(( (BASH_VERSINFO[0] << 8) + BASH_VERSINFO[1] )) \
+			-lt $(( (4 << 8) + 2 )) ]] ; then
+		_MULTIBUILD_ANCIENT_BASH=1
+		return 0
+	fi
+
+	declare -g -A _MULTIBUILD_EXPORTED_PHASES || die
+	local p
+	for p; do
+		_MULTIBUILD_EXPORTED_PHASES[${p}]=${ECLASS}_${p}
+	done
+}
+
+# @FUNCTION: multibuild_get_phase_function
+# @USAGE: <phase-name>
+# @DESCRIPTION:
+# Find the topmost handler for the named phase. It can be either
+# user-defined phase handler (with the same name as the phase)
+# or a handler exported most recently by an eclass.
+#
+# Prints the function name to stdout or null string if there is
+# no handler for the phase.
+multibuild_get_phase_function() {
+	debug-print-function ${FUNCNAME} "${@}"
+
+	[[ ${#} -ne 1 ]] && die "Usage: multibuild_get_phase_function <phase-name>"
+	if [[ ${_MULTIBUILD_ANCIENT_BASH} ]]; then
+		die "Old bash (<4.2) used to start the build, please restart emerge."
+	fi
+
+	local phase=${1}
+
+	# user-defined phase
+	if ! declare -f "${phase}" >/dev/null; then
+		phase=${_MULTIBUILD_EXPORTED_PHASES[${phase}]}
+
+		if [[ ! ${phase} ]]; then
+			return
+		elif ! declare -f "${phase}" >/dev/null; then
+			die "Phase function ${phase} exported but never defined!"
+		fi
+	fi
+
+	echo "${phase}"
+}
+
 _MULTIBUILD=1
 fi
-- 
1.8.1.5

[gentoo-dev] [PATCH 2/7] distutils-r1: use multibuild phase helpers.

[Michał Górny]

---
 gx86/eclass/distutils-r1.eclass | 71 ++++++++++++++++-------------------------
 1 file changed, 27 insertions(+), 44 deletions(-)

diff --git a/gx86/eclass/distutils-r1.eclass b/gx86/eclass/distutils-r1.eclass
index 264ce9c..ce518a7 100644
--- a/gx86/eclass/distutils-r1.eclass
+++ b/gx86/eclass/distutils-r1.eclass
@@ -92,6 +92,8 @@ fi
 if [[ ! ${DISTUTILS_OPTIONAL} ]]; then
 	EXPORT_FUNCTIONS src_prepare src_configure src_compile src_test src_install
 fi
+multibuild_export_phases \
+	python_prepare_all python_compile python_install python_install_all
 
 if [[ ! ${_DISTUTILS_R1} ]]; then
 
@@ -559,6 +561,9 @@ distutils-r1_run_phase() {
 # If in-source build is used, the command will be run in the copy
 # of sources made for the best Python interpreter.
 _distutils-r1_run_common_phase() {
+	# not all phases need be set
+	[[ ${1} ]] || return
+
 	local DISTUTILS_ORIG_BUILD_DIR=${BUILD_DIR}
 
 	local MULTIBUILD_VARIANTS
@@ -576,6 +581,9 @@ _distutils-r1_run_common_phase() {
 _distutils-r1_run_foreach_impl() {
 	debug-print-function ${FUNCNAME} "${@}"
 
+	# not all phases need be set
+	[[ ${1} ]] || return
+
 	set -- distutils-r1_run_phase "${@}"
 
 	if [[ ! ${DISTUTILS_SINGLE_IMPL} ]]; then
@@ -599,68 +607,43 @@ _distutils-r1_run_foreach_impl() {
 distutils-r1_src_prepare() {
 	debug-print-function ${FUNCNAME} "${@}"
 
-	# common preparations
-	if declare -f python_prepare_all >/dev/null; then
-		python_prepare_all
-	else
-		distutils-r1_python_prepare_all
-	fi
-
-	if declare -f python_prepare >/dev/null; then
-		_distutils-r1_run_foreach_impl python_prepare
-	fi
+	"$(multibuild_get_phase_function python_prepare_all)"
+	_distutils-r1_run_foreach_impl \
+		"$(multibuild_get_phase_function python_prepare)"
 }
 
 distutils-r1_src_configure() {
-	if declare -f python_configure >/dev/null; then
-		_distutils-r1_run_foreach_impl python_configure
-	fi
-
-	if declare -f python_configure_all >/dev/null; then
-		_distutils-r1_run_common_phase python_configure_all
-	fi
+	_distutils-r1_run_foreach_impl \
+		"$(multibuild_get_phase_function python_configure)"
+	_distutils-r1_run_common_phase \
+		"$(multibuild_get_phase_function python_configure_all)"
 }
 
 distutils-r1_src_compile() {
 	debug-print-function ${FUNCNAME} "${@}"
 
-	if declare -f python_compile >/dev/null; then
-		_distutils-r1_run_foreach_impl python_compile
-	else
-		_distutils-r1_run_foreach_impl distutils-r1_python_compile
-	fi
-
-	if declare -f python_compile_all >/dev/null; then
-		_distutils-r1_run_common_phase python_compile_all
-	fi
+	_distutils-r1_run_foreach_impl \
+		"$(multibuild_get_phase_function python_compile)"
+	_distutils-r1_run_common_phase \
+		"$(multibuild_get_phase_function python_compile_all)"
 }
 
 distutils-r1_src_test() {
 	debug-print-function ${FUNCNAME} "${@}"
 
-	if declare -f python_test >/dev/null; then
-		_distutils-r1_run_foreach_impl python_test
-	fi
-
-	if declare -f python_test_all >/dev/null; then
-		_distutils-r1_run_common_phase python_test_all
-	fi
+	_distutils-r1_run_foreach_impl \
+		"$(multibuild_get_phase_function python_test)"
+	_distutils-r1_run_common_phase \
+		"$(multibuild_get_phase_function python_test_all)"
 }
 
 distutils-r1_src_install() {
 	debug-print-function ${FUNCNAME} "${@}"
 
-	if declare -f python_install >/dev/null; then
-		_distutils-r1_run_foreach_impl python_install
-	else
-		_distutils-r1_run_foreach_impl distutils-r1_python_install
-	fi
-
-	if declare -f python_install_all >/dev/null; then
-		_distutils-r1_run_common_phase python_install_all
-	else
-		_distutils-r1_run_common_phase distutils-r1_python_install_all
-	fi
+	_distutils-r1_run_foreach_impl \
+		"$(multibuild_get_phase_function python_install)"
+	_distutils-r1_run_common_phase \
+		"$(multibuild_get_phase_function python_install_all)"
 }
 
 _DISTUTILS_R1=1
-- 
1.8.1.5

[gentoo-dev] [PATCH 3/7] multilib-minimal: split out mkdir to unify sub-functions.

[Michał Górny]

---
 gx86/eclass/multilib-minimal.eclass | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/gx86/eclass/multilib-minimal.eclass b/gx86/eclass/multilib-minimal.eclass
index 070425f..a77368e 100644
--- a/gx86/eclass/multilib-minimal.eclass
+++ b/gx86/eclass/multilib-minimal.eclass
@@ -30,14 +30,18 @@ case ${EAPI:-0} in
 esac
 
 
-inherit multilib-build
+inherit multibuild multilib-build
 
 EXPORT_FUNCTIONS src_configure src_compile src_test src_install
 
 
 multilib-minimal_src_configure() {
-	multilib-minimal_abi_src_configure() {
+	_multilib-minimal_mkdir() {
 		mkdir -p "${BUILD_DIR}" || die
+	}
+	multilib_foreach_abi _multilib-minimal_mkdir
+
+	multilib-minimal_abi_src_configure() {
 		pushd "${BUILD_DIR}" >/dev/null || die
 		if declare -f multilib_src_configure >/dev/null ; then
 			multilib_src_configure
-- 
1.8.1.5

[gentoo-dev] [PATCH 4/7] multilib-minimal: reuse run_in_build_dir.

[Michał Górny]

---
 gx86/eclass/multilib-minimal.eclass | 20 ++++++++------------
 1 file changed, 8 insertions(+), 12 deletions(-)

diff --git a/gx86/eclass/multilib-minimal.eclass b/gx86/eclass/multilib-minimal.eclass
index a77368e..2510578 100644
--- a/gx86/eclass/multilib-minimal.eclass
+++ b/gx86/eclass/multilib-minimal.eclass
@@ -35,6 +35,10 @@ inherit multibuild multilib-build
 EXPORT_FUNCTIONS src_configure src_compile src_test src_install
 
 
+_multilib-minimal_wrap_phase() {
+	run_in_build_dir "${@}"
+}
+
 multilib-minimal_src_configure() {
 	_multilib-minimal_mkdir() {
 		mkdir -p "${BUILD_DIR}" || die
@@ -42,58 +46,50 @@ multilib-minimal_src_configure() {
 	multilib_foreach_abi _multilib-minimal_mkdir
 
 	multilib-minimal_abi_src_configure() {
-		pushd "${BUILD_DIR}" >/dev/null || die
 		if declare -f multilib_src_configure >/dev/null ; then
 			multilib_src_configure
 		else
 			default_src_configure
 		fi
-		popd >/dev/null || die
 	}
 
-	multilib_foreach_abi multilib-minimal_abi_src_configure
+	multilib_foreach_abi _multilib-minimal_wrap_phase multilib-minimal_abi_src_configure
 }
 
 multilib-minimal_src_compile() {
 	multilib-minimal_abi_src_compile() {
-		pushd "${BUILD_DIR}" >/dev/null || die
 		if declare -f multilib_src_compile >/dev/null ; then
 			multilib_src_compile
 		else
 			default_src_compile
 		fi
-		popd >/dev/null || die
 	}
 
-	multilib_foreach_abi multilib-minimal_abi_src_compile
+	multilib_foreach_abi _multilib-minimal_wrap_phase multilib-minimal_abi_src_compile
 }
 
 multilib-minimal_src_test() {
 	multilib-minimal_abi_src_test() {
-		pushd "${BUILD_DIR}" >/dev/null || die
 		if declare -f multilib_src_test >/dev/null ; then
 			multilib_src_test
 		else
 			default_src_test
 		fi
-		popd >/dev/null || die
 	}
 
-	multilib_foreach_abi multilib-minimal_abi_src_test
+	multilib_foreach_abi _multilib-minimal_wrap_phase multilib-minimal_abi_src_test
 }
 
 multilib-minimal_src_install() {
 	multilib-minimal_abi_src_install() {
-		pushd "${BUILD_DIR}" >/dev/null || die
 		if declare -f multilib_src_install >/dev/null ; then
 			multilib_src_install
 		else
 			default_src_install	
 		fi
 		multilib_check_headers
-		popd >/dev/null || die
 	}
-	multilib_foreach_abi multilib-minimal_abi_src_install
+	multilib_foreach_abi _multilib-minimal_wrap_phase multilib-minimal_abi_src_install
 
 	if declare -f multilib_src_install_all >/dev/null ; then
 		multilib_src_install_all
-- 
1.8.1.5

[gentoo-dev] [PATCH 5/7] multilib-minimal: reuse multibuild phase function handlers.

[Michał Górny]

---
 gx86/eclass/multilib-minimal.eclass | 54 ++++++++++++-------------------------
 1 file changed, 17 insertions(+), 37 deletions(-)

diff --git a/gx86/eclass/multilib-minimal.eclass b/gx86/eclass/multilib-minimal.eclass
index 2510578..2a707b1 100644
--- a/gx86/eclass/multilib-minimal.eclass
+++ b/gx86/eclass/multilib-minimal.eclass
@@ -45,53 +45,33 @@ multilib-minimal_src_configure() {
 	}
 	multilib_foreach_abi _multilib-minimal_mkdir
 
-	multilib-minimal_abi_src_configure() {
-		if declare -f multilib_src_configure >/dev/null ; then
-			multilib_src_configure
-		else
-			default_src_configure
-		fi
-	}
-
-	multilib_foreach_abi _multilib-minimal_wrap_phase multilib-minimal_abi_src_configure
+	local phase=$(multibuild_get_phase_function multilib_src_configure)
+	multilib_foreach_abi \
+		_multilib-minimal_wrap_phase "${phase:-default_src_configure}"
 }
 
 multilib-minimal_src_compile() {
-	multilib-minimal_abi_src_compile() {
-		if declare -f multilib_src_compile >/dev/null ; then
-			multilib_src_compile
-		else
-			default_src_compile
-		fi
-	}
-
-	multilib_foreach_abi _multilib-minimal_wrap_phase multilib-minimal_abi_src_compile
+	local phase=$(multibuild_get_phase_function multilib_src_compile)
+	multilib_foreach_abi \
+		_multilib-minimal_wrap_phase "${phase:-default_src_compile}"
 }
 
 multilib-minimal_src_test() {
-	multilib-minimal_abi_src_test() {
-		if declare -f multilib_src_test >/dev/null ; then
-			multilib_src_test
-		else
-			default_src_test
-		fi
-	}
-
-	multilib_foreach_abi _multilib-minimal_wrap_phase multilib-minimal_abi_src_test
+	local phase=$(multibuild_get_phase_function multilib_src_test)
+	multilib_foreach_abi \
+		_multilib-minimal_wrap_phase "${phase:-default_src_test}"
 }
 
 multilib-minimal_src_install() {
-	multilib-minimal_abi_src_install() {
-		if declare -f multilib_src_install >/dev/null ; then
-			multilib_src_install
-		else
-			default_src_install	
-		fi
+	_multilib-minimal_wrap_install_phase() {
+		_multilib-minimal_wrap_phase "${@}"
 		multilib_check_headers
 	}
-	multilib_foreach_abi _multilib-minimal_wrap_phase multilib-minimal_abi_src_install
 
-	if declare -f multilib_src_install_all >/dev/null ; then
-		multilib_src_install_all
-	fi
+	local phase=$(multibuild_get_phase_function multilib_src_install)
+	multilib_foreach_abi \
+		_multilib-minimal_wrap_install_phase "${phase:-default_src_install}"
+
+	phase=$(multibuild_get_phase_function multilib_src_install_all)
+	[[ ${phase} ]] && "${phase}"
 }
-- 
1.8.1.5

[gentoo-dev] [PATCH 6/7] multilib-minimal: run multilib_src_configure in parallel.

[Michał Górny]

---
 gx86/eclass/multilib-minimal.eclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/gx86/eclass/multilib-minimal.eclass b/gx86/eclass/multilib-minimal.eclass
index 2a707b1..cfe4eef 100644
--- a/gx86/eclass/multilib-minimal.eclass
+++ b/gx86/eclass/multilib-minimal.eclass
@@ -46,7 +46,7 @@ multilib-minimal_src_configure() {
 	multilib_foreach_abi _multilib-minimal_mkdir
 
 	local phase=$(multibuild_get_phase_function multilib_src_configure)
-	multilib_foreach_abi \
+	multilib_parallel_foreach_abi \
 		_multilib-minimal_wrap_phase "${phase:-default_src_configure}"
 }
 
-- 
1.8.1.5

[gentoo-dev] [PATCH 7/7] autotools-multilib: reuse phase functions from multilib-minimal.

[Michał Górny]

This makes replacing sub-phase functions much easier.
---
 gx86/eclass/autotools-multilib.eclass | 34 ++++++++++++++++++++++++----------
 1 file changed, 24 insertions(+), 10 deletions(-)

diff --git a/gx86/eclass/autotools-multilib.eclass b/gx86/eclass/autotools-multilib.eclass
index d7372b0..d04fd1f 100644
--- a/gx86/eclass/autotools-multilib.eclass
+++ b/gx86/eclass/autotools-multilib.eclass
@@ -29,33 +29,47 @@ if [[ ${AUTOTOOLS_IN_SOURCE_BUILD} ]]; then
 	die "${ECLASS}: multilib support requires out-of-source builds."
 fi
 
-inherit autotools-utils multilib-build
+inherit autotools-utils multibuild multilib-minimal
 
 EXPORT_FUNCTIONS src_prepare src_configure src_compile src_test src_install
+multibuild_export_phases \
+	multilib_src_configure multilib_src_compile multilib_src_test \
+	multilib_src_install
 
 autotools-multilib_src_prepare() {
 	autotools-utils_src_prepare "${@}"
 }
 
 autotools-multilib_src_configure() {
-	multilib_parallel_foreach_abi autotools-utils_src_configure "${@}"
+	multilib-minimal_src_configure "${@}"
 }
 
 autotools-multilib_src_compile() {
-	multilib_foreach_abi autotools-utils_src_compile "${@}"
+	multilib-minimal_src_compile "${@}"
 }
 
 autotools-multilib_src_test() {
-	multilib_foreach_abi autotools-utils_src_test "${@}"
+	multilib-minimal_src_test "${@}"
 }
 
 autotools-multilib_src_install() {
-	autotools-multilib_secure_install() {
-		autotools-utils_src_install "${@}"
+	multilib-minimal_src_install "${@}"
+}
+
+# multilib-minimal phases
+
+autotools-multilib_multilib_src_configure() {
+	autotools-utils_src_configure "${@}"
+}
 
-		# Make sure all headers are the same for each ABI.
-		multilib_check_headers
-	}
+autotools-multilib_multilib_src_compile() {
+	autotools-utils_src_compile "${@}"
+}
+
+autotools-multilib_multilib_src_test() {
+	autotools-utils_src_test "${@}"
+}
 
-	multilib_foreach_abi autotools-multilib_secure_install "${@}"
+autotools-multilib_multilib_src_install() {
+	autotools-utils_src_install "${@}"
 }
-- 
1.8.1.5

Re: [gentoo-dev] [PATCH 1/7] multibuild: introduce a generic framework for custom phase functions.

[Ulrich Mueller]

>>>>> On Sun, 10 Mar 2013, Michał Górny wrote:
 
> +DEPEND=">=app-shells/bash-4.2"
> +

Why is this needed?

Ulrich

Re: [gentoo-dev] [PATCH 1/7] multibuild: introduce a generic framework for custom phase functions.

[Ulrich Mueller]

>>>>> On Sun, 10 Mar 2013, Ulrich Mueller wrote:

>>>>> On Sun, 10 Mar 2013, Michał Górny wrote:
>> +DEPEND=">=app-shells/bash-4.2"
>> +

> Why is this needed?

Seems it's because of this:

>> +	declare -g -A _MULTIBUILD_EXPORTED_PHASES || die
>> +	local p
>> +	for p; do
>> +		_MULTIBUILD_EXPORTED_PHASES[${p}]=${ECLASS}_${p}
>> +	done

Associative arrays are not supported in bash 3.2, so they must not be
used in EAPIs 0 to 5.

Ulrich

Re: [gentoo-dev] [PATCH 1/7] multibuild: introduce a generic framework for custom phase functions.

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 684 bytes --]

On Sun, 10 Mar 2013 13:16:46 +0100
Ulrich Mueller <ulm@gentoo.org> wrote:
> >>>>> On Sun, 10 Mar 2013, Ulrich Mueller wrote:
> 
> >>>>> On Sun, 10 Mar 2013, Michał Górny wrote:
> >> +DEPEND=">=app-shells/bash-4.2"
> >> +
> 
> > Why is this needed?
> 
> Seems it's because of this:
> 
> >> +	declare -g -A _MULTIBUILD_EXPORTED_PHASES || die
> >> +	local p
> >> +	for p; do
> >> +		_MULTIBUILD_EXPORTED_PHASES[${p}]=${ECLASS}_${p}
> >> +	done
> 
> Associative arrays are not supported in bash 3.2, so they must not be
> used in EAPIs 0 to 5.

A dependency upon a newer bash doesn't guarantee that the package
mangler will use that bash.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] [PATCH 1/7] multibuild: introduce a generic framework for custom phase functions.

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 926 bytes --]

On Sun, 10 Mar 2013 12:18:25 +0000
Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:

> On Sun, 10 Mar 2013 13:16:46 +0100
> Ulrich Mueller <ulm@gentoo.org> wrote:
> > >>>>> On Sun, 10 Mar 2013, Ulrich Mueller wrote:
> > 
> > >>>>> On Sun, 10 Mar 2013, Michał Górny wrote:
> > >> +DEPEND=">=app-shells/bash-4.2"
> > >> +
> > 
> > > Why is this needed?
> > 
> > Seems it's because of this:
> > 
> > >> +	declare -g -A _MULTIBUILD_EXPORTED_PHASES || die
> > >> +	local p
> > >> +	for p; do
> > >> +		_MULTIBUILD_EXPORTED_PHASES[${p}]=${ECLASS}_${p}
> > >> +	done
> > 
> > Associative arrays are not supported in bash 3.2, so they must not be
> > used in EAPIs 0 to 5.
> 
> A dependency upon a newer bash doesn't guarantee that the package
> mangler will use that bash.

Yep. That's why the non-quoted fragment has a safety check and dies if
it doesn't.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]

Re: [gentoo-dev] [PATCH 1/7] multibuild: introduce a generic framework for custom phase functions.

[Ulrich Mueller]

>>>>> On Sun, 10 Mar 2013, Michał Górny wrote:

> Yep. That's why the non-quoted fragment has a safety check and dies
> if it doesn't.

It doesn't matter if there's a safety check. Bash 4 features are
simply not allowed in the tree.

Ulrich

Re: [gentoo-dev] [PATCH 1/7] multibuild: introduce a generic framework for custom phase functions.

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 694 bytes --]

On Sun, 10 Mar 2013 14:44:42 +0100
Ulrich Mueller <ulm@gentoo.org> wrote:

> >>>>> On Sun, 10 Mar 2013, Michał Górny wrote:
> 
> > Yep. That's why the non-quoted fragment has a safety check and dies
> > if it doesn't.
> 
> It doesn't matter if there's a safety check. Bash 4 features are
> simply not allowed in the tree.

Is there a technical reason for that? As far as I understand it,
the method used in the ebuild should guarantee that in the worst case
user would have to restart emerge once.

Well, unless we're talking about a theoretical package mangler which
intentionally uses internal, old version of bash to prove the point.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]

Re: [gentoo-dev] [PATCH 1/7] multibuild: introduce a generic framework for custom phase functions.

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 444 bytes --]

On Sun, 10 Mar 2013 14:48:06 +0100
Michał Górny <mgorny@gentoo.org> wrote:
> Well, unless we're talking about a theoretical package mangler which
> intentionally uses internal, old version of bash to prove the point.

That's a good idea, maybe we'll do that. Sounds like a good way of
doing better input validation. Perhaps we could patch our internal bash
to make it easier to catch certain other errors too.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] [PATCH 1/7] multibuild: introduce a generic framework for custom phase functions.

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1010 bytes --]

On Sun, 10 Mar 2013 16:46:41 +0100
Michał Górny <mgorny@gentoo.org> wrote:
> On Sun, 10 Mar 2013 15:26:29 +0000
> Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
> > On Sun, 10 Mar 2013 14:48:06 +0100
> > Michał Górny <mgorny@gentoo.org> wrote:
> > > Well, unless we're talking about a theoretical package mangler
> > > which intentionally uses internal, old version of bash to prove
> > > the point.
> > 
> > That's a good idea, maybe we'll do that. Sounds like a good way of
> > doing better input validation. Perhaps we could patch our internal
> > bash to make it easier to catch certain other errors too.
> 
> Please don't forget to bundle a few rootkits inside, so your users
> won't have to wait for security issues to be found in the ye ol' bash
> version you'll use.

You mean, in the bash that will be being run as root, that is
accessible exclusively to packages, all of which are allowed to run
things as root, install set*id binaries, etc?

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] [PATCH 1/7] multibuild: introduce a generic framework for custom phase functions.

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 731 bytes --]

On Sun, 10 Mar 2013 15:26:29 +0000
Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:

> On Sun, 10 Mar 2013 14:48:06 +0100
> Michał Górny <mgorny@gentoo.org> wrote:
> > Well, unless we're talking about a theoretical package mangler which
> > intentionally uses internal, old version of bash to prove the point.
> 
> That's a good idea, maybe we'll do that. Sounds like a good way of
> doing better input validation. Perhaps we could patch our internal bash
> to make it easier to catch certain other errors too.

Please don't forget to bundle a few rootkits inside, so your users
won't have to wait for security issues to be found in the ye ol' bash
version you'll use.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]

[gentoo-dev] [PATCH] multibuild: introduce a generic framework for custom phase functions.

[Michał Górny]

The framework provides functions to declare, export and obtain custom
phase functions.

Each of the custom phases can be defined by eclasses and ebuilds
in a manner similar to regular phases. The eclasses define
${ECLASS}_${phase} function and run 'multibuild_export_phases' to
register them. The ebuilds define ${phase} function and it automatically
takes precedence over eclass-defined ones.
---
 gx86/eclass/multibuild.eclass | 66 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 66 insertions(+)

diff --git a/gx86/eclass/multibuild.eclass b/gx86/eclass/multibuild.eclass
index bc510e9..3187c9e 100644
--- a/gx86/eclass/multibuild.eclass
+++ b/gx86/eclass/multibuild.eclass
@@ -245,5 +245,71 @@ run_in_build_dir() {
 	return ${ret}
 }
 
+# @ECLASS-VARIABLE: _MULTIBUILD_EXPORTED_PHASES
+# @INTERNAL
+# @DESCRIPTION:
+# The list of currently exported phase functions.
+#
+# Each function is stored in the form of 'eclass:phase-name'.
+# New exports are prepended to the list, so the first matching value
+# is the most recent one.
+_MULTIBUILD_EXPORTED_PHASES=()
+
+# @FUNCTION: multibuild_export_phases
+# @USAGE: <phase-name>...
+# @DESCRIPTION:
+# Export the eclass phase functions for named phases. The functions need
+# be named ${ECLASS}_<phase-name>. The exported functions will override
+# any previously exported phases.
+multibuild_export_phases() {
+	debug-print-function ${FUNCNAME} "${@}"
+
+	[[ ${#} -eq 0 ]] && die "Usage: multibuild_export_phases <phase-name>..."
+
+	# just prepend to the list
+	_MULTIBUILD_EXPORTED_PHASES=(
+		"${@/#/${ECLASS}:}"
+		"${_MULTIBUILD_EXPORTED_PHASES[@]}"
+	)
+}
+
+# @FUNCTION: multibuild_get_phase_function
+# @USAGE: <phase-name>
+# @DESCRIPTION:
+# Find the topmost handler for the named phase. It can be either
+# user-defined phase handler (with the same name as the phase)
+# or a handler exported most recently by an eclass.
+#
+# Prints the function name to stdout or null string if there is
+# no handler for the phase.
+multibuild_get_phase_function() {
+	debug-print-function ${FUNCNAME} "${@}"
+
+	[[ ${#} -ne 1 ]] && die "Usage: multibuild_get_phase_function <phase-name>"
+
+	local phase=${1}
+
+	# user-defined phase
+	if ! declare -f "${phase}" >/dev/null; then
+		local found p
+		for p in "${_MULTIBUILD_EXPORTED_PHASES[@]}"; do
+			if [[ ${p} == *:${phase} ]]; then
+				# we're breaking out, so we can overwrite it.
+				phase=${p/:/_}
+				found=1
+				break
+			fi
+		done
+
+		if [[ ! ${found} ]]; then
+			return
+		elif ! declare -f "${phase}" >/dev/null; then
+			die "Phase function ${phase} exported but never defined!"
+		fi
+	fi
+
+	echo "${phase}"
+}
+
 _MULTIBUILD=1
 fi
-- 
1.8.1.5

Re: [gentoo-dev] [PATCH] multibuild: introduce a generic framework for custom phase functions.

[Alec Warner]

On Sun, Mar 10, 2013 at 8:50 AM, Michał Górny <mgorny@gentoo.org> wrote:
> The framework provides functions to declare, export and obtain custom
> phase functions.

Thanks for fixing this up.

-A

>
> Each of the custom phases can be defined by eclasses and ebuilds
> in a manner similar to regular phases. The eclasses define
> ${ECLASS}_${phase} function and run 'multibuild_export_phases' to
> register them. The ebuilds define ${phase} function and it automatically
> takes precedence over eclass-defined ones.
> ---
>  gx86/eclass/multibuild.eclass | 66 +++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 66 insertions(+)
>
> diff --git a/gx86/eclass/multibuild.eclass b/gx86/eclass/multibuild.eclass
> index bc510e9..3187c9e 100644
> --- a/gx86/eclass/multibuild.eclass
> +++ b/gx86/eclass/multibuild.eclass
> @@ -245,5 +245,71 @@ run_in_build_dir() {
>         return ${ret}
>  }
>
> +# @ECLASS-VARIABLE: _MULTIBUILD_EXPORTED_PHASES
> +# @INTERNAL
> +# @DESCRIPTION:
> +# The list of currently exported phase functions.
> +#
> +# Each function is stored in the form of 'eclass:phase-name'.
> +# New exports are prepended to the list, so the first matching value
> +# is the most recent one.
> +_MULTIBUILD_EXPORTED_PHASES=()
> +
> +# @FUNCTION: multibuild_export_phases
> +# @USAGE: <phase-name>...
> +# @DESCRIPTION:
> +# Export the eclass phase functions for named phases. The functions need
> +# be named ${ECLASS}_<phase-name>. The exported functions will override
> +# any previously exported phases.
> +multibuild_export_phases() {
> +       debug-print-function ${FUNCNAME} "${@}"
> +
> +       [[ ${#} -eq 0 ]] && die "Usage: multibuild_export_phases <phase-name>..."
> +
> +       # just prepend to the list
> +       _MULTIBUILD_EXPORTED_PHASES=(
> +               "${@/#/${ECLASS}:}"
> +               "${_MULTIBUILD_EXPORTED_PHASES[@]}"
> +       )
> +}
> +
> +# @FUNCTION: multibuild_get_phase_function
> +# @USAGE: <phase-name>
> +# @DESCRIPTION:
> +# Find the topmost handler for the named phase. It can be either
> +# user-defined phase handler (with the same name as the phase)
> +# or a handler exported most recently by an eclass.
> +#
> +# Prints the function name to stdout or null string if there is
> +# no handler for the phase.
> +multibuild_get_phase_function() {
> +       debug-print-function ${FUNCNAME} "${@}"
> +
> +       [[ ${#} -ne 1 ]] && die "Usage: multibuild_get_phase_function <phase-name>"
> +
> +       local phase=${1}
> +
> +       # user-defined phase
> +       if ! declare -f "${phase}" >/dev/null; then
> +               local found p
> +               for p in "${_MULTIBUILD_EXPORTED_PHASES[@]}"; do
> +                       if [[ ${p} == *:${phase} ]]; then
> +                               # we're breaking out, so we can overwrite it.
> +                               phase=${p/:/_}
> +                               found=1
> +                               break
> +                       fi
> +               done
> +
> +               if [[ ! ${found} ]]; then
> +                       return
> +               elif ! declare -f "${phase}" >/dev/null; then
> +                       die "Phase function ${phase} exported but never defined!"
> +               fi
> +       fi
> +
> +       echo "${phase}"
> +}
> +
>  _MULTIBUILD=1
>  fi
> --
> 1.8.1.5
>
>

Re: [gentoo-dev] [PATCHES] multibuild.eclass: custom phase function helpers

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 964 bytes --]

On Sun, 10 Mar 2013 11:16:44 +0100
Michał Górny <mgorny@gentoo.org> wrote:

> distutils-r1 (and previously python-distutils-ng) was using custom
> phase functions for a while. Recently, hasufell added multilib-minimal
> which does the same. Since in both cases the custom functions are
> closely related to building multiple variants of the package, I'm
> thinking of adding a few helper functions to the multibuild.eclass.
> 
> The framework I'm suggesting will make the custom phase functions
> behave similarly to regular ones. Most importantly, eclasses will be
> provided with a function to 'export' them.

Due to the issue of non-generic design of multilib-minimal eclass,
I have decided not to apply those patches. The autotools-multilib can't
sanely benefit from it, and then there's no consumer for the eclass
framework.

We can get back here if there is a new eclass which needs
the framework.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]