[gentoo-dev] [PATCHES] multilib-build: public API for header wrapping

[gentoo-dev] [PATCHES] multilib-build: public API for header wrapping

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2627 bytes --]

Hello,

Following the introduction of header wrapping in autotools-multilib,
I'm submitting two patches: one providing a public API for it
in multilib-build, and the other one using it in multilib-minimal. Both
patches will be sent in reply to this mail.

The API consists of two functions: multilib_prepare_wrappers
and multilib_install_wrappers. Although currently they handle header
wrapping only, they could be theoretically extended to support more
kinds of wrappers.

Both functions take an optional <install-root> argument which defaults
to ${ED}. That argument is used for obtaining wrapped files and storing
the wrappers. However, it does not affect the temporary storage
location which is '${ED}/tmp' unconditionally.

Both functions use environment variables to obtain the list of files
to be wrapped. The MULTILIB_WRAPPED_HEADERS variable is used, the same
as in the initial approach. In future, more variables can be used to
support more kinds of wrappers.

The multilib_prepare_wrappers function needs to be called for each ABI
after the files for that ABI were installed, with the root to which
they were installed. It moves the necessary out of the root,
and prepares the wrapper in the temporary area.

The multilib_install_wrappers functions needs to be called once
after all the wrappers were prepared and they can be installed.
The wrappers are installed in the root passed to it, and temporary area
is cleaned.

That said, there are two generic approaches to wrapping:

a) install-and-wrap approach as used by the current eclasses:

  foo_install() {
    some_foo_install # installs to ${D}
    multilib_prepare_wrappers
  }
  multilib_foreach_abi foo_install
  multilib_install_wrappers

In this approach, each successive ABI potentially overwrites files
installed by the previous one. The wrapper preparation needs to follow
the install phase so that wrapped files are preserved in their original
form. Afterwards, wrappers are installed on top of the install.

b) install-then-merge approach:

  bar_install() {
    local myfakeroot=${ED}/_${ABI}
    some_bar_install "${myfakeroot}" # installs to the fake root
    multilib_prepare_wrappers "${myfakeroot}"
    multibuild_merge_root "${myfakeroot}" "${ED}"
  }
  multilib_parallel_foreach_abi bar_install
  multilib_install_wrappers

This time, all per-ABI installs are done in separate directories. This
allows them to be done in parallel. The wrapper preparation needs to be
done on the separate copies, then those can be merged onto the real
root.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]

[gentoo-dev] [PATCH 1/2] Move header wrapping code into multilib-build.

[Michał Górny]

---
 gx86/eclass/autotools-multilib.eclass |  82 +----------------------
 gx86/eclass/multilib-build.eclass     | 121 ++++++++++++++++++++++++++++++++++
 2 files changed, 124 insertions(+), 79 deletions(-)

diff --git a/gx86/eclass/autotools-multilib.eclass b/gx86/eclass/autotools-multilib.eclass
index 55d32d7..6b0960f 100644
--- a/gx86/eclass/autotools-multilib.eclass
+++ b/gx86/eclass/autotools-multilib.eclass
@@ -33,28 +33,6 @@ inherit autotools-utils multilib-build
 
 EXPORT_FUNCTIONS src_prepare src_configure src_compile src_test src_install
 
-# @ECLASS-VARIABLE: MULTILIB_WRAPPED_HEADERS
-# @DESCRIPTION:
-# A list of headers to wrap for multilib support. The listed headers
-# will be moved to a non-standard location and replace with a file
-# including them conditionally to current ABI.
-#
-# This variable has to be a bash array. Paths shall be relative to
-# installation root (${ED}), and name regular files. Recursive wrapping
-# is not supported.
-#
-# Please note that header wrapping is *discouraged*. It is preferred to
-# install all headers in a subdirectory of libdir and use pkg-config to
-# locate the headers. Some C preprocessors will not work with wrapped
-# headers.
-#
-# Example:
-# @CODE
-# MULTILIB_WRAPPED_HEADERS=(
-#	/usr/include/foobar/config.h
-# )
-# @CODE
-
 autotools-multilib_src_prepare() {
 	autotools-utils_src_prepare "${@}"
 }
@@ -71,71 +49,17 @@ autotools-multilib_src_test() {
 	multilib_foreach_abi autotools-utils_src_test "${@}"
 }
 
-_autotools-multilib_wrap_headers() {
-	debug-print-function ${FUNCNAME} "$@"
-	local f
-
-	for f in "${MULTILIB_WRAPPED_HEADERS[@]}"; do
-		# drop leading slash if it's there
-		f=${f#/}
-
-		if [[ ${f} != usr/include/* ]]; then
-			die "Wrapping headers outside of /usr/include is not supported at the moment."
-		fi
-		# and then usr/include
-		f=${f#usr/include}
-
-		local dir=${f%/*}
-
-		# $CHOST shall be set by multilib_toolchain_setup
-		dodir "/tmp/multilib-include/${CHOST}${dir}"
-		mv "${ED}/usr/include${f}" "${ED}/tmp/multilib-include/${CHOST}${dir}/" || die
-
-		if [[ ! -f ${ED}/tmp/multilib-include${f} ]]; then
-			dodir "/tmp/multilib-include${dir}"
-			# a generic template
-			cat > "${ED}/tmp/multilib-include${f}" <<_EOF_ || die
-/* This file is auto-generated by autotools-multilib.eclass
- * as a multilib-friendly wrapper. For the original content,
- * please see the files that are #included below.
- */
-
-#if defined(__x86_64__) /* amd64 */
-#	if defined(__ILP32__) /* x32 ABI */
-#		error "x86_x32 not supported by the package."
-#	else /* 64-bit ABI */
-#		error "x86_64 not supported by the package."
-#	endif
-#elif defined(__i386__) /* plain x86 */
-#	error "x86_32 not supported by the package."
-#else
-#	error "No ABI matched, please report a bug to bugs.gentoo.org"
-#endif
-_EOF_
-		fi
-
-		# Note: match a space afterwards to avoid collision potential.
-		sed -e "/${MULTILIB_ABI} /s&error.*&include <${CHOST}/${f}>&" \
-			-i "${ED}/tmp/multilib-include${f}" || die
-	done
-}
-
 autotools-multilib_src_install() {
 	autotools-multilib_secure_install() {
 		autotools-utils_src_install "${@}"
 
-		_autotools-multilib_wrap_headers
+		multilib_prepare_wrappers
 		# Make sure all headers are the same for each ABI.
 		multilib_check_headers
 	}
 
 	multilib_foreach_abi autotools-multilib_secure_install "${@}"
 
-	# merge the wrapped headers
-	if [[ -d "${ED}"/tmp/multilib-include ]]; then
-		multibuild_merge_root \
-			"${ED}"/tmp/multilib-include "${ED}"/usr/include
-		# it can fail if something else uses /tmp
-		rmdir "${ED}"/tmp &>/dev/null
-	fi
+	# merge the wrappers
+	multilib_install_wrappers
 }
diff --git a/gx86/eclass/multilib-build.eclass b/gx86/eclass/multilib-build.eclass
index fdaed6b..f0c433b 100644
--- a/gx86/eclass/multilib-build.eclass
+++ b/gx86/eclass/multilib-build.eclass
@@ -235,5 +235,126 @@ multilib_copy_sources() {
 	multibuild_copy_sources
 }
 
+# @ECLASS-VARIABLE: MULTILIB_WRAPPED_HEADERS
+# @DESCRIPTION:
+# A list of headers to wrap for multilib support. The listed headers
+# will be moved to a non-standard location and replaced with a file
+# including them conditionally to current ABI.
+#
+# This variable has to be a bash array. Paths shall be relative to
+# installation root (${ED}), and name regular files. Recursive wrapping
+# is not supported.
+#
+# Please note that header wrapping is *discouraged*. It is preferred to
+# install all headers in a subdirectory of libdir and use pkg-config to
+# locate the headers. Some C preprocessors will not work with wrapped
+# headers.
+#
+# Example:
+# @CODE
+# MULTILIB_WRAPPED_HEADERS=(
+#	/usr/include/foobar/config.h
+# )
+# @CODE
+
+# @FUNCTION: multilib_prepare_wrappers
+# @USAGE: [<install-root>]
+# @DESCRIPTION:
+# Perform the preparation of all kinds of wrappers for the current ABI.
+# This function shall be called once per each ABI, after installing
+# the files to be wrapped.
+#
+# Takes an optional custom <install-root> from which files will be
+# used. If no root is specified, uses ${ED}.
+#
+# The files to be wrapped are specified using separate variables,
+# e.g. MULTILIB_WRAPPED_HEADERS. Those variables shall not be changed
+# between the successive calls to multilib_prepare_wrappers
+# and multilib_install_wrappers.
+#
+# After all wrappers are prepared, multilib_install_wrappers shall
+# be called to commit them to the installation tree.
+multilib_prepare_wrappers() {
+	debug-print-function ${FUNCNAME} "${@}"
+
+	[[ ${#} -le 1 ]] || die "${FUNCNAME}: too many arguments"
+
+	local root=${1:-${ED}}
+	local f
+
+	for f in "${MULTILIB_WRAPPED_HEADERS[@]}"; do
+		# drop leading slash if it's there
+		f=${f#/}
+
+		if [[ ${f} != usr/include/* ]]; then
+			die "Wrapping headers outside of /usr/include is not supported at the moment."
+		fi
+		# and then usr/include
+		f=${f#usr/include}
+
+		local dir=${f%/*}
+
+		# $CHOST shall be set by multilib_toolchain_setup
+		dodir "/tmp/multilib-include/${CHOST}${dir}"
+		mv "${root}/usr/include${f}" "${ED}/tmp/multilib-include/${CHOST}${dir}/" || die
+
+		if [[ ! -f ${ED}/tmp/multilib-include${f} ]]; then
+			dodir "/tmp/multilib-include${dir}"
+			# a generic template
+			cat > "${ED}/tmp/multilib-include${f}" <<_EOF_ || die
+/* This file is auto-generated by multilib-build.eclass
+ * as a multilib-friendly wrapper. For the original content,
+ * please see the files that are #included below.
+ */
+
+#if defined(__x86_64__) /* amd64 */
+#	if defined(__ILP32__) /* x32 ABI */
+#		error "x86_x32 not supported by the package."
+#	else /* 64-bit ABI */
+#		error "x86_64 not supported by the package."
+#	endif
+#elif defined(__i386__) /* plain x86 */
+#	error "x86_32 not supported by the package."
+#else
+#	error "No ABI matched, please report a bug to bugs.gentoo.org"
+#endif
+_EOF_
+		fi
+
+		# Note: match a space afterwards to avoid collision potential.
+		sed -e "/${MULTILIB_ABI} /s&error.*&include <${CHOST}/${f}>&" \
+			-i "${ED}/tmp/multilib-include${f}" || die
+	done
+}
+
+# @FUNCTION: multilib_install_wrappers
+# @USAGE: [<install-root>]
+# @DESCRIPTION:
+# Install the previously-prepared wrappers. This function shall
+# be called once, after all wrappers were prepared.
+#
+# Takes an optional custom <install-root> to which the wrappers will be
+# installed. If no root is specified, uses ${ED}. There is no need to
+# use the same root as when preparing the wrappers.
+#
+# The files to be wrapped are specified using separate variables,
+# e.g. MULTILIB_WRAPPED_HEADERS. Those variables shall not be changed
+# between the calls to multilib_prepare_wrappers
+# and multilib_install_wrappers.
+multilib_install_wrappers() {
+	debug-print-function ${FUNCNAME} "${@}"
+
+	[[ ${#} -le 1 ]] || die "${FUNCNAME}: too many arguments"
+
+	local root=${1:-${ED}}
+
+	if [[ -d "${ED}"/tmp/multilib-include ]]; then
+		multibuild_merge_root \
+			"${ED}"/tmp/multilib-include "${root}"/usr/include
+		# it can fail if something else uses /tmp
+		rmdir "${ED}"/tmp &>/dev/null
+	fi
+}
+
 _MULTILIB_BUILD=1
 fi
-- 
1.8.1.5

[gentoo-dev] [PATCH 2/2] multilib-minimal: enable installing wrappers.

[Michał Górny]

---
 gx86/eclass/multilib-minimal.eclass | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/gx86/eclass/multilib-minimal.eclass b/gx86/eclass/multilib-minimal.eclass
index 070425f..44d6ed3 100644
--- a/gx86/eclass/multilib-minimal.eclass
+++ b/gx86/eclass/multilib-minimal.eclass
@@ -86,10 +86,12 @@ multilib-minimal_src_install() {
 		else
 			default_src_install	
 		fi
+		multilib_prepare_wrappers
 		multilib_check_headers
 		popd >/dev/null || die
 	}
 	multilib_foreach_abi multilib-minimal_abi_src_install
+	multilib_install_wrappers
 
 	if declare -f multilib_src_install_all >/dev/null ; then
 		multilib_src_install_all
-- 
1.8.1.5

Re: [gentoo-dev] [PATCHES] multilib-build: public API for header wrapping

[Alexis Ballier]

At a first look, this seems all good. Thanks!

Alexis.

Re: [gentoo-dev] [PATCHES] multilib-build: public API for header wrapping

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 455 bytes --]

On Thu, 4 Apr 2013 22:50:46 +0200
Michał Górny <mgorny@gentoo.org> wrote:

> Following the introduction of header wrapping in autotools-multilib,
> I'm submitting two patches: one providing a public API for it
> in multilib-build, and the other one using it in multilib-minimal. Both
> patches will be sent in reply to this mail.

Rebased on the tree without the $ABI->$MULTILIB_ABI patch,
and committed.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]