[gentoo-python] [PATCHES] distutils-r1: making in-source builds more compatible with out-of-source builds

[gentoo-python] [PATCHES] distutils-r1: making in-source builds more compatible with out-of-source builds

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2655 bytes --]

Hello,

Currently in-source builds differ a lot from out-of-source builds. Most
notably:

- BUILD_DIR points to source dir rather than build dir,

- build directories are not overriden,

- PYTHONPATH is not set.


With the net outcome being that:

- in-source builds are theoretically safer for weird build system hacks,

- in-source build ebuilds are harder to write,

- switching between one and the other requires a fair amount of changes.


If we consider then that enabling in-source build should be considered
a quick and possibly temporary hack to get the package to build, points
2/3 seem to really outweigh the benefit of 1.

For that reason, I'd like to 'unify' the behavior between the two.
The patches which I will send in reply to this mail will:


1) make BUILD_DIR point to the 'build' subdirectory in in-source builds,

2) override build locations the same way in both kinds of builds,

3) set PYTHONPATH in in-source builds,

4) [optional] run *_all() phases in implementation-specific sources
copy rather than ${S}.


This is an incompatible change but seems to be the least troublesome
possibility. Most importantly, any ebuild using BUILD_DIR
in python_*_all() phase will become at least partially broken.

Hopefully, this ain't as bad as it seems. Those ebuilds which used
BUILD_DIR there to append to PYTHONPATH will still work. While
the appending will no longer be correct, the newly-introduced
PYTHONPATH will take precedence and solve the issue for now.

There is also the potential of packages being broken because of
--build-* overrides. But the most common case of relying on standard
locations would probably still work because the new paths will mostly be
absolute-path variants of the default locations.

In any case, I'm testing this a bit now and will let you know what gets
broken.


Rationale:

The code appends '/build' subdirectory to BUILD_DIR in distutils-r1
(python-r1's BUILD_DIR still points to the sources). This seemed like
the best possible way of achieving the goal since:

1) BUILD_DIR is not used commonly in in-source build ebuilds,

2) using ${BUILD_DIR}/lib with BUILD_DIR pointing to source root is
likely to collide with package sources,

3) using ${BUILD_DIR}/build/lib conditionally would cause
incompatibility with out-of-source builds,

4) using ${BUILD_DIR}/build/lib unconditionally would break a fair
number of out-of-source build ebuilds using '${BUILD_DIR}/lib'.

Therefore, appending the 'build' subdir to BUILD_DIR seems like
the least evil necessary to fix the past mistake.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]

[gentoo-python] [PATCH distutils-r1 1/4] In-source builds: make BUILD_DIR point to the 'build' subdir.

[Michał Górny]

Currently BUILD_DIR points to the 'top' source directory. It is not
really suitable for adjusting the build paths since 'lib', 'test',
'scripts' sub-directories would likely collide.

The 'safest' way of making BUILD_DIR usable for build locations would be
to make it point to the 'build' subdirectory. That subdirectory is used
by default for builds and therefore it is very unlikely to cause any
collisions.

Note that this will actually break a few ebuilds relying on BUILD_DIR
pointing to the top of source tree. We will need to fix those but that
will be straightforward.
---
 gx86/eclass/distutils-r1.eclass | 11 ++++-------
 1 file changed, 4 insertions(+), 7 deletions(-)

diff --git a/gx86/eclass/distutils-r1.eclass b/gx86/eclass/distutils-r1.eclass
index df46d59..7a1551b 100644
--- a/gx86/eclass/distutils-r1.eclass
+++ b/gx86/eclass/distutils-r1.eclass
@@ -285,11 +285,6 @@ distutils_install_for_testing() {
 	# 5) 'install' needs to go before 'bdist_egg' or the latter would
 	#    re-set install paths.
 
-	if [[ ${DISTUTILS_IN_SOURCE_BUILD} ]]; then
-		# use 'build' subdirectory to reduce the risk of collisions
-		local BUILD_DIR=${BUILD_DIR}/build
-	fi
-
 	TEST_DIR=${BUILD_DIR}/test
 	local bindir=${TEST_DIR}/scripts
 	local libdir=${TEST_DIR}/lib
@@ -540,8 +535,9 @@ distutils-r1_python_install_all() {
 # directory, with BUILD_DIR pointing at the build directory
 # and PYTHONPATH having an entry for the module build directory.
 #
-# If in-source builds are used, the command is executed in the BUILD_DIR
-# (the directory holding per-implementation copy of sources).
+# If in-source builds are used, the command is executed in the directory
+# holding the per-implementation copy of sources. BUILD_DIR points
+# to the 'build' subdirectory.
 distutils-r1_run_phase() {
 	debug-print-function ${FUNCNAME} "${@}"
 
@@ -549,6 +545,7 @@ distutils-r1_run_phase() {
 		if [[ ! ${DISTUTILS_SINGLE_IMPL} ]]; then
 			pushd "${BUILD_DIR}" >/dev/null || die
 		fi
+		local BUILD_DIR=${BUILD_DIR}/build
 	else
 		local PYTHONPATH="${BUILD_DIR}/lib:${PYTHONPATH}"
 		export PYTHONPATH
-- 
1.8.1.4

[gentoo-python] [PATCH distutils-r1 2/4] In-source builds: override build locations as well.

[Michał Górny]

Since BUILD_DIR is now useful and collision-free there, we can use it to
obtain expectable install locations.
---
 gx86/eclass/distutils-r1.eclass | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/gx86/eclass/distutils-r1.eclass b/gx86/eclass/distutils-r1.eclass
index 7a1551b..757c094 100644
--- a/gx86/eclass/distutils-r1.eclass
+++ b/gx86/eclass/distutils-r1.eclass
@@ -228,8 +228,8 @@ esetup.py() {
 	debug-print-function ${FUNCNAME} "${@}"
 
 	local add_args=()
-	if [[ ! ${DISTUTILS_IN_SOURCE_BUILD} ]]; then
-		if [[ ! ${BUILD_DIR} ]]; then
+	if [[ ! ${BUILD_DIR} ]]; then
+		if [[ ! ${DISTUTILS_IN_SOURCE_BUILD} ]]; then
 			die 'Out-of-source build requested, yet BUILD_DIR unset.'
 		fi
 
-- 
1.8.1.4

[gentoo-python] [PATCH distutils-r1 3/4] In-source builds: set PYTHONPATH properly.

[Michał Górny]

Another step into increasing compatibility between in-source
and out-of-source builds.
---
 gx86/eclass/distutils-r1.eclass | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/gx86/eclass/distutils-r1.eclass b/gx86/eclass/distutils-r1.eclass
index 757c094..0c5abff 100644
--- a/gx86/eclass/distutils-r1.eclass
+++ b/gx86/eclass/distutils-r1.eclass
@@ -546,10 +546,8 @@ distutils-r1_run_phase() {
 			pushd "${BUILD_DIR}" >/dev/null || die
 		fi
 		local BUILD_DIR=${BUILD_DIR}/build
-	else
-		local PYTHONPATH="${BUILD_DIR}/lib:${PYTHONPATH}"
-		export PYTHONPATH
 	fi
+	local -x PYTHONPATH="${BUILD_DIR}/lib:${PYTHONPATH}"
 
 	local TMPDIR=${T}/${EPYTHON}
 
-- 
1.8.1.4

[gentoo-python] [PATCH distutils-r1 4/4] In-source builds: run *_all() phases in best-impl sources copy.

[Michał Górny]

This seems a bit safer, and provides users with expectable way of
obtaining the location of those sources. The original sources directory
can be still obtained via ${S}.
---
 gx86/eclass/distutils-r1.eclass | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/gx86/eclass/distutils-r1.eclass b/gx86/eclass/distutils-r1.eclass
index 0c5abff..7a295c2 100644
--- a/gx86/eclass/distutils-r1.eclass
+++ b/gx86/eclass/distutils-r1.eclass
@@ -580,6 +580,9 @@ distutils-r1_run_phase() {
 # @INTERNAL
 # @DESCRIPTION:
 # Run the given command, restoring the best-implementation state.
+#
+# If in-source build is used, the command will be run in the copy
+# of sources made for the best Python interpreter.
 _distutils-r1_run_common_phase() {
 	local DISTUTILS_ORIG_BUILD_DIR=${BUILD_DIR}
 
@@ -590,8 +593,16 @@ _distutils-r1_run_common_phase() {
 
 	export EPYTHON PYTHON PYTHONPATH
 
+	if [[ ${DISTUTILS_IN_SOURCE_BUILD} ]]; then
+		pushd "${BUILD_DIR}"/.. >/dev/null || die
+	fi
+
 	einfo "common: running ${1}"
 	"${@}"
+
+	if [[ ${DISTUTILS_IN_SOURCE_BUILD} ]]; then
+		popd >/dev/null || die
+	fi
 }
 
 # @FUNCTION: _distutils-r1_multijob_init
-- 
1.8.1.4