[gentoo-python] Alternate solution for 'python', 'python2' and 'pkg-config python' in ebuilds

[gentoo-python] Alternate solution for 'python', 'python2' and 'pkg-config python' in ebuilds

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1231 bytes --]

Hello,

Currently, we're using python-wrapper to properly respect EPYTHON
in ebuilds. This is a bit hacky since every call to Python has to go
through wrapper which chooses the same implementation over and over
again.

There was a request to handle 'pkg-config --cflags --libs python'
through eselect-python. Of course, we can't wrap pkg-config like we do
Python (well, we can but...).


The alternate solution is something similar to what vala people do. It
addresses all packages calling 'python', 'python2', 'python3'
or 'pkg-config python' and using any of the -r1 eclasses.

The idea is to create a ${T}/bin and ${T}/pkgconfig directories.
In the former, place 'python' and either 'python2' or 'python3'
symlinks proper to current ${EPYTHON}. In the latter, 'python.pc'
and possibly 'python2.pc' or 'python3.pc'. Add the former to ${PATH},
the latter to ${PKG_CONFIG_PATH}.

In -single-r1 and -any-r1 the directory creation and environment
manipulation would be done in pkg_setup(). In python-r1 it would have
to be done in python_export_best (with separate directory)
and python_foreach_impl.


Does this sound like a good solution? Should I run some tests?

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]

Re: [gentoo-python] Alternate solution for 'python', 'python2' and 'pkg-config python' in ebuilds

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 3220 bytes --]

On Fri, 29 Mar 2013 20:57:02 +0100
Michał Górny <mgorny@gentoo.org> wrote:

> Does this sound like a good solution? Should I run some tests?

And preceding the warm welcome of the idea, I've prepared a patch
for testing. I will send it in reply, as usual.

The generated tree looks like:

├── pypy-c1.9
│   ├── bin
│   │   ├── 2to3
│   │   ├── python -> /usr/bin/pypy-c1.9
│   │   ├── python2 -> python
│   │   └── python-config
│   └── pkgconfig
├── pypy-c2.0
│   ├── bin
│   │   ├── 2to3
│   │   ├── python -> /usr/bin/pypy-c2.0
│   │   ├── python2 -> python
│   │   └── python-config
│   └── pkgconfig
├── python2.5
│   ├── bin
│   │   ├── 2to3
│   │   ├── python -> /usr/bin/python2.5
│   │   ├── python2 -> python
│   │   └── python-config -> /usr/bin/python2.5-config
│   └── pkgconfig
│       ├── python2.pc -> python.pc
│       └── python.pc -> /dev/null
├── python2.6
│   ├── bin
│   │   ├── 2to3 -> /usr/bin/2to3-2.6
│   │   ├── python -> /usr/bin/python2.6
│   │   ├── python2 -> python
│   │   └── python-config -> /usr/bin/python2.6-config
│   └── pkgconfig
│       ├── python2.pc -> python.pc
│       └── python.pc -> /dev/null
├── python2.7
│   ├── bin
│   │   ├── 2to3 -> /usr/bin/2to3-2.7
│   │   ├── python -> /usr/bin/python2.7
│   │   ├── python2 -> python
│   │   └── python-config -> /usr/bin/python2.7-config
│   └── pkgconfig
│       ├── python2.pc -> python.pc
│       └── python.pc -> /usr/lib64/pkgconfig/python-2.7.pc
├── python3.1
│   ├── bin
│   │   ├── 2to3 -> /usr/bin/2to3-3.1
│   │   ├── python -> /usr/bin/python3.1
│   │   ├── python3 -> python
│   │   └── python-config -> /usr/bin/python3.1-config
│   └── pkgconfig
│       ├── python3.pc -> python.pc
│       └── python.pc -> /usr/lib64/pkgconfig/python-3.1.pc
├── python3.2
│   ├── bin
│   │   ├── 2to3 -> /usr/bin/2to3-3.2
│   │   ├── python -> /usr/bin/python3.2
│   │   ├── python3 -> python
│   │   └── python-config -> /usr/bin/python3.2-config
│   └── pkgconfig
│       ├── python3.pc -> python.pc
│       └── python.pc -> /usr/lib64/pkgconfig/python-3.2.pc
└── python3.3
    ├── bin
    │   ├── 2to3 -> /usr/bin/2to3-3.3
    │   ├── python -> /usr/bin/python3.3
    │   ├── python3 -> python
    │   └── python-config -> /usr/bin/python3.3-config
    └── pkgconfig
        ├── python3.pc -> python.pc
        └── python.pc -> /usr/lib64/pkgconfig/python-3.3.pc

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]

[gentoo-python] [PATCH] Create a temporary links for Python execs and pkg-config.

[Michał Górny]

With this patch, all apps which call 'python', 'python2', 'python3',
'2to3', 'python-config' or 'pkg-config ... python' start to work
properly without any kind of wrapper.

The wrappers are set up in python_foreach_impl, python_export_best
and pkg_setup phases of python-any-r1 and python-single-r1.

python_foreach_impl uses ${T}/${EPYTHON} subdirs for the wrappers.
The remaining functions set them 'globally' in ${T}.
---
 gx86/eclass/python-any-r1.eclass    |  5 +-
 gx86/eclass/python-r1.eclass        |  3 ++
 gx86/eclass/python-single-r1.eclass |  1 +
 gx86/eclass/python-utils-r1.eclass  | 94 +++++++++++++++++++++++++++++++++++++
 4 files changed, 102 insertions(+), 1 deletion(-)

diff --git a/gx86/eclass/python-any-r1.eclass b/gx86/eclass/python-any-r1.eclass
index 5d9a3d1..4761221 100644
--- a/gx86/eclass/python-any-r1.eclass
+++ b/gx86/eclass/python-any-r1.eclass
@@ -205,7 +205,10 @@ python-any-r1_pkg_setup() {
 	local PYTHON_PKG_DEP
 	for i in "${rev_impls[@]}"; do
 		python_export "${i}" PYTHON_PKG_DEP EPYTHON PYTHON
-		ROOT=/ has_version "${PYTHON_PKG_DEP}" && return
+		if ROOT=/ has_version "${PYTHON_PKG_DEP}"; then
+			python_wrapper_setup "${T}"
+			return
+		fi
 	done
 }
 
diff --git a/gx86/eclass/python-r1.eclass b/gx86/eclass/python-r1.eclass
index 4ba80a9..a3fdd0e 100644
--- a/gx86/eclass/python-r1.eclass
+++ b/gx86/eclass/python-r1.eclass
@@ -634,7 +634,9 @@ _python_multibuild_wrapper() {
 	debug-print-function ${FUNCNAME} "${@}"
 
 	local -x EPYTHON PYTHON
+	local -x PATH=${PATH} PKG_CONFIG_PATH=${PKG_CONFIG_PATH}
 	python_export "${MULTIBUILD_VARIANT}" EPYTHON PYTHON
+	python_wrapper_setup "${T}/${EPYTHON}"
 
 	"${@}"
 }
@@ -707,6 +709,7 @@ python_export_best() {
 
 	debug-print "${FUNCNAME}: Best implementation is: ${best}"
 	python_export "${best}" "${@}"
+	python_wrapper_setup "${T}"
 }
 
 # @FUNCTION: python_replicate_script
diff --git a/gx86/eclass/python-single-r1.eclass b/gx86/eclass/python-single-r1.eclass
index 6235b66..950bf52 100644
--- a/gx86/eclass/python-single-r1.eclass
+++ b/gx86/eclass/python-single-r1.eclass
@@ -207,6 +207,7 @@ python-single-r1_pkg_setup() {
 			fi
 
 			python_export "${impl}" EPYTHON PYTHON
+			python_wrapper_setup "${T}"
 		fi
 	done
 
diff --git a/gx86/eclass/python-utils-r1.eclass b/gx86/eclass/python-utils-r1.eclass
index a375546..6aa4a64 100644
--- a/gx86/eclass/python-utils-r1.eclass
+++ b/gx86/eclass/python-utils-r1.eclass
@@ -818,5 +818,99 @@ python_doheader() {
 	doins -r "${@}" || die
 }
 
+# @FUNCTION: python_wrapper_setup
+# @USAGE: <path> [<impl>]
+# @DESCRIPTION:
+# Create proper 'python' executable and pkg-config wrappers
+# (if available) in the directory named by <path>. Set up PATH
+# and PKG_CONFIG_PATH appropriately.
+#
+# The wrappers will be created for implementation named by <impl>,
+# or for one named by ${EPYTHON} if no <impl> passed.
+#
+# If the named directory contains a python symlink already, it will
+# be assumed to contain proper wrappers already and only environment
+# setup will be done. If wrapper update is requested, the directory
+# shall be removed first.
+python_wrapper_setup() {
+	debug-print-function ${FUNCNAME} "${@}"
+
+	local workdir=${1}
+	local impl=${2:-${EPYTHON}}
+
+	[[ ${workdir} ]] || die "${FUNCNAME}: no workdir specified."
+	[[ ${impl} ]] || die "${FUNCNAME}: no impl nor EPYTHON specified."
+
+	if [[ ! -x ${workdir}/bin/python ]]; then
+		mkdir -p "${workdir}"/{bin,pkgconfig} || die
+
+		# Clean up, in case we were supposed to do a cheap update.
+		rm -f "${workdir}"/bin/python{,2,3,-config}
+		rm -f "${workdir}"/bin/2to3
+		rm -f "${workdir}"/pkgconfig/python{,2,3}.pc
+
+		local EPYTHON PYTHON
+		python_export "${impl}" EPYTHON PYTHON
+
+		local pyver
+		if [[ ${EPYTHON} == python3* ]]; then
+			pyver=3
+		else # includes pypy & jython
+			pyver=2
+		fi
+
+		# Python interpreter
+		ln -s "${PYTHON}" "${workdir}"/bin/python || die
+		ln -s python "${workdir}"/bin/python${pyver} || die
+
+		local nonsupp=()
+
+		# CPython-specific
+		if [[ ${EPYTHON} == python* ]]; then
+			ln -s "${PYTHON}-config" "${workdir}"/bin/python-config || die
+
+			# Python 2.6+.
+			if [[ ${EPYTHON} != python2.5 ]]; then
+				ln -s "${PYTHON/python/2to3-}" "${workdir}"/bin/2to3 || die
+			else
+				nonsupp+=( 2to3 )
+			fi
+
+			# Python 2.7+.
+			if [[ ${EPYTHON} != python2.[56] ]]; then
+				ln -s "${EPREFIX}"/usr/$(get_libdir)/pkgconfig/${EPYTHON/n/n-}.pc \
+					"${workdir}"/pkgconfig/python.pc || die
+			else
+				# XXX?
+				ln -s /dev/null "${workdir}"/pkgconfig/python.pc || die
+			fi
+			ln -s python.pc "${workdir}"/pkgconfig/python${pyver}.pc || die
+		else
+			nonsupp+=( 2to3 python-config )
+		fi
+
+		local x
+		for x in "${nonsupp[@]}"; do
+			echo >"${workdir}"/bin/${x} <<__EOF__ || die
+#!/bin/sh
+echo "${x} is not supported by ${EPYTHON}" >&2
+exit 1
+__EOF__
+			chmod +x "${workdir}"/bin/${x} || die
+		done
+
+		# Now, set the environment.
+		# But note that ${workdir} may be shared with something else,
+		# and thus already on top of PATH.
+		if [[ ${PATH##:*} != ${workdir}/bin ]]; then
+			PATH=${workdir}/bin${PATH:+:${PATH}}
+		fi
+		if [[ ${PKG_CONFIG_PATH##:*} != ${workdir}/pkgconfig ]]; then
+			PKG_CONFIG_PATH=${workdir}/pkgconfig${PKG_CONFIG_PATH:+:${PKG_CONFIG_PATH}}
+		fi
+		export PATH PKG_CONFIG_PATH
+	fi
+}
+
 _PYTHON_UTILS_R1=1
 fi
-- 
1.8.1.5

Re: [gentoo-python] [PATCH] Create a temporary links for Python execs and pkg-config.

[Mike Gilbert]

[-- Attachment #1: Type: text/plain, Size: 643 bytes --]

On 03/29/2013 06:39 PM, Michał Górny wrote:
> With this patch, all apps which call 'python', 'python2', 'python3',
> '2to3', 'python-config' or 'pkg-config ... python' start to work
> properly without any kind of wrapper.
> 
> The wrappers are set up in python_foreach_impl, python_export_best
> and pkg_setup phases of python-any-r1 and python-single-r1.
> 
> python_foreach_impl uses ${T}/${EPYTHON} subdirs for the wrappers.
> The remaining functions set them 'globally' in ${T}.

I tried to follow this line-by-line, but symlinks make my head hurt. ;)

I'm ok with the patch, although I have no idea what it might break.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 230 bytes --]

Re: [gentoo-python] [PATCH] Create a temporary links for Python execs and pkg-config.

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 581 bytes --]

On Fri, 29 Mar 2013 23:39:30 +0100
Michał Górny <mgorny@gentoo.org> wrote:

> With this patch, all apps which call 'python', 'python2', 'python3',
> '2to3', 'python-config' or 'pkg-config ... python' start to work
> properly without any kind of wrapper.
> 
> The wrappers are set up in python_foreach_impl, python_export_best
> and pkg_setup phases of python-any-r1 and python-single-r1.
> 
> python_foreach_impl uses ${T}/${EPYTHON} subdirs for the wrappers.
> The remaining functions set them 'globally' in ${T}.

Committed.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]