[gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[Michał Górny]

[-- Attachment #1.1: Type: text/plain, Size: 272 bytes --]

Hello,

Instead of the floating patches and p-d-ng modifications I sent
earlier, here are the two complete (so far, well, initial :P) eclasses
for review.

They are designed as 'mostly' drop-in python-distutils-ng replacement.

-- 
Best regards,
Michał Górny

[-- Attachment #1.2: python-r1.eclass --]
[-- Type: text/plain, Size: 5952 bytes --]

# Copyright 1999-2012 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: $

# @ECLASS: python-r1
# @MAINTAINER:
# Python herd <python@gentoo.org>
# @AUTHOR:
# Author: Michał Górny <mgorny@gentoo.org>
# Based on work of: Krzysztof Pawlik <nelchael@gentoo.org>
# @BLURB: A common, simple eclass for Python packages.
# @DESCRIPTION:
# A common eclass providing helper functions to build and install
# packages supporting being installed for multiple Python
# implementations.
#
# This eclass sets correct IUSE and REQUIRED_USE. It exports PYTHON_DEPS
# and PYTHON_USEDEP so you can create correct dependencies for your
# package easily. It also provides methods to easily run a command for
# each enabled Python implementation and duplicate the sources for them.

case "${EAPI}" in
	0|1|2|3)
		die "Unsupported EAPI=${EAPI} (too old) for ${ECLASS}"
		;;
	4)
		# EAPI=4 needed for REQUIRED_USE
		;;
	*)
		die "Unsupported EAPI=${EAPI} (unknown) for ${ECLASS}"
		;;
esac

# @ECLASS-VARIABLE: _PYTHON_ALL_IMPLS
# @INTERNAL
# @DESCRIPTION:
# All supported Python implementations, most preferred last.
_PYTHON_ALL_IMPLS=(
	jython2_5
	pypy1_8 pypy1_9
	python3_1 python3_2
	python2_5 python2_6 python2_7
)

# @ECLASS-VARIABLE: PYTHON_COMPAT
# @DESCRIPTION:
# This variable contains a space separated list of Python
# implementations a package supports. It must be set before
# the `inherit' call.  The default is to enable all implementations.
#
# PYTHON_COMPAT can be either a scalar or an array. If it's a scalar,
# the eclass will implicitly convert it to an array.
: ${PYTHON_COMPAT:=${_PYTHON_ALL_IMPLS[@]}}

# @ECLASS-VARIABLE: PYTHON_REQ_USE
# @DEFAULT_UNSET
# @DESCRIPTION:
# The list of USEflags required to be enabled on the chosen Python
# implementations, formed as a USE-dependency string. It should be valid
# for all implementations in PYTHON_COMPAT, so it may be necessary to
# use USE defaults.
#
# Example:
# @CODE
# PYTHON_REQ_USE="gdbm,ncurses(-)?"
# @CODE
#
# Will cause the Python dependencies to look like:
# @CODE
# python_targets_pythonX_Y? (
#   dev-lang/python:X_Y[gdbm,ncurses(-)?] )
# @CODE

# @ECLASS-VARIABLE: PYTHON_DEPS
# @DESCRIPTION:
# This is an eclass-generated Python dependency string for all
# implementations listed in PYTHON_COMPAT. It should be used
# in RDEPEND and/or DEPEND like:
#
# @CODE
# RDEPEND="${PYTHON_DEPS}
#   dev-foo/mydep"
# DEPEND="${RDEPEND}"
# @CODE

# @ECLASS-VARIABLE: PYTHON_USEDEP
# @DESCRIPTION:
# This is an eclass-generated USE-dependency string which can be used to
# depend on another Python package being built for the same Python
# implementations. It should be used like:
#
# @CODE
# RDEPEND="dev-python/foo[${PYTHON_USEDEP}]"
# @CODE

PYTHON_COMPAT=( ${PYTHON_COMPAT[@]} )

_python_set_globals() {
	local flags=( "${PYTHON_COMPAT[@]/#/python_targets_}" )
	local optflags=${flags[@]/%/?}

	IUSE=${flags[*]}
	REQUIRED_USE="|| ( ${flags[*]} )"
	PYTHON_USEDEP=${optflags// /,}

	PYTHON_DEPS=
	local i
	for i in ${PYTHON_COMPAT[@]}; do
		local d
		case ${i} in
			python*)
				d='dev-lang/python';;
			jython*)
				d='dev-java/jython';;
			pypy*)
				d='dev-python/pypy';;
			*)
				die "Invalid implementation: ${i}"
		esac

		local v=${i##*[a-z]}
		local usestr
		[[ ${PYTHON_REQ_USE} ]] && usestr="[${PYTHON_REQ_USE}]"
		PYTHON_DEPS+=" python_targets_${i}? (
			${d}:${v/_/.}${usestr} )"
	done
}
_python_set_globals

# @FUNCTION: python_get_PYTHON
# @USAGE: <impl>
# @DESCRIPTION:
# Get the Python executable name for the given implementation. Please
# note that this this function returns the 'basename' only.
# If using it for a hashbang, please use #!/usr/bin/env.
python_get_PYTHON() {
	local impl=${1/_/.}

	case "${impl}" in
		python*|jython*)
			echo ${impl}
			;;
		pypy*)
			echo pypy-c${impl#pypy}
			;;
		*)
			die "Invalid argument to python_get_PYTHON: ${1}"
			;;
	esac
}

# @FUNCTION: python_copy_sources
# @DESCRIPTION:
# Create a single copy of the package sources (${S}) for each enabled
# Python implementation.
python_copy_sources() {
	local impl
	local bdir=${BUILD_DIR:-${S}}

	einfo "Will copy sources from ${S}"
	# the order is irrelevant here
	for impl in ${PYTHON_COMPAT[@]}; do
		if use python_targets_${impl}
		then
			local BUILD_DIR=${bdir%%/}-${impl}

			einfo "${impl}: copying to ${BUILD_DIR}"
			cp -pr "${S}" "${BUILD_DIR}" || die
		fi
	done
}

# @FUNCTION: python_foreach_impl
# @USAGE: <command> [<args>...]
# @DESCRIPTION:
# Run the given command for each of the enabled Python implementations.
# If additional parameters are passed, they will be passed through
# to the command. If the command fails, python_foreach_impl dies.
# If necessary, use ':' to force a successful return.
#
# Before the command is run, EPYTHON is set to the name of the current
# Python implementation, PYTHON is set to the correct Python executable
# name and exported, and BUILD_DIR is set to a 'default' build directory
# for given implementation (e.g. ${BUILD_DIR:-${S}}-python2_7).
#
# The command is run inside the build directory. If it doesn't exist
# yet, it is created (as an empty directory!). If your build system does
# not support out-of-source builds, you will likely want to use
# python_copy_sources first.
python_foreach_impl() {
	local impl
	local bdir=${BUILD_DIR:-${S}}

	for impl in ${_PYTHON_ALL_IMPLS[@]}; do
		if has ${impl} ${PYTHON_COMPAT[@]} && use python_targets_${impl}
		then
			local PYTHON=$(python_get_PYTHON "${impl}")
			local EPYTHON=${impl}
			local BUILD_DIR=${bdir%%/}-${impl}
			export PYTHON

			mkdir -p "${BUILD_DIR}" || die
			pushd "${BUILD_DIR}" &>/dev/null || die
			einfo "${EPYTHON}: running ${@}"
			"${@}" || die "${EPYTHON}: ${1} failed"
			popd || die
		fi
	done
}

[-- Attachment #1.3: distutils-r1.eclass --]
[-- Type: text/plain, Size: 6367 bytes --]

# Copyright 1999-2012 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: $

# @ECLASS: distutils-r1
# @MAINTAINER:
# Python herd <python@gentoo.org>
# @AUTHOR:
# Author: Michał Górny <mgorny@gentoo.org>
# Based on the work of: Krzysztof Pawlik <nelchael@gentoo.org>
# @BLURB: A simple eclass to build Python packages using distutils.
# @DESCRIPTION:
# A simple eclass providing functions to build Python packages using
# the distutils build system. It exports phase functions for all
# the src_* phases. Each of the phases runs two pseudo-phases:
# python_..._all() (e.g. python_prepare_all()) once in ${S}, then
# python_...() (e.g. python_prepare()) for each implementation
# (see: python_foreach_impl() in python-r1).
#
# In distutils-r1_src_prepare(), the 'all' function is run before
# per-implementation ones (because it creates the implementations),
# per-implementation functions are run in a random order.
#
# In remaining phase functions, the per-implementation functions are run
# before the 'all' one, and they are ordered from the least to the most
# preferred implementation (so that 'better' files overwrite 'worse'
# ones).
#
# If the ebuild doesn't specify a particular pseudo-phase function,
# the default one will be used (distutils-r1_...). Defaults are provided
# for all per-implementation pseudo-phases, python_prepare_all()
# and python_install_all(); whenever writing your own pseudo-phase
# functions, you should consider calling the defaults (and especially
# distutils-r1_python_prepare_all).
#
# Please note that distutils-r1 sets RDEPEND and DEPEND for you.

case "${EAPI}" in
	0|1|2|3)
		die "Unsupported EAPI=${EAPI} (too old) for ${ECLASS}"
		;;
	4)
		;;
	*)
		die "Unsupported EAPI=${EAPI} (unknown) for ${ECLASS}"
		;;
esac

inherit eutils python-r1

EXPORT_FUNCTIONS src_prepare src_configure src_compile src_test src_install

RDEPEND=${PYTHON_DEPS}
DEPEND=${PYTHON_DEPS}

# @FUNCTION: distutils-r1_python_prepare_all
# @DESCRIPTION:
# The default python_prepare_all(). It applies the patches from PATCHES
# array, then user patches and finally calls python_copy_sources to
# create copies of resulting sources for each Python implementation.
distutils-r1_python_prepare_all() {
	[[ ${PATCHES} ]] && epatch "${PATCHES[@]}"

	epatch_user

	# create source copies for each implementation
	python_copy_sources
}

# @FUNCTION: distutils-r1_python_prepare
# @DESCRIPTION:
# The default python_prepare(). Currently it is a no-op
# but in the future it may apply implementation-specific quirks
# to the build system.
distutils-r1_python_prepare() {
	:
}

# @FUNCTION: distutils-r1_python_configure
# @DESCRIPTION:
# The default python_configure(). Currently a no-op.
distutils-r1_python_configure() {
	:
}

# @FUNCTION: distutils-r1_python_compile
# @DESCRIPTION:
# The default python_compile(). Runs 'setup.py build' using the correct
# Python implementation.
distutils-r1_python_compile() {
	cd "${BUILD_DIR}" || die
	"${PYTHON}" setup.py build || die
}

# @FUNCTION: distutils-r1_python_test
# @DESCRIPTION:
# The default python_test(). Currently a no-op.
distutils-r1_python_test() {
	:
}

# @FUNCTION: distutils-r1_rename_scripts
# @DESCRIPTION:
# Renames installed Python scripts to be implementation-suffixed.
# ${PYTHON} has to be set to the expected Python executable (which
# hashbang will be grepped for), and ${EPYTHON} to the implementation
# name (for new name).
distutils-r1_rename_scripts() {
	local dir f
	local wrapdir=${T}/distutils_wrappers

	# XXX: change this if we ever allow directories in bin/sbin
	for dir in bin sbin usr/bin usr/sbin; do
		for f in "${D}"/${dir}/*; do
			if [[ -x ${f} ]]; then
				if [[ "$(head -n 1 "${f}")" == '#!'*${PYTHON}* ]]
				then
					local newf=${f}-${EPYTHON}
					mv "${f}" "${newf}" || die

					mkdir -p "${wrapdir}"/${dir}
					ln -fs "${newf##*/}" \
						"${wrapdir}"/${dir}/"${f##*/}" || die
				fi
			fi
		done
	done
}

# @FUNCTION: distutils-r1_python_install
# @DESCRIPTION:
# The default python_install(). Runs 'setup.py install' using
# the correct Python implementation, and appending the optimization
# flags. Then calls distutils-r1_rename_scripts.
distutils-r1_python_install() {
	local flags

	case "${EPYTHON}" in
		jython*)
			flags='--compile';;
		*)
			flags='--compile -O2';;
	esac

	# XXX: where does it come from?!
	unset PYTHONDONTWRITEBYTECODE

	cd "${BUILD_DIR}" || die
	"${PYTHON}" setup.py install ${flags} --root="${D}" || die

	distutils-r1_rename_scripts
}

# @FUNCTION: distutils-r1_python_install_all
# @DESCRIPTION:
# The default python_install_all(). It creates wrappers
# for the implementation-suffixed executables.
distutils-r1_python_install_all() {
	local wrapdir=${T}/distutils_wrappers

	if [[ -d ${wrapdir} ]]; then
		cp -r "${wrapdir}/." "${D}" || die
	fi
}

distutils-r1_src_prepare() {
	# common preparations
	if declare -f python_prepare_all >/dev/null; then
		python_prepare_all
	else
		distutils-r1_python_prepare_all
	fi

	if declare -f python_prepare >/dev/null; then
		python_foreach_impl python_prepare
	else
		distutils-r1_python_prepare
	fi
}

distutils-r1_src_configure() {
	if declare -f python_configure >/dev/null; then
		python_foreach_impl python_configure
	else
		distutils-r1_python_configure
	fi

	if declare -f python_configure_all >/dev/null; then
		python_configure_all
	fi
}

distutils-r1_src_compile() {
	if declare -f python_compile >/dev/null; then
		python_foreach_impl python_compile
	else
		python_foreach_impl distutils-r1_python_compile
	fi

	if declare -f python_compile_all >/dev/null; then
		python_compile_all
	fi
}

distutils-r1_src_test() {
	if declare -f python_test >/dev/null; then
		python_foreach_impl python_test
	else
		distutils-r1_python_test
	fi

	if declare -f python_test_all >/dev/null; then
		python_test_all
	fi
}

distutils-r1_src_install() {
	if declare -f python_install >/dev/null; then
		python_foreach_impl python_install
	else
		python_foreach_impl distutils-r1_python_install
	fi

	if declare -f python_install_all >/dev/null; then
		python_install_all
	else
		distutils-r1_python_install_all
	fi
}

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[Tomáš Chvátal]

2012/9/29 Michał Górny <mgorny@gentoo.org>:
> Hello,
>
> Instead of the floating patches and p-d-ng modifications I sent
> earlier, here are the two complete (so far, well, initial :P) eclasses
> for review.
>
> They are designed as 'mostly' drop-in python-distutils-ng replacement.
>
Hi,

the eclasses look pretty, so good job,
just one question tho, would it be possible to use more
debug-something calls so the log file would be populated automatically
with whats going on (same like eg git-2 eclass)?

Cheers

Tom

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 708 bytes --]

On Sat, 29 Sep 2012 12:00:18 +0200
Tomáš Chvátal <tomas.chvatal@gmail.com> wrote:

> 2012/9/29 Michał Górny <mgorny@gentoo.org>:
> > Hello,
> >
> > Instead of the floating patches and p-d-ng modifications I sent
> > earlier, here are the two complete (so far, well, initial :P) eclasses
> > for review.
> >
> > They are designed as 'mostly' drop-in python-distutils-ng replacement.
> >
> Hi,
> 
> the eclasses look pretty, so good job,
> just one question tho, would it be possible to use more
> debug-something calls so the log file would be populated automatically
> with whats going on (same like eg git-2 eclass)?

Can do. Was just lazy ;P.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[Michał Górny]

[-- Attachment #1.1: Type: text/plain, Size: 694 bytes --]

On Sat, 29 Sep 2012 12:00:18 +0200
Tomáš Chvátal <tomas.chvatal@gmail.com> wrote:

> 2012/9/29 Michał Górny <mgorny@gentoo.org>:
> > Hello,
> >
> > Instead of the floating patches and p-d-ng modifications I sent
> > earlier, here are the two complete (so far, well, initial :P) eclasses
> > for review.
> >
> > They are designed as 'mostly' drop-in python-distutils-ng replacement.
> >
> Hi,
> 
> the eclasses look pretty, so good job,
> just one question tho, would it be possible to use more
> debug-something calls so the log file would be populated automatically
> with whats going on (same like eg git-2 eclass)?

Try now :P.

-- 
Best regards,
Michał Górny

[-- Attachment #1.2: python-r1.eclass --]
[-- Type: text/plain, Size: 6392 bytes --]

# Copyright 1999-2012 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: $

# @ECLASS: python-r1
# @MAINTAINER:
# Python herd <python@gentoo.org>
# @AUTHOR:
# Author: Michał Górny <mgorny@gentoo.org>
# Based on work of: Krzysztof Pawlik <nelchael@gentoo.org>
# @BLURB: A common, simple eclass for Python packages.
# @DESCRIPTION:
# A common eclass providing helper functions to build and install
# packages supporting being installed for multiple Python
# implementations.
#
# This eclass sets correct IUSE and REQUIRED_USE. It exports PYTHON_DEPS
# and PYTHON_USEDEP so you can create correct dependencies for your
# package easily. It also provides methods to easily run a command for
# each enabled Python implementation and duplicate the sources for them.

case "${EAPI}" in
	0|1|2|3)
		die "Unsupported EAPI=${EAPI} (too old) for ${ECLASS}"
		;;
	4)
		# EAPI=4 needed for REQUIRED_USE
		;;
	*)
		die "Unsupported EAPI=${EAPI} (unknown) for ${ECLASS}"
		;;
esac

# @ECLASS-VARIABLE: _PYTHON_ALL_IMPLS
# @INTERNAL
# @DESCRIPTION:
# All supported Python implementations, most preferred last.
_PYTHON_ALL_IMPLS=(
	jython2_5
	pypy1_8 pypy1_9
	python3_1 python3_2
	python2_5 python2_6 python2_7
)

# @ECLASS-VARIABLE: PYTHON_COMPAT
# @DESCRIPTION:
# This variable contains a space separated list of Python
# implementations a package supports. It must be set before
# the `inherit' call.  The default is to enable all implementations.
#
# PYTHON_COMPAT can be either a scalar or an array. If it's a scalar,
# the eclass will implicitly convert it to an array.
: ${PYTHON_COMPAT:=${_PYTHON_ALL_IMPLS[@]}}

# @ECLASS-VARIABLE: PYTHON_REQ_USE
# @DEFAULT_UNSET
# @DESCRIPTION:
# The list of USEflags required to be enabled on the chosen Python
# implementations, formed as a USE-dependency string. It should be valid
# for all implementations in PYTHON_COMPAT, so it may be necessary to
# use USE defaults.
#
# Example:
# @CODE
# PYTHON_REQ_USE="gdbm,ncurses(-)?"
# @CODE
#
# Will cause the Python dependencies to look like:
# @CODE
# python_targets_pythonX_Y? (
#   dev-lang/python:X_Y[gdbm,ncurses(-)?] )
# @CODE

# @ECLASS-VARIABLE: PYTHON_DEPS
# @DESCRIPTION:
# This is an eclass-generated Python dependency string for all
# implementations listed in PYTHON_COMPAT. It should be used
# in RDEPEND and/or DEPEND like:
#
# @CODE
# RDEPEND="${PYTHON_DEPS}
#   dev-foo/mydep"
# DEPEND="${RDEPEND}"
# @CODE

# @ECLASS-VARIABLE: PYTHON_USEDEP
# @DESCRIPTION:
# This is an eclass-generated USE-dependency string which can be used to
# depend on another Python package being built for the same Python
# implementations. It should be used like:
#
# @CODE
# RDEPEND="dev-python/foo[${PYTHON_USEDEP}]"
# @CODE

PYTHON_COMPAT=( ${PYTHON_COMPAT[@]} )

_python_set_globals() {
	local flags=( "${PYTHON_COMPAT[@]/#/python_targets_}" )
	local optflags=${flags[@]/%/?}

	IUSE=${flags[*]}
	REQUIRED_USE="|| ( ${flags[*]} )"
	PYTHON_USEDEP=${optflags// /,}

	PYTHON_DEPS=
	local i
	for i in ${PYTHON_COMPAT[@]}; do
		local d
		case ${i} in
			python*)
				d='dev-lang/python';;
			jython*)
				d='dev-java/jython';;
			pypy*)
				d='dev-python/pypy';;
			*)
				die "Invalid implementation: ${i}"
		esac

		local v=${i##*[a-z]}
		local usestr
		[[ ${PYTHON_REQ_USE} ]] && usestr="[${PYTHON_REQ_USE}]"
		PYTHON_DEPS+=" python_targets_${i}? (
			${d}:${v/_/.}${usestr} )"
	done
}
_python_set_globals

# @FUNCTION: python_get_PYTHON
# @USAGE: <impl>
# @DESCRIPTION:
# Get the Python executable name for the given implementation. Please
# note that this this function returns the 'basename' only.
# If using it for a hashbang, please use #!/usr/bin/env.
python_get_PYTHON() {
	debug-print-function ${FUNCNAME} "${@}"

	local impl=${1/_/.}
	local ret

	case "${impl}" in
		python*|jython*)
			ret=${impl}
			;;
		pypy*)
			ret=pypy-c${impl#pypy}
			;;
		*)
			die "Invalid argument to python_get_PYTHON: ${1}"
			;;
	esac

	debug-print "${FUNCNAME}: ${impl} -> ${ret}"
	echo "${ret}"
}

# @FUNCTION: python_copy_sources
# @DESCRIPTION:
# Create a single copy of the package sources (${S}) for each enabled
# Python implementation.
python_copy_sources() {
	debug-print-function ${FUNCNAME} "${@}"

	local impl
	local bdir=${BUILD_DIR:-${S}}

	debug-print "${FUNCNAME}: bdir = ${bdir}"
	einfo "Will copy sources from ${S}"
	# the order is irrelevant here
	for impl in ${PYTHON_COMPAT[@]}; do
		if use python_targets_${impl}
		then
			local BUILD_DIR=${bdir%%/}-${impl}

			einfo "${impl}: copying to ${BUILD_DIR}"
			debug-print "${FUNCNAME}: [${impl}] cp ${S} => ${BUILD_DIR}"
			cp -pr "${S}" "${BUILD_DIR}" || die
		fi
	done
}

# @FUNCTION: python_foreach_impl
# @USAGE: <command> [<args>...]
# @DESCRIPTION:
# Run the given command for each of the enabled Python implementations.
# If additional parameters are passed, they will be passed through
# to the command. If the command fails, python_foreach_impl dies.
# If necessary, use ':' to force a successful return.
#
# Before the command is run, EPYTHON is set to the name of the current
# Python implementation, PYTHON is set to the correct Python executable
# name and exported, and BUILD_DIR is set to a 'default' build directory
# for given implementation (e.g. ${BUILD_DIR:-${S}}-python2_7).
#
# The command is run inside the build directory. If it doesn't exist
# yet, it is created (as an empty directory!). If your build system does
# not support out-of-source builds, you will likely want to use
# python_copy_sources first.
python_foreach_impl() {
	debug-print-function ${FUNCNAME} "${@}"

	local impl
	local bdir=${BUILD_DIR:-${S}}

	debug-print "${FUNCNAME}: bdir = ${bdir}"
	for impl in ${_PYTHON_ALL_IMPLS[@]}; do
		if has ${impl} ${PYTHON_COMPAT[@]} && use python_targets_${impl}
		then
			local PYTHON=$(python_get_PYTHON "${impl}")
			local EPYTHON=${impl}
			local BUILD_DIR=${bdir%%/}-${impl}
			export PYTHON

			debug-print "${FUNCNAME}: [${impl}] build_dir = ${BUILD_DIR}"

			mkdir -p "${BUILD_DIR}" || die
			pushd "${BUILD_DIR}" &>/dev/null || die
			einfo "${EPYTHON}: running ${@}"
			"${@}" || die "${EPYTHON}: ${1} failed"
			popd &>/dev/null || die
		fi
	done
}

[-- Attachment #1.3: distutils-r1.eclass --]
[-- Type: text/plain, Size: 7245 bytes --]

# Copyright 1999-2012 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: $

# @ECLASS: distutils-r1
# @MAINTAINER:
# Python herd <python@gentoo.org>
# @AUTHOR:
# Author: Michał Górny <mgorny@gentoo.org>
# Based on the work of: Krzysztof Pawlik <nelchael@gentoo.org>
# @BLURB: A simple eclass to build Python packages using distutils.
# @DESCRIPTION:
# A simple eclass providing functions to build Python packages using
# the distutils build system. It exports phase functions for all
# the src_* phases. Each of the phases runs two pseudo-phases:
# python_..._all() (e.g. python_prepare_all()) once in ${S}, then
# python_...() (e.g. python_prepare()) for each implementation
# (see: python_foreach_impl() in python-r1).
#
# In distutils-r1_src_prepare(), the 'all' function is run before
# per-implementation ones (because it creates the implementations),
# per-implementation functions are run in a random order.
#
# In remaining phase functions, the per-implementation functions are run
# before the 'all' one, and they are ordered from the least to the most
# preferred implementation (so that 'better' files overwrite 'worse'
# ones).
#
# If the ebuild doesn't specify a particular pseudo-phase function,
# the default one will be used (distutils-r1_...). Defaults are provided
# for all per-implementation pseudo-phases, python_prepare_all()
# and python_install_all(); whenever writing your own pseudo-phase
# functions, you should consider calling the defaults (and especially
# distutils-r1_python_prepare_all).
#
# Please note that distutils-r1 sets RDEPEND and DEPEND for you.

case "${EAPI}" in
	0|1|2|3)
		die "Unsupported EAPI=${EAPI} (too old) for ${ECLASS}"
		;;
	4)
		;;
	*)
		die "Unsupported EAPI=${EAPI} (unknown) for ${ECLASS}"
		;;
esac

inherit eutils python-r1

EXPORT_FUNCTIONS src_prepare src_configure src_compile src_test src_install

RDEPEND=${PYTHON_DEPS}
DEPEND=${PYTHON_DEPS}

# @FUNCTION: distutils-r1_python_prepare_all
# @DESCRIPTION:
# The default python_prepare_all(). It applies the patches from PATCHES
# array, then user patches and finally calls python_copy_sources to
# create copies of resulting sources for each Python implementation.
distutils-r1_python_prepare_all() {
	debug-print-function ${FUNCNAME} "${@}"

	[[ ${PATCHES} ]] && epatch "${PATCHES[@]}"

	epatch_user

	# create source copies for each implementation
	python_copy_sources
}

# @FUNCTION: distutils-r1_python_prepare
# @DESCRIPTION:
# The default python_prepare(). Currently it is a no-op
# but in the future it may apply implementation-specific quirks
# to the build system.
distutils-r1_python_prepare() {
	debug-print-function ${FUNCNAME} "${@}"

	:
}

# @FUNCTION: distutils-r1_python_configure
# @DESCRIPTION:
# The default python_configure(). Currently a no-op.
distutils-r1_python_configure() {
	debug-print-function ${FUNCNAME} "${@}"

	:
}

# @FUNCTION: distutils-r1_python_compile
# @DESCRIPTION:
# The default python_compile(). Runs 'setup.py build' using the correct
# Python implementation.
distutils-r1_python_compile() {
	debug-print-function ${FUNCNAME} "${@}"

	cd "${BUILD_DIR}" || die
	set -- "${PYTHON}" setup.py build
	echo "${@}"
	"${@}" || die
}

# @FUNCTION: distutils-r1_python_test
# @DESCRIPTION:
# The default python_test(). Currently a no-op.
distutils-r1_python_test() {
	debug-print-function ${FUNCNAME} "${@}"

	:
}

# @FUNCTION: distutils-r1_rename_scripts
# @DESCRIPTION:
# Renames installed Python scripts to be implementation-suffixed.
# ${PYTHON} has to be set to the expected Python executable (which
# hashbang will be grepped for), and ${EPYTHON} to the implementation
# name (for new name).
distutils-r1_rename_scripts() {
	debug-print-function ${FUNCNAME} "${@}"

	local dir f
	local wrapdir=${T}/distutils_wrappers

	# XXX: change this if we ever allow directories in bin/sbin
	for dir in bin sbin usr/bin usr/sbin; do
		for f in "${D}"/${dir}/*; do
			if [[ -x ${f} ]]; then
				debug-print "${FUNCNAME}: found executable at ${f#${D}/}"

				if [[ "$(head -n 1 "${f}")" == '#!'*${PYTHON}* ]]
				then
					debug-print "${FUNCNAME}: matching shebang: $(head -n 1 "${f}")"

					local newf=${f}-${EPYTHON}
					debug-print "${FUNCNAME}: renamed to ${newf#${D}/}"
					mv "${f}" "${newf}" || die

					debug-print "${FUNCNAME}: symlink to ${wrapdir}/${dir}/${f##*/}"

					mkdir -p "${wrapdir}"/${dir}
					ln -fs "${newf##*/}" \
						"${wrapdir}/${dir}/${f##*/}" || die
				fi
			fi
		done
	done
}

# @FUNCTION: distutils-r1_python_install
# @DESCRIPTION:
# The default python_install(). Runs 'setup.py install' using
# the correct Python implementation, and appending the optimization
# flags. Then calls distutils-r1_rename_scripts.
distutils-r1_python_install() {
	debug-print-function ${FUNCNAME} "${@}"

	local flags

	case "${EPYTHON}" in
		jython*)
			flags='--compile';;
		*)
			flags='--compile -O2';;
	esac
	debug-print "${FUNCNAME}: [${EPYTHON}] flags: ${flags}"

	unset PYTHONDONTWRITEBYTECODE

	cd "${BUILD_DIR}" || die
	set -- "${PYTHON}" setup.py install ${flags} --root="${D}"
	echo "${@}"
	"${@}" || die

	distutils-r1_rename_scripts
}

# @FUNCTION: distutils-r1_python_install_all
# @DESCRIPTION:
# The default python_install_all(). It creates wrappers
# for the implementation-suffixed executables.
distutils-r1_python_install_all() {
	debug-print-function ${FUNCNAME} "${@}"

	local wrapdir=${T}/distutils_wrappers

	if [[ -d ${wrapdir} ]]; then
		cp -r "${wrapdir}/." "${D}" || die
	fi
}

distutils-r1_src_prepare() {
	debug-print-function ${FUNCNAME} "${@}"

	# common preparations
	if declare -f python_prepare_all >/dev/null; then
		python_prepare_all
	else
		distutils-r1_python_prepare_all
	fi

	if declare -f python_prepare >/dev/null; then
		python_foreach_impl python_prepare
	else
		distutils-r1_python_prepare
	fi
}

distutils-r1_src_configure() {
	if declare -f python_configure >/dev/null; then
		python_foreach_impl python_configure
	else
		distutils-r1_python_configure
	fi

	if declare -f python_configure_all >/dev/null; then
		python_configure_all
	fi
}

distutils-r1_src_compile() {
	debug-print-function ${FUNCNAME} "${@}"

	if declare -f python_compile >/dev/null; then
		python_foreach_impl python_compile
	else
		python_foreach_impl distutils-r1_python_compile
	fi

	if declare -f python_compile_all >/dev/null; then
		python_compile_all
	fi
}

distutils-r1_src_test() {
	debug-print-function ${FUNCNAME} "${@}"

	if declare -f python_test >/dev/null; then
		python_foreach_impl python_test
	else
		distutils-r1_python_test
	fi

	if declare -f python_test_all >/dev/null; then
		python_test_all
	fi
}

distutils-r1_src_install() {
	debug-print-function ${FUNCNAME} "${@}"

	if declare -f python_install >/dev/null; then
		python_foreach_impl python_install
	else
		python_foreach_impl distutils-r1_python_install
	fi

	if declare -f python_install_all >/dev/null; then
		python_install_all
	else
		distutils-r1_python_install_all
	fi
}

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[Luca Barbato]

On 09/29/2012 12:45 PM, Michał Górny wrote:
> On Sat, 29 Sep 2012 12:00:18 +0200
> Tomáš Chvátal <tomas.chvatal@gmail.com> wrote:
> 
>> 2012/9/29 Michał Górny <mgorny@gentoo.org>:
>>> Hello,
>>>
>>> Instead of the floating patches and p-d-ng modifications I sent
>>> earlier, here are the two complete (so far, well, initial :P) eclasses
>>> for review.
>>>
>>> They are designed as 'mostly' drop-in python-distutils-ng replacement.
>>>
>> Hi,
>>
>> the eclasses look pretty, so good job,
>> just one question tho, would it be possible to use more
>> debug-something calls so the log file would be populated automatically
>> with whats going on (same like eg git-2 eclass)?
> 
> Try now :P.
> 
Looks even nicer, great!

lu

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[Markos Chandras]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On 09/29/2012 09:53 AM, Micha? Górny wrote:
> Hello,
> 
> Instead of the floating patches and p-d-ng modifications I sent 
> earlier, here are the two complete (so far, well, initial :P)
> eclasses for review.
> 
> They are designed as 'mostly' drop-in python-distutils-ng
> replacement.
> 
Hi,

Are you saying that you are going to remove the python-distutils-ng
eclass in favour of the new eclasses? I don't quite understand the
reasons to be honest.

- -- 
Regards,
Markos Chandras / Gentoo Linux Developer / Key ID: B4AFF2C2
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iQIcBAEBCgAGBQJQZstvAAoJEPqDWhW0r/LCC4gQAJmmseSriDS8ahCnkUNBm+61
NGxGijft0zE8qFvdLDH+koQ+ym8/KrMvZcp5U1pLEGDOBVXYj33nOQ9kA24rKBvz
Ro2Ydvxcj0Zo/nXGmLB0Mr9IaW4oWY8A16iY24qCpmPV1AWJXdoB0gdtXcDS4MmD
+LT19sRMqJstjUXbS02xrWS8lrbcPxVqVkfAPtFo82+/rtceHu+ymSVgJwkTQgBV
9999tKLdI16hK3x9pvceMgaYlC1W32yWq3YUOn3mvFjh4EjANrZ64ED1Q3A+QCYe
pUSwx3BGTZnQK6WQtSer/8O/oW7FoR5DooJTNsg1xrPTEdPd79D28TThy3ollACP
F5GtKyZO5gFK4SxQdbYfo9Su1Wa0XYybLHihnJDUVHuF2/AchSFW4CtGHfxQ4A0g
HWtEgdfRzk5kTUwR2LUhrisei3c6qpB0KGcZZCpi5FF9s150Si2wlKi2PbffAfil
0vLML3mnZnez5bWzPB1DqHf5eWdM00/BFaG4IQJkipwC3jJQ/rye9ruAv2UTM/Rs
UH6FQCfTRbScaf0E2WCGMIr33//HsIzy7KPAMAyfJWmxsBGwYHtouci7rQgUZIxn
+MdLQuzTYkUgybdnkhLmEeThl8coNwVeEzXwoQZGlciszz6cniSu66WbEEAUxgTP
5F9+vTb5XF1Jq+ZXrrb8
=7EZ5
-----END PGP SIGNATURE-----

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 942 bytes --]

On Sat, 29 Sep 2012 11:20:31 +0100
Markos Chandras <hwoarang@gentoo.org> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA512
> 
> On 09/29/2012 09:53 AM, Micha? Górny wrote:
> > Hello,
> > 
> > Instead of the floating patches and p-d-ng modifications I sent 
> > earlier, here are the two complete (so far, well, initial :P)
> > eclasses for review.
> > 
> > They are designed as 'mostly' drop-in python-distutils-ng
> > replacement.
> > 
> Hi,
> 
> Are you saying that you are going to remove the python-distutils-ng
> eclass in favour of the new eclasses? I don't quite understand the
> reasons to be honest.

The reason is simple -- I can't fix it without changing the API.
Changing the API on a live eclass is confusing, and considering that it
is not used by many packages, it's easier to lastrite it.

Also, this fixes the name not to have any '-ng' nor '-ds9'.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[hasufell]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 09/29/2012 12:49 PM, Michał Górny wrote:
> On Sat, 29 Sep 2012 11:20:31 +0100 Markos Chandras
> <hwoarang@gentoo.org> wrote:
> 
>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512
>> 
>> On 09/29/2012 09:53 AM, Micha? Górny wrote:
>>> Hello,
>>> 
>>> Instead of the floating patches and p-d-ng modifications I sent
>>>  earlier, here are the two complete (so far, well, initial :P) 
>>> eclasses for review.
>>> 
>>> They are designed as 'mostly' drop-in python-distutils-ng 
>>> replacement.
>>> 
>> Hi,
>> 
>> Are you saying that you are going to remove the
>> python-distutils-ng eclass in favour of the new eclasses? I don't
>> quite understand the reasons to be honest.
> 
> The reason is simple -- I can't fix it without changing the API. 
> Changing the API on a live eclass is confusing, and considering
> that it is not used by many packages, it's easier to lastrite it.
> 
> Also, this fixes the name not to have any '-ng' nor '-ds9'.
> 

What are the reasons to change the API in the first place? There has
to be a good reason, cause this will involve yet another migration of
many ebuilds. I don't see any bugreports.

I fear this will cause more confusion, i.e. some ebuilds using the old
distutils, some using python-distutils-ng and some using distutils-r1
resulting in weird tree behavior.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJQZvxsAAoJEFpvPKfnPDWzL1IH/iaChrfPET4KArZzaViXJjlL
4pM2tmgByJNAgtFjwLwz3c6lqdJGAV8Uf4VpPTec+Z7lj0v0SDj04YI63CgFZ2N/
R7edGAoJdGOFDv/oOY+bP38PeWpnuguOvEaKI8EEqaJgLne29wPEQ7+KcWe8M6hM
tA41lIIFpK2PN+kIHdItbSl8aRZmm9NorfUCukFfs1odwV+C0B3rP4mJZQW+TnbR
DmDqFCFQF/r1k+n17wARqvcCL+hBqs/CvTG7K8Z/mNWD/Dove1vMB1ir8p8KnYSO
TULgpcVEK4VuXjHrccLhmO4ODWZiXn/yWKws3z+XmRwIwBB7m9IhC8G32aeyqoE=
=gU6H
-----END PGP SIGNATURE-----

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 29/09/12 09:49 AM, hasufell wrote:
> On 09/29/2012 12:49 PM, Michał Górny wrote:
>> On Sat, 29 Sep 2012 11:20:31 +0100 Markos Chandras 
>> <hwoarang@gentoo.org> wrote:
> 
>>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512
>>> 
>>> On 09/29/2012 09:53 AM, Micha? Górny wrote:
>>>> Hello,
>>>> 
>>>> Instead of the floating patches and p-d-ng modifications I
>>>> sent earlier, here are the two complete (so far, well,
>>>> initial :P) eclasses for review.
>>>> 
>>>> They are designed as 'mostly' drop-in python-distutils-ng 
>>>> replacement.
>>>> 
>>> Hi,
>>> 
>>> Are you saying that you are going to remove the 
>>> python-distutils-ng eclass in favour of the new eclasses? I
>>> don't quite understand the reasons to be honest.
> 
>> The reason is simple -- I can't fix it without changing the API.
>>  Changing the API on a live eclass is confusing, and considering 
>> that it is not used by many packages, it's easier to lastrite
>> it.
> 
>> Also, this fixes the name not to have any '-ng' nor '-ds9'.
> 
> 
> What are the reasons to change the API in the first place? There
> has to be a good reason, cause this will involve yet another
> migration of many ebuilds. I don't see any bugreports.
> 
> I fear this will cause more confusion, i.e. some ebuilds using the
> old distutils, some using python-distutils-ng and some using
> distutils-r1 resulting in weird tree behavior.
> 

Given that at present, distutils-r1 and python-distutils-ng have
identical end-results, I think that the introduction of distutils-r1
to the tree should also involve a sed against all the existing ebuilds
using python-distutils-ng to move them to the new eclass.  Then
python-distutils-ng only needs to remain to support overlays.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iF4EAREIAAYFAlBnA1YACgkQ2ugaI38ACPBtCgD/UXW804+tsTOnI0RtfWfhiewK
a0W9DXplPRprWYZg4mQBAIWbRf+AJDrIqGvELiwt3p0FXChbCYypHDmm3tb8ljxL
=isBB
-----END PGP SIGNATURE-----

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[hasufell]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 09/29/2012 04:19 PM, Ian Stakenvicius wrote:
> On 29/09/12 09:49 AM, hasufell wrote:
>> On 09/29/2012 12:49 PM, Michał Górny wrote:
>>> On Sat, 29 Sep 2012 11:20:31 +0100 Markos Chandras 
>>> <hwoarang@gentoo.org> wrote:
> 
>>>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512
>>>> 
>>>> On 09/29/2012 09:53 AM, Micha? Górny wrote:
>>>>> Hello,
>>>>> 
>>>>> Instead of the floating patches and p-d-ng modifications I 
>>>>> sent earlier, here are the two complete (so far, well, 
>>>>> initial :P) eclasses for review.
>>>>> 
>>>>> They are designed as 'mostly' drop-in python-distutils-ng 
>>>>> replacement.
>>>>> 
>>>> Hi,
>>>> 
>>>> Are you saying that you are going to remove the 
>>>> python-distutils-ng eclass in favour of the new eclasses? I 
>>>> don't quite understand the reasons to be honest.
> 
>>> The reason is simple -- I can't fix it without changing the
>>> API. Changing the API on a live eclass is confusing, and
>>> considering that it is not used by many packages, it's easier
>>> to lastrite it.
> 
>>> Also, this fixes the name not to have any '-ng' nor '-ds9'.
> 
> 
>> What are the reasons to change the API in the first place? There 
>> has to be a good reason, cause this will involve yet another 
>> migration of many ebuilds. I don't see any bugreports.
> 
>> I fear this will cause more confusion, i.e. some ebuilds using
>> the old distutils, some using python-distutils-ng and some using 
>> distutils-r1 resulting in weird tree behavior.
> 
> 
> Given that at present, distutils-r1 and python-distutils-ng have 
> identical end-results, I think that the introduction of
> distutils-r1 to the tree should also involve a sed against all the
> existing ebuilds using python-distutils-ng to move them to the new
> eclass.  Then python-distutils-ng only needs to remain to support
> overlays.
> 
> 

That still does not explain the reasons why this work was initiated.

If there is any way to fix the current eclass, that should be preferred.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJQZwUtAAoJEFpvPKfnPDWz+g4IAIL0eFfX6rMHKBxtNkCGt7yo
dnPMiAjlbRwDVkpCBnorATwLpnHhsRfsHtHXkQrNXWzgGvSgOETpvGzmFgvPzr4L
lvOs3ND8BFZz3OiQuw3K2GrwInbQCXg1oFlKdBuLOom7WxtePVXeJsK3Ck4coGcH
NIfYlQNLaISp0CvUhGg3yF6/PjSCZ9vwfIN7muY1OVspE0DWXCRIZoOs623RixTS
cwzFRIdlxeJgw+JEuLN8wSsXe+Ir3bmmFOiRF+FD6LzjoYdh0xRyGX6Qgg974F7f
yb2aOT2MCMANWrMgdYiNuRZGJNvUagZ78PRIGHWNw+PzDaNC3jXqrTBsGpkk2Fc=
=azK1
-----END PGP SIGNATURE-----

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[Dirkjan Ochtman]

On Sat, Sep 29, 2012 at 4:26 PM, hasufell <hasufell@gentoo.org> wrote:
> That still does not explain the reasons why this work was initiated.
>
> If there is any way to fix the current eclass, that should be preferred.

I tend to agree. Michał, let me first say I value the time you have
invested to make the eclasses better. However, at this point I have a
strong feeling that we have more people willing to write code to fix
things than we have people building consensus on what
features/policies/mechanisms we need to make it easy to write
high-quality ebuilds for Python/distutils. I would prefer discussions
on problems that the current ebuilds have and discussions on how to
solve them, not at the code level, but that the mechanism level.

Cheers,

Dirkjan

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1305 bytes --]

On Sat, 29 Sep 2012 16:37:15 +0200
Dirkjan Ochtman <djc@gentoo.org> wrote:

> On Sat, Sep 29, 2012 at 4:26 PM, hasufell <hasufell@gentoo.org> wrote:
> > That still does not explain the reasons why this work was initiated.
> >
> > If there is any way to fix the current eclass, that should be preferred.
> 
> I tend to agree. Michał, let me first say I value the time you have
> invested to make the eclasses better. However, at this point I have a
> strong feeling that we have more people willing to write code to fix
> things than we have people building consensus on what
> features/policies/mechanisms we need to make it easy to write
> high-quality ebuilds for Python/distutils. I would prefer discussions
> on problems that the current ebuilds have and discussions on how to
> solve them, not at the code level, but that the mechanism level.

The main issue: noone wants to even touch python.eclass or anything
nearby.

The second issue: python-distutils-ng isn't good enough. It has too
many things hard-wired. I think I have already pointed enough problems
with it. Not that many people cared to respond.

It's sad that people don't care to respond when you point the issues
out but then complain when you do something to fix them.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[hasufell]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 09/29/2012 08:39 PM, Michał Górny wrote:
> On Sat, 29 Sep 2012 16:37:15 +0200 Dirkjan Ochtman <djc@gentoo.org>
> wrote:
> 
>> On Sat, Sep 29, 2012 at 4:26 PM, hasufell <hasufell@gentoo.org>
>> wrote:
>>> That still does not explain the reasons why this work was
>>> initiated.
>>> 
>>> If there is any way to fix the current eclass, that should be
>>> preferred.
>> 
>> I tend to agree. Michał, let me first say I value the time you
>> have invested to make the eclasses better. However, at this point
>> I have a strong feeling that we have more people willing to write
>> code to fix things than we have people building consensus on
>> what features/policies/mechanisms we need to make it easy to
>> write high-quality ebuilds for Python/distutils. I would prefer
>> discussions on problems that the current ebuilds have and
>> discussions on how to solve them, not at the code level, but that
>> the mechanism level.
> 
> The main issue: noone wants to even touch python.eclass or
> anything nearby.
> 
> The second issue: python-distutils-ng isn't good enough. It has
> too many things hard-wired. I think I have already pointed enough
> problems with it. Not that many people cared to respond.
> 
> It's sad that people don't care to respond when you point the
> issues out but then complain when you do something to fix them.
> 

Did you CC gentoo-dev? I cannot find the tread.

> [example needed]
> 

??

I meant that not all tree ebuilds use the same python-eclass
implementation which IS a problem. Adding another implementation does
not really improve that situation.

> Please list the features. Preferably, order them by usefulness,
> with exact use cases.
> 

As I said, I think the python herd already did a list on this.

Btw. could you give exact examples on how to convert widely used
python ebuilds with your eclasses?
E.g. dev-python/pygobject dev-python/setuptools or dev-libs/boost?
How can I convert shebangs consistently and recursively?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJQZ16AAAoJEFpvPKfnPDWzzMgH/0dGzePj8eSrZ3OZDzwMYE3B
Vx8BiD2Vd+OnGyq2w0infJpN8lDlGu+8yxGwWervZJ7tIxqabhQokI03tBSyLRgt
em5R+AgSiR6GSiIRfMNoFYj+5zR8vz4gHtCzrI47O8W6R6e3fRj3JKShY7+T4Djz
vBMyD4ZuxLg0CnvJ05rrc41CEvmAY/aWysS5WNoevdx8Jf8exaVtfp6TXGu/q+JV
7py4gFA5wXmb7UCv9Tcw0IxiglVAfEJRzvRh68TComBKWuUw0YhGd/x2VxaLZ0dr
ghCt4XBjyi5g4q1rcDedEPowth1Q/9M6VpP6fT5ZTPVIs5r49G9vMcRymppWetM=
=5M+2
-----END PGP SIGNATURE-----

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2912 bytes --]

On Sat, 29 Sep 2012 22:48:00 +0200
hasufell <hasufell@gentoo.org> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On 09/29/2012 08:39 PM, Michał Górny wrote:
> > On Sat, 29 Sep 2012 16:37:15 +0200 Dirkjan Ochtman <djc@gentoo.org>
> > wrote:
> > 
> >> On Sat, Sep 29, 2012 at 4:26 PM, hasufell <hasufell@gentoo.org>
> >> wrote:
> >>> That still does not explain the reasons why this work was
> >>> initiated.
> >>> 
> >>> If there is any way to fix the current eclass, that should be
> >>> preferred.
> >> 
> >> I tend to agree. Michał, let me first say I value the time you
> >> have invested to make the eclasses better. However, at this point
> >> I have a strong feeling that we have more people willing to write
> >> code to fix things than we have people building consensus on
> >> what features/policies/mechanisms we need to make it easy to
> >> write high-quality ebuilds for Python/distutils. I would prefer
> >> discussions on problems that the current ebuilds have and
> >> discussions on how to solve them, not at the code level, but that
> >> the mechanism level.
> > 
> > The main issue: noone wants to even touch python.eclass or
> > anything nearby.
> > 
> > The second issue: python-distutils-ng isn't good enough. It has
> > too many things hard-wired. I think I have already pointed enough
> > problems with it. Not that many people cared to respond.
> > 
> > It's sad that people don't care to respond when you point the
> > issues out but then complain when you do something to fix them.
> > 
> 
> Did you CC gentoo-dev? I cannot find the tread.

Maybe. People interested in Python should be either on the Python ml
or on the python@ alias, I believe.

> > [example needed]
> > 
> 
> ??
> 
> I meant that not all tree ebuilds use the same python-eclass
> implementation which IS a problem. Adding another implementation does
> not really improve that situation.

As long as they can inter-operate correctly, I don't see any problem.

> > Please list the features. Preferably, order them by usefulness,
> > with exact use cases.
> > 
> 
> As I said, I think the python herd already did a list on this.
> 
> Btw. could you give exact examples on how to convert widely used
> python ebuilds with your eclasses?
> E.g. dev-python/pygobject dev-python/setuptools or dev-libs/boost?

I have prepared and tested an ebuild for pygobject and setuptools here:

http://git.overlays.gentoo.org/gitweb/?p=dev/mgorny.git;a=tree;f=dev-python;h=557b96c041bb6da969574dd75dcdfd4022ba636b;hb=refs/heads/python-r1

I will take a look at boost a while later, since I have to have much
more time to compile it :P.

> How can I convert shebangs consistently and recursively?

Converting shebangs applies to packages supporting multiple Python
implementations or those not doing so?

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 29/09/12 10:26 AM, hasufell wrote:
> On 09/29/2012 04:19 PM, Ian Stakenvicius wrote:
>> On 29/09/12 09:49 AM, hasufell wrote:
>>> On 09/29/2012 12:49 PM, Michał Górny wrote:
>>>> On Sat, 29 Sep 2012 11:20:31 +0100 Markos Chandras 
>>>> <hwoarang@gentoo.org> wrote:
> 
>>>>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512
>>>>> 
>>>>> On 09/29/2012 09:53 AM, Micha? Górny wrote:
>>>>>> Hello,
>>>>>> 
>>>>>> Instead of the floating patches and p-d-ng modifications
>>>>>> I sent earlier, here are the two complete (so far, well,
>>>>>>  initial :P) eclasses for review.
>>>>>> 
>>>>>> They are designed as 'mostly' drop-in python-distutils-ng
>>>>>>  replacement.
>>>>>> 
>>>>> Hi,
>>>>> 
>>>>> Are you saying that you are going to remove the 
>>>>> python-distutils-ng eclass in favour of the new eclasses? I
>>>>>  don't quite understand the reasons to be honest.
> 
>>>> The reason is simple -- I can't fix it without changing the 
>>>> API. Changing the API on a live eclass is confusing, and 
>>>> considering that it is not used by many packages, it's
>>>> easier to lastrite it.
> 
>>>> Also, this fixes the name not to have any '-ng' nor '-ds9'.
> 
> 
>>> What are the reasons to change the API in the first place?
>>> There has to be a good reason, cause this will involve yet
>>> another migration of many ebuilds. I don't see any bugreports.
> 
>>> I fear this will cause more confusion, i.e. some ebuilds using 
>>> the old distutils, some using python-distutils-ng and some
>>> using distutils-r1 resulting in weird tree behavior.
> 
> 
>> Given that at present, distutils-r1 and python-distutils-ng have
>>  identical end-results, I think that the introduction of 
>> distutils-r1 to the tree should also involve a sed against all
>> the existing ebuilds using python-distutils-ng to move them to
>> the new eclass.  Then python-distutils-ng only needs to remain to
>> support overlays.
> 
> 
> 
> That still does not explain the reasons why this work was
> initiated.
> 
> If there is any way to fix the current eclass, that should be
> preferred.
> 

There isn't so much a problem with the current python-distutils-ng
eclass but rather it's to expand it to a more comprehensive
replacement for both distutils and python eclasses.  In order to do
that efficiently, most of the core functionality should be moved so
that the new distutils is more like a wrapper to the new python.

This could certainly be done by patching the existing eclass, but
mgorny wants to use new eclass names instead of keeping the current
one.  Hence the rename.  I think that's about it..
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iF4EAREIAAYFAlBnFbcACgkQ2ugaI38ACPAirwD/SqHvaJfc73pYzxSoow0ORPJY
mSe1aS9kNk7SGT4ey1EA/jLPc1+of8Rwh3BFxeGfk0H1Go4mr/AbqhLDPnkxO2Sn
=QUTg
-----END PGP SIGNATURE-----

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[hasufell]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 09/29/2012 05:37 PM, Ian Stakenvicius wrote:
> 
> There isn't so much a problem with the current python-distutils-ng 
> eclass but rather it's to expand it to a more comprehensive 
> replacement for both distutils and python eclasses.  In order to
> do that efficiently, most of the core functionality should be moved
> so that the new distutils is more like a wrapper to the new
> python.
> 
> This could certainly be done by patching the existing eclass, but 
> mgorny wants to use new eclass names instead of keeping the
> current one.  Hence the rename.  I think that's about it..
> 

In that case we are missing 95% of the features of python.eclass.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJQZxeDAAoJEFpvPKfnPDWzmRAH+wThUWb1jzE+jFUbTeuByKca
a4wVAFWy8ruGIQI82+/9bY5zZqitiqU1MijAixbdgwLwGeFurD6UBx+7vAUJ01YR
G5ALZOqxK7js0TJ3pv9wXiNihhoGjXGby+8MtbUogJ5mqB9r9vYaZaOUMRpaOpkg
VOgpVXX2YGixuder8JRo2snVQkd+hpMoZ3EI4w0EaSofhNGEV8f1BP27OUNgts1K
iH3EuVU3CF5STqGK4Fo7wwNwhsbzkQbMBVe/V9zBvJQEZyUVU9EuQ0+bvnedzAMB
PPgEXmNdxxbALxIR3xSpi7o/dyl21feJK968C9ObPpwMMloaNfQewQYB0MNPrY4=
=CEMA
-----END PGP SIGNATURE-----

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 29/09/12 11:45 AM, hasufell wrote:
> On 09/29/2012 05:37 PM, Ian Stakenvicius wrote:
> 
>> There isn't so much a problem with the current
>> python-distutils-ng eclass but rather it's to expand it to a more
>> comprehensive replacement for both distutils and python eclasses.
>> In order to do that efficiently, most of the core functionality
>> should be moved so that the new distutils is more like a wrapper
>> to the new python.
> 
>> This could certainly be done by patching the existing eclass, but
>>  mgorny wants to use new eclass names instead of keeping the 
>> current one.  Hence the rename.  I think that's about it..
> 
> 
> In that case we are missing 95% of the features of python.eclass.
> 

Aha, but do we really -need- 95% of the features?  :)   Personally
(this is my investment in this project) I'd just like to see
non-distutils ebuilds needing python support using PYTHON_TARGETS.
Also, I'd like to see that anything at all which new-distutils
(whatever it be called) might need the old python.eclass for be
integrated into new-distutils (probably via new-python).
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iF4EAREIAAYFAlBnGKkACgkQ2ugaI38ACPCUGQD/f2tf7bjyxkq52In7OrH+nDSL
nWLUWc9btwRm3Uuyd3AA/3tFI8uOiqpAVS7Ze4ScCt1UHi7LdvXgYGZRxPbJUbvS
=4o7s
-----END PGP SIGNATURE-----

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[hasufell]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 09/29/2012 05:50 PM, Ian Stakenvicius wrote:
> On 29/09/12 11:45 AM, hasufell wrote:
>> On 09/29/2012 05:37 PM, Ian Stakenvicius wrote:
> 
>>> There isn't so much a problem with the current 
>>> python-distutils-ng eclass but rather it's to expand it to a
>>> more comprehensive replacement for both distutils and python
>>> eclasses. In order to do that efficiently, most of the core
>>> functionality should be moved so that the new distutils is more
>>> like a wrapper to the new python.
> 
>>> This could certainly be done by patching the existing eclass,
>>> but mgorny wants to use new eclass names instead of keeping the
>>>  current one.  Hence the rename.  I think that's about it..
> 
> 
>> In that case we are missing 95% of the features of
>> python.eclass.
> 
> 
> Aha, but do we really -need- 95% of the features?

Yeah, maybe not.

But that should be discussed beforehand imo. So that we really design
this eclass together. Otherwise I fear we will end up getting a 4th
implementation...
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJQZxmzAAoJEFpvPKfnPDWzi2AIAKQjjlbuf8K3TUFGmXPldTi4
sQJhwZjo4sngQF1zql4K2RJHnx2R6qsr/YteZ/4ek2yTg356oqkMjAyk5BV5Dpv8
shJugkgvAd3iZWVOUqQEMjxl+Rd3wRmgAw5oC+EEXrck6vOOgQbla/RdLwYFstkP
5Pmp+0hnksRcAnGhOiw7W0JLBJzuhPjoeUGdXVVVwuaPIzlgis0Jv8fSMeP4jxpT
vM5EOuvrwpM+gJzkpH0ABCdVHMyigufXCKED11JOrRTUA1fgT2XoWBMCblBoMXtH
e6WzUT5NojLlHn4E3psqy3kA4bqzRdMm6u3Iw4bPLFnl0vUaCGYqVfvL1OYkTU4=
=gbNx
-----END PGP SIGNATURE-----

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[Ciaran McCreesh]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sat, 29 Sep 2012 17:45:07 +0200
hasufell <hasufell@gentoo.org> wrote:
> In that case we are missing 95% of the features of python.eclass.

You say that like it's a bad thing...

Seriously, most of the problem with python.eclass (and several other
problematic eclasses) is that it tries to do far too much all in one
place. More smaller eclasses is a good thing.

- -- 
Ciaran McCreesh
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iEYEARECAAYFAlBnGacACgkQ96zL6DUtXhFFCgCfXr4BzUaN7L/WaAtYV//JOkjW
ES4AoNQU0/PwOBdzBTgspOt45V/2FDxG
=fvDp
-----END PGP SIGNATURE-----

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 964 bytes --]

On Sat, 29 Sep 2012 17:45:07 +0200
hasufell <hasufell@gentoo.org> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On 09/29/2012 05:37 PM, Ian Stakenvicius wrote:
> > 
> > There isn't so much a problem with the current python-distutils-ng 
> > eclass but rather it's to expand it to a more comprehensive 
> > replacement for both distutils and python eclasses.  In order to
> > do that efficiently, most of the core functionality should be moved
> > so that the new distutils is more like a wrapper to the new
> > python.
> > 
> > This could certainly be done by patching the existing eclass, but 
> > mgorny wants to use new eclass names instead of keeping the
> > current one.  Hence the rename.  I think that's about it..
> > 
> 
> In that case we are missing 95% of the features of python.eclass.

Please list the features. Preferably, order them by usefulness, with
exact use cases.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[Pacho Ramos]

[-- Attachment #1: Type: text/plain, Size: 1362 bytes --]

El sáb, 29-09-2012 a las 20:40 +0200, Michał Górny escribió:
> On Sat, 29 Sep 2012 17:45:07 +0200
> hasufell <hasufell@gentoo.org> wrote:
> 
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> > 
> > On 09/29/2012 05:37 PM, Ian Stakenvicius wrote:
> > > 
> > > There isn't so much a problem with the current python-distutils-ng 
> > > eclass but rather it's to expand it to a more comprehensive 
> > > replacement for both distutils and python eclasses.  In order to
> > > do that efficiently, most of the core functionality should be moved
> > > so that the new distutils is more like a wrapper to the new
> > > python.
> > > 
> > > This could certainly be done by patching the existing eclass, but 
> > > mgorny wants to use new eclass names instead of keeping the
> > > current one.  Hence the rename.  I think that's about it..
> > > 
> > 
> > In that case we are missing 95% of the features of python.eclass.
> 
> Please list the features. Preferably, order them by usefulness, with
> exact use cases.
> 

Personally, I usually run:
- python_clean_py-compile_files -> Clean py-compile files to disable
byte-compilation allowing us to drop all various ways of doing this that
were living in the tree some time ago.
- python_convert_shebangs -> but, I guess this is handled in a different
way in your eclass, no? :/


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2085 bytes --]

On Sat, 29 Sep 2012 21:20:00 +0200
Pacho Ramos <pacho@gentoo.org> wrote:

> El sáb, 29-09-2012 a las 20:40 +0200, Michał Górny escribió:
> > On Sat, 29 Sep 2012 17:45:07 +0200
> > hasufell <hasufell@gentoo.org> wrote:
> > 
> > > -----BEGIN PGP SIGNED MESSAGE-----
> > > Hash: SHA1
> > > 
> > > On 09/29/2012 05:37 PM, Ian Stakenvicius wrote:
> > > > 
> > > > There isn't so much a problem with the current python-distutils-ng 
> > > > eclass but rather it's to expand it to a more comprehensive 
> > > > replacement for both distutils and python eclasses.  In order to
> > > > do that efficiently, most of the core functionality should be moved
> > > > so that the new distutils is more like a wrapper to the new
> > > > python.
> > > > 
> > > > This could certainly be done by patching the existing eclass, but 
> > > > mgorny wants to use new eclass names instead of keeping the
> > > > current one.  Hence the rename.  I think that's about it..
> > > > 
> > > 
> > > In that case we are missing 95% of the features of python.eclass.
> > 
> > Please list the features. Preferably, order them by usefulness, with
> > exact use cases.
> > 
> 
> Personally, I usually run:
> - python_clean_py-compile_files -> Clean py-compile files to disable
> byte-compilation allowing us to drop all various ways of doing this that
> were living in the tree some time ago.

Hmm, what's the problem with compiling them? Do you mean some case when
the results of the compilation are different from the way done
by the eclass?

> - python_convert_shebangs -> but, I guess this is handled in a different
> way in your eclass, no? :/

Depends on what you need. To be honest, I haven't added any code for
custom script handling yet, just the usual distutils case.

A package which does not explicitly support multiple Python
implementations is a completely different things, needing more
discussion first and which actually may be handled through a separate
eclass if most code of python-r1 proves useless for it.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[Pacho Ramos]

[-- Attachment #1: Type: text/plain, Size: 2529 bytes --]

El sáb, 29-09-2012 a las 22:34 +0200, Michał Górny escribió:
> On Sat, 29 Sep 2012 21:20:00 +0200
> Pacho Ramos <pacho@gentoo.org> wrote:
> 
> > El sáb, 29-09-2012 a las 20:40 +0200, Michał Górny escribió:
> > > On Sat, 29 Sep 2012 17:45:07 +0200
> > > hasufell <hasufell@gentoo.org> wrote:
> > > 
> > > > -----BEGIN PGP SIGNED MESSAGE-----
> > > > Hash: SHA1
> > > > 
> > > > On 09/29/2012 05:37 PM, Ian Stakenvicius wrote:
> > > > > 
> > > > > There isn't so much a problem with the current python-distutils-ng 
> > > > > eclass but rather it's to expand it to a more comprehensive 
> > > > > replacement for both distutils and python eclasses.  In order to
> > > > > do that efficiently, most of the core functionality should be moved
> > > > > so that the new distutils is more like a wrapper to the new
> > > > > python.
> > > > > 
> > > > > This could certainly be done by patching the existing eclass, but 
> > > > > mgorny wants to use new eclass names instead of keeping the
> > > > > current one.  Hence the rename.  I think that's about it..
> > > > > 
> > > > 
> > > > In that case we are missing 95% of the features of python.eclass.
> > > 
> > > Please list the features. Preferably, order them by usefulness, with
> > > exact use cases.
> > > 
> > 
> > Personally, I usually run:
> > - python_clean_py-compile_files -> Clean py-compile files to disable
> > byte-compilation allowing us to drop all various ways of doing this that
> > were living in the tree some time ago.
> 
> Hmm, what's the problem with compiling them? Do you mean some case when
> the results of the compilation are different from the way done
> by the eclass?
> 

Well, if I don't misremember, we currently prefer to compile them at
postinst phase instead of during src_compile, but maybe this is no
longer needed (no idea :( )

> > - python_convert_shebangs -> but, I guess this is handled in a different
> > way in your eclass, no? :/
> 
> Depends on what you need. To be honest, I haven't added any code for
> custom script handling yet, just the usual distutils case.
> 
> A package which does not explicitly support multiple Python
> implementations is a completely different things, needing more
> discussion first and which actually may be handled through a separate
> eclass if most code of python-r1 proves useless for it.
> 

I was thinking on a lot of packages still being only compatible with
python2... I guess will need to still use python.eclass for them then

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[Fabian Groffen]

[-- Attachment #1: Type: text/plain, Size: 1157 bytes --]

On 30-09-2012 10:31:17 +0200, Pacho Ramos wrote:
> > > Personally, I usually run:
> > > - python_clean_py-compile_files -> Clean py-compile files to disable
> > > byte-compilation allowing us to drop all various ways of doing this that
> > > were living in the tree some time ago.
> > 
> > Hmm, what's the problem with compiling them? Do you mean some case when
> > the results of the compilation are different from the way done
> > by the eclass?
> > 
> 
> Well, if I don't misremember, we currently prefer to compile them at
> postinst phase instead of during src_compile, but maybe this is no
> longer needed (no idea :( )

The files are indeed cache, and should be generated on the system that
installs the files, not the system that builds them.  They are currently
outside of VDB.  pyc files store the path to the original files, so
generating in ${ROOT} yields in wrong paths.  Python sometimes
regenerates the files when it runs.  It may as well just ignore them
(since they are newer but non-matching, unclear to me).  In the worst
case it returns "Bad marshalling data".


-- 
Fabian Groffen
Gentoo on a different level

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[Pacho Ramos]

[-- Attachment #1: Type: text/plain, Size: 1304 bytes --]

El dom, 30-09-2012 a las 10:58 +0200, Fabian Groffen escribió:
> On 30-09-2012 10:31:17 +0200, Pacho Ramos wrote:
> > > > Personally, I usually run:
> > > > - python_clean_py-compile_files -> Clean py-compile files to disable
> > > > byte-compilation allowing us to drop all various ways of doing this that
> > > > were living in the tree some time ago.
> > > 
> > > Hmm, what's the problem with compiling them? Do you mean some case when
> > > the results of the compilation are different from the way done
> > > by the eclass?
> > > 
> > 
> > Well, if I don't misremember, we currently prefer to compile them at
> > postinst phase instead of during src_compile, but maybe this is no
> > longer needed (no idea :( )
> 
> The files are indeed cache, and should be generated on the system that
> installs the files, not the system that builds them.  They are currently
> outside of VDB.  pyc files store the path to the original files, so
> generating in ${ROOT} yields in wrong paths.  Python sometimes
> regenerates the files when it runs.  It may as well just ignore them
> (since they are newer but non-matching, unclear to me).  In the worst
> case it returns "Bad marshalling data".
> 
> 

Then, I guess having such function would still be useful :) Thanks for
the info

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1362 bytes --]

On Sun, 30 Sep 2012 10:58:06 +0200
Fabian Groffen <grobian@gentoo.org> wrote:

> On 30-09-2012 10:31:17 +0200, Pacho Ramos wrote:
> > > > Personally, I usually run:
> > > > - python_clean_py-compile_files -> Clean py-compile files to disable
> > > > byte-compilation allowing us to drop all various ways of doing this that
> > > > were living in the tree some time ago.
> > > 
> > > Hmm, what's the problem with compiling them? Do you mean some case when
> > > the results of the compilation are different from the way done
> > > by the eclass?
> > > 
> > 
> > Well, if I don't misremember, we currently prefer to compile them at
> > postinst phase instead of during src_compile, but maybe this is no
> > longer needed (no idea :( )
> 
> The files are indeed cache, and should be generated on the system that
> installs the files, not the system that builds them.  They are currently
> outside of VDB.  pyc files store the path to the original files, so
> generating in ${ROOT} yields in wrong paths.  Python sometimes
> regenerates the files when it runs.  It may as well just ignore them
> (since they are newer but non-matching, unclear to me).  In the worst
> case it returns "Bad marshalling data".

Hmm, so python-distutils-ng was wrong from the beginning and nobody
cared to point it out?

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[Fabian Groffen]

[-- Attachment #1: Type: text/plain, Size: 843 bytes --]

On 30-09-2012 15:28:47 +0200, Michał Górny wrote:
> > The files are indeed cache, and should be generated on the system that
> > installs the files, not the system that builds them.  They are currently
> > outside of VDB.  pyc files store the path to the original files, so
> > generating in ${ROOT} yields in wrong paths.  Python sometimes
> > regenerates the files when it runs.  It may as well just ignore them
> > (since they are newer but non-matching, unclear to me).  In the worst
> > case it returns "Bad marshalling data".
> 
> Hmm, so python-distutils-ng was wrong from the beginning and nobody
> cared to point it out?

I'm not reviewing each and everything in detail, sorry.  It's just
that Pacho raised the point in this thread that made me recall the
above.


-- 
Fabian Groffen
Gentoo on a different level

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1462 bytes --]

On Sun, 30 Sep 2012 10:58:06 +0200
Fabian Groffen <grobian@gentoo.org> wrote:

> On 30-09-2012 10:31:17 +0200, Pacho Ramos wrote:
> > > > Personally, I usually run:
> > > > - python_clean_py-compile_files -> Clean py-compile files to disable
> > > > byte-compilation allowing us to drop all various ways of doing this that
> > > > were living in the tree some time ago.
> > > 
> > > Hmm, what's the problem with compiling them? Do you mean some case when
> > > the results of the compilation are different from the way done
> > > by the eclass?
> > > 
> > 
> > Well, if I don't misremember, we currently prefer to compile them at
> > postinst phase instead of during src_compile, but maybe this is no
> > longer needed (no idea :( )
> 
> The files are indeed cache, and should be generated on the system that
> installs the files, not the system that builds them.  They are currently
> outside of VDB.  pyc files store the path to the original files, so
> generating in ${ROOT} yields in wrong paths.  Python sometimes
> regenerates the files when it runs.  It may as well just ignore them
> (since they are newer but non-matching, unclear to me).  In the worst
> case it returns "Bad marshalling data".

I'm afraid you are wrong here.

I've just tested an ebuild using distutils (flaggie) and autotools
(pygobject) and both install .pyc files with correct paths (i.e. with
DESTDIR stripped).

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[Fabian Groffen]

[-- Attachment #1: Type: text/plain, Size: 2247 bytes --]

I'm on the list, obviously, so PLEASE stop Cc-ing me!

On 30-09-2012 15:57:16 +0200, Michał Górny wrote:
> > The files are indeed cache, and should be generated on the system that
> > installs the files, not the system that builds them.  They are currently
> > outside of VDB.  pyc files store the path to the original files, so
> > generating in ${ROOT} yields in wrong paths.  Python sometimes
> > regenerates the files when it runs.  It may as well just ignore them
> > (since they are newer but non-matching, unclear to me).  In the worst
> > case it returns "Bad marshalling data".
> 
> I'm afraid you are wrong here.
> 
> I've just tested an ebuild using distutils (flaggie) and autotools
> (pygobject) and both install .pyc files with correct paths (i.e. with
> DESTDIR stripped).

Could be, if it uses relative paths, but they won't be absolute then
either.  From some experiments I did, I saw Python ignoring wrong paths
(entire cache?), or updating the pyc/pyo files.  Regardless whether or
not the data is wrong, the discussion is more if it hurts (and makes
sense).

Back then, we found it hurt us (Bad marshalling data) when using
binpkgs.  Not sure if it still does today with Python 2.7.  Somehow we
reached a consensus with the Python maintainer at that time that cache
stuff shouldn't be in VDB, also because it depends on system and perhaps
Python version.  Since embedded systems might not even want it (it's
just cache afterall, wasting their sometimes precious disk space), it
had to be out of the package contents for them as well.

That's how we came to the situation as it was before python eclass
rewrites.  I'd much prefer Python to write its cache to some
/var/cache/python dir so it can create whatever it wants, but on a
place which is not of any concern, as its obvious one can throw it away,
and doesn't pollute the live fs.  But Python doesn't work that way.

Whatever the new way will be, for whatever reason, please make sure it
is consistent.  And be aware that this doesn't just mean changing
eclasses, since this rule is effectuated in certain ebuilds too
(dev-lang/python itself being the most obvious example).


-- 
Fabian Groffen
Gentoo on a different level

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[Zac Medico]

On 09/30/2012 07:12 AM, Fabian Groffen wrote:
> Back then, we found it hurt us (Bad marshalling data) when using
> binpkgs.  Not sure if it still does today with Python 2.7.  Somehow we
> reached a consensus with the Python maintainer at that time that cache
> stuff shouldn't be in VDB, also because it depends on system and perhaps
> Python version.  Since embedded systems might not even want it (it's
> just cache afterall, wasting their sometimes precious disk space), it
> had to be out of the package contents for them as well.

They can use INSTALL_MASK="*.py[co]" if they need to. In portage we have
a default COLLISION_IGNORE="*.py[co]" setting, so that there are no
related issues with file collisions for *.py[co] if the user removes the
INSTALL_MASK setting later.
-- 
Thanks,
Zac

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[Brian Harring]

On Sun, Sep 30, 2012 at 10:58:06AM +0200, Fabian Groffen wrote:
> On 30-09-2012 10:31:17 +0200, Pacho Ramos wrote:
> > > > Personally, I usually run:
> > > > - python_clean_py-compile_files -> Clean py-compile files to disable
> > > > byte-compilation allowing us to drop all various ways of doing this that
> > > > were living in the tree some time ago.
> > > 
> > > Hmm, what's the problem with compiling them? Do you mean some case when
> > > the results of the compilation are different from the way done
> > > by the eclass?
> > > 
> > 
> > Well, if I don't misremember, we currently prefer to compile them at
> > postinst phase instead of during src_compile, but maybe this is no
> > longer needed (no idea :( )
> 
> The files are indeed cache, and should be generated on the system that
> installs the files, not the system that builds them.  They are currently
> outside of VDB.  pyc files store the path to the original files, so
> generating in ${ROOT} yields in wrong paths.  Python sometimes
> regenerates the files when it runs.  It may as well just ignore them
> (since they are newer but non-matching, unclear to me).

.pyc should be compatible within slotted python versions; 
recompilation occurs if .pyc/.pyo mtime differs from the originating 
.py.  Via EAPI=3 mtime gurantees, that should be addressed- below 
that, compilation via pkg_postinst is necessary due to the lack of 
mtime guarantees.

> In the worst case it returns "Bad marshalling data".

Examples wanted for this.  If this occurs, that's a python bug- one 
exception... portage (figures).  They install into a non 
/usr/lib/python* location, meaning the .pyc/.pyo from py2.6 is 
exposed/accessed for py2.7 for example.

That said, a .pyc/.pyo from an older python version causing a blow up 
in a differing python version hasn't occurred since the 2.3 -> 2.4 
transition (if memory serves).

Either way, examples desired please.
~harring

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[Fabian Groffen]

[-- Attachment #1: Type: text/plain, Size: 626 bytes --]

On 30-09-2012 14:47:17 -0700, Brian Harring wrote:
> > In the worst case it returns "Bad marshalling data".
> 
> Examples wanted for this.  If this occurs, that's a python bug- one 
> exception... portage (figures).  They install into a non 
> /usr/lib/python* location, meaning the .pyc/.pyo from py2.6 is 
> exposed/accessed for py2.7 for example.

https://bugs.gentoo.org/show_bug.cgi?id=300922

I doubt whether it's a Python bug, we have to mess with the files.  But
then again, I did some toying, and it seems Python doesn't care about
this (any more?).


-- 
Fabian Groffen
Gentoo on a different level

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[Brian Harring]

On Mon, Oct 01, 2012 at 08:36:12AM +0200, Fabian Groffen wrote:
> On 30-09-2012 14:47:17 -0700, Brian Harring wrote:
> > > In the worst case it returns "Bad marshalling data".
> > 
> > Examples wanted for this.  If this occurs, that's a python bug- one 
> > exception... portage (figures).  They install into a non 
> > /usr/lib/python* location, meaning the .pyc/.pyo from py2.6 is 
> > exposed/accessed for py2.7 for example.
> 
> https://bugs.gentoo.org/show_bug.cgi?id=300922
> 
> I doubt whether it's a Python bug, we have to mess with the files.  But
> then again, I did some toying, and it seems Python doesn't care about
> this (any more?).

Well, offhand that bug is pre EAPI3 (eapi3 was approved 01/18/10, and 
adoption was slow- lot of people skipped straight to eapi4) - so the 
mtime wouldn't have been guaranteed preserved for a long while.  
Meaning the bugs data I don't trust to be relevant due to timing, and 
age.

As you said, this needs revisiting- minimally, portage is screwing 
around contents there, and I don't trust the python eclass to /not/ be 
forcing a compileall after the fact anyways.

Suggest backing down the various protections for a full test, and 
resuming that bug- if you can replicate it, I'm definitely interested 
(dealt with this when it occurred for 2.3->2.4 for example).

~harring

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[Marien Zwart]

(pardon any awkward formatting, writing from webinterface instead of a
"proper" client. Please yell at me off-list or on irc if this post is
excessively broken.)

> https://bugs.gentoo.org/show_bug.cgi?id=300922

From that bug:

> - chpathtool does some "simple" heuristic to switch the build EPREFIX
>   to the new target EPREFIX
>
> - python (seemingly) doesn't like when the pyc files are changed and *not* regenerated.
>   You will see the message: "Bad marshalling data"

I'm not sure if I'm interpreting this correctly, but it sounds as if
your chpathtool edits the .pyc file to replace the path? If yes: that
will indeed not work, as strings are stored length-prefixed. You
cannot sensibly edit .pyc files, you should normally just rewrite
them.

The .pyc/.pyo file store a version-dependent magic, the mtime of their
corresponding .py file, and (since recently) the size of the
correspondig .py file. If either changes the .pyc/.pyo is regenerated.
The magic does not normally change in released pythons, so that's not
much of a problem. The mtime is probably no longer a problem these
days either.

There used to be a more subtle problem: the python objects you import
have the path to their source embedded into them, and if you move the
.pyc/.pyo file around that path would be wrong, confusing some (mostly
debugging) tools (see http://bugs.python.org/issue1180193 ). This was
fixed fairly recently (in python 3, python 2.7 and python 2.6.2), so
it is probably no longer worth worrying about, but it's probably one
reason these files were handled this way in the past.

-- 
Marien Zwart (marienz)

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[Fabian Groffen]

[-- Attachment #1: Type: text/plain, Size: 737 bytes --]

On 01-10-2012 17:17:40 +0200, Marien Zwart wrote:
> There used to be a more subtle problem: the python objects you import
> have the path to their source embedded into them, and if you move the
> .pyc/.pyo file around that path would be wrong, confusing some (mostly
> debugging) tools (see http://bugs.python.org/issue1180193 ). This was
> fixed fairly recently (in python 3, python 2.7 and python 2.6.2), so
> it is probably no longer worth worrying about, but it's probably one
> reason these files were handled this way in the past.

This most likely is the reason why my tests with Python 2.7 worked fine
when I tinkered with the .pyc and/or moved it around.

Thanks

-- 
Fabian Groffen
Gentoo on a different level

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[hasufell]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 09/29/2012 05:37 PM, Ian Stakenvicius wrote:
> 
> There isn't so much a problem with the current python-distutils-ng 
> eclass but rather it's to expand it to a more comprehensive 
> replacement for both distutils and python eclasses.  In order to
> do that efficiently, most of the core functionality should be moved
> so that the new distutils is more like a wrapper to the new
> python.
> 
> This could certainly be done by patching the existing eclass, but 
> mgorny wants to use new eclass names instead of keeping the
> current one.  Hence the rename.  I think that's about it..
> 

To be a bit more precise:

if we really want to replace python.eclass we should start the new
eclass by gathering a list of features the python herd and developers
want. I think there already is something like that, no?

I don't see a point in adding a draft-eclass to the tree.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJQZxguAAoJEFpvPKfnPDWz29QH/2mylpPs2759z27RKqvmdGh8
X8bV+CMRqWBPl1+blXpGlX9Bf6Er7MRQD1XarqpgvT+1ALhJYL0SZO/MA5DTxvkJ
1EdhdlIVK2ew6UTOmH0jin+wSuspBE1ZpLJCLLWQ94PQgLScaVmAj+XWMLuCbSOF
GfKW1thACamIKl3ej1foxMzD9mtSvufqI+eZQd0V341jHR1v5JF8LeqfC3C8c8nR
AalRvqbh1QltzcX7ao8wWeq4cYUAGrYACrjQqiEtEnMuUkk2upQMzdjt0I41D5vT
oOJtlsk742SLdE4ZIHCsXbjeEOKaFiLBlDjHvcvkWl4MkbKQ6pGpnsTYSpDm8+k=
=iY6x
-----END PGP SIGNATURE-----

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1937 bytes --]

On Sat, 29 Sep 2012 15:49:32 +0200
hasufell <hasufell@gentoo.org> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On 09/29/2012 12:49 PM, Michał Górny wrote:
> > On Sat, 29 Sep 2012 11:20:31 +0100 Markos Chandras
> > <hwoarang@gentoo.org> wrote:
> > 
> >> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512
> >> 
> >> On 09/29/2012 09:53 AM, Micha? Górny wrote:
> >>> Hello,
> >>> 
> >>> Instead of the floating patches and p-d-ng modifications I sent
> >>>  earlier, here are the two complete (so far, well, initial :P) 
> >>> eclasses for review.
> >>> 
> >>> They are designed as 'mostly' drop-in python-distutils-ng 
> >>> replacement.
> >>> 
> >> Hi,
> >> 
> >> Are you saying that you are going to remove the
> >> python-distutils-ng eclass in favour of the new eclasses? I don't
> >> quite understand the reasons to be honest.
> > 
> > The reason is simple -- I can't fix it without changing the API. 
> > Changing the API on a live eclass is confusing, and considering
> > that it is not used by many packages, it's easier to lastrite it.
> > 
> > Also, this fixes the name not to have any '-ng' nor '-ds9'.
> > 
> 
> What are the reasons to change the API in the first place? There has
> to be a good reason, cause this will involve yet another migration of
> many ebuilds. I don't see any bugreports.

I have pointed out what changes to the API are _necessary_ to introduce
a good eclass on gentoo-python@.

Otherwise, the eclass would have to have at least two almost identical
functions doing the same thing, one universal and one for specific case
where specific parameters are passed (and not used in a single ebuild).

> I fear this will cause more confusion, i.e. some ebuilds using the old
> distutils, some using python-distutils-ng and some using distutils-r1
> resulting in weird tree behavior.

[example needed]

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[Ben de Groot]

On 29 September 2012 18:20, Markos Chandras <hwoarang@gentoo.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA512
>
> On 09/29/2012 09:53 AM, Micha? Górny wrote:
>> Hello,
>>
>> Instead of the floating patches and p-d-ng modifications I sent
>> earlier, here are the two complete (so far, well, initial :P)
>> eclasses for review.
>>
>> They are designed as 'mostly' drop-in python-distutils-ng
>> replacement.
>>
> Hi,
>
> Are you saying that you are going to remove the python-distutils-ng
> eclass in favour of the new eclasses? I don't quite understand the
> reasons to be honest.

The current python.eclass is too horribly complicated, and should go
the way of the dodo ASAP; while on the other hand python-distutils-ng
is too limited (e.g. for non-distutils multiple-implementation python
needing packages) and oddly named.

I fully support this attempt to improve the current situation, and as
I understand so does most of our python team.

-- 
Cheers,

Ben | yngwin
Gentoo developer
Gentoo Qt project lead, Gentoo Wiki admin

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 584 bytes --]

On Sat, 29 Sep 2012 10:53:29 +0200
Michał Górny <mgorny@gentoo.org> wrote:

> Instead of the floating patches and p-d-ng modifications I sent
> earlier, here are the two complete (so far, well, initial :P) eclasses
> for review.
> 
> They are designed as 'mostly' drop-in python-distutils-ng replacement.

Conversion of all ebuilds in my overlay:
http://git.overlays.gentoo.org/gitweb/?p=dev/mgorny.git;a=commitdiff;h=c0c8c4ea

Conversion of all ebuilds in gx86:
https://bitbucket.org/mgorny/gx86-working-tree/changeset/6442b5d5b

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[hasufell]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

1. there is still no way to convert shebangs via python-r1
2. there are no equivalent functions to the python_get_* stuff which
are sometimes necessary for broken build systems or test phases
3. there is no equivalent for python_mod_optimize. Having that in
distutils-r1 does not suffice for non-distutils packages where I might
have to do that stuff manually.
5. distutils-r1_rename_scripts does not allow to specify a location
possibly breaking non-standard locations (e.g. games location)
6. How can I make the python dependencies optional, e.g. that
python:3.2 is only pulled in when gtk useflag for this package is set?
7. are we sure now that it's safe to ship the .pyc/.pyo files inside
the package without side-effects?
8. how would I manually generate implementation-suffixed scripts
without distutils-r1?
example: x11-misc/redshift-1.7-r1
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJQiWvKAAoJEFpvPKfnPDWzwicIALNXQl4CySuYUhtcFdCuPC9X
Dcckhf8cJ1SvHl88MV2NJAg5GmoW+ZYB3BPxPY6z/QS1JD8Qxig/ZLBcvo3hfNoL
yAds48rRDr11TWpABNP+2HWxOkvNo2WyxXIxGKbqWJHqInYiSdZnjCBbdee/iVYt
DGazs/H2vrv59QPKvDVVoDCuGHBQ84Be4DUHBDoyLAi+Zao2br/vtbq8WMFSoaJn
IV7jwlnyoajQyLwcXH35f5Avx5pSqQmVe52JHe0z5sYs/irFtnH0EdUpyQQKp4Pt
YOE3HrMG5NHpoI089rSHRfMO6HwKTc4hZJP4Cr/NB+NB3+wmgR1fmGKswiTgQ6c=
=Kwhu
-----END PGP SIGNATURE-----

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 25/10/12 12:41 PM, hasufell wrote:
> 6. How can I make the python dependencies optional, e.g. that 
> python:3.2 is only pulled in when gtk useflag for this package is
> set?

that's not so much an optional dependency as an enforcement of a
PYHTON_TARGETS, isn't it?  or is it the other way around, ie,
PYTHON_COMPAT="python3.2" is not possible unless USE="+gtk" ?


Afaik the way python-r1 and distutils-r1 is meant to work is that
it'll build for whatever is supplied in PYTHON_TARGETS , in accordance
with whatever is in PYTHON_COMPAT.  It's supposed to be a complete
set, though.  So if python-3.2 is in both TARGETS and COMPAT then it's
built for that, otherwise it isn't.  For your USE=gtk issue , either
you would need to complain via either

REQUIRED_USE="gtk? ( python_targets_python3_2 )"

or

REQUIRED_USE="python_targets_python3_2? ( gtk )"

..depending on which situation applies.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iF4EAREIAAYFAlCJcoUACgkQ2ugaI38ACPA1lgD+KuSlC92KqsnHcePQJhtJCGY2
iXGhvboB2h1qlHoPAEUA/jP/XcWmjaRSjIxtEJMUqq76qqqMCaBwphrMBkbReGVT
=euK2
-----END PGP SIGNATURE-----

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2758 bytes --]

On Thu, 25 Oct 2012 18:41:47 +0200
hasufell <hasufell@gentoo.org> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> 1. there is still no way to convert shebangs via python-r1

What conversion do you expect? The docs say it clearly that the eclass
will be extended on request, so please file a clear request what you
want with an example use case.

> 2. there are no equivalent functions to the python_get_* stuff which
> are sometimes necessary for broken build systems or test phases

There is python_export(). I will be happy to extend it / add
python_get*() wrappers when someone makes a clear list of what
is needed.

Example use cases will be appreciated again. Good examples will make it
possible to choose good variable names.

> 3. there is no equivalent for python_mod_optimize. Having that in
> distutils-r1 does not suffice for non-distutils packages where I might
> have to do that stuff manually.

There is a lot of stuff missing for packages which try to install
Python stuff by hand rather than using proper setup. I will be happy to
provide more when I know what is actually needed and how it will be
used.

> 5. distutils-r1_rename_scripts does not allow to specify a location
> possibly breaking non-standard locations (e.g. games location)

It's a quick function. Adding additional paths or changing
the algorithm won't be hard. Just don't expect me to do random stuff
just because someone may want that someday.

FYI: I've added that mindless games/bin to the paths.

> 6. How can I make the python dependencies optional, e.g. that
> python:3.2 is only pulled in when gtk useflag for this package is set?

inherit python-r1

RDEPEND='gtk? ( ${PYTHON_DEPS} )'

> 7. are we sure now that it's safe to ship the .pyc/.pyo files inside
> the package without side-effects?

There are side effects and they are well known. Still, none of them is
an issue important enough to outweigh the benefit.

> 8. how would I manually generate implementation-suffixed scripts
> without distutils-r1?

You would use the function I would introduce when I got the request
filed and potential discussion done. Most importantly, whether there
should be a way to call 2to3 on the scripts.

By the way, did you just request two partial features instead
of requesting a way to manually install Python scripts?

> example: x11-misc/redshift-1.7-r1

Thanks. I will take a look at that package and see what is necessary to
make it buildable with python-r1.

By the way, do I understand correctly that right now you are building
stuff for a random Python implementation and removing it afterwards?
I believe that's something really worth fixing.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[hasufell]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 10/25/2012 07:43 PM, Michał Górny wrote:
> On Thu, 25 Oct 2012 18:41:47 +0200 hasufell <hasufell@gentoo.org>
> wrote:
> 
>> 1. there is still no way to convert shebangs via python-r1
> 
> What conversion do you expect? The docs say it clearly that the
> eclass will be extended on request, so please file a clear request
> what you want with an example use case.

request: A function that converts the shebang to a version specific
python shebang chosen by me.

usecase: python scripts installed by a non-distutils build system
which need appropriate shebang conversion (Unless we can fix that in a
different way.)

> 
>> 2. there are no equivalent functions to the python_get_* stuff
>> which are sometimes necessary for broken build systems or test
>> phases
> 
> There is python_export(). I will be happy to extend it / add 
> python_get*() wrappers when someone makes a clear list of what is
> needed.
> 
> Example use cases will be appreciated again. Good examples will
> make it possible to choose good variable names.

example:
x11-misc/redshift-1.7-r1

I'll give more examples as I come across those again. This is much
"afair".

> 
>> 3. there is no equivalent for python_mod_optimize. Having that
>> in distutils-r1 does not suffice for non-distutils packages where
>> I might have to do that stuff manually.
> 
> There is a lot of stuff missing for packages which try to install 
> Python stuff by hand rather than using proper setup. I will be
> happy to provide more when I know what is actually needed and how
> it will be used.
> 

example:
x11-misc/redshift-1.7-r1

Again, I'll give more examples as I come across those.

>> 5. distutils-r1_rename_scripts does not allow to specify a
>> location possibly breaking non-standard locations (e.g. games
>> location)
> 
> It's a quick function. Adding additional paths or changing the
> algorithm won't be hard. Just don't expect me to do random stuff 
> just because someone may want that someday.
> 
> FYI: I've added that mindless games/bin to the paths.

that games/bin change is not sufficient since the games variables can
be modified by the user and this is supported by the games eclass. So
you have to pass an option to the ebuild developer.


> 
>> 8. how would I manually generate implementation-suffixed scripts 
>> without distutils-r1?
> 
> You would use the function I would introduce when I got the
> request filed and potential discussion done. Most importantly,
> whether there should be a way to call 2to3 on the scripts.
> 
> By the way, did you just request two partial features instead of
> requesting a way to manually install Python scripts?
> 
>> example: x11-misc/redshift-1.7-r1
> 
> Thanks. I will take a look at that package and see what is
> necessary to make it buildable with python-r1.
> 
> By the way, do I understand correctly that right now you are
> building stuff for a random Python implementation and removing it
> afterwards? I believe that's something really worth fixing.
> 

That way was chosen to avoid an extensive patch I have already
written. The maintainer of the package did not want to keep that
around through version bumps which is understandable.

- -----

Currently you seem to have focused more on distutils when writing
python-r1 which makes this eclass a bit raw.
Waiting for other developers to file feature requests instead of
figuring out those yourself before they even consider porting their
ebuild to your new eclasses seems like a questionable policy to me.
They might not be too excited about it to start discussions and
feature requests just to switch from an already working implementation.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJQiYspAAoJEFpvPKfnPDWzP/UH/i3GYZjrN1J+p5XjLmB7a20F
zYpZzjRSKJqv5V2xB3obngTEHT+qvSaQ4xMUdhrwuF/XpyEjD18z93kmzq1gPvan
ofnek3WiTHxaB0AJ3RoOdYB8gPRXFZpWxtySDOFgrP322mY+sBNyHMLVQ4CIJwq5
UsRwSlJNM4yddKSmnTxdRt8unmNm9mhYcmWGtebJ7MDU1720RR/RP4TxtumvdP3y
i4EDM61+kdoyKiXIfCIkKknj4CztiPZlyrRiGTFZ63+99mbJJt0qYCPYYzf9l+MV
nCrM+zUjqGvmvMeLhMulpxbn2SW7E8PpJobwTiTR8I99voTCcHH3M9NFUhfWrbs=
=//Y7
-----END PGP SIGNATURE-----

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1252 bytes --]

On Thu, 25 Oct 2012 20:55:37 +0200
hasufell <hasufell@gentoo.org> wrote:

> Currently you seem to have focused more on distutils when writing
> python-r1 which makes this eclass a bit raw.
> Waiting for other developers to file feature requests instead of
> figuring out those yourself before they even consider porting their
> ebuild to your new eclasses seems like a questionable policy to me.
> They might not be too excited about it to start discussions and
> feature requests just to switch from an already working implementation.

As you may have failed to notice, most of Python packages actually are
using distutils. Thus, the core goal for distutils-r1/python-r1 was to
correctly support those packages.

Now that distutils is supported well, I can start thinking about
supporting random hackish build systems. I will review redshift and
give you my thoughts.

Just note that your attitude is not motivating at all. I haven't killed
any of your kitten or forced anyone to use python-r1. Most of you
didn't even care to give a single word of feedback throughout
the development process, so your position of 'this eclass doesn't give
me shiny functions I want' is at least impolite.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[hasufell]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 10/25/2012 10:13 PM, Michał Górny wrote:
> On Thu, 25 Oct 2012 20:55:37 +0200 hasufell <hasufell@gentoo.org>
> wrote:
> 
>> Currently you seem to have focused more on distutils when
>> writing python-r1 which makes this eclass a bit raw. Waiting for
>> other developers to file feature requests instead of figuring out
>> those yourself before they even consider porting their ebuild to
>> your new eclasses seems like a questionable policy to me. They
>> might not be too excited about it to start discussions and 
>> feature requests just to switch from an already working
>> implementation.
> 
> As you may have failed to notice, most of Python packages actually
> are using distutils. Thus, the core goal for distutils-r1/python-r1
> was to correctly support those packages.
> 
> Now that distutils is supported well, I can start thinking about 
> supporting random hackish build systems. I will review redshift
> and give you my thoughts.
> 
> Just note that your attitude is not motivating at all. I haven't
> killed any of your kitten or forced anyone to use python-r1. Most
> of you didn't even care to give a single word of feedback
> throughout the development process, so your position of 'this
> eclass doesn't give me shiny functions I want' is at least
> impolite.
> 

Sorry, I thought the main goal was to deprecate python.eclass at some
(very distant) point.

If that is true, then we need to support _all_ build systems
using/related to python.

I was just trying to say that you shouldn't wait for developers to
point out all these cases. Best way would be to start converting
exotic ebuilds and grep for packages that use python, but _not_ distutils.

Otherwise I am missing the point why you created two eclasses instead
of one (namely just distutils-r1).
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJQitZQAAoJEFpvPKfnPDWzuOEIAJp9siMgIh4mdDP/kMNvCvpw
jOJqML6ZMq9fEl5g8y7D46Vlw3cpQOUErh7fR7iNhN21EZsetNiAfi+s25+cnIWV
XF/zrmdGxGJLqgJLwRI8sWwobCiQWpzQC+wJND6DDyCEk5NsJNMuCfFvgIO3l6YY
Q6Amtn3QRwNGaZdCF6jWnSScqyJIK5x6ih6UVe99tgwPasNzDWxLesyr1LbkW4sB
yohyQGN+JuSWlbOrM9BCs2M5VBFSMlnXTdwJqB4wxEY60FPsFQ33+5Mhx39Wvzd+
oDB+0NpqsVU6SZZ3K2hCr70T5M4j/CFGP1AaX5Z1nYoXxeJRkrat+95lcTCXeo0=
=awW1
-----END PGP SIGNATURE-----

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 5266 bytes --]

On Thu, 25 Oct 2012 20:55:37 +0200
hasufell <hasufell@gentoo.org> wrote:

> On 10/25/2012 07:43 PM, Michał Górny wrote:
> > On Thu, 25 Oct 2012 18:41:47 +0200 hasufell <hasufell@gentoo.org>
> > wrote:
> > 
> >> 1. there is still no way to convert shebangs via python-r1
> > 
> > What conversion do you expect? The docs say it clearly that the
> > eclass will be extended on request, so please file a clear request
> > what you want with an example use case.
> 
> request: A function that converts the shebang to a version specific
> python shebang chosen by me.
> 
> usecase: python scripts installed by a non-distutils build system
> which need appropriate shebang conversion (Unless we can fix that in a
> different way.)

That's a fair request. Last thing which I have to think about is
whether we want this in python-r1, or a more general shebang replacement
function in eutils.

> >> 2. there are no equivalent functions to the python_get_* stuff
> >> which are sometimes necessary for broken build systems or test
> >> phases
> > 
> > There is python_export(). I will be happy to extend it / add 
> > python_get*() wrappers when someone makes a clear list of what is
> > needed.
> > 
> > Example use cases will be appreciated again. Good examples will
> > make it possible to choose good variable names.
> 
> example:
> x11-misc/redshift-1.7-r1
> 
> I'll give more examples as I come across those again. This is much
> "afair".

Sorry, didn't notice the sitedir use.

> >> 3. there is no equivalent for python_mod_optimize. Having that
> >> in distutils-r1 does not suffice for non-distutils packages where
> >> I might have to do that stuff manually.
> > 
> > There is a lot of stuff missing for packages which try to install 
> > Python stuff by hand rather than using proper setup. I will be
> > happy to provide more when I know what is actually needed and how
> > it will be used.
> > 
> 
> example:
> x11-misc/redshift-1.7-r1
> 
> Again, I'll give more examples as I come across those.

Err, do you expect the eclass to provide a function to restore
the py-compile script you're removing?

> >> 5. distutils-r1_rename_scripts does not allow to specify a
> >> location possibly breaking non-standard locations (e.g. games
> >> location)
> > 
> > It's a quick function. Adding additional paths or changing the
> > algorithm won't be hard. Just don't expect me to do random stuff 
> > just because someone may want that someday.
> > 
> > FYI: I've added that mindless games/bin to the paths.
> 
> that games/bin change is not sufficient since the games variables can
> be modified by the user and this is supported by the games eclass. So
> you have to pass an option to the ebuild developer.

User can do many stupid things, and some eclasses are just stupid
and should be killed with fire. A lot of fire. Passing an option is
just inventing a minigun to shot a mosquito.

I will be happy to fix that whenever someone stumbles around that.
If ever. If ever distutils setup.py will actually install anything
to /usr/randomlocation/bin which will be actually supposed to be
rewritten.

> >> 8. how would I manually generate implementation-suffixed scripts 
> >> without distutils-r1?
> > 
> > You would use the function I would introduce when I got the
> > request filed and potential discussion done. Most importantly,
> > whether there should be a way to call 2to3 on the scripts.
> > 
> > By the way, did you just request two partial features instead of
> > requesting a way to manually install Python scripts?

Ok, looking at redshift, I'm not really sure if we really need or even
want to install implementation-suffixed scripts there. That's something
to discuss.

The main reason for implementation-suffixing of scripts is that
distutils can mangle the scripts for various implementations (i.e. 2to3
them). The side reason is ability to force a particular implementation
when no other suitable way is available (which is pretty rare).

To be honest, redshift doesn't fit either. I doubt anyone will really
need to put 'gtk-redshift-python2.6' anywhere, and both scripts will be
identical. So it may be just enough or even better just to mangle
the script to have 'python2' shebang.

> >> example: x11-misc/redshift-1.7-r1
> > 
> > Thanks. I will take a look at that package and see what is
> > necessary to make it buildable with python-r1.
> > 
> > By the way, do I understand correctly that right now you are
> > building stuff for a random Python implementation and removing it
> > afterwards? I believe that's something really worth fixing.
> > 
> 
> That way was chosen to avoid an extensive patch I have already
> written. The maintainer of the package did not want to keep that
> around through version bumps which is understandable.

You again fail to see an interim solution which will solve the issue
without much trouble, similar to one used for multilib. You let
the build system work with the default implementation, then repeat
install for remaining implementations.

Of course, right now it's not supported by python-r1. I will think how
to solve it in a simple way.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[hasufell]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 10/25/2012 06:41 PM, hasufell wrote:
> 1. there is still no way to convert shebangs via python-r1 2. there
> are no equivalent functions to the python_get_* stuff which are
> sometimes necessary for broken build systems or test phases 3.
> there is no equivalent for python_mod_optimize. Having that in 
> distutils-r1 does not suffice for non-distutils packages where I
> might have to do that stuff manually. 5.
> distutils-r1_rename_scripts does not allow to specify a location 
> possibly breaking non-standard locations (e.g. games location) 6.
> How can I make the python dependencies optional, e.g. that 
> python:3.2 is only pulled in when gtk useflag for this package is
> set? 7. are we sure now that it's safe to ship the .pyc/.pyo files
> inside the package without side-effects? 8. how would I manually
> generate implementation-suffixed scripts without distutils-r1? 
> example: x11-misc/redshift-1.7-r1
> 

4. no equivalent to python_set_active_version

feature request: let the ebuild developer choose a python
implementation for the whole emerge process.

usecase: packages not using distutils and not supporting multiple
python abis, but detecting installed python in some way and working
with all python:2.* slots.

examples that come to my mind right now:
games-strategy/freeorion
games-action/openclonk
net-im/pidgin
app-office/libreoffice

python_foreach_impl cannot fit that
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJQiZDJAAoJEFpvPKfnPDWzwaIIAKAIlQlrtxWL3gPbUZqR/XKA
H/tov9nk+yb9VoqB8Hn4k0gZ1ZicGNgPIqY777/eJ2G/F9IdT8866Lztq4lWyno8
r9LZMsesNnXxZDnDLgbIyt1DXm9ddmiC3bTi3iCpgQ54sats+icgcsZ40ya/9+k5
ouZCxnxBI7LhgWPKmtloZC2Rundcgoxk7K5le37O6P6nhH5HPI8vdXtnUmNDJ/oY
ryb6VD1hkSnkY/pcIQ7MVUspGqw45LN/INhQAMvakla4y1r9pa12rsKgMAKnoYv6
LjhWtXLJEeYx+crJvLoHjdrXBX6TS8JhBc3BOVgtU1LzccDqEkqPOoLBLHUx7Tw=
=D5Ee
-----END PGP SIGNATURE-----

Re: [gentoo-dev] [RFC] Initial python-r1.eclass & distutils-r1.eclass

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1958 bytes --]

On Thu, 25 Oct 2012 21:19:37 +0200
hasufell <hasufell@gentoo.org> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On 10/25/2012 06:41 PM, hasufell wrote:
> > 1. there is still no way to convert shebangs via python-r1 2. there
> > are no equivalent functions to the python_get_* stuff which are
> > sometimes necessary for broken build systems or test phases 3.
> > there is no equivalent for python_mod_optimize. Having that in 
> > distutils-r1 does not suffice for non-distutils packages where I
> > might have to do that stuff manually. 5.
> > distutils-r1_rename_scripts does not allow to specify a location 
> > possibly breaking non-standard locations (e.g. games location) 6.
> > How can I make the python dependencies optional, e.g. that 
> > python:3.2 is only pulled in when gtk useflag for this package is
> > set? 7. are we sure now that it's safe to ship the .pyc/.pyo files
> > inside the package without side-effects? 8. how would I manually
> > generate implementation-suffixed scripts without distutils-r1? 
> > example: x11-misc/redshift-1.7-r1
> > 
> 
> 4. no equivalent to python_set_active_version
> 
> feature request: let the ebuild developer choose a python
> implementation for the whole emerge process.
> 
> usecase: packages not using distutils and not supporting multiple
> python abis, but detecting installed python in some way and working
> with all python:2.* slots.
> 
> examples that come to my mind right now:
> games-strategy/freeorion
> games-action/openclonk
> net-im/pidgin
> app-office/libreoffice
> 
> python_foreach_impl cannot fit that

Ah, sorry, I must have missed when we started supporting packages for
single Python implementation in python-r1. That probably happened when
nobody replied to the my mail requesting ideas on handling those
packages [1].

[1]:http://article.gmane.org/gmane.linux.gentoo.python/22

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]