[gentoo-dev] New eclass for Python

[gentoo-dev] New eclass for Python

[Krzysztof Pawlik]

[-- Attachment #1.1: Type: text/plain, Size: 1403 bytes --]

Hello,

After some work during weekend on Python packages I've decided to start a
rewrite of Python/distutils eclass for installing Python packages. My main goal
was simplicity and functionality similar to ruby-ng.eclass (thanks Ruby team for
your great work!). Python team members already contributed comments and
suggestions and helped me to make the eclass better, thank you!

Highlights:
 - *SIMPLE*next
 - uses PYTHON_TARGETS use-expand (no more python-updater, whoooo!)
 - EAPI4 required, uses REQUIRED_USE
 - <400 lines of code including documentation
 - should work for >95% of packages (my educated guess)
 - did I mention it's *SIMPLE*?
 - easy to maintain & read so it's also easy to use

Important thing: I'm not aiming at having 100% functionality of current
python.eclass+distutils.eclass in the new one, I think that simplicity is more
important that supporting every possible, obscure case that's out there.

I'm attaching the eclass itself and two ebuilds using it, code is also available
in my overlay at http://git.overlays.gentoo.org/gitweb/?p=dev/nelchael.git;a=summary

If there are no objections then during the weekend (March 3, 4) I will add this
to portage (after finishing remaining TODO items, PyPy requires 4G of RAM(!!)).

-- 
Krzysztof Pawlik  <nelchael at gentoo.org>  key id: 0xF6A80E46
desktop-misc, java, vim, kernel, python, apache...


[-- Attachment #1.2: xlwt-0.7.2-r256.ebuild --]
[-- Type: text/plain, Size: 735 bytes --]

# Copyright 1999-2012 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header$

EAPI="4"

inherit python-distutils-ng

DESCRIPTION="Python library to create spreadsheet files compatible with Excel"
HOMEPAGE="http://pypi.python.org/pypi/xlwt"
SRC_URI="mirror://pypi/${PN:0:1}/${PN}/${P}.tar.gz"

LICENSE="BSD"
SLOT="0"
KEYWORDS="~amd64 ~ppc-aix ~hppa-hpux ~ia64-hpux ~x86-interix ~x86-linux ~sparc-solaris ~x86-solaris"
IUSE="examples"

DEPEND=""
RDEPEND=""

python_prepare_all() {
	sed -i \
		-e "s,'doc,# 'doc,g" \
		-e "s,'exa,# 'exa,g" \
		setup.py || die
}

python_install_all() {
	dohtml xlwt/doc/*.html
	if use examples; then
		insinto "/usr/share/doc/${PF}"
		doins -r xlwt/examples
	fi
}

[-- Attachment #1.3: xlrd-0.7.1-r256.ebuild --]
[-- Type: text/plain, Size: 838 bytes --]

# Copyright 1999-2012 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header$

EAPI="4"

inherit python-distutils-ng

DESCRIPTION="Library for developers to extract data from Microsoft Excel (tm) spreadsheet files"
HOMEPAGE="http://pypi.python.org/pypi/xlrd"
SRC_URI="mirror://pypi/${PN:0:1}/${PN}/${P}.tar.gz"

LICENSE="BSD"
SLOT="0"
KEYWORDS="~amd64 ~ppc-aix ~hppa-hpux ~ia64-hpux ~x86-interix ~x86-linux ~sparc-solaris ~x86-solaris"
IUSE="examples"

DEPEND=""
RDEPEND=""

python_prepare_all() {
	sed -i \
		-e "s,'doc,# 'doc,g" \
		-e "s,'exa,# 'exa,g" \
		setup.py || die
}

python_install_all() {
	rm -f "${D}/usr/bin"/*.py || die

	python-distutils-ng_doscript scripts/runxlrd.py

	dohtml xlrd/doc/*.html
	if use examples; then
		insinto "/usr/share/doc/${PF}"
		doins -r xlrd/examples
	fi
}

[-- Attachment #1.4: python_targets.desc --]
[-- Type: text/plain, Size: 455 bytes --]

# Copyright 2009-2012 Gentoo Foundation.
# Distributed under the terms of the GNU General Public License v2
# $Header$

# This file contains descriptions of PYTHON_TARGETS USE_EXPAND flags.

python2_5 - Build with Python 2.5
python2_6 - Build with Python 2.6
python2_7 - Build with Python 2.7
python3_1 - Build with Python 3.1
python3_2 - Build with Python 3.2
jython2_5 - Build with Jython 2.5
pypy1_7 - Build with PyPy 1.7
pypy1_8 - Build with PyPy 1.8

[-- Attachment #1.5: python-distutils-ng.eclass --]
[-- Type: text/plain, Size: 11741 bytes --]

# Copyright 1999-2012 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header$

# @ECLASS: python-distutils-ng
# @MAINTAINER:
# Python herd <python@gentoo.org>
# @AUTHOR:
# Author: Krzysztof Pawlik <nelchael@gentoo.org>
# @BLURB: An eclass for installing Python packages using distutils with proper
# support for multiple Python slots.
# @DESCRIPTION:
# The Python eclass is designed to allow an easier installation of Python
# packages and their incorporation into the Gentoo Linux system.
#
# TODO: Document implementations!

# @ECLASS-VARIABLE: PYTHON_COMPAT
# @DESCRIPTION:
# This variable contains a space separated list of implementations (see above) a
# package is compatible to. It must be set before the `inherit' call. The
# default is to enable all implementations.

if [[ -z "${PYTHON_COMPAT}" ]]; then
	# Default: pure python, support all implementations
	PYTHON_COMPAT="  python2_5 python2_6 python2_7"
	PYTHON_COMPAT+=" python3_1 python3_2"
	PYTHON_COMPAT+=" jython2_5"
	PYTHON_COMPAT+=" pypy1_7 pypy1_8"
fi

# @ECLASS-VARIABLE: PYTHON_OPTIONAL
# @DESCRIPTION:
# Set the value to "yes" to make the dependency on a Python interpreter
# optional.

# @ECLASS-VARIABLE: PYTHON_DISABLE_COMPILATION
# @DESCRIPTION:
# Set the value to "yes" to skip compilation and/or optimization of Python
# modules.

EXPORT_FUNCTIONS src_unpack src_prepare src_configure src_compile src_test src_install

case "${EAPI}" in
	0|1|2|3)
		die "Unsupported EAPI=${EAPI} (too old) for python-distutils-ng.eclass" ;;
	4)
		# EAPI=4 needed for REQUIRED_USE
		S="${S:-${WORKDIR}/${P}}"
		;;
	*)
		die "Unsupported EAPI=${EAPI} (unknown) for python-distutils-ng.eclass" ;;
esac

# @FUNCTION: _python-distutils-ng_generate_depend
# @USAGE: implementation
# @RETURN: Package atom of a Python implementation for *DEPEND.
# @DESCRIPTION:
# This function returns the full package atom of a Python implementation.
#
# `implementation' has to be one of the valid values for PYTHON_COMPAT.
_python-distutils-ng_generate_depend() {
	local impl="${1/_/.}"
	case "${impl}" in
		python?.?)
			echo "dev-lang/${impl::-3}:${impl: -3}" ;;
		jython?.?)
			echo "dev-java/${impl::-3}:${impl: -3}" ;;
		pypy?.?)
			echo "dev-python/${impl::-3}:${impl: -3}" ;;
		*)
			die "Unsupported implementation: ${1}" ;;
	esac
}

# @FUNCTION: _python-distutils-ng_get_binary_for_implementation
# @USAGE: implementation
# @RETURN: Full path to Python binary for given implementation.
# @DESCRIPTION:
# This function returns full path for Python binary for given implementation.
#
# Binary returned by this function should be used instead of simply calling
# `python'.
_python-distutils-ng_get_binary_for_implementation() {
	local impl="${1/_/.}"
	case "${impl}" in
		python?.?|jython?.?)
			echo "/usr/bin/${impl}" ;;
		pypy?.?)
			echo "TODO" ;;
		*)
			die "Unsupported implementation: ${1}" ;;
	esac
}

required_use_str=" || (
	python_targets_python2_5 python_targets_python2_6 python_targets_python2_7
	python_targets_python3_1 python_targets_python3_2
	python_targets_jython2_5
	python_targets_pypy1_7 python_targets_pypy1_8 )"
if [[ "${PYTHON_OPTIONAL}" = "yes" ]]; then
	IUSE+="python"
	REQUIRED_USE+=" python? ( ${required_use_str} )"
else
	REQUIRED_USE+="${required_use_str}"
fi

for impl in ${PYTHON_COMPAT}; do
	IUSE+=" python_targets_${impl} "
	local dep_str="python_targets_${impl}? ( $(_python-distutils-ng_generate_depend "${impl}") )"

	if [[ "${PYTHON_OPTIONAL}" = "yes" ]]; then
		RDEPEND="${RDEPEND} python? ( ${dep_str} )"
		DEPEND="${DEPEND} python? ( ${dep_str} )"
	else
		RDEPEND="${RDEPEND} ${dep_str}"
		DEPEND="${DEPEND} ${dep_str}"
	fi
done

PACKAGE_SPECIFIC_S="${S#${WORKDIR}/}"

# @FUNCTION: _python-distutils-ng_run_for_impl
# @USAGE: implementation command_to_run
# @DESCRIPTION:
# Run command_to_run using specified Python implementation.
#
# This will run the command_to_run in implementation-specific working directory.
_python-distutils-ng_run_for_impl() {
	local impl="${1}"
	local command="${2}"

	S="${WORKDIR}/impl_${impl}/${PACKAGE_SPECIFIC_S}"
	PYTHON="$(_python-distutils-ng_get_binary_for_implementation "${impl}")"

	einfo "Running ${command} in ${S} for ${impl}"

	pushd "${S}" &> /dev/null
	"${command}" "${impl}" "${PYTHON}"
	popd &> /dev/null
}

# @FUNCTION: _python-distutils-ng_run_for_all_impls
# @USAGE: command_to_run
# @DESCRIPTION:
# Run command_to_run for all enabled Python implementations.
#
# See also _python-distutils-ng_run_for_impl
_python-distutils-ng_run_for_all_impls() {
	local command="${1}"

	for impl in ${PYTHON_COMPAT}; do
		use "python_targets_${impl}" ${PYTHON_COMPAT} || continue
		_python-distutils-ng_run_for_impl "${impl}" "${command}"
	done
}

# @FUNCTION: _python-distutils-ng_default_distutils_compile
# @DESCRIPTION:
# Default src_compile for distutils-based packages.
_python-distutils-ng_default_distutils_compile() {
	"${PYTHON}" setup.py build || die
}

# @FUNCTION: _python-distutils-ng_default_distutils_install
# @DESCRIPTION:
# Default src_install for distutils-based packages.
_python-distutils-ng_default_distutils_install() {
	"${PYTHON}" setup.py install --no-compile --root="${D}/" || die
}

# @FUNCTION: _python-distutils-ng_has_compileall
# @USAGE: implementation
# @RETURN: 0 if given implementation has compileall module
# @DESCRIPTION:
# This function is used to decide whenever to compile Python modules for given
# implementation.
_python-distutils-ng_has_compileall() {
	case "${1}" in
		python?_?|jython?_?)
			return 0 ;;
		*)
			return 1 ;;
	esac
}

# @FUNCTION: _python-distutils-ng_has_compileall_opt
# @USAGE: implementation
# @RETURN: 0 if given implementation has compileall module and supports # optimizations
# @DESCRIPTION:
# This function is used to decide whenever to compile and optimize Python
# modules for given implementation.
_python-distutils-ng_has_compileall_opt() {
	case "${1}" in
		python?_?)
			return 0 ;;
		*)
			return 1 ;;
	esac
}

# @FUNCTION: python-distutils-ng_doscript
# @USAGE: script_file_name
# @DESCRIPTION:
# Install given script file in /usr/bin/ for all enabled implementations using
# original script name as a base name.
#
# See also python-distutils-ng_newscript
python-distutils-ng_doscript() {
	python-distutils-ng_newscript "${1}" "$(basename "${1}")"
}

# @FUNCTION: python-distutils-ng_newscript
# @USAGE: script_file_name new_file_name
# @DESCRIPTION:
# Install given script file in /usr/bin/ for all enabled implementations using
# new_file_name as a base name.
#
# Each script copy will have the name mangled to "new_file_name-IMPLEMENTATION"
# and new hash-bang line will be inserted to reference specific Python
# interpreter.
#
# There will be also a symlink with name equal to new_file_name that will be a
# symlink to default implementation, which defaults to value of
# PYTHON_DEFAULT_IMPLEMENTATION, if not specified the function will pick default
# implementation: it will the be first enabled from the following list:
#   python2_7, python2_6, python2_5, python3_2, python3_1, pypy1_8, pypy1_7, jython2_5
python-distutils-ng_newscript() {
	[[ -n "${1}" ]] || die "Missing source file name"
	[[ -n "${2}" ]] || die "Missing destination file name"
	local source_file="${1}"
	local destination_file="${2}"
	local default_impl="${PYTHON_DEFAULT_IMPLEMENTATION}"

	if [[ -z "${default_impl}" ]]; then
		# TODO: Pick default implementation
		for impl in python{2_7,2_6,2_5,3_2,2_1} pypy{1_8,1_7} jython2_5; do
			use "python_targets_${impl}" || continue
			default_impl="${impl}"
			break;
		done
	else
		use "python_targets_${impl}" || \
			die "default implementation ${default_impl} not enabled"
	fi

	[[ -n "${default_impl}" ]] || die "Could not select default implementation"

	einfo "Installing ${source_file} for multiple implementations (default: ${default_impl})"
	insinto /usr/bin
	for impl in ${PYTHON_COMPAT}; do
		use "python_targets_${impl}" ${PYTHON_COMPAT} || continue

		newins "${source_file}" "${destination_file}-${impl}"
		fperms 755 "/usr/bin/${destination_file}-${impl}"
		sed -i \
			-e "1i#!$(_python-distutils-ng_get_binary_for_implementation "${impl}")" \
			"${D}/usr/bin/${destination_file}-${impl}" || die
	done

	dosym "${destination_file}-${default_impl}" "/usr/bin/${destination_file}"
}

# Phase function: src_unpack
python-distutils-ng_src_unpack() {
	[[ "${PYTHON_OPTIONAL}" = "yes" ]] && { use python || return; }

	if type python_unpack &> /dev/null; then
		# This should not run anything specific to any single Python
		# implementation, keep it generic:
		python_unpack_all
	else
		[[ -n ${A} ]] && unpack ${A}
	fi
}

# Phase function: src_prepare
python-distutils-ng_src_prepare() {
	[[ "${PYTHON_OPTIONAL}" = "yes" ]] && { use python || return; }

	# Try to run binary for each implementation:
	for impl in ${PYTHON_COMPAT}; do
		use "python_targets_${impl}" ${PYTHON_COMPAT} || continue
		$(_python-distutils-ng_get_binary_for_implementation "${impl}") \
			-c "import sys" || die
	done

	# Run prepare shared by all implementations:
	if type python_prepare_all &> /dev/null; then
		einfo "Running python_prepare_all in ${S} for all"
		python_prepare_all
	fi

	# Create a copy of S for each implementation:
	for impl in ${PYTHON_COMPAT}; do
		use "python_targets_${impl}" ${PYTHON_COMPAT} || continue

		einfo "Creating copy for ${impl} in ${WORKDIR}/impl_${impl}"
		mkdir -p "${WORKDIR}/impl_${impl}" || die
		cp -pr "${S}" "${WORKDIR}/impl_${impl}/${PACKAGE_SPECIFIC_S}" || die
	done

	# Run python_prepare for each implementation:
	if type python_prepare &> /dev/null; then
		_python-distutils-ng_run_for_all_impls python_prepare
	fi
}

# Phase function: src_configure
python-distutils-ng_src_configure() {
	[[ "${PYTHON_OPTIONAL}" = "yes" ]] && { use python || return; }

	if type python_configure &> /dev/null; then
		_python-distutils-ng_run_for_all_impls python_configure
	fi
}

# Phase function: src_compile
python-distutils-ng_src_compile() {
	[[ "${PYTHON_OPTIONAL}" = "yes" ]] && { use python || return; }

	if type python_compile &> /dev/null; then
		_python-distutils-ng_run_for_all_impls python_compile
	else
		_python-distutils-ng_run_for_all_impls \
			_python-distutils-ng_default_distutils_compile
	fi
}

# Phase function: src_test
python-distutils-ng_src_test() {
	[[ "${PYTHON_OPTIONAL}" = "yes" ]] && { use python || return; }

	if type python_test &> /dev/null; then
		_python-distutils-ng_run_for_all_impls python_test
	fi
}

# Phase function: src_install
python-distutils-ng_src_install() {
	[[ "${PYTHON_OPTIONAL}" = "yes" ]] && { use python || return; }

	if type python_install &> /dev/null; then
		_python-distutils-ng_run_for_all_impls python_install
	else
		_python-distutils-ng_run_for_all_impls \
			_python-distutils-ng_default_distutils_install
	fi

	S="${WORKDIR}/${PACKAGE_SPECIFIC_S}"
	if type python_install_all &> /dev/null; then
		einfo "Running python_install_all in ${S} for all"
		python_install_all
	fi

	for impl in ${PYTHON_COMPAT}; do
		[[ "${PYTHON_DISABLE_COMPILATION}" = "yes" ]] && continue
		use "python_targets_${impl}" ${PYTHON_COMPAT} || continue

		PYTHON="$(_python-distutils-ng_get_binary_for_implementation "${impl}")"
		for accessible_path in $(${PYTHON} -c 'import sys; print " ".join(sys.path)'); do
			[[ -d "${D}/${accessible_path}" ]] || continue

			_python-distutils-ng_has_compileall "${impl}" || continue
			ebegin "Compiling ${accessible_path} for ${impl}"
			${PYTHON} \
				-m compileall -q -f "${D}/${accessible_path}" || die
			eend $?

			_python-distutils-ng_has_compileall_opt "${impl}" || continue
			ebegin "Optimizing ${accessible_path} for ${impl}"
			PYTHONOPTIMIZE=1 ${PYTHON} \
				-m compileall -q -f "${D}/${accessible_path}" || die
			eend $?
		done;
	done
}

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 554 bytes --]

Re: [gentoo-dev] New eclass for Python

[Sergei Trofimovich]

[-- Attachment #1: Type: text/plain, Size: 608 bytes --]

On Tue, 28 Feb 2012 22:13:36 +0100
Krzysztof Pawlik <nelchael@gentoo.org> wrote:

> I'm attaching the eclass itself and two ebuilds using it, code is also available
> in my overlay at http://git.overlays.gentoo.org/gitweb/?p=dev/nelchael.git;a=summary
Nice!

> eclass/python-distutils-ng.eclass
> 236                 # TODO: Pick default implementation
> 237                 for impl in python{2_7,2_6,2_5,3_2,2_1} pypy{1_8,1_7} jython2_5; do

Looks like 2_1 should be 3_1

minor suggestion:
s/_python-distutils-ng_run_for_all_impls/_python-distutils-ng_run_for_each_impl/g

-- 

  Sergei

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[gentoo-dev] Re: [gentoo-python] New eclass for Python

[Mike Gilbert]

On Tue, Feb 28, 2012 at 4:13 PM, Krzysztof Pawlik <nelchael@gentoo.org> wrote:
> Hello,
>
> After some work during weekend on Python packages I've decided to start a
> rewrite of Python/distutils eclass for installing Python packages. My main goal
> was simplicity and functionality similar to ruby-ng.eclass (thanks Ruby team for
> your great work!). Python team members already contributed comments and
> suggestions and helped me to make the eclass better, thank you!
>
> Highlights:
>  - *SIMPLE*next
>  - uses PYTHON_TARGETS use-expand (no more python-updater, whoooo!)
>  - EAPI4 required, uses REQUIRED_USE
>  - <400 lines of code including documentation
>  - should work for >95% of packages (my educated guess)
>  - did I mention it's *SIMPLE*?
>  - easy to maintain & read so it's also easy to use
>
> Important thing: I'm not aiming at having 100% functionality of current
> python.eclass+distutils.eclass in the new one, I think that simplicity is more
> important that supporting every possible, obscure case that's out there.
>
> I'm attaching the eclass itself and two ebuilds using it, code is also available
> in my overlay at http://git.overlays.gentoo.org/gitweb/?p=dev/nelchael.git;a=summary
>
> If there are no objections then during the weekend (March 3, 4) I will add this
> to portage (after finishing remaining TODO items, PyPy requires 4G of RAM(!!)).
>
> --
> Krzysztof Pawlik  <nelchael at gentoo.org>  key id: 0xF6A80E46
> desktop-misc, java, vim, kernel, python, apache...
>

# Phase function: src_unpack
python-distutils-ng_src_unpack() {
	[[ "${PYTHON_OPTIONAL}" = "yes" ]] && { use python || return; }

	if type python_unpack &> /dev/null; then
		# This should not run anything specific to any single Python
		# implementation, keep it generic:
		python_unpack_all
	else
		[[ -n ${A} ]] && unpack ${A}
	fi
}

I think you meant to write "if type python_unpack_all".

More to the point, I don't actually understand why this function
exists. It doesn't actually do anything that default_src_unpack does
not do already. Exporting it will clobber any vcs eclasses if the
inherit order is wrong.

Re: [gentoo-dev] New eclass for Python

["Paweł Hajdan, Jr."]

[-- Attachment #1: Type: text/plain, Size: 901 bytes --]

On 2/28/12 10:13 PM, Krzysztof Pawlik wrote:
> Highlights:
>  - <400 lines of code including documentation
>  - should work for >95% of packages (my educated guess)
>  - did I mention it's *SIMPLE*?
>  - easy to maintain & read so it's also easy to use

This is awesome! Compare that to over 3000 LOC of python.eclass. :)

> Important thing: I'm not aiming at having 100% functionality of current
> python.eclass+distutils.eclass in the new one, I think that simplicity is more
> important that supporting every possible, obscure case that's out there.

Exactly.

> If there are no objections then during the weekend (March 3, 4) I will add this
> to portage (after finishing remaining TODO items, PyPy requires 4G of RAM(!!)).

I second this so much (didn't review the eclass in detail, but it's
obviously better than python.eclass).

What's the plan to retire python.eclass?


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 203 bytes --]

[gentoo-dev] Re: [gentoo-python] New eclass for Python

[Dirkjan Ochtman]

On Tue, Feb 28, 2012 at 22:13, Krzysztof Pawlik <nelchael@gentoo.org> wrote:
> If there are no objections then during the weekend (March 3, 4) I will add this
> to portage (after finishing remaining TODO items, PyPy requires 4G of RAM(!!)).

Can we perhaps just name it python-r2 rather than python-distutils-ng?
Seems descriptive enough...

Cheers,

Dirkjan

Re: [gentoo-dev] New eclass for Python

[Fabian Groffen]

[-- Attachment #1: Type: text/plain, Size: 408 bytes --]

On 28-02-2012 22:13:36 +0100, Krzysztof Pawlik wrote:
[good stuff]

Much appreciated!

From 2nd example ebuild:
> python_install_all() {
> 	rm -f "${D}/usr/bin"/*.py || die

s/D/ED/ here for Prefix :)

I haven't checked the eclass in detail, but did you intend to make it
Prefix aware at all?  I guess someone from us should test your work.


-- 
Fabian Groffen
Gentoo on a different level

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [gentoo-dev] New eclass for Python

[Krzysztof Pawlik]

[-- Attachment #1: Type: text/plain, Size: 731 bytes --]

On 29/02/12 09:17, Fabian Groffen wrote:
> On 28-02-2012 22:13:36 +0100, Krzysztof Pawlik wrote:
> [good stuff]
> 
> Much appreciated!
> 
> From 2nd example ebuild:
>> python_install_all() {
>> 	rm -f "${D}/usr/bin"/*.py || die
> 
> s/D/ED/ here for Prefix :)
> 
> I haven't checked the eclass in detail, but did you intend to make it
> Prefix aware at all?  I guess someone from us should test your work.

Yes, please - take a look at my overlay (it always has the latest version) and
check for prefix related issues. Take a look also at xlwt and xlrd in dev-python
in the same overlay.

-- 
Krzysztof Pawlik  <nelchael at gentoo.org>  key id: 0xF6A80E46
desktop-misc, java, vim, kernel, python, apache...


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 554 bytes --]

Re: [gentoo-dev] New eclass for Python

[Krzysztof Pawlik]

[-- Attachment #1: Type: text/plain, Size: 820 bytes --]

On 29/02/12 04:21, Sergei Trofimovich wrote:
> On Tue, 28 Feb 2012 22:13:36 +0100
> Krzysztof Pawlik <nelchael@gentoo.org> wrote:
> 
>> I'm attaching the eclass itself and two ebuilds using it, code is also available
>> in my overlay at http://git.overlays.gentoo.org/gitweb/?p=dev/nelchael.git;a=summary
> Nice!
> 
>> eclass/python-distutils-ng.eclass
>> 236                 # TODO: Pick default implementation
>> 237                 for impl in python{2_7,2_6,2_5,3_2,2_1} pypy{1_8,1_7} jython2_5; do
> 
> Looks like 2_1 should be 3_1

Yes! Thank you.

> minor suggestion:
> s/_python-distutils-ng_run_for_all_impls/_python-distutils-ng_run_for_each_impl/g

LGTM, changed.

-- 
Krzysztof Pawlik  <nelchael at gentoo.org>  key id: 0xF6A80E46
desktop-misc, java, vim, kernel, python, apache...


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 554 bytes --]

Re: [gentoo-dev] Re: [gentoo-python] New eclass for Python

[Krzysztof Pawlik]

[-- Attachment #1: Type: text/plain, Size: 2489 bytes --]

On 29/02/12 06:13, Mike Gilbert wrote:
> On Tue, Feb 28, 2012 at 4:13 PM, Krzysztof Pawlik <nelchael@gentoo.org> wrote:
>> Hello,
>>
>> After some work during weekend on Python packages I've decided to start a
>> rewrite of Python/distutils eclass for installing Python packages. My main goal
>> was simplicity and functionality similar to ruby-ng.eclass (thanks Ruby team for
>> your great work!). Python team members already contributed comments and
>> suggestions and helped me to make the eclass better, thank you!
>>
>> Highlights:
>>  - *SIMPLE*next
>>  - uses PYTHON_TARGETS use-expand (no more python-updater, whoooo!)
>>  - EAPI4 required, uses REQUIRED_USE
>>  - <400 lines of code including documentation
>>  - should work for >95% of packages (my educated guess)
>>  - did I mention it's *SIMPLE*?
>>  - easy to maintain & read so it's also easy to use
>>
>> Important thing: I'm not aiming at having 100% functionality of current
>> python.eclass+distutils.eclass in the new one, I think that simplicity is more
>> important that supporting every possible, obscure case that's out there.
>>
>> I'm attaching the eclass itself and two ebuilds using it, code is also available
>> in my overlay at http://git.overlays.gentoo.org/gitweb/?p=dev/nelchael.git;a=summary
>>
>> If there are no objections then during the weekend (March 3, 4) I will add this
>> to portage (after finishing remaining TODO items, PyPy requires 4G of RAM(!!)).
>>
>> --
>> Krzysztof Pawlik  <nelchael at gentoo.org>  key id: 0xF6A80E46
>> desktop-misc, java, vim, kernel, python, apache...
>>
> 
> # Phase function: src_unpack
> python-distutils-ng_src_unpack() {
> 	[[ "${PYTHON_OPTIONAL}" = "yes" ]] && { use python || return; }
> 
> 	if type python_unpack &> /dev/null; then
> 		# This should not run anything specific to any single Python
> 		# implementation, keep it generic:
> 		python_unpack_all
> 	else
> 		[[ -n ${A} ]] && unpack ${A}
> 	fi
> }
> 
> I think you meant to write "if type python_unpack_all".
> 
> More to the point, I don't actually understand why this function
> exists. It doesn't actually do anything that default_src_unpack does
> not do already. Exporting it will clobber any vcs eclasses if the
> inherit order is wrong.

You're right, I've killed the python-distutils-ng_src_unpack() function.

-- 
Krzysztof Pawlik  <nelchael at gentoo.org>  key id: 0xF6A80E46
desktop-misc, java, vim, kernel, python, apache...


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 554 bytes --]

Re: [gentoo-dev] New eclass for Python

[Krzysztof Pawlik]

[-- Attachment #1: Type: text/plain, Size: 1697 bytes --]

On 29/02/12 08:49, "Paweł Hajdan, Jr." wrote:
> On 2/28/12 10:13 PM, Krzysztof Pawlik wrote:
>> Highlights:
>>  - <400 lines of code including documentation
>>  - should work for >95% of packages (my educated guess)
>>  - did I mention it's *SIMPLE*?
>>  - easy to maintain & read so it's also easy to use
> 
> This is awesome! Compare that to over 3000 LOC of python.eclass. :)

Count distutils.eclass too:

$ wc -l python-distutils-ng.eclass python.eclass distutils.eclass
   364 python-distutils-ng.eclass
  3168 python.eclass
   592 distutils.eclass

>> Important thing: I'm not aiming at having 100% functionality of current
>> python.eclass+distutils.eclass in the new one, I think that simplicity is more
>> important that supporting every possible, obscure case that's out there.
> 
> Exactly.
> 
>> If there are no objections then during the weekend (March 3, 4) I will add this
>> to portage (after finishing remaining TODO items, PyPy requires 4G of RAM(!!)).
> 
> I second this so much (didn't review the eclass in detail, but it's
> obviously better than python.eclass).

Thank you :)

> What's the plan to retire python.eclass?

I'm not entirely sure - there's no pressure to migrate off it, both can coexist
happily. I don't think it can be retired until be have all functionality from
python.eclass+distutils.eclass somewhere (especially tests!).

If the package is distutils-based then migration should be pretty painless and
new packages (or version/revision bumps) can start using it as soon as it hits
portage.

-- 
Krzysztof Pawlik  <nelchael at gentoo.org>  key id: 0xF6A80E46
desktop-misc, java, vim, kernel, python, apache...


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 554 bytes --]

Re: [gentoo-dev] Re: [gentoo-python] New eclass for Python

[Krzysztof Pawlik]

[-- Attachment #1: Type: text/plain, Size: 788 bytes --]

On 29/02/12 09:11, Dirkjan Ochtman wrote:
> On Tue, Feb 28, 2012 at 22:13, Krzysztof Pawlik <nelchael@gentoo.org> wrote:
>> If there are no objections then during the weekend (March 3, 4) I will add this
>> to portage (after finishing remaining TODO items, PyPy requires 4G of RAM(!!)).
> 
> Can we perhaps just name it python-r2 rather than python-distutils-ng?
> Seems descriptive enough...

Yes and no - it's named python-distutils because main focus was this
combination, it can work for other cases too, it's just that it combines
functionality from both old eclasses. Besides I like the name, but I can be
convinced to name it python-r2.eclass.

-- 
Krzysztof Pawlik  <nelchael at gentoo.org>  key id: 0xF6A80E46
desktop-misc, java, vim, kernel, python, apache...


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 554 bytes --]

Re: [gentoo-dev] New eclass for Python

[Jeroen Roovers]

On Wed, 29 Feb 2012 18:38:22 +0100
Krzysztof Pawlik <nelchael@gentoo.org> wrote:

> On 29/02/12 08:49, "Paweł Hajdan, Jr." wrote:
> > This is awesome! Compare that to over 3000 LOC of python.eclass. :)
> 
> Count distutils.eclass too:
> 
> $ wc -l python-distutils-ng.eclass python.eclass distutils.eclass
>    364 python-distutils-ng.eclass
>   3168 python.eclass
>    592 distutils.eclass

cvs/gentoo-x86/eclass $ cloc --by-file \
 --force-lang="Bourne Again Shell",eclass python.eclass \
 distutils .eclass python-distutils-ng.eclass
 3 text files.
 3 unique files.
 0 files ignored.

http://cloc.sourceforge.net v 1.55  T=0.5 s (6.0 files/s, 8294.0
lines/s)
-----------------------------------------------
File                       blank comment  code
-----------------------------------------------
python.eclass                396     288  2494
distutils.eclass             111      86   395
python-distutils-ng.eclass    51     110   216
-----------------------------------------------
SUM:                         558     484  3105
-----------------------------------------------


     jer

Re: [gentoo-dev] Re: [gentoo-python] New eclass for Python

[Mike Gilbert]

On Wed, Feb 29, 2012 at 12:39 PM, Krzysztof Pawlik <nelchael@gentoo.org> wrote:
> On 29/02/12 09:11, Dirkjan Ochtman wrote:
>> On Tue, Feb 28, 2012 at 22:13, Krzysztof Pawlik <nelchael@gentoo.org> wrote:
>>> If there are no objections then during the weekend (March 3, 4) I will add this
>>> to portage (after finishing remaining TODO items, PyPy requires 4G of RAM(!!)).
>>
>> Can we perhaps just name it python-r2 rather than python-distutils-ng?
>> Seems descriptive enough...
>
> Yes and no - it's named python-distutils because main focus was this
> combination, it can work for other cases too, it's just that it combines
> functionality from both old eclasses. Besides I like the name, but I can be
> convinced to name it python-r2.eclass.
>

The distutils-based packages are a good starting point. The name is a
good fit for that.

Looking forward, I think it might make sense to move some of the
functionality to a "core" python eclass that would just be for utility
functions -- similar to the python/distutils split we have now. This
would be used in ebuilds that are not primarily based around distutils
(setup.py).

Re: [gentoo-dev] New eclass for Python

[Alexandre Rostovtsev]

On Tue, 2012-02-28 at 22:13 +0100, Krzysztof Pawlik wrote:
> Hello,
> 
> After some work during weekend on Python packages I've decided to start a
> rewrite of Python/distutils eclass for installing Python packages. My main goal
> was simplicity and functionality similar to ruby-ng.eclass (thanks Ruby team for
> your great work!). Python team members already contributed comments and
> suggestions and helped me to make the eclass better, thank you!
> 
> Highlights:
>  - *SIMPLE*next
>  - uses PYTHON_TARGETS use-expand (no more python-updater, whoooo!)
>  - EAPI4 required, uses REQUIRED_USE
>  - <400 lines of code including documentation
>  - should work for >95% of packages (my educated guess)
>  - did I mention it's *SIMPLE*?
>  - easy to maintain & read so it's also easy to use
> 
> Important thing: I'm not aiming at having 100% functionality of current
> python.eclass+distutils.eclass in the new one, I think that simplicity is more
> important that supporting every possible, obscure case that's out there.
> 
> I'm attaching the eclass itself and two ebuilds using it, code is also available
> in my overlay at http://git.overlays.gentoo.org/gitweb/?p=dev/nelchael.git;a=summary
> 
> If there are no objections then during the weekend (March 3, 4) I will add this
> to portage (after finishing remaining TODO items, PyPy requires 4G of RAM(!!)).

The proposed eclass omits three features from python.eclass which are
heavily used in the gnome stack.

First, it does not set EPYTHON, which is a problem for the many packages
whose build systems execute /usr/bin/python and assume that it's a
generic python2 or the same version of python2.x for which the package
is being built.

Second, there doesn't seem to be any support for packages that do not
install in python's site-packages and do not allow multiple python ABIs.
If I have, for example, a package that installs python modules
in /usr/lib/appname or /usr/share/appname, how can I specify that
PYTHON_TARGETS="python2.6" or "python2.7" or "python3.2" is allowed, but
something like PYTHON_TARGETS="python2.7 python3.2" is not?

Third, python-distutils-ng_doscript() installs only one file at a time
(no built-in support for multiple files at a time or for recursing
through a directory tree), installs it only in /usr/bin (a package might
want it in e.g. /usr/libexec or /usr/share/appname/scripts), and always
creates scriptname-IMPLEMENTATION (polluting tab-completion in /usr/bin
for the common case of programs that do not support multiple python
ABIs). As a result, it is far less useful than python.eclass's
python_convert_shebangs().

-Alexandre.

Re: [gentoo-dev] New eclass for Python

[Krzysztof Pawlik]

[-- Attachment #1: Type: text/plain, Size: 4224 bytes --]

On 29/02/12 20:51, Alexandre Rostovtsev wrote:
> On Tue, 2012-02-28 at 22:13 +0100, Krzysztof Pawlik wrote:
>> Hello,
>>
>> After some work during weekend on Python packages I've decided to start a
>> rewrite of Python/distutils eclass for installing Python packages. My main goal
>> was simplicity and functionality similar to ruby-ng.eclass (thanks Ruby team for
>> your great work!). Python team members already contributed comments and
>> suggestions and helped me to make the eclass better, thank you!
>>
>> Highlights:
>>  - *SIMPLE*next
>>  - uses PYTHON_TARGETS use-expand (no more python-updater, whoooo!)
>>  - EAPI4 required, uses REQUIRED_USE
>>  - <400 lines of code including documentation
>>  - should work for >95% of packages (my educated guess)
>>  - did I mention it's *SIMPLE*?
>>  - easy to maintain & read so it's also easy to use
>>
>> Important thing: I'm not aiming at having 100% functionality of current
>> python.eclass+distutils.eclass in the new one, I think that simplicity is more
>> important that supporting every possible, obscure case that's out there.
>>
>> I'm attaching the eclass itself and two ebuilds using it, code is also available
>> in my overlay at http://git.overlays.gentoo.org/gitweb/?p=dev/nelchael.git;a=summary
>>
>> If there are no objections then during the weekend (March 3, 4) I will add this
>> to portage (after finishing remaining TODO items, PyPy requires 4G of RAM(!!)).
> 
> The proposed eclass omits three features from python.eclass which are
> heavily used in the gnome stack.

Correct me if I'm wrong, but Gnome doesn't use standard distutils?

Regardless of this eclass you can still use python.eclass, it not going anywhere
anytime soon (just like I wrote in one of previous e-mails).

> First, it does not set EPYTHON, which is a problem for the many packages
> whose build systems execute /usr/bin/python and assume that it's a
> generic python2 or the same version of python2.x for which the package
> is being built.

Setting EPYTHON is not a problem -- just another variable.

> Second, there doesn't seem to be any support for packages that do not
> install in python's site-packages and do not allow multiple python ABIs.
> If I have, for example, a package that installs python modules
> in /usr/lib/appname or /usr/share/appname, how can I specify that
> PYTHON_TARGETS="python2.6" or "python2.7" or "python3.2" is allowed, but
> something like PYTHON_TARGETS="python2.7 python3.2" is not?

You're correct, note that I've stressed that this eclass is mainly for
distutils-based packages. I'm not using Gnome, so can you provide some package
examples that I can look at?

<personal opinion>
If package decides to use given language then please, please play by the rules
set by the rest of world (Ruby -> gems, Python -> distutils, Perl -> CPAN, PHP
-> PEAR).

I don't like installing Python code outside of site-packages, the only exception
to that rule is portage (at least for now).
</personal opinion>

> Third, python-distutils-ng_doscript() installs only one file at a time
> (no built-in support for multiple files at a time or for recursing
> through a directory tree),

Yes, that why bash has `for' loop ;)

> installs it only in /usr/bin (a package might
> want it in e.g. /usr/libexec or /usr/share/appname/scripts)

Yes, I might add --destdir (or something like that) later on.

> and always
> creates scriptname-IMPLEMENTATION (polluting tab-completion in /usr/bin
> for the common case of programs that do not support multiple python
> ABIs). As a result, it is far less useful than python.eclass's
> python_convert_shebangs().

Yes, that "pollutes" this name space, but I'm not buying this argument. Do you
know how CVS and .svn "pollute" my tab-completion list? I even set FIGNORE so I
can live with it.

I'd be happy to hear how to solve this - what prefix or suffix to use? One way
would be quite trivial: if only one implementation is enabled do not create
script-${impl}, go with single file, does that sound good?

-- 
Krzysztof Pawlik  <nelchael at gentoo.org>  key id: 0xF6A80E46
desktop-misc, java, vim, kernel, python, apache...


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 554 bytes --]

Re: [gentoo-dev] New eclass for Python

[Andreas K. Huettel]

[-- Attachment #1: Type: Text/Plain, Size: 1630 bytes --]

Am Mittwoch 29 Februar 2012, 21:24:49 schrieb Krzysztof Pawlik:
> > Second, there doesn't seem to be any support for packages that do not
> > install in python's site-packages and do not allow multiple python ABIs.
> > If I have, for example, a package that installs python modules
> > in /usr/lib/appname or /usr/share/appname, how can I specify that
> > PYTHON_TARGETS="python2.6" or "python2.7" or "python3.2" is allowed, but
> > something like PYTHON_TARGETS="python2.7 python3.2" is not?
> 
> You're correct, note that I've stressed that this eclass is mainly for
> distutils-based packages. I'm not using Gnome, so can you provide some
> package examples that I can look at?
> 
> <personal opinion>
> If package decides to use given language then please, please play by the
> rules set by the rest of world (Ruby -> gems, Python -> distutils, Perl ->
> CPAN, PHP -> PEAR).
> 
> I don't like installing Python code outside of site-packages, the only
> exception to that rule is portage (at least for now).
> </personal opinion>

We will hit the same problem with KDE (actually we already hit it): it has 
various types of scripting support, and each installs a KDE library linked to 
whatever language interpreter.

(Now, that library- is it a Python/Ruby library or a KDE library? Because it 
is at the proper place for KDE stuff :)

It's not just about calling an external language but also about embedding the 
interpreter for in-app scripting... and KDE rather heavily relies on python.

-- 

Andreas K. Huettel
Gentoo Linux developer 
dilfridge@gentoo.org
http://www.akhuettel.de/


[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] New eclass for Python

[Krzysztof Pawlik]

[-- Attachment #1: Type: text/plain, Size: 1970 bytes --]

On 29/02/12 22:08, Andreas K. Huettel wrote:
> Am Mittwoch 29 Februar 2012, 21:24:49 schrieb Krzysztof Pawlik:
>>> Second, there doesn't seem to be any support for packages that do not
>>> install in python's site-packages and do not allow multiple python ABIs.
>>> If I have, for example, a package that installs python modules
>>> in /usr/lib/appname or /usr/share/appname, how can I specify that
>>> PYTHON_TARGETS="python2.6" or "python2.7" or "python3.2" is allowed, but
>>> something like PYTHON_TARGETS="python2.7 python3.2" is not?
>>
>> You're correct, note that I've stressed that this eclass is mainly for
>> distutils-based packages. I'm not using Gnome, so can you provide some
>> package examples that I can look at?
>>
>> <personal opinion>
>> If package decides to use given language then please, please play by the
>> rules set by the rest of world (Ruby -> gems, Python -> distutils, Perl ->
>> CPAN, PHP -> PEAR).
>>
>> I don't like installing Python code outside of site-packages, the only
>> exception to that rule is portage (at least for now).
>> </personal opinion>
> 
> We will hit the same problem with KDE (actually we already hit it): it has 
> various types of scripting support, and each installs a KDE library linked to 
> whatever language interpreter.
> 
> (Now, that library- is it a Python/Ruby library or a KDE library? Because it 
> is at the proper place for KDE stuff :)
> 
> It's not just about calling an external language but also about embedding the 
> interpreter for in-app scripting... and KDE rather heavily relies on python.

I agree with last sentence - it's about embedding Python/Ruby/Foo, *this eclass
is about installing packages for Python*. Compiling against an implementation is
something completely different -- for example try building against Jython.

-- 
Krzysztof Pawlik  <nelchael at gentoo.org>  key id: 0xF6A80E46
desktop-misc, java, vim, kernel, python, apache...


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 554 bytes --]

Re: [gentoo-dev] New eclass for Python

[Alexandre Rostovtsev]

On Wed, 2012-02-29 at 21:24 +0100, Krzysztof Pawlik wrote:
> On 29/02/12 20:51, Alexandre Rostovtsev wrote:
> > The proposed eclass omits three features from python.eclass which are
> > heavily used in the gnome stack.
> 
> Correct me if I'm wrong, but Gnome doesn't use standard distutils?

Gnome is mostly written in C and therefore uses standard autotools :)

> > Second, there doesn't seem to be any support for packages that do not
> > install in python's site-packages and do not allow multiple python ABIs.
> > If I have, for example, a package that installs python modules
> > in /usr/lib/appname or /usr/share/appname, how can I specify that
> > PYTHON_TARGETS="python2.6" or "python2.7" or "python3.2" is allowed, but
> > something like PYTHON_TARGETS="python2.7 python3.2" is not?
> 
> You're correct, note that I've stressed that this eclass is mainly for
> distutils-based packages. I'm not using Gnome, so can you provide some package
> examples that I can look at?
> 
> <personal opinion>
> If package decides to use given language then please, please play by the rules
> set by the rest of world (Ruby -> gems, Python -> distutils, Perl -> CPAN, PHP
> -> PEAR).
> 
> I don't like installing Python code outside of site-packages, the only exception
> to that rule is portage (at least for now).
> </personal opinion>

Some non-python packages allow python-based plugins. Obviously these
plugins live in the package's plugin directory (not in python's
site-packages) and use the package's main build system (not distutils),
and multiple python ABIs cannot be supported because that would result
in colliding plugins. Typical examples are app-editors/gedit,
media-gfx/gimp, media-sound/rhythmbox, or media-video/totem.

Some packages install a C library that links to a specific version of
libpython or that defines a particular python version string at compile
time, making it impossible to use the package with multiple python ABIs.
Examples I know are dev-libs/libpeas and dev-python/nautilus-python.

And then there are packages which could support e.g. multiple python2
ABIs in theory, but doing so in practice would require a fair bit of
patching, taking substantial effort with no real benefit for end users.
An example that springs to mind here is gnome-extra/zeitgeist.

> I'd be happy to hear how to solve this - what prefix or suffix to use? One way
> would be quite trivial: if only one implementation is enabled do not create
> script-${impl}, go with single file, does that sound good?

That would be the ideal solution.

-Alexandre.

Re: [gentoo-dev] New eclass for Python

[Krzysztof Pawlik]

[-- Attachment #1: Type: text/plain, Size: 3380 bytes --]

On 29/02/12 22:57, Alexandre Rostovtsev wrote:
> On Wed, 2012-02-29 at 21:24 +0100, Krzysztof Pawlik wrote:
>> On 29/02/12 20:51, Alexandre Rostovtsev wrote:
>>> The proposed eclass omits three features from python.eclass which are
>>> heavily used in the gnome stack.
>>
>> Correct me if I'm wrong, but Gnome doesn't use standard distutils?
> 
> Gnome is mostly written in C and therefore uses standard autotools :)

Ok, thank you for this information.

>>> Second, there doesn't seem to be any support for packages that do not
>>> install in python's site-packages and do not allow multiple python ABIs.
>>> If I have, for example, a package that installs python modules
>>> in /usr/lib/appname or /usr/share/appname, how can I specify that
>>> PYTHON_TARGETS="python2.6" or "python2.7" or "python3.2" is allowed, but
>>> something like PYTHON_TARGETS="python2.7 python3.2" is not?
>>
>> You're correct, note that I've stressed that this eclass is mainly for
>> distutils-based packages. I'm not using Gnome, so can you provide some package
>> examples that I can look at?
>>
>> <personal opinion>
>> If package decides to use given language then please, please play by the rules
>> set by the rest of world (Ruby -> gems, Python -> distutils, Perl -> CPAN, PHP
>> -> PEAR).
>>
>> I don't like installing Python code outside of site-packages, the only exception
>> to that rule is portage (at least for now).
>> </personal opinion>
> 
> Some non-python packages allow python-based plugins. Obviously these
> plugins live in the package's plugin directory (not in python's
> site-packages) and use the package's main build system (not distutils),
> and multiple python ABIs cannot be supported because that would result
> in colliding plugins. Typical examples are app-editors/gedit,
> media-gfx/gimp, media-sound/rhythmbox, or media-video/totem.
> 
> Some packages install a C library that links to a specific version of
> libpython or that defines a particular python version string at compile
> time, making it impossible to use the package with multiple python ABIs.
> Examples I know are dev-libs/libpeas and dev-python/nautilus-python.
> 
> And then there are packages which could support e.g. multiple python2
> ABIs in theory, but doing so in practice would require a fair bit of
> patching, taking substantial effort with no real benefit for end users.
> An example that springs to mind here is gnome-extra/zeitgeist.

I see - so it's the same case as with KDE (like Andreas wrote) - it's actually
not a "python package" but rather an embedded Python. That's very different case
than I'm targeting with this eclass (at least for now).

>> I'd be happy to hear how to solve this - what prefix or suffix to use? One way
>> would be quite trivial: if only one implementation is enabled do not create
>> script-${impl}, go with single file, does that sound good?
> 
> That would be the ideal solution.

Ok, I've implemented this solution, now if one implementation is enabled there
will be only one script, no foo-IMPL mangling. I've added also ability to
install the script in other directory than /usr/bin. Hope this solves this case.

Thank you for your comments and suggestions :)

-- 
Krzysztof Pawlik  <nelchael at gentoo.org>  key id: 0xF6A80E46
desktop-misc, java, vim, kernel, python, apache...


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 554 bytes --]

[gentoo-dev] Re: [gentoo-python] New eclass for Python

[Arfrever Frehtes Taifersar Arahesis]

[-- Attachment #1: Type: Text/Plain, Size: 241 bytes --]

2012-02-28 22:13:36 Krzysztof Pawlik napisał(a):
>  - uses PYTHON_TARGETS use-expand

You cannot painlessly use USE flags for this purpose in gentoo-x86 without support for use.unsatisfiable.

-- 
Arfrever Frehtes Taifersar Arahesis

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] Re: [gentoo-python] New eclass for Python

[Zac Medico]

On 03/02/2012 11:59 PM, Arfrever Frehtes Taifersar Arahesis wrote:
> 2012-02-28 22:13:36 Krzysztof Pawlik napisał(a):
>>  - uses PYTHON_TARGETS use-expand
> 
> You cannot painlessly use USE flags for this purpose in gentoo-x86 without support for use.unsatisfiable.

Maybe we should change the repoman behavior so that it's just a warning
when a USE-conditional dep is only satisfiable by a masked package. The
probably makes more sense given that emerge --autounmask is enabled by
default these days.
-- 
Thanks,
Zac

Re: [gentoo-dev] New eclass for Python

[Krzysztof Pawlik]

[-- Attachment #1: Type: text/plain, Size: 481 bytes --]

On 28/02/12 22:13, Krzysztof Pawlik wrote:
> If there are no objections then during the weekend (March 3, 4) I will add this
> to portage (after finishing remaining TODO items, PyPy requires 4G of RAM(!!)).

Hello,

Slightly late due to Real Life™ but finally it's in the main tree :)

(and yes - I've tested it with pypy - works as expected :)

-- 
Krzysztof Pawlik  <nelchael at gentoo.org>  key id: 0xF6A80E46
desktop-misc, java, vim, kernel, python, apache...


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 554 bytes --]

Re: [gentoo-dev] New eclass for Python

[Luca Barbato]

On 3/25/12 11:56 AM, Krzysztof Pawlik wrote:
> On 28/02/12 22:13, Krzysztof Pawlik wrote:
>> If there are no objections then during the weekend (March 3, 4) I will add this
>> to portage (after finishing remaining TODO items, PyPy requires 4G of RAM(!!)).
>
> Hello,
>
> Slightly late due to Real Life™ but finally it's in the main tree :)
>
> (and yes - I've tested it with pypy - works as expected :)

Wonderful =)

I'd split away the python eclass to have a trimmed down eclass specific 
for the implementations btw. (the mess with the eselect-python shown we 
could fix that better)

lu

Re: [gentoo-dev] New eclass for Python

[justin]

[-- Attachment #1: Type: text/plain, Size: 805 bytes --]

On 25/03/12 20:56, Krzysztof Pawlik wrote:
> On 28/02/12 22:13, Krzysztof Pawlik wrote:
>> If there are no objections then during the weekend (March 3, 4) I will add this
>> to portage (after finishing remaining TODO items, PyPy requires 4G of RAM(!!)).
> 
> Hello,
> 
> Slightly late due to Real Life™ but finally it's in the main tree :)
> 
> (and yes - I've tested it with pypy - works as expected :)
> 

Hi,

is there any documentation beside the man page somewhere?

I tried to port some ebuilds but as soon I set

PYTHON_COMPAT="python2_7 python2_6 python2_5 pypy1_8"

inherit python-distutils-ng

I get

  REQUIRED_USE: USE flag 'python_targets_python3_1' is not in IUSE


Did I do something wrong, or is there something not straight in the eclass?

Thanks justin


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: [gentoo-dev] New eclass for Python

[justin]

[-- Attachment #1: Type: text/plain, Size: 490 bytes --]

On 25/03/12 20:56, Krzysztof Pawlik wrote:
> On 28/02/12 22:13, Krzysztof Pawlik wrote:
>> If there are no objections then during the weekend (March 3, 4) I will add this
>> to portage (after finishing remaining TODO items, PyPy requires 4G of RAM(!!)).
> 
> Hello,
> 
> Slightly late due to Real Life™ but finally it's in the main tree :)
> 
> (and yes - I've tested it with pypy - works as expected :)
> 

Hi,

is it okay if I start fixing things for prefix?

jusitn


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: [gentoo-dev] New eclass for Python

[Krzysztof Pawlik]

[-- Attachment #1: Type: text/plain, Size: 770 bytes --]

On 26/03/12 09:21, justin wrote:
> On 25/03/12 20:56, Krzysztof Pawlik wrote:
>> On 28/02/12 22:13, Krzysztof Pawlik wrote:
>>> If there are no objections then during the weekend (March 3, 4) I will add this
>>> to portage (after finishing remaining TODO items, PyPy requires 4G of RAM(!!)).
>>
>> Hello,
>>
>> Slightly late due to Real Life™ but finally it's in the main tree :)
>>
>> (and yes - I've tested it with pypy - works as expected :)
>>
> 
> Hi,
> 
> is it okay if I start fixing things for prefix?

Yes, by all means - please send the diff to this thread.

> jusitn
  ^^^^^^ your e-mail says "justin" -- typo?

-- 
Krzysztof Pawlik  <nelchael at gentoo.org>  key id: 0xF6A80E46
desktop-misc, java, vim, kernel, python, apache...


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 554 bytes --]

Re: [gentoo-dev] New eclass for Python

[Krzysztof Pawlik]

[-- Attachment #1: Type: text/plain, Size: 1231 bytes --]

On 26/03/12 09:20, justin wrote:
> On 25/03/12 20:56, Krzysztof Pawlik wrote:
>> On 28/02/12 22:13, Krzysztof Pawlik wrote:
>>> If there are no objections then during the weekend (March 3, 4) I will add this
>>> to portage (after finishing remaining TODO items, PyPy requires 4G of RAM(!!)).
>>
>> Hello,
>>
>> Slightly late due to Real Life™ but finally it's in the main tree :)
>>
>> (and yes - I've tested it with pypy - works as expected :)
>>
> 
> Hi,
> 
> is there any documentation beside the man page somewhere?

No.

> I tried to port some ebuilds but as soon I set
> 
> PYTHON_COMPAT="python2_7 python2_6 python2_5 pypy1_8"
> 
> inherit python-distutils-ng
> 
> I get
> 
>   REQUIRED_USE: USE flag 'python_targets_python3_1' is not in IUSE
> 
> Did I do something wrong, or is there something not straight in the eclass?

Can you send me the whole ebuild off-list?

There are two ebuilds using the eclass that I've used as tests:
http://git.overlays.gentoo.org/gitweb/?p=dev/nelchael.git;a=tree;f=dev-python;h=f1a8e00e3e6df33806d8972c8898f1187163bd3d;hb=HEAD

-- 
Krzysztof Pawlik  <nelchael at gentoo.org>  key id: 0xF6A80E46
desktop-misc, java, vim, kernel, python, apache...


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 554 bytes --]

Re: [gentoo-dev] New eclass for Python

[Krzysztof Pawlik]

[-- Attachment #1: Type: text/plain, Size: 2565 bytes --]

On 26/03/12 18:11, Krzysztof Pawlik wrote:
> On 26/03/12 09:20, justin wrote:
>> On 25/03/12 20:56, Krzysztof Pawlik wrote:
>>> On 28/02/12 22:13, Krzysztof Pawlik wrote:
>>>> If there are no objections then during the weekend (March 3, 4) I will add this
>>>> to portage (after finishing remaining TODO items, PyPy requires 4G of RAM(!!)).
>>>
>>> Hello,
>>>
>>> Slightly late due to Real Life™ but finally it's in the main tree :)
>>>
>>> (and yes - I've tested it with pypy - works as expected :)
>>>
>>
>> Hi,
>>
>> is there any documentation beside the man page somewhere?
> 
> No.
> 
>> I tried to port some ebuilds but as soon I set
>>
>> PYTHON_COMPAT="python2_7 python2_6 python2_5 pypy1_8"
>>
>> inherit python-distutils-ng
>>
>> I get
>>
>>   REQUIRED_USE: USE flag 'python_targets_python3_1' is not in IUSE
>>
>> Did I do something wrong, or is there something not straight in the eclass?
> 
> Can you send me the whole ebuild off-list?
> 
> There are two ebuilds using the eclass that I've used as tests:
> http://git.overlays.gentoo.org/gitweb/?p=dev/nelchael.git;a=tree;f=dev-python;h=f1a8e00e3e6df33806d8972c8898f1187163bd3d;hb=HEAD

Ok, found a bug: REQUIRED_USE can't contain elements not in USE, so if you
excluded python3_1 from PYTHON_COMPAT it didn't appear in IUSE too ->
REQUIRED_USE contained invalid value. Fixed by below patch:

nelchael@s-lappy ~/.../gentoo-x86/eclass$ cvs diff
Index: python-distutils-ng.eclass
===================================================================
RCS file: /var/cvsroot/gentoo-x86/eclass/python-distutils-ng.eclass,v
retrieving revision 1.2
diff -u -r1.2 python-distutils-ng.eclass
--- python-distutils-ng.eclass  26 Mar 2012 06:12:53 -0000      1.2
+++ python-distutils-ng.eclass  26 Mar 2012 16:20:52 -0000
@@ -105,11 +105,11 @@
        esac
 }

-required_use_str=" || (
-       python_targets_python2_5 python_targets_python2_6 python_targets_python2_7
-       python_targets_python3_1 python_targets_python3_2
-       python_targets_jython2_5
-       python_targets_pypy1_7 python_targets_pypy1_8 )"
+required_use_str=""
+for impl in ${PYTHON_COMPAT}; do
+       required_use_str="${required_use_str} python_targets_${impl}"
+done
+required_use_str=" || ( ${required_use_str} )"
 if [[ "${PYTHON_OPTIONAL}" = "yes" ]]; then
        IUSE+="python"
        REQUIRED_USE+=" python? ( ${required_use_str} )"


-- 
Krzysztof Pawlik  <nelchael at gentoo.org>  key id: 0xF6A80E46
desktop-misc, java, vim, kernel, python, apache...


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 554 bytes --]

Re: [gentoo-dev] New eclass for Python

[Corentin Chary]

On Mon, Mar 26, 2012 at 6:23 PM, Krzysztof Pawlik <nelchael@gentoo.org> wrote:
> On 26/03/12 18:11, Krzysztof Pawlik wrote:
>> On 26/03/12 09:20, justin wrote:
>>> On 25/03/12 20:56, Krzysztof Pawlik wrote:
>>>> On 28/02/12 22:13, Krzysztof Pawlik wrote:
>>>>> If there are no objections then during the weekend (March 3, 4) I will add this
>>>>> to portage (after finishing remaining TODO items, PyPy requires 4G of RAM(!!)).
>>>>
>>>> Hello,
>>>>
>>>> Slightly late due to Real Life™ but finally it's in the main tree :)
>>>>
>>>> (and yes - I've tested it with pypy - works as expected :)
>>>>
>>>
>>> Hi,
>>>
>>> is there any documentation beside the man page somewhere?
>>
>> No.
>>
>>> I tried to port some ebuilds but as soon I set
>>>
>>> PYTHON_COMPAT="python2_7 python2_6 python2_5 pypy1_8"
>>>
>>> inherit python-distutils-ng
>>>
>>> I get
>>>
>>>   REQUIRED_USE: USE flag 'python_targets_python3_1' is not in IUSE
>>>
>>> Did I do something wrong, or is there something not straight in the eclass?
>>
>> Can you send me the whole ebuild off-list?
>>
>> There are two ebuilds using the eclass that I've used as tests:
>> http://git.overlays.gentoo.org/gitweb/?p=dev/nelchael.git;a=tree;f=dev-python;h=f1a8e00e3e6df33806d8972c8898f1187163bd3d;hb=HEAD
>
> Ok, found a bug: REQUIRED_USE can't contain elements not in USE, so if you
> excluded python3_1 from PYTHON_COMPAT it didn't appear in IUSE too ->
> REQUIRED_USE contained invalid value. Fixed by below patch:
>
> nelchael@s-lappy ~/.../gentoo-x86/eclass$ cvs diff
> Index: python-distutils-ng.eclass
> ===================================================================
> RCS file: /var/cvsroot/gentoo-x86/eclass/python-distutils-ng.eclass,v
> retrieving revision 1.2
> diff -u -r1.2 python-distutils-ng.eclass
> --- python-distutils-ng.eclass  26 Mar 2012 06:12:53 -0000      1.2
> +++ python-distutils-ng.eclass  26 Mar 2012 16:20:52 -0000
> @@ -105,11 +105,11 @@
>        esac
>  }
>
> -required_use_str=" || (
> -       python_targets_python2_5 python_targets_python2_6 python_targets_python2_7
> -       python_targets_python3_1 python_targets_python3_2
> -       python_targets_jython2_5
> -       python_targets_pypy1_7 python_targets_pypy1_8 )"
> +required_use_str=""
> +for impl in ${PYTHON_COMPAT}; do
> +       required_use_str="${required_use_str} python_targets_${impl}"
> +done
> +required_use_str=" || ( ${required_use_str} )"
>  if [[ "${PYTHON_OPTIONAL}" = "yes" ]]; then
>        IUSE+="python"
>        REQUIRED_USE+=" python? ( ${required_use_str} )"
>
>
> --
> Krzysztof Pawlik  <nelchael at gentoo.org>  key id: 0xF6A80E46
> desktop-misc, java, vim, kernel, python, apache...
>

I have a feature request for distutil-ng (or maybe it's already
possible but I don't know how).

I have a package that depends on python-dateutil:python-2 for
python2_x and python-dateutil:python-3 for python3_x.
Would it be possible to have virtual targets like "python, python2,
python3, pypi, jithon" ?

Thanks,


-- 
Corentin Chary
http://xf.iksaif.net

Re: [gentoo-dev] New eclass for Python

[Mike Gilbert]

On Wed, Apr 4, 2012 at 4:50 AM, Corentin Chary <corentin.chary@gmail.com> wrote:
> I have a package that depends on python-dateutil:python-2 for
> python2_x and python-dateutil:python-3 for python3_x.
> Would it be possible to have virtual targets like "python, python2,
> python3, pypi, jithon" ?
>

With regards to python-dateutil: As of python-dateutil-2.1, there are
no longer separate slots for python-2 and python-3. As well, I masked
the only version (2.0) with SLOT=python-3.

For future compatibility, you should remove the slot from your
dependencies and just depend on dev-python/python-dateutil.

Re: [gentoo-dev] New eclass for Python

[Corentin Chary]

On Wed, Apr 4, 2012 at 4:22 PM, Mike Gilbert <floppym@gentoo.org> wrote:
> On Wed, Apr 4, 2012 at 4:50 AM, Corentin Chary <corentin.chary@gmail.com> wrote:
>> I have a package that depends on python-dateutil:python-2 for
>> python2_x and python-dateutil:python-3 for python3_x.
>> Would it be possible to have virtual targets like "python, python2,
>> python3, pypi, jithon" ?
>>
>
> With regards to python-dateutil: As of python-dateutil-2.1, there are
> no longer separate slots for python-2 and python-3. As well, I masked
> the only version (2.0) with SLOT=python-3.
>
> For future compatibility, you should remove the slot from your
> dependencies and just depend on dev-python/python-dateutil.
>

Yep, I just saw that. But well, celery is full of examples like thats
since it have different dependencies for python2, python2.5, python2.5
+ tests, jython, jython + tests, etc..
Having a way to group similar python would be great.


-- 
Corentin Chary
http://xf.iksaif.net

Re: [gentoo-dev] New eclass for Python

[Mike Gilbert]

On Wed, Apr 4, 2012 at 4:50 AM, Corentin Chary <corentin.chary@gmail.com> wrote:
> I have a feature request for distutil-ng (or maybe it's already
> possible but I don't know how).
>
> I have a package that depends on python-dateutil:python-2 for
> python2_x and python-dateutil:python-3 for python3_x.
> Would it be possible to have virtual targets like "python, python2,
> python3, pypi, jithon" ?
>

One way to implement this would be to have a separate function that
you would call to generate abi-specific dependencies.

Re: [gentoo-dev] New eclass for Python

[Brian Harring]

On Wed, Apr 04, 2012 at 10:41:03AM -0400, Mike Gilbert wrote:
> On Wed, Apr 4, 2012 at 4:50 AM, Corentin Chary <corentin.chary@gmail.com> wrote:
> > I have a feature request for distutil-ng (or maybe it's already
> > possible but I don't know how).
> >
> > I have a package that depends on python-dateutil:python-2 for
> > python2_x and python-dateutil:python-3 for python3_x.
> > Would it be possible to have virtual targets like "python, python2,
> > python3, pypi, jithon" ?
> >
> 
> One way to implement this would be to have a separate function that
> you would call to generate abi-specific dependencies.

Think through the performance implications of that, and the fact that 
moves control/decisions outside of the PM resolvers purview.

If it's use controlled and is part of metadata, sure, but anything 
else?  Not so much. :)

~harring

Re: [gentoo-dev] New eclass for Python

[Mike Gilbert]

On Wed, Apr 4, 2012 at 8:07 PM, Brian Harring <ferringb@gmail.com> wrote:
> On Wed, Apr 04, 2012 at 10:41:03AM -0400, Mike Gilbert wrote:
>> On Wed, Apr 4, 2012 at 4:50 AM, Corentin Chary <corentin.chary@gmail.com> wrote:
>> > I have a feature request for distutil-ng (or maybe it's already
>> > possible but I don't know how).
>> >
>> > I have a package that depends on python-dateutil:python-2 for
>> > python2_x and python-dateutil:python-3 for python3_x.
>> > Would it be possible to have virtual targets like "python, python2,
>> > python3, pypi, jithon" ?
>> >
>>
>> One way to implement this would be to have a separate function that
>> you would call to generate abi-specific dependencies.
>
> Think through the performance implications of that, and the fact that
> moves control/decisions outside of the PM resolvers purview.
>
> If it's use controlled and is part of metadata, sure, but anything
> else?  Not so much. :)

All I'm talking about is a function or function(s) to generate deps like these:

python2_6? ( dev-python/python-dateutil[python2_6] )
python2_7? ( dev-python/python-dateutil[python2_7] )

I'm not sure how that would affect "performance".

Re: [gentoo-dev] New eclass for Python

[Brian Harring]

On Wed, Apr 04, 2012 at 08:36:37PM -0400, Mike Gilbert wrote:
> On Wed, Apr 4, 2012 at 8:07 PM, Brian Harring <ferringb@gmail.com> wrote:
> > On Wed, Apr 04, 2012 at 10:41:03AM -0400, Mike Gilbert wrote:
> >> On Wed, Apr 4, 2012 at 4:50 AM, Corentin Chary <corentin.chary@gmail.com> wrote:
> >> > I have a feature request for distutil-ng (or maybe it's already
> >> > possible but I don't know how).
> >> >
> >> > I have a package that depends on python-dateutil:python-2 for
> >> > python2_x and python-dateutil:python-3 for python3_x.
> >> > Would it be possible to have virtual targets like "python, python2,
> >> > python3, pypi, jithon" ?
> >> >
> >>
> >> One way to implement this would be to have a separate function that
> >> you would call to generate abi-specific dependencies.
> >
> > Think through the performance implications of that, and the fact that
> > moves control/decisions outside of the PM resolvers purview.
> >
> > If it's use controlled and is part of metadata, sure, but anything
> > else? ??Not so much. :)
> 
> All I'm talking about is a function or function(s) to generate deps like these:
> 
> python2_6? ( dev-python/python-dateutil[python2_6] )
> python2_7? ( dev-python/python-dateutil[python2_7] )
> 
> I'm not sure how that would affect "performance".

Nah, that's fine- that's metadata level representation of it.

Sorry, aparenytly I misread your original email- interpretted it as 
shelling out for that decision (something people propose 
occasionally, and usually needs to be stomped out w/ maximal force), 
rather than representing it properly in dependencies.

What you're intending there is perfectly sane/acceptable; pardon the 
noise ;)

~brian