public inbox for gentoo-dev@lists.gentoo.org
 help / color / mirror / Atom feed
* [gentoo-dev] [New eclass] twisted-r1.eclass
@ 2013-08-03 15:13 Michał Górny
  2013-08-03 15:54 ` Ulrich Mueller
                   ` (4 more replies)
  0 siblings, 5 replies; 13+ messages in thread
From: Michał Górny @ 2013-08-03 15:13 UTC (permalink / raw
  To: Gentoo Developer Mailing List; +Cc: python


[-- Attachment #1.1: Type: text/plain, Size: 1328 bytes --]

Hello,

We've been working with yac for a while to get the old twisted.eclass
converted to be compliant with distutils-r1 both in design
and in spirit. This is the first version we'd like to submit for review.

A few notes:

1. The eclass aims to be less conditional than the old one. Especially
we've dropped all the ${CATEGORY}/${PN} checks. The code still sets all
the funny defaults for Twisted suite but those aren't incremental
and can easily be overrode in ebuilds. And in most cases, they simple
are (SRC_URI, LICENSE).

2. The eclass comes with a pure bash-3.2 CamelCase converter
for changing PNs like 'twisted-foo' into 'TwistedFoo'. The relevant
code can be moved to eutils as portable replacements for bash-4 ${foo^}
and friends.

3. The eclass provides a reusable twisted-r1_python_test and sets it as
default python_test. If someone needs to override it, he can still call
it using the former name.

4. Cache updating hack is based off twisted.eclass. Sadly, it's not
something we can do without postrm/postinst. Similarly to the old
eclass, TWISTED_PLUGINS needs to list the plugin locations. Since most
ebuilds install to twisted.plugins, it defaults to that. If an ebuild
does not install plugins at all, it needs to set empty TWISTED_PLUGINS.

-- 
Best regards,
Michał Górny

[-- Attachment #1.2: twisted-r1.eclass --]
[-- Type: text/plain, Size: 5918 bytes --]

# Copyright 1999-2011 Gentoo Foundation
# Distributed under the terms of the GNU General Public License, v2 or later
# $Header: /var/cvsroot/gentoo-x86/eclass/twisted.eclass,v 1.10 2011/12/27 06:54:23 floppym Exp $

# @ECLASS: twisted-r1.eclass
# @MAINTAINER:
# Gentoo Python Project <python@gentoo.org>
# @AUTHOR:
# Author: Michał Górny <mgorny@gentoo.org>
# Author: Jan Matejka <yac@gentoo.org>
# @BLURB: Eclass for Twisted packages
# @DESCRIPTION:
# The twisted eclass defines phase functions for Twisted packages.

case "${EAPI:-0}" in
	0|1|2|3)
		die "Unsupported EAPI=${EAPI:-0} (too old) for ${ECLASS}"
		;;
	4|5)
		;;
	*)
		die "Unsupported EAPI=${EAPI} (unknown) for ${ECLASS}"
		;;
esac

if [[ ! ${_TWISTED_R1} ]]; then

inherit distutils-r1 versionator

fi # ! ${_TWISTED_R1}

EXPORT_FUNCTIONS src_install pkg_postinst pkg_postrm

if [[ ! ${_TWISTED_R1} ]]; then

# @FUNCTION: _twisted_camelcase
# @USAGE: <package-name>
# @DESCRIPTION:
# Convert dash-separated package name to CamelCase. In pure bash.
# Really.
_twisted_camelcase() {
	local pn=${1}
	local save_IFS=${IFS}
	local IFS=-

	# IFS=- splits words by -.
	local w words=( ${pn} )

	local IFS=${save_IFS}
	for w in "${words[@]}"; do
		local fl=${w:0:1}
		# obtain octal ASCII code for the first letter.
		local ord=$(printf '%o' "'${fl}")

		# check if it's [a-z]. ASCII codes are locale-safe.
		if [[ ${ord} -ge 141 && ${ord} -le 172 ]]; then
			# now substract 040 to make it upper-case.
			# fun fact: in range 0141..0172, decimal '- 40' is fine.
			local ord=$(( ${ord} - 40))
			# and convert it back to the character.
			fl=$(printf '\'${ord})
		fi

		echo -n "${fl}${w:1}"
	done
}

# @ECLASS-VARIABLE: TWISTED_PN
# @DESCRIPTION:
# The Twisted CamelCase converted form of package name.
#
# Example: TwistedCore
TWISTED_PN=$(_twisted_camelcase ${PN})

# @ECLASS-VARIABLE: TWISTED_P
# @DESCRIPTION:
# The Twisted CamelCase package name & version.
#
# Example: TwistedCore-1.2.3
TWISTED_P=${TWISTED_PN}-${PV}

HOMEPAGE="http://www.twistedmatrix.com/"
SRC_URI="http://twistedmatrix.com/Releases/${TWISTED_PN}"
SRC_URI="${SRC_URI}/$(get_version_component_range 1-2 ${PV})"
SRC_URI="${SRC_URI}/${TWISTED_P}.tar.bz2"

LICENSE="MIT"
SLOT="0"
IUSE=""

S=${WORKDIR}/${TWISTED_P}

# @ECLASS-VARIABLE: TWISTED_PLUGINS
# @DESCRIPTION:
# An array of Twisted plugins, whose cache is regenerated
# in pkg_postinst() and pkg_postrm() phases.
#
# If no plugins are installed, set to empty array.
[[ ${TWISTED_PLUGINS[@]} ]] || TWISTED_PLUGINS=( twisted.plugins )


# @FUNCTION: twisted-r1_python_test
# @DESCRIPTION:
# The common python_test() implementation that suffices Twisted
# packages.
twisted-r1_python_test() {
	local sitedir=$(python_get_sitedir)

	# Copy modules of other Twisted packages from site-packages
	# directory to the temporary directory.
	local libdir=${BUILD_DIR}/test/lib
	mkdir -p "${libdir}" || die
	cp -r "${ROOT}${sitedir}"/twisted "${libdir}" || die
	# Drop the installed module in case previous version conflicts with
	# the new one somehow.
	rm -fr "${libdir}/${PN/-//}" || die

	distutils_install_for_testing || die

	cd "${TEST_DIR}"/lib || die
	trial ${PN/-/.} || die "Tests fail with ${EPYTHON}"
}

# @FUNCTION: python_test
# @DESCRIPTION:
# Default python_test() for Twisted packages. If you need to override
# it, you can access the original implementation
# via twisted-r1_python_test.
python_test() {
	twisted-r1_python_test
}

# @FUNCTION: twisted-r1_src_install
# @DESCRIPTION:
# Default src_install() for Twisted packages. Automatically handles HTML
# docs and manpages in Twisted packages
twisted-r1_src_install() {
	# TODO: doesn't this accidentially involve installing manpages? ;f
	if [[ ${CATEGORY}/${PN} == dev-python/twisted* && -d doc ]]; then
		local HTML_DOCS=( doc/. )
	fi

	distutils-r1_src_install

	[[ -d doc/man ]] && doman doc/man/*.[[:digit:]]
}

# @FUNCTION: _twisted-r1_create_caches
# @USAGE: <packages>...
# @DESCRIPTION:
# Create dropin.cache for plugins in specified packages. The packages
# are to be listed in standard dotted Python syntax.
_twisted-r1_create_caches() {
	# http://twistedmatrix.com/documents/current/core/howto/plugin.html
	"${PYTHON}" -c \
"import sys
sys.path.insert(0, '${ROOT}$(python_get_sitedir)')

fail = False

try:
	from twisted.plugin import getPlugins, IPlugin
except ImportError as e:
	if '${EBUILD_PHASE}' == 'postinst':
		raise
else:
	for module in sys.argv[1:]:
		try:
			__import__(module, globals())
		except ImportError as e:
			if '${EBUILD_PHASE}' == 'postinst':
				raise
		else:
			list(getPlugins(IPlugin, sys.modules[module]))
" \
		"${@}" || die "twisted plugin cache update failed"
}

# @FUNCTION: twisted-r1_update_plugin_cache
# @DESCRIPTION:
# Update and clean up plugin caches for packages listed
# in TWISTED_PLUGINS.
twisted-r1_update_plugin_cache() {
	local subdirs=( "${TWISTED_PLUGINS[@]//.//}" )
	local paths=( "${subdirs[@]/#/${ROOT}$(python_get_sitedir)/}" )
	local caches=( "${paths[@]/%//dropin.cache}" )

	# First, delete existing (possibly stray) caches.
	rm -f "${caches[@]}" || die

	# Now, let's see which ones we can regenerate.
	_twisted-r1_create_caches "${TWISTED_PLUGINS[@]}"

	# Finally, drop empty parent directories.
	rmdir -p "${paths[@]}" 2>/dev/null
}

# @FUNCTION: twisted-r1_pkg_postinst
# @DESCRIPTION:
# Post-installation hook for twisted-r1. Updates plugin caches.
twisted-r1_pkg_postinst() {
	_distutils-r1_run_foreach_impl twisted-r1_update_plugin_cache
}

# @FUNCTION: twisted-r1_pkg_postrm
# @DESCRIPTION:
# Post-removal hook for twisted-r1. Updates plugin caches.
twisted-r1_pkg_postrm() {
	_distutils-r1_run_foreach_impl twisted-r1_update_plugin_cache
}

_TWISTED_R1=1

fi # ! ${_TWISTED_R1}

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [gentoo-dev] [New eclass] twisted-r1.eclass
  2013-08-03 15:13 [gentoo-dev] [New eclass] twisted-r1.eclass Michał Górny
@ 2013-08-03 15:54 ` Ulrich Mueller
  2013-08-03 18:29   ` Michał Górny
  2013-08-03 21:57 ` Michał Górny
                   ` (3 subsequent siblings)
  4 siblings, 1 reply; 13+ messages in thread
From: Ulrich Mueller @ 2013-08-03 15:54 UTC (permalink / raw
  To: gentoo-dev; +Cc: python

>>>>> On Sat, 3 Aug 2013, Michał Górny wrote:

> 2. The eclass comes with a pure bash-3.2 CamelCase converter for
> changing PNs like 'twisted-foo' into 'TwistedFoo'. The relevant code
> can be moved to eutils as portable replacements for bash-4 ${foo^}
> and friends.

>		# obtain octal ASCII code for the first letter.
>		local ord=$(printf '%o' "'${fl}")
>
>		# check if it's [a-z]. ASCII codes are locale-safe.
>		if [[ ${ord} -ge 141 && ${ord} -le 172 ]]; then
>			# now substract 040 to make it upper-case.
>			# fun fact: in range 0141..0172, decimal '- 40' is fine.
>			local ord=$(( ${ord} - 40))
>			# and convert it back to the character.
>			fl=$(printf '\'${ord})
>		fi

This looks just horrible. You do decimal arithmetic on octal numbers?

Ulrich


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [gentoo-dev] [New eclass] twisted-r1.eclass
  2013-08-03 15:54 ` Ulrich Mueller
@ 2013-08-03 18:29   ` Michał Górny
  2013-08-03 19:37     ` Ulrich Mueller
  2013-08-03 19:37     ` Alex Xu
  0 siblings, 2 replies; 13+ messages in thread
From: Michał Górny @ 2013-08-03 18:29 UTC (permalink / raw
  To: gentoo-dev; +Cc: ulm, python

[-- Attachment #1: Type: text/plain, Size: 1103 bytes --]

Dnia 2013-08-03, o godz. 17:54:42
Ulrich Mueller <ulm@gentoo.org> napisał(a):

> >>>>> On Sat, 3 Aug 2013, Michał Górny wrote:
> 
> > 2. The eclass comes with a pure bash-3.2 CamelCase converter for
> > changing PNs like 'twisted-foo' into 'TwistedFoo'. The relevant code
> > can be moved to eutils as portable replacements for bash-4 ${foo^}
> > and friends.
> 
> >		# obtain octal ASCII code for the first letter.
> >		local ord=$(printf '%o' "'${fl}")
> >
> >		# check if it's [a-z]. ASCII codes are locale-safe.
> >		if [[ ${ord} -ge 141 && ${ord} -le 172 ]]; then
> >			# now substract 040 to make it upper-case.
> >			# fun fact: in range 0141..0172, decimal '- 40' is fine.
> >			local ord=$(( ${ord} - 40))
> >			# and convert it back to the character.
> >			fl=$(printf '\'${ord})
> >		fi
> 
> This looks just horrible. You do decimal arithmetic on octal numbers?

Yes. Bash wasn't really happy to do octal arithmetic for me. Yet
in this particular case, with proper assumptions, decimal arithmetic is
practically equivalent.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [gentoo-dev] [New eclass] twisted-r1.eclass
  2013-08-03 18:29   ` Michał Górny
@ 2013-08-03 19:37     ` Ulrich Mueller
  2013-08-03 19:42       ` Michał Górny
  2013-08-03 19:37     ` Alex Xu
  1 sibling, 1 reply; 13+ messages in thread
From: Ulrich Mueller @ 2013-08-03 19:37 UTC (permalink / raw
  To: Michał Górny; +Cc: gentoo-dev, ulm, python

>>>>> On Sat, 3 Aug 2013, Michał Górny wrote:

> Dnia 2013-08-03, o godz. 17:54:42 Ulrich Mueller <ulm@gentoo.org>
> napisał(a):

>> This looks just horrible. You do decimal arithmetic on octal
>> numbers?

> Yes. Bash wasn't really happy to do octal arithmetic for me. Yet in
> this particular case, with proper assumptions, decimal arithmetic is
> practically equivalent.

Do you consider that in future maybe someone else has to maintain that
code? IMHO we should avoid such hacks.

Why don't you do the calculations in decimal throughout? You can
convert to octal (using printf) in the last step.

Ulrich


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [gentoo-dev] [New eclass] twisted-r1.eclass
  2013-08-03 18:29   ` Michał Górny
  2013-08-03 19:37     ` Ulrich Mueller
@ 2013-08-03 19:37     ` Alex Xu
  2013-08-03 19:47       ` Alex Xu
  2013-08-03 19:49       ` Michał Górny
  1 sibling, 2 replies; 13+ messages in thread
From: Alex Xu @ 2013-08-03 19:37 UTC (permalink / raw
  To: gentoo-dev

[-- Attachment #1: Type: text/plain, Size: 1765 bytes --]

On 03/08/13 02:29 PM, Michał Górny wrote:
> Dnia 2013-08-03, o godz. 17:54:42
> Ulrich Mueller <ulm@gentoo.org> napisał(a):
> 
>>>>>>> On Sat, 3 Aug 2013, Michał Górny wrote:
>>
>>> 2. The eclass comes with a pure bash-3.2 CamelCase converter for
>>> changing PNs like 'twisted-foo' into 'TwistedFoo'. The relevant code
>>> can be moved to eutils as portable replacements for bash-4 ${foo^}
>>> and friends.
>>
>>> 		# obtain octal ASCII code for the first letter.
>>> 		local ord=$(printf '%o' "'${fl}")
>>>
>>> 		# check if it's [a-z]. ASCII codes are locale-safe.
>>> 		if [[ ${ord} -ge 141 && ${ord} -le 172 ]]; then
>>> 			# now substract 040 to make it upper-case.
>>> 			# fun fact: in range 0141..0172, decimal '- 40' is fine.
>>> 			local ord=$(( ${ord} - 40))
>>> 			# and convert it back to the character.
>>> 			fl=$(printf '\'${ord})
>>> 		fi
>>
>> This looks just horrible. You do decimal arithmetic on octal numbers?
> 
> Yes. Bash wasn't really happy to do octal arithmetic for me. Yet
> in this particular case, with proper assumptions, decimal arithmetic is
> practically equivalent.
> 

		# obtain decimal ASCII code for the first letter.
		local fl=$(printf '%d' "'${w}")

		# check if it's [a-z]. ASCII codes are locale-safe.
		if [[ ${ord} -ge 97 && ${ord} -le 122 ]]; then
			local ord=$(( ${ord} - 32 ))
			# and convert it back to the character.
			fl=$(printf '\'${ord})
		fi

		echo -n "${fl}${w:1}"

Probably var names should be adjusted, I'm not too familiar with bash
locals.

printf '%d' "'twisted" outputs "116" as expected, similar to
printf("%d", *"asdf qwerty") in C.

Tested in Bash 4.2.45.

Now time to sit back and wait for it to break in bash
<obscure-version-here>.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [gentoo-dev] [New eclass] twisted-r1.eclass
  2013-08-03 19:37     ` Ulrich Mueller
@ 2013-08-03 19:42       ` Michał Górny
  0 siblings, 0 replies; 13+ messages in thread
From: Michał Górny @ 2013-08-03 19:42 UTC (permalink / raw
  To: gentoo-dev; +Cc: ulm, python

[-- Attachment #1: Type: text/plain, Size: 1214 bytes --]

Dnia 2013-08-03, o godz. 21:37:10
Ulrich Mueller <ulm@gentoo.org> napisał(a):

> >>>>> On Sat, 3 Aug 2013, Michał Górny wrote:
> 
> > Dnia 2013-08-03, o godz. 17:54:42 Ulrich Mueller <ulm@gentoo.org>
> > napisał(a):
> 
> >> This looks just horrible. You do decimal arithmetic on octal
> >> numbers?
> 
> > Yes. Bash wasn't really happy to do octal arithmetic for me. Yet in
> > this particular case, with proper assumptions, decimal arithmetic is
> > practically equivalent.
> 
> Do you consider that in future maybe someone else has to maintain that
> code? IMHO we should avoid such hacks.

Well, the code isn't really supposed to need much maintenance. And it
becomes obsolete as soon as we switch to bash-4. I've already added
additional:

  if [[ ${BASH_VERSINFO[0]} -ge 4 ]]; then
    echo -n "${w^}"
  else
    # current snippet
  fi

Anyway, given the comments and specific case, I doubt it should be
a real issue. 

> Why don't you do the calculations in decimal throughout? You can
> convert to octal (using printf) in the last step.

I consider that a pointless conversion, esp. that it doesn't 
make a difference code-wise.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [gentoo-dev] [New eclass] twisted-r1.eclass
  2013-08-03 19:37     ` Alex Xu
@ 2013-08-03 19:47       ` Alex Xu
  2013-08-03 19:49       ` Michał Górny
  1 sibling, 0 replies; 13+ messages in thread
From: Alex Xu @ 2013-08-03 19:47 UTC (permalink / raw
  To: gentoo-dev

[-- Attachment #1: Type: text/plain, Size: 1933 bytes --]

On 03/08/13 03:37 PM, Alex Xu wrote:
> On 03/08/13 02:29 PM, Michał Górny wrote:
>> Dnia 2013-08-03, o godz. 17:54:42
>> Ulrich Mueller <ulm@gentoo.org> napisał(a):
>>
>>>>>>>> On Sat, 3 Aug 2013, Michał Górny wrote:
>>>
>>>> 2. The eclass comes with a pure bash-3.2 CamelCase converter for
>>>> changing PNs like 'twisted-foo' into 'TwistedFoo'. The relevant code
>>>> can be moved to eutils as portable replacements for bash-4 ${foo^}
>>>> and friends.
>>>
>>>> 		# obtain octal ASCII code for the first letter.
>>>> 		local ord=$(printf '%o' "'${fl}")
>>>>
>>>> 		# check if it's [a-z]. ASCII codes are locale-safe.
>>>> 		if [[ ${ord} -ge 141 && ${ord} -le 172 ]]; then
>>>> 			# now substract 040 to make it upper-case.
>>>> 			# fun fact: in range 0141..0172, decimal '- 40' is fine.
>>>> 			local ord=$(( ${ord} - 40))
>>>> 			# and convert it back to the character.
>>>> 			fl=$(printf '\'${ord})
>>>> 		fi
>>>
>>> This looks just horrible. You do decimal arithmetic on octal numbers?
>>
>> Yes. Bash wasn't really happy to do octal arithmetic for me. Yet
>> in this particular case, with proper assumptions, decimal arithmetic is
>> practically equivalent.
>>
> 
> 		# obtain decimal ASCII code for the first letter.
> 		local fl=$(printf '%d' "'${w}")
> 
> 		# check if it's [a-z]. ASCII codes are locale-safe.
> 		if [[ ${ord} -ge 97 && ${ord} -le 122 ]]; then
> 			local ord=$(( ${ord} - 32 ))
> 			# and convert it back to the character.
> 			fl=$(printf '\'${ord})
> 		fi
> 
> 		echo -n "${fl}${w:1}"
> 
> Probably var names should be adjusted, I'm not too familiar with bash
> locals.
> 
> printf '%d' "'twisted" outputs "116" as expected, similar to
> printf("%d", *"asdf qwerty") in C.
> 
> Tested in Bash 4.2.45.
> 
> Now time to sit back and wait for it to break in bash
> <obscure-version-here>.
> 

I am dumb. Please disregard the previous message.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [gentoo-dev] [New eclass] twisted-r1.eclass
  2013-08-03 19:37     ` Alex Xu
  2013-08-03 19:47       ` Alex Xu
@ 2013-08-03 19:49       ` Michał Górny
  1 sibling, 0 replies; 13+ messages in thread
From: Michał Górny @ 2013-08-03 19:49 UTC (permalink / raw
  To: gentoo-dev; +Cc: alex_y_xu

[-- Attachment #1: Type: text/plain, Size: 2038 bytes --]

Dnia 2013-08-03, o godz. 15:37:54
Alex Xu <alex_y_xu@yahoo.ca> napisał(a):

> On 03/08/13 02:29 PM, Michał Górny wrote:
> > Dnia 2013-08-03, o godz. 17:54:42
> > Ulrich Mueller <ulm@gentoo.org> napisał(a):
> > 
> >>>>>>> On Sat, 3 Aug 2013, Michał Górny wrote:
> >>
> >>> 2. The eclass comes with a pure bash-3.2 CamelCase converter for
> >>> changing PNs like 'twisted-foo' into 'TwistedFoo'. The relevant code
> >>> can be moved to eutils as portable replacements for bash-4 ${foo^}
> >>> and friends.
> >>
> >>> 		# obtain octal ASCII code for the first letter.
> >>> 		local ord=$(printf '%o' "'${fl}")
> >>>
> >>> 		# check if it's [a-z]. ASCII codes are locale-safe.
> >>> 		if [[ ${ord} -ge 141 && ${ord} -le 172 ]]; then
> >>> 			# now substract 040 to make it upper-case.
> >>> 			# fun fact: in range 0141..0172, decimal '- 40' is fine.
> >>> 			local ord=$(( ${ord} - 40))
> >>> 			# and convert it back to the character.
> >>> 			fl=$(printf '\'${ord})
> >>> 		fi
> >>
> >> This looks just horrible. You do decimal arithmetic on octal numbers?
> > 
> > Yes. Bash wasn't really happy to do octal arithmetic for me. Yet
> > in this particular case, with proper assumptions, decimal arithmetic is
> > practically equivalent.
> > 
> 
> 		# obtain decimal ASCII code for the first letter.
> 		local fl=$(printf '%d' "'${w}")
> 
> 		# check if it's [a-z]. ASCII codes are locale-safe.
> 		if [[ ${ord} -ge 97 && ${ord} -le 122 ]]; then
> 			local ord=$(( ${ord} - 32 ))
> 			# and convert it back to the character.
> 			fl=$(printf '\'${ord})
> 		fi
> 
> 		echo -n "${fl}${w:1}"
> 
> Probably var names should be adjusted, I'm not too familiar with bash
> locals.
> 
> printf '%d' "'twisted" outputs "116" as expected, similar to
> printf("%d", *"asdf qwerty") in C.
> 
> Tested in Bash 4.2.45.

You could test the whole snippet, not just the beginning. Then you
would know that you're passing decimal to '\ooo' which expects octal.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [gentoo-dev] [New eclass] twisted-r1.eclass
  2013-08-03 15:13 [gentoo-dev] [New eclass] twisted-r1.eclass Michał Górny
  2013-08-03 15:54 ` Ulrich Mueller
@ 2013-08-03 21:57 ` Michał Górny
  2013-08-03 22:28 ` Michał Górny
                   ` (2 subsequent siblings)
  4 siblings, 0 replies; 13+ messages in thread
From: Michał Górny @ 2013-08-03 21:57 UTC (permalink / raw
  To: gentoo-dev; +Cc: python


[-- Attachment #1.1: Type: text/plain, Size: 431 bytes --]

Dnia 2013-08-03, o godz. 17:13:03
Michał Górny <mgorny@gentoo.org> napisał(a):

> We've been working with yac for a while to get the old twisted.eclass
> converted to be compliant with distutils-r1 both in design
> and in spirit. This is the first version we'd like to submit for review.

Second version, with much more faster camelcase magic (about 1.5 times)
and bash-4 support.

-- 
Best regards,
Michał Górny

[-- Attachment #1.2: twisted-r1.eclass --]
[-- Type: text/plain, Size: 5853 bytes --]

# Copyright 1999-2011 Gentoo Foundation
# Distributed under the terms of the GNU General Public License, v2 or later
# $Header: /var/cvsroot/gentoo-x86/eclass/twisted.eclass,v 1.10 2011/12/27 06:54:23 floppym Exp $

# @ECLASS: twisted-r1.eclass
# @MAINTAINER:
# Gentoo Python Project <python@gentoo.org>
# @AUTHOR:
# Author: Michał Górny <mgorny@gentoo.org>
# Author: Jan Matejka <yac@gentoo.org>
# @BLURB: Eclass for Twisted packages
# @DESCRIPTION:
# The twisted eclass defines phase functions for Twisted packages.

case "${EAPI:-0}" in
	0|1|2|3)
		die "Unsupported EAPI=${EAPI:-0} (too old) for ${ECLASS}"
		;;
	4|5)
		;;
	*)
		die "Unsupported EAPI=${EAPI} (unknown) for ${ECLASS}"
		;;
esac

if [[ ! ${_TWISTED_R1} ]]; then

inherit distutils-r1 versionator

fi # ! ${_TWISTED_R1}

EXPORT_FUNCTIONS src_install pkg_postinst pkg_postrm

if [[ ! ${_TWISTED_R1} ]]; then

# @FUNCTION: _twisted-r1_camelcase_pn
# @DESCRIPTION:
# Convert dash-separated ${PN} to CamelCase ${TWISTED_PN}. In pure bash.
# Really.
_twisted-r1_camelcase_pn() {
	local save_IFS=${IFS}
	local IFS=-

	# IFS=- splits words by -.
	local w words=( ${PN} )

	TWISTED_PN=

	local IFS=${save_IFS}
	local LC_COLLATE=C

	local uc='ABCDEFGHIJKLMNOPQRSTUVWXYZ'

	for w in "${words[@]}"; do
		if [[ ${BASH_VERSINFO[0]} -ge 4 ]]; then
			TWISTED_PN+=${w^}
		else
			local fl=${w:0:1}

			# Danger: magic starts here. Please close your eyes.
			# In base 36, a..z represents digits 10..35. We substract 10
			# and get array subscripts for uc.

			[[ ${fl} == [a-z] ]] && fl=${uc:36#${fl} - 10:1}

			TWISTED_PN+="${fl}${w:1}"
		fi
	done
}

# @ECLASS-VARIABLE: TWISTED_PN
# @DESCRIPTION:
# The Twisted CamelCase converted form of package name.
#
# Example: TwistedCore
_twisted-r1_camelcase_pn

# @ECLASS-VARIABLE: TWISTED_P
# @DESCRIPTION:
# The Twisted CamelCase package name & version.
#
# Example: TwistedCore-1.2.3
TWISTED_P=${TWISTED_PN}-${PV}

HOMEPAGE="http://www.twistedmatrix.com/"
SRC_URI="http://twistedmatrix.com/Releases/${TWISTED_PN}"
SRC_URI="${SRC_URI}/$(get_version_component_range 1-2 ${PV})"
SRC_URI="${SRC_URI}/${TWISTED_P}.tar.bz2"

LICENSE="MIT"
SLOT="0"
IUSE=""

S=${WORKDIR}/${TWISTED_P}

# @ECLASS-VARIABLE: TWISTED_PLUGINS
# @DESCRIPTION:
# An array of Twisted plugins, whose cache is regenerated
# in pkg_postinst() and pkg_postrm() phases.
#
# If no plugins are installed, set to empty array.
[[ ${TWISTED_PLUGINS[@]} ]] || TWISTED_PLUGINS=( twisted.plugins )


# @FUNCTION: twisted-r1_python_test
# @DESCRIPTION:
# The common python_test() implementation that suffices Twisted
# packages.
twisted-r1_python_test() {
	local sitedir=$(python_get_sitedir)

	# Copy modules of other Twisted packages from site-packages
	# directory to the temporary directory.
	local libdir=${BUILD_DIR}/test/lib
	mkdir -p "${libdir}" || die
	cp -r "${ROOT}${sitedir}"/twisted "${libdir}" || die
	# Drop the installed module in case previous version conflicts with
	# the new one somehow.
	rm -fr "${libdir}/${PN/-//}" || die

	distutils_install_for_testing || die

	cd "${TEST_DIR}"/lib || die
	trial ${PN/-/.} || die "Tests fail with ${EPYTHON}"
}

# @FUNCTION: python_test
# @DESCRIPTION:
# Default python_test() for Twisted packages. If you need to override
# it, you can access the original implementation
# via twisted-r1_python_test.
python_test() {
	twisted-r1_python_test
}

# @FUNCTION: twisted-r1_src_install
# @DESCRIPTION:
# Default src_install() for Twisted packages. Automatically handles HTML
# docs and manpages in Twisted packages
twisted-r1_src_install() {
	# TODO: doesn't this accidentially involve installing manpages? ;f
	if [[ ${CATEGORY}/${PN} == dev-python/twisted* && -d doc ]]; then
		local HTML_DOCS=( doc/. )
	fi

	distutils-r1_src_install

	[[ -d doc/man ]] && doman doc/man/*.[[:digit:]]
}

# @FUNCTION: _twisted-r1_create_caches
# @USAGE: <packages>...
# @DESCRIPTION:
# Create dropin.cache for plugins in specified packages. The packages
# are to be listed in standard dotted Python syntax.
_twisted-r1_create_caches() {
	# http://twistedmatrix.com/documents/current/core/howto/plugin.html
	"${PYTHON}" -c \
"import sys
sys.path.insert(0, '${ROOT}$(python_get_sitedir)')

fail = False

try:
	from twisted.plugin import getPlugins, IPlugin
except ImportError as e:
	if '${EBUILD_PHASE}' == 'postinst':
		raise
else:
	for module in sys.argv[1:]:
		try:
			__import__(module, globals())
		except ImportError as e:
			if '${EBUILD_PHASE}' == 'postinst':
				raise
		else:
			list(getPlugins(IPlugin, sys.modules[module]))
" \
		"${@}" || die "twisted plugin cache update failed"
}

# @FUNCTION: twisted-r1_update_plugin_cache
# @DESCRIPTION:
# Update and clean up plugin caches for packages listed
# in TWISTED_PLUGINS.
twisted-r1_update_plugin_cache() {
	local subdirs=( "${TWISTED_PLUGINS[@]//.//}" )
	local paths=( "${subdirs[@]/#/${ROOT}$(python_get_sitedir)/}" )
	local caches=( "${paths[@]/%//dropin.cache}" )

	# First, delete existing (possibly stray) caches.
	rm -f "${caches[@]}" || die

	# Now, let's see which ones we can regenerate.
	_twisted-r1_create_caches "${TWISTED_PLUGINS[@]}"

	# Finally, drop empty parent directories.
	rmdir -p "${paths[@]}" 2>/dev/null
}

# @FUNCTION: twisted-r1_pkg_postinst
# @DESCRIPTION:
# Post-installation hook for twisted-r1. Updates plugin caches.
twisted-r1_pkg_postinst() {
	_distutils-r1_run_foreach_impl twisted-r1_update_plugin_cache
}

# @FUNCTION: twisted-r1_pkg_postrm
# @DESCRIPTION:
# Post-removal hook for twisted-r1. Updates plugin caches.
twisted-r1_pkg_postrm() {
	_distutils-r1_run_foreach_impl twisted-r1_update_plugin_cache
}

_TWISTED_R1=1

fi # ! ${_TWISTED_R1}

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [gentoo-dev] [New eclass] twisted-r1.eclass
  2013-08-03 15:13 [gentoo-dev] [New eclass] twisted-r1.eclass Michał Górny
  2013-08-03 15:54 ` Ulrich Mueller
  2013-08-03 21:57 ` Michał Górny
@ 2013-08-03 22:28 ` Michał Górny
  2013-08-04  5:15 ` [gentoo-dev] " Marien Zwart
  2013-08-04 12:37 ` [gentoo-dev] " Michał Górny
  4 siblings, 0 replies; 13+ messages in thread
From: Michał Górny @ 2013-08-03 22:28 UTC (permalink / raw
  To: gentoo-dev; +Cc: python


[-- Attachment #1.1: Type: text/plain, Size: 368 bytes --]

Dnia 2013-08-03, o godz. 17:13:03
Michał Górny <mgorny@gentoo.org> napisał(a):

> We've been working with yac for a while to get the old twisted.eclass
> converted to be compliant with distutils-r1 both in design
> and in spirit. This is the first version we'd like to submit for review.

One more bit of optimization.

-- 
Best regards,
Michał Górny

[-- Attachment #1.2: twisted-r1.eclass --]
[-- Type: text/plain, Size: 5921 bytes --]

# Copyright 1999-2011 Gentoo Foundation
# Distributed under the terms of the GNU General Public License, v2 or later
# $Header: /var/cvsroot/gentoo-x86/eclass/twisted.eclass,v 1.10 2011/12/27 06:54:23 floppym Exp $

# @ECLASS: twisted-r1.eclass
# @MAINTAINER:
# Gentoo Python Project <python@gentoo.org>
# @AUTHOR:
# Author: Michał Górny <mgorny@gentoo.org>
# Author: Jan Matejka <yac@gentoo.org>
# @BLURB: Eclass for Twisted packages
# @DESCRIPTION:
# The twisted eclass defines phase functions for Twisted packages.

case "${EAPI:-0}" in
	0|1|2|3)
		die "Unsupported EAPI=${EAPI:-0} (too old) for ${ECLASS}"
		;;
	4|5)
		;;
	*)
		die "Unsupported EAPI=${EAPI} (unknown) for ${ECLASS}"
		;;
esac

if [[ ! ${_TWISTED_R1} ]]; then

inherit distutils-r1 versionator

fi # ! ${_TWISTED_R1}

EXPORT_FUNCTIONS src_install pkg_postinst pkg_postrm

if [[ ! ${_TWISTED_R1} ]]; then

# @FUNCTION: _twisted-r1_camelcase_pn
# @DESCRIPTION:
# Convert dash-separated ${PN} to CamelCase ${TWISTED_PN}. In pure bash.
# Really.
_twisted-r1_camelcase_pn() {
	local IFS=-

	# IFS=- splits words by -.
	local words=( ${PN} )

	# we can't keep '-' as it collides with [a-z] check
	# and '' is used by bash-4 words[*], so let's just set globally
	IFS=

	if [[ ${BASH_VERSINFO[0]} -ge 4 ]]; then
		TWISTED_PN=${words[*]^}
		return
	fi

	local w LC_COLLATE=C uc='ABCDEFGHIJKLMNOPQRSTUVWXYZ'

	TWISTED_PN=
	for w in "${words[@]}"; do
		local fl=${w:0:1}

		# Danger: magic starts here. Please close your eyes.
		# In base 36, a..z represents digits 10..35. We substract 10
		# and get array subscripts for uc.

		[[ ${fl} == [a-z] ]] && fl=${uc:36#${fl} - 10:1}

		TWISTED_PN+=${fl}${w:1}
	done
}

# @ECLASS-VARIABLE: TWISTED_PN
# @DESCRIPTION:
# The Twisted CamelCase converted form of package name.
#
# Example: TwistedCore
_twisted-r1_camelcase_pn

# @ECLASS-VARIABLE: TWISTED_P
# @DESCRIPTION:
# The Twisted CamelCase package name & version.
#
# Example: TwistedCore-1.2.3
TWISTED_P=${TWISTED_PN}-${PV}

HOMEPAGE="http://www.twistedmatrix.com/"
SRC_URI="http://twistedmatrix.com/Releases/${TWISTED_PN}"
SRC_URI="${SRC_URI}/$(get_version_component_range 1-2 ${PV})"
SRC_URI="${SRC_URI}/${TWISTED_P}.tar.bz2"

LICENSE="MIT"
SLOT="0"
IUSE=""

S=${WORKDIR}/${TWISTED_P}

# @ECLASS-VARIABLE: TWISTED_PLUGINS
# @DESCRIPTION:
# An array of Twisted plugins, whose cache is regenerated
# in pkg_postinst() and pkg_postrm() phases.
#
# If no plugins are installed, set to empty array.
[[ ${TWISTED_PLUGINS[@]} ]] || TWISTED_PLUGINS=( twisted.plugins )


# @FUNCTION: twisted-r1_python_test
# @DESCRIPTION:
# The common python_test() implementation that suffices Twisted
# packages.
twisted-r1_python_test() {
	local sitedir=$(python_get_sitedir)

	# Copy modules of other Twisted packages from site-packages
	# directory to the temporary directory.
	local libdir=${BUILD_DIR}/test/lib
	mkdir -p "${libdir}" || die
	cp -r "${ROOT}${sitedir}"/twisted "${libdir}" || die
	# Drop the installed module in case previous version conflicts with
	# the new one somehow.
	rm -fr "${libdir}/${PN/-//}" || die

	distutils_install_for_testing || die

	cd "${TEST_DIR}"/lib || die
	trial ${PN/-/.} || die "Tests fail with ${EPYTHON}"
}

# @FUNCTION: python_test
# @DESCRIPTION:
# Default python_test() for Twisted packages. If you need to override
# it, you can access the original implementation
# via twisted-r1_python_test.
python_test() {
	twisted-r1_python_test
}

# @FUNCTION: twisted-r1_src_install
# @DESCRIPTION:
# Default src_install() for Twisted packages. Automatically handles HTML
# docs and manpages in Twisted packages
twisted-r1_src_install() {
	# TODO: doesn't this accidentially involve installing manpages? ;f
	if [[ ${CATEGORY}/${PN} == dev-python/twisted* && -d doc ]]; then
		local HTML_DOCS=( doc/. )
	fi

	distutils-r1_src_install

	[[ -d doc/man ]] && doman doc/man/*.[[:digit:]]
}

# @FUNCTION: _twisted-r1_create_caches
# @USAGE: <packages>...
# @DESCRIPTION:
# Create dropin.cache for plugins in specified packages. The packages
# are to be listed in standard dotted Python syntax.
_twisted-r1_create_caches() {
	# http://twistedmatrix.com/documents/current/core/howto/plugin.html
	"${PYTHON}" -c \
"import sys
sys.path.insert(0, '${ROOT}$(python_get_sitedir)')

fail = False

try:
	from twisted.plugin import getPlugins, IPlugin
except ImportError as e:
	if '${EBUILD_PHASE}' == 'postinst':
		raise
else:
	for module in sys.argv[1:]:
		try:
			__import__(module, globals())
		except ImportError as e:
			if '${EBUILD_PHASE}' == 'postinst':
				raise
		else:
			list(getPlugins(IPlugin, sys.modules[module]))
" \
		"${@}" || die "twisted plugin cache update failed"
}

# @FUNCTION: twisted-r1_update_plugin_cache
# @DESCRIPTION:
# Update and clean up plugin caches for packages listed
# in TWISTED_PLUGINS.
twisted-r1_update_plugin_cache() {
	local subdirs=( "${TWISTED_PLUGINS[@]//.//}" )
	local paths=( "${subdirs[@]/#/${ROOT}$(python_get_sitedir)/}" )
	local caches=( "${paths[@]/%//dropin.cache}" )

	# First, delete existing (possibly stray) caches.
	rm -f "${caches[@]}" || die

	# Now, let's see which ones we can regenerate.
	_twisted-r1_create_caches "${TWISTED_PLUGINS[@]}"

	# Finally, drop empty parent directories.
	rmdir -p "${paths[@]}" 2>/dev/null
}

# @FUNCTION: twisted-r1_pkg_postinst
# @DESCRIPTION:
# Post-installation hook for twisted-r1. Updates plugin caches.
twisted-r1_pkg_postinst() {
	_distutils-r1_run_foreach_impl twisted-r1_update_plugin_cache
}

# @FUNCTION: twisted-r1_pkg_postrm
# @DESCRIPTION:
# Post-removal hook for twisted-r1. Updates plugin caches.
twisted-r1_pkg_postrm() {
	_distutils-r1_run_foreach_impl twisted-r1_update_plugin_cache
}

_TWISTED_R1=1

fi # ! ${_TWISTED_R1}

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]

^ permalink raw reply	[flat|nested] 13+ messages in thread

* [gentoo-dev] Re: [New eclass] twisted-r1.eclass
  2013-08-03 15:13 [gentoo-dev] [New eclass] twisted-r1.eclass Michał Górny
                   ` (2 preceding siblings ...)
  2013-08-03 22:28 ` Michał Górny
@ 2013-08-04  5:15 ` Marien Zwart
  2013-08-04  7:34   ` Michał Górny
  2013-08-04 12:37 ` [gentoo-dev] " Michał Górny
  4 siblings, 1 reply; 13+ messages in thread
From: Marien Zwart @ 2013-08-04  5:15 UTC (permalink / raw
  To: Michał Górny; +Cc: Gentoo Developer Mailing List, python

On Sun, Aug 4, 2013 at 1:13 AM, Michał Górny <mgorny@gentoo.org> wrote:
> We've been working with yac for a while to get the old twisted.eclass
> converted to be compliant with distutils-r1 both in design
> and in spirit. This is the first version we'd like to submit for review.

Thanks for doing this. If memory serves (and cvs log says it does) I'm
to blame for the first version of this eclass so perhaps I should
review this one :) twisted.eclass changed quite a bit since I last
looked at it, though.

I've reviewed the "One more bit of optimization" version.

> 1. The eclass aims to be less conditional than the old one. Especially
> we've dropped all the ${CATEGORY}/${PN} checks. The code still sets all
> the funny defaults for Twisted suite but those aren't incremental
> and can easily be overrode in ebuilds. And in most cases, they simple
> are (SRC_URI, LICENSE).

The original version just set those unconditionally, the conditionals
I think you're referring to were added by Arfrever in revision 1.8.
It's not clear to me why. Does someone on this list remember?

> 2. The eclass comes with a pure bash-3.2 CamelCase converter
> for changing PNs like 'twisted-foo' into 'TwistedFoo'. The relevant
> code can be moved to eutils as portable replacements for bash-4 ${foo^}
> and friends.

That was considered when the original was committed but rejected as
getting too messy. Two questions: have you tested this contraption
with the oldest version of bash we still care about, and have you
considered making it take the input as an argument and echoing the
result (making it work the way versionator.eclass functions do)? If
you want this to be usable as a portable utility function you'd have
to write it that way, might as well do that from the start.

I'm only ok with this code because we'll eventually end up requiring
bash 4 at which point this can be written sanely.

> 3. The eclass provides a reusable twisted-r1_python_test and sets it as
> default python_test. If someone needs to override it, he can still call
> it using the former name.

Kind of a shame EXPORT_FUNCTIONS only works on actual phase functions.
The rm -rf in there is slightly hacky: its target will legitimately
not exist if this is the initial install or if we're not in a
twisted-* package (in which case the package should probably not be
hitting this function, but it will if not overridden). There's
potential for confusion there if an upgrade drops or renames a
twisted/plugins/twisted_blah.py file, but that seems like enough of an
edge case it's not worth worrying about.

I was going to recommend adding variables that control what gets
copied and removed, but I can't think of any current users of such
variables. So it's probably not worth the hassle.

If I read this right it'll break if distutils_install_for_testing ever
changes its mind on where to install (and its docstring says to use
TEST_DIR, not which path that'll be). So I'd add a check for
${TEST_DIR}/lib being equal to $libdir after the call to
distutils_install_for_testing, and print a noisy QA message about
updating the eclass if they're different.

> 4. Cache updating hack is based off twisted.eclass. Sadly, it's not
> something we can do without postrm/postinst. Similarly to the old
> eclass, TWISTED_PLUGINS needs to list the plugin locations. Since most
> ebuilds install to twisted.plugins, it defaults to that. If an ebuild
> does not install plugins at all, it needs to set empty TWISTED_PLUGINS.

If I read that right you can just __import__(module), you don't need
__import__(module, globals()). Also, maybe do the loop over plugin
locations in bash. I think if you do that you don't need __import__
and sys.modules anymore, you can just generate "import $module" and
"list(getPlugins(IPlugin, $module))".

I wouldn't worry too much about using pkg_post* for this. It's what
they're there for (twisted's isn't the only plugin system with a file
like this, see for example gdk-pixbuf).

It seems that if TWISTED_PLUGINS is set to the empty array, "[[
${TWISTED_PLUGINS[@]} ]] || TWISTED_PLUGINS=( twisted.plugins )" will
set it to twisted.plugins. Is that intentional? It's not necessarily a
problem (you can just set TWISTED_PLUGINS back to the empty array
after the inherit) but it's a bit confusing and I don't think it's
what you intended (why bother with that conditional at all?)

I was going to claim importing from twisted.plugin should never fail,
but then I realized dev-python/twisted might want to use these
functions too.

I might add a comment to the functions operating on that array to
remind people it might be the empty array. It wasn't immediately
obvious to me upon reading the code it'd safely do nothing (today I
learned "rm -f" (with no further arguments) successfully does nothing,
while "rm" (with no arguments) fails because of a missing operand).
-- 

Marien Zwart (marienz)


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [gentoo-dev] Re: [New eclass] twisted-r1.eclass
  2013-08-04  5:15 ` [gentoo-dev] " Marien Zwart
@ 2013-08-04  7:34   ` Michał Górny
  0 siblings, 0 replies; 13+ messages in thread
From: Michał Górny @ 2013-08-04  7:34 UTC (permalink / raw
  To: gentoo-dev; +Cc: marienz, python

[-- Attachment #1: Type: text/plain, Size: 5686 bytes --]

Dnia 2013-08-04, o godz. 15:15:12
Marien Zwart <marienz@gentoo.org> napisał(a):

> On Sun, Aug 4, 2013 at 1:13 AM, Michał Górny <mgorny@gentoo.org> wrote:
> > 2. The eclass comes with a pure bash-3.2 CamelCase converter
> > for changing PNs like 'twisted-foo' into 'TwistedFoo'. The relevant
> > code can be moved to eutils as portable replacements for bash-4 ${foo^}
> > and friends.
> 
> That was considered when the original was committed but rejected as
> getting too messy. Two questions: have you tested this contraption
> with the oldest version of bash we still care about, and have you
> considered making it take the input as an argument and echoing the
> result (making it work the way versionator.eclass functions do)? If
> you want this to be usable as a portable utility function you'd have
> to write it that way, might as well do that from the start.

Yes, it's compliant with bash-3.2. That's why it has conditional
on bash-4. And the original version worked that way but subshells
impact performance, and that's not something we'd like to do
in the global scope. However, if it was to be reused, then it will
probably work that way.

> I'm only ok with this code because we'll eventually end up requiring
> bash 4 at which point this can be written sanely.
> 
> > 3. The eclass provides a reusable twisted-r1_python_test and sets it as
> > default python_test. If someone needs to override it, he can still call
> > it using the former name.
> 
> Kind of a shame EXPORT_FUNCTIONS only works on actual phase functions.

I had an eclass creating framework for custom phase functions but it
was rejected by the ml as introducing too much abstraction.

> The rm -rf in there is slightly hacky: its target will legitimately
> not exist if this is the initial install or if we're not in a
> twisted-* package (in which case the package should probably not be
> hitting this function, but it will if not overridden).

Hmm, considering the specifics of the function, I think we could make
the cp/rm hack conditional to package name. Non-twisted packages could
still reuse plain 'trial' call.

> There's potential for confusion there if an upgrade drops or renames a
> twisted/plugins/twisted_blah.py file, but that seems like enough of an
> edge case it's not worth worrying about.

True.

> I was going to recommend adding variables that control what gets
> copied and removed, but I can't think of any current users of such
> variables. So it's probably not worth the hassle.

During the tests? Sounds like overkill.

> If I read this right it'll break if distutils_install_for_testing ever
> changes its mind on where to install (and its docstring says to use
> TEST_DIR, not which path that'll be). So I'd add a check for
> ${TEST_DIR}/lib being equal to $libdir after the call to
> distutils_install_for_testing, and print a noisy QA message about
> updating the eclass if they're different.

Sounds reasonable. I was wondering about moving TEST_DIR earlier
in distutils-r1 but this is probably cleaner.

> > 4. Cache updating hack is based off twisted.eclass. Sadly, it's not
> > something we can do without postrm/postinst. Similarly to the old
> > eclass, TWISTED_PLUGINS needs to list the plugin locations. Since most
> > ebuilds install to twisted.plugins, it defaults to that. If an ebuild
> > does not install plugins at all, it needs to set empty TWISTED_PLUGINS.
> 
> If I read that right you can just __import__(module), you don't need
> __import__(module, globals()). Also, maybe do the loop over plugin
> locations in bash. I think if you do that you don't need __import__
> and sys.modules anymore, you can just generate "import $module" and
> "list(getPlugins(IPlugin, $module))".

But you'll call Python a few times. Looping in Python is less
expensive. Plus inlining variables into Python code is really ugly.

> I wouldn't worry too much about using pkg_post* for this. It's what
> they're there for (twisted's isn't the only plugin system with a file
> like this, see for example gdk-pixbuf).
> 
> It seems that if TWISTED_PLUGINS is set to the empty array, "[[
> ${TWISTED_PLUGINS[@]} ]] || TWISTED_PLUGINS=( twisted.plugins )" will
> set it to twisted.plugins. Is that intentional? It's not necessarily a
> problem (you can just set TWISTED_PLUGINS back to the empty array
> after the inherit) but it's a bit confusing and I don't think it's
> what you intended (why bother with that conditional at all?)

It's all yac's fault :P. I didn't even think about that line.

Well, in fact TWISTED_PLUGINS will usually be set after the inherit.
But indeed we should use some kind of 'declare' to check if it was
declared instead.

> I was going to claim importing from twisted.plugin should never fail,
> but then I realized dev-python/twisted might want to use these
> functions too.

And yes, it does. It's responsible for dropping the plugin cache too.

> I might add a comment to the functions operating on that array to
> remind people it might be the empty array. It wasn't immediately
> obvious to me upon reading the code it'd safely do nothing (today I
> learned "rm -f" (with no further arguments) successfully does nothing,
> while "rm" (with no arguments) fails because of a missing operand).

Yes, the code would use more checks. The 'empty array' was rather
a last-minute invention since originally the eclass wasn't intended to
be used by packages not installing twisted plugins. But then, at least
CamelCase name generator and possibly trial testing would still benefit
those packages.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [gentoo-dev] [New eclass] twisted-r1.eclass
  2013-08-03 15:13 [gentoo-dev] [New eclass] twisted-r1.eclass Michał Górny
                   ` (3 preceding siblings ...)
  2013-08-04  5:15 ` [gentoo-dev] " Marien Zwart
@ 2013-08-04 12:37 ` Michał Górny
  4 siblings, 0 replies; 13+ messages in thread
From: Michał Górny @ 2013-08-04 12:37 UTC (permalink / raw
  To: gentoo-dev; +Cc: python


[-- Attachment #1.1: Type: text/plain, Size: 738 bytes --]

Dnia 2013-08-03, o godz. 17:13:03
Michał Górny <mgorny@gentoo.org> napisał(a):

> We've been working with yac for a while to get the old twisted.eclass
> converted to be compliant with distutils-r1 both in design
> and in spirit. This is the first version we'd like to submit for review.

Following comments from marienz:

1. Restored the other subshell obtaining TWISTED_PN from
_twisted_camelcase,

2. Fixed handling empty TWISTED_PLUGINS,

3. Added integrity check for TEST_DIR consistency,

4. Made TWISTED_P* & HTML_DOCS overridable,

5. Added TWISTED_RELEASE that contains major+minor version as it's used
to build SRC_URI and often in dependencies,

6. Improved docs.

-- 
Best regards,
Michał Górny

[-- Attachment #1.2: twisted-r1.eclass --]
[-- Type: text/plain, Size: 6550 bytes --]

# Copyright 1999-2013 Gentoo Foundation
# Distributed under the terms of the GNU General Public License, v2 or later
# $Header: /var/cvsroot/gentoo-x86/eclass/twisted.eclass,v 1.10 2011/12/27 06:54:23 floppym Exp $

# @ECLASS: twisted-r1.eclass
# @MAINTAINER:
# Gentoo Python Project <python@gentoo.org>
# @AUTHOR:
# Author: Michał Górny <mgorny@gentoo.org>
# Author: Jan Matejka <yac@gentoo.org>
# @BLURB: Eclass for Twisted packages
# @DESCRIPTION:
# The twisted eclass defines phase functions for Twisted packages.

case "${EAPI:-0}" in
	0|1|2|3)
		die "Unsupported EAPI=${EAPI:-0} (too old) for ${ECLASS}"
		;;
	4|5)
		;;
	*)
		die "Unsupported EAPI=${EAPI} (unknown) for ${ECLASS}"
		;;
esac

if [[ ! ${_TWISTED_R1} ]]; then

inherit distutils-r1 versionator

fi # ! ${_TWISTED_R1}

EXPORT_FUNCTIONS src_install pkg_postinst pkg_postrm

if [[ ! ${_TWISTED_R1} ]]; then

# @FUNCTION: _twisted-r1_camelcase
# @USAGE: <pn>
# @DESCRIPTION:
# Convert dash-separated <pn> to CamelCase name suitable for Twisted.
# In pure bash, therefore safe for global scope execution.
_twisted-r1_camelcase() {
	local IFS=-

	# IFS=- splits words by -.
	local words=( ${1} )

	# we can't keep '-' as it collides with [a-z] check
	# and '' is used by bash-4 words[*], so let's just set globally
	IFS=

	if [[ ${BASH_VERSINFO[0]} -ge 4 ]]; then
		echo "${words[*]^}"
		return
	fi

	local w LC_COLLATE=C uc='ABCDEFGHIJKLMNOPQRSTUVWXYZ'

	local out
	for w in "${words[@]}"; do
		local fl=${w:0:1}

		# Danger: magic starts here. Please close your eyes.
		# In base 36, a..z represents digits 10..35. We substract 10
		# and get array subscripts for uc.

		[[ ${fl} == [a-z] ]] && fl=${uc:36#${fl} - 10:1}

		out+=${fl}${w:1}
	done

	echo "${out}"
}

# @ECLASS-VARIABLE: TWISTED_PN
# @DESCRIPTION:
# The real package name. Default to camel-case conversion of ${PN}.
#
# Example: TwistedCore
: ${TWISTED_PN:=$(_twisted-r1_camelcase ${PN})}

# @ECLASS-VARIABLE: TWISTED_P
# @DESCRIPTION:
# The real package name with version appended.
#
# It is used to build the default SRC_URI and S values.
#
# Example: TwistedCore-1.2.3
: ${TWISTED_P:=${TWISTED_PN}-${PV}}

# @ECLASS-VARIABLE: TWISTED_RELEASE
# @DESCRIPTION:
# The 'release' of Twisted. Defaults to the major & minor version
# number from ${PV}.
#
# It is used to build the default SRC_URI. It may be also used
# in dependencies against other Twisted packages.
#
# Example: 1.2
: ${TWISTED_RELEASE:=$(get_version_component_range 1-2 ${PV})}

HOMEPAGE="http://www.twistedmatrix.com/"
SRC_URI="http://twistedmatrix.com/Releases/${TWISTED_PN}"
SRC_URI="${SRC_URI}/${TWISTED_RELEASE}/${TWISTED_P}.tar.bz2"

LICENSE="MIT"
SLOT="0"
IUSE=""

S=${WORKDIR}/${TWISTED_P}

# @ECLASS-VARIABLE: TWISTED_PLUGINS
# @DESCRIPTION:
# An array of Twisted plugins, whose cache is regenerated
# in pkg_postinst() and pkg_postrm() phases.
#
# If no plugins are installed, set to empty array.
declare -p TWISTED_PLUGINS &>/dev/null || TWISTED_PLUGINS=( twisted.plugins )

# @FUNCTION: twisted-r1_python_test
# @DESCRIPTION:
# The common python_test() implementation that suffices for Twisted
# packages.
twisted-r1_python_test() {
	local sitedir=$(python_get_sitedir)

	# Copy modules of other Twisted packages from site-packages
	# directory to the temporary directory.
	local libdir=${BUILD_DIR}/test/lib
	mkdir -p "${libdir}" || die
	cp -r "${ROOT}${sitedir}"/twisted "${libdir}" || die
	# Drop the installed module in case previous version conflicts with
	# the new one somehow.
	rm -fr "${libdir}/${PN/-//}" || die

	distutils_install_for_testing || die

	if [[ ${TEST_DIR} != ${BUILD_DIR}/test ]]; then
		eqawarn "twisted-r1 integrity check failed."
		eqawarn "TEST_DIR: ${TEST_DIR}"
		eqawarn "expected: ${BUILD_DIR}/test"
	fi

	cd "${TEST_DIR}"/lib || die
	trial ${PN/-/.} || die "Tests fail with ${EPYTHON}"
}

# @FUNCTION: python_test
# @DESCRIPTION:
# Default python_test() for Twisted packages. If you need to override
# it, you can access the original implementation
# via twisted-r1_python_test.
python_test() {
	twisted-r1_python_test
}

# @FUNCTION: twisted-r1_src_install
# @DESCRIPTION:
# Default src_install() for Twisted packages. Automatically handles HTML
# docs (unless HTML_DOCS is set explicitly) and manpages in Twisted
# packages.
twisted-r1_src_install() {
	[[ -d doc ]] && local HTML_DOCS=( "${HTML_DOCS[@]:-doc/.}" )
	[[ -d doc/man ]] && doman doc/man/*.[[:digit:]]

	distutils-r1_src_install
}

# @FUNCTION: _twisted-r1_create_caches
# @USAGE: <packages>...
# @DESCRIPTION:
# Create dropin.cache for plugins in specified packages. The packages
# are to be listed in standard dotted Python syntax.
_twisted-r1_create_caches() {
	# http://twistedmatrix.com/documents/current/core/howto/plugin.html
	"${PYTHON}" -c \
"import sys
sys.path.insert(0, '${ROOT}$(python_get_sitedir)')

fail = False

try:
	from twisted.plugin import getPlugins, IPlugin
except ImportError as e:
	if '${EBUILD_PHASE}' == 'postinst':
		raise
else:
	for module in sys.argv[1:]:
		try:
			__import__(module, globals())
		except ImportError as e:
			if '${EBUILD_PHASE}' == 'postinst':
				raise
		else:
			list(getPlugins(IPlugin, sys.modules[module]))
" \
		"${@}" || die "twisted plugin cache update failed"
}

# @FUNCTION: twisted-r1_update_plugin_cache
# @DESCRIPTION:
# Update and clean up plugin caches for packages listed
# in TWISTED_PLUGINS.
twisted-r1_update_plugin_cache() {
	[[ ${TWISTED_PLUGINS[@]} ]] || return

	local subdirs=( "${TWISTED_PLUGINS[@]//.//}" )
	local paths=( "${subdirs[@]/#/${ROOT}$(python_get_sitedir)/}" )
	local caches=( "${paths[@]/%//dropin.cache}" )

	# First, delete existing (possibly stray) caches.
	rm -f "${caches[@]}" || die

	# Now, let's see which ones we can regenerate.
	_twisted-r1_create_caches "${TWISTED_PLUGINS[@]}"

	# Finally, drop empty parent directories.
	rmdir -p "${paths[@]}" 2>/dev/null
}

# @FUNCTION: twisted-r1_pkg_postinst
# @DESCRIPTION:
# Post-installation hook for twisted-r1. Updates plugin caches.
twisted-r1_pkg_postinst() {
	_distutils-r1_run_foreach_impl twisted-r1_update_plugin_cache
}

# @FUNCTION: twisted-r1_pkg_postrm
# @DESCRIPTION:
# Post-removal hook for twisted-r1. Updates plugin caches.
twisted-r1_pkg_postrm() {
	_distutils-r1_run_foreach_impl twisted-r1_update_plugin_cache
}

_TWISTED_R1=1

fi # ! ${_TWISTED_R1}

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]

^ permalink raw reply	[flat|nested] 13+ messages in thread

end of thread, other threads:[~2013-08-04 12:37 UTC | newest]

Thread overview: 13+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-08-03 15:13 [gentoo-dev] [New eclass] twisted-r1.eclass Michał Górny
2013-08-03 15:54 ` Ulrich Mueller
2013-08-03 18:29   ` Michał Górny
2013-08-03 19:37     ` Ulrich Mueller
2013-08-03 19:42       ` Michał Górny
2013-08-03 19:37     ` Alex Xu
2013-08-03 19:47       ` Alex Xu
2013-08-03 19:49       ` Michał Górny
2013-08-03 21:57 ` Michał Górny
2013-08-03 22:28 ` Michał Górny
2013-08-04  5:15 ` [gentoo-dev] " Marien Zwart
2013-08-04  7:34   ` Michał Górny
2013-08-04 12:37 ` [gentoo-dev] " Michał Górny

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox