[gentoo-python] twisted-r1.eclass postrm failure for new twisted ebuilds

[gentoo-python] twisted-r1.eclass postrm failure for new twisted ebuilds

[Brian Dolbec]

[-- Attachment #1: Type: text/plain, Size: 1777 bytes --]


OK, I've got the new twisted ebuild almost ready.  I have the tests
working, only diabling the network test group and one other that was
disabled in the split pkg ebuild.

But, the twisted-r1 postrm function is not working correctly.  I have
been overriding it in the ebuild so far.  It seems that there is an
EPYTHON issue...

I'll let the traceback speak for itself.

 * python2_7: running distutils-r1_run_phase twisted-r1_update_plugin_cache
 * python3_4: running distutils-r1_run_phase twisted-r1_update_plugin_cache
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib64/python3.4/distutils/sysconfig.py", line 13, in <module>
    import re
  File "/usr/lib64/python3.4/re.py", line 336, in <module>
    import copyreg
  File "/usr/lib64/python2.7/site-packages/copyreg/__init__.py", line 7, in <module>
    raise ImportError('This package should not be accessible on Python 3. '
ImportError: This package should not be accessible on Python 3. 
Either you are trying to run from the python-future src folder or your installation
 of python-future is corrupted.



What I can't figure out is of it isn't EPYTHON that is screwed up, how the hell is 
this happening...  a simple py3.4 import re causes the 3.4 re.py to try and import a 
py2.7 version of copyreq.

I can't reproduce this in a terminal window with py3.4 session.  "import re" is fine and
works correctly.  So, it can't be a bad python install.

I'll work on pushing the changes to the twisted ebuilds in the morning to my buildbot-dev overlay.
But I have attached the WIP ebuild for 16.3.0.  You can grab the files dir from [1] if you want 
to try emerging it.

[1] https://github.com/dol-sen/buildbot-dev/tree/master/dev-python/twisted/files
-- 
Brian Dolbec <dolsen>


[-- Attachment #2: twisted-16.3.0.ebuild --]
[-- Type: application/octet-stream, Size: 5850 bytes --]

# Copyright 1999-2016 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Id$

EAPI=5
PYTHON_COMPAT=( python2_7 python3_{4,5})
PYTHON_REQ_USE="threads(+)"

inherit eutils flag-o-matic twisted-r1

DESCRIPTION="An asynchronous networking framework written in Python"
HOMEPAGE="https://github.com/twisted/twisted https://pypi.python.org/pypi/Twisted"
SRC_URI="mirror://pypi/${P:0:1}/twisted/Twisted-${PV}.tar.bz2 -> ${P}.tar.bz2"

LICENSE="MIT"
SLOT="0"
#KEYWORDS="~alpha ~amd64 ~arm ~arm64 ~hppa ~ia64 ~m68k ~ppc ~ppc64 ~s390 ~sh ~x86 ~x86-fbsd ~ia64-hpux ~x86-interix ~amd64-linux ~x86-linux ~ppc-macos ~x64-macos ~x86-macos ~sparc-solaris ~sparc64-solaris ~x64-solaris ~x86-solaris"
KEYWORDS="~amd64 ~x86"
IUSE="conch crypt dev http2 serial +soap test"

RDEPEND=">=dev-python/zope-interface-3.6.0[${PYTHON_USEDEP}]
	conch? (
		dev-python/gmpy[${PYTHON_USEDEP}]
		dev-python/pyasn1[${PYTHON_USEDEP}]
		>=dev-python/cryptography-0.9.1[${PYTHON_USEDEP}]
		>=dev-python/appdirs-1.4.0[${PYTHON_USEDEP}]
	)
	crypt? (
		>=dev-python/pyopenssl-0.13[${PYTHON_USEDEP}]
		dev-python/service_identity[${PYTHON_USEDEP}]
		dev-python/idna[${PYTHON_USEDEP}]
	)
	dev? (
		>=dev-python/twistedchecker-0.4.0[${PYTHON_USEDEP}]
		>=dev-python/pyflakes-1.0.0[${PYTHON_USEDEP}]
		$(python_gen_cond_dep '>=dev-python/twisted-dev-tools-0.0.2[${PYTHON_USEDEP}]' python2_7)
		dev-python/subunit[${PYTHON_USEDEP}]
		>=dev-python/sphinx-1.3.1[${PYTHON_USEDEP}]
		$(python_gen_cond_dep '>=dev-python/pydoctor-15.0.0[${PYTHON_USEDEP}]' python2_7)
	)
	serial? ( dev-python/pyserial[${PYTHON_USEDEP}] )
	soap? ( $(python_gen_cond_dep 'dev-python/soappy[${PYTHON_USEDEP}]' python2_7) )
	http2? (
		>=dev-python/hyper-h2-2.3.0[${PYTHON_USEDEP}]
		<dev-python/hyper-h2-3.0[${PYTHON_USEDEP}]
		>=dev-python/priority-1.1.0[${PYTHON_USEDEP}]
		<dev-python/priority-2.0[${PYTHON_USEDEP}]
	)
"
DEPEND="
	!dev-python/twisted-core
	!dev-python/twisted-conch
	!dev-python/twisted-mail
	!dev-python/twisted-names
	!dev-python/twisted-words
	!dev-python/twisted-web
	test? (
		dev-python/gmpy[${PYTHON_USEDEP}]
		dev-python/pyasn1[${PYTHON_USEDEP}]
		>=dev-python/cryptography-0.9.1[${PYTHON_USEDEP}]
		>=dev-python/appdirs-1.4.0[${PYTHON_USEDEP}]
		>=dev-python/pyopenssl-0.13[${PYTHON_USEDEP}]
		dev-python/service_identity[${PYTHON_USEDEP}]
		dev-python/idna[${PYTHON_USEDEP}]
		>=dev-python/pyflakes-1.0.0[${PYTHON_USEDEP}]
		dev-python/subunit[${PYTHON_USEDEP}]
		>=dev-python/sphinx-1.3.1[${PYTHON_USEDEP}]
		dev-python/pyserial[${PYTHON_USEDEP}]
	)
"

PATCHES=(
	# Respect TWISTED_DISABLE_WRITING_OF_PLUGIN_CACHE variable.
	"${FILESDIR}/${PN}-core-9.0.0-respect_TWISTED_DISABLE_WRITING_OF_PLUGIN_CACHE.patch"
)

python_prepare_all() {
	rm -R twisted/pair/test/test_*.py || die "rm twisted/pair/test/test_*.py FAILED"
	# Possibly due to over taxing of the distutils_install_for_testing function
	rm twisted/python/test/test_release.py || die "rm twisted/python/test/test_release.py FAILED"
	if [[ "${EUID}" -eq 0 ]]; then
		# Disable tests failing with root permissions.
		sed \
			-e "s/test_newPluginsOnReadOnlyPath/_&/" \
			-e "s/test_deployedMode/_&/" \
			-i twisted/test/test_plugin.py
	fi

	distutils-r1_python_prepare_all
}

python_compile() {
	if ! python_is_python3; then
		# Needed to make the sendmsg extension work
		# (see http://twistedmatrix.com/trac/ticket/5701 )
		local -x CFLAGS="${CFLAGS} -fno-strict-aliasing"
		local -x CXXFLAGS="${CXXFLAGS} -fno-strict-aliasing"
	fi

	distutils-r1_python_compile
}

python_test() {
	distutils_install_for_testing

	pushd "${TEST_DIR}"/lib > /dev/null || die

	# Skip broken tests.

	# http://twistedmatrix.com/trac/ticket/5375
	#sed -e "/class ZshIntegrationTestCase/,/^$/d" -i twisted/scripts/test/test_scripts.py \
	#	|| die "sed failed"

	# Prevent it from pulling in plugins from already installed twisted packages.
	#rm -f twisted/plugins/__init__.py

	# An empty file doesn't work because the tests check for doc strings in all packages.
	#echo "'''plugins stub'''" > twisted/plugins/__init__.py || die

	# Requires twisted-lore creating a circ. dep and fail even if installed.
	# test_loreDeprecation and test_exist failures appeared in version 14.0.0.
	# Possibly due to over taxing of the distutils_install_for_testing function
	#rm -f twisted/python/test/test_release.py || die
	#sed -e 's:test_loreDeprecation:_&:' -i twisted/test/test_twisted.py || die
	#sed -e 's:test_exist:_&:' -i twisted/python/test/test_dist3.py || die

	# Requires connection to the network
	#sed -e 's:test_multiListen:_&:' -i twisted/test/test_udp.py || die

	# Appeared in version 14.0.0; https://twistedmatrix.com/trac/ticket/7422; known failures
	# Upstream somewhat unreceptive
	# The last of these invokes a hang or a long delay
	#sed -e 's:test_dataReceivedThrows:_&:' \
	#	-e 's:test_resumeProducingThrows:_&:' \
	#	-e 's:test_resumeProducingAbortLater:_&:' \
	#	-e 's:test_resumeProducingAbort:_&:' \
	#	-e 's:test_fullWriteBufferAfterByteExchange:_&:' \
	#	-i twisted/internet/test/test_tcp.py || die
	#sed -e 's:test_logPrefix:_&:' -i twisted/internet/test/connectionmixins.py || die

	if ! "${TEST_DIR}"/scripts/trial twisted; then
		die "Tests failed with ${EPYTHON}"
	fi

	popd > /dev/null || die
}

python_install() {
	distutils-r1_python_install

	cd "${D%/}$(python_get_sitedir)" || die

	# create 'Twisted' egg wrt bug #299736
	#local egg=( Twisted_Core*.egg-info )
	#[[ -f ${egg[0]} ]] || die "Twisted_Core*.egg-info not found"
	#ln -s "${egg[0]}" "${egg[0]/_Core/}" || die

	# own the dropin.cache so we don't leave orphans
	touch twisted/plugins/dropin.cache || die
}

python_install_all() {
	distutils-r1_python_install_all

	newconfd "${FILESDIR}/twistd.conf" twistd
	newinitd "${FILESDIR}/twistd.init" twistd
}

pkg_postinst() {
	einfo "Install complete"
}

pkg_postrm(){
	einfo ""
}

Re: [gentoo-python] twisted-r1.eclass postrm failure for new twisted ebuilds

[Michał Górny]

Dnia 29 września 2016 09:06:59 CEST, Brian Dolbec <dolsen@gentoo.org> napisał(a):
>
>OK, I've got the new twisted ebuild almost ready.  I have the tests
>working, only diabling the network test group and one other that was
>disabled in the split pkg ebuild.
>
>But, the twisted-r1 postrm function is not working correctly.  I have
>been overriding it in the ebuild so far.  It seems that there is an
>EPYTHON issue...
>
>I'll let the traceback speak for itself.
>
>* python2_7: running distutils-r1_run_phase
>twisted-r1_update_plugin_cache
>* python3_4: running distutils-r1_run_phase
>twisted-r1_update_plugin_cache
>Traceback (most recent call last):
>  File "<string>", line 1, in <module>
>File "/usr/lib64/python3.4/distutils/sysconfig.py", line 13, in
><module>
>    import re
>  File "/usr/lib64/python3.4/re.py", line 336, in <module>
>    import copyreg
>File "/usr/lib64/python2.7/site-packages/copyreg/__init__.py", line 7,
>in <module>
>raise ImportError('This package should not be accessible on Python 3. '
>ImportError: This package should not be accessible on Python 3. 
>Either you are trying to run from the python-future src folder or your
>installation
> of python-future is corrupted.

At a first glance, it looks like you have some package installed that gets auto-imported magically and is broken itself, therefore breaking everything.

>
>
>
>What I can't figure out is of it isn't EPYTHON that is screwed up, how
>the hell is 
>this happening...  a simple py3.4 import re causes the 3.4 re.py to try
>and import a 
>py2.7 version of copyreq.
>
>I can't reproduce this in a terminal window with py3.4 session. 
>"import re" is fine and
>works correctly.  So, it can't be a bad python install.
>
>I'll work on pushing the changes to the twisted ebuilds in the morning
>to my buildbot-dev overlay.
>But I have attached the WIP ebuild for 16.3.0.  You can grab the files
>dir from [1] if you want 
>to try emerging it.
>
>[1]
>https://github.com/dol-sen/buildbot-dev/tree/master/dev-python/twisted/files


-- 
Best regards,
Michał Górny (by phone)

Re: [gentoo-python] twisted-r1.eclass postrm failure for new twisted ebuilds

[Mike Gilbert]

On Thu, Sep 29, 2016 at 3:06 AM, Brian Dolbec <dolsen@gentoo.org> wrote:
>
> OK, I've got the new twisted ebuild almost ready.  I have the tests
> working, only diabling the network test group and one other that was
> disabled in the split pkg ebuild.
>
> But, the twisted-r1 postrm function is not working correctly.  I have
> been overriding it in the ebuild so far.  It seems that there is an
> EPYTHON issue...
>
> I'll let the traceback speak for itself.
>
>  * python2_7: running distutils-r1_run_phase twisted-r1_update_plugin_cache
>  * python3_4: running distutils-r1_run_phase twisted-r1_update_plugin_cache
> Traceback (most recent call last):
>   File "<string>", line 1, in <module>
>   File "/usr/lib64/python3.4/distutils/sysconfig.py", line 13, in <module>
>     import re
>   File "/usr/lib64/python3.4/re.py", line 336, in <module>
>     import copyreg
>   File "/usr/lib64/python2.7/site-packages/copyreg/__init__.py", line 7, in <module>
>     raise ImportError('This package should not be accessible on Python 3. '
> ImportError: This package should not be accessible on Python 3.
> Either you are trying to run from the python-future src folder or your installation
>  of python-future is corrupted.
>
>
>
> What I can't figure out is of it isn't EPYTHON that is screwed up, how the hell is
> this happening...  a simple py3.4 import re causes the 3.4 re.py to try and import a
> py2.7 version of copyreq.
>
> I can't reproduce this in a terminal window with py3.4 session.  "import re" is fine and
> works correctly.  So, it can't be a bad python install.

See portage bug 595028.

https://bugs.gentoo.org/show_bug.cgi?id=595028

You can workaround the problem by adding something like this to any
pkg_* functions.

cd "${HOME}"