[gentoo-user] Installing python packages for different versions

[gentoo-user] Installing python packages for different versions

[Florian Philipp]

[-- Attachment #1: Type: text/plain, Size: 1052 bytes --]

Hi!

This is actually a follow-up for my thread "Trouble installing Plone".

Following scenario: I have packages which run on python-2.4 and other
packages which work with 2.5. Zope is a prominent example of the 2.4 gang.

The problem: When I emerge a python package, for example
dev-python/imaging, it is only installed in the
/usr/lib/python-2.*/site-packages directory of the python version which
is currently enabled by eselect.

Naturally, this is the most recent version: 2.5. However, as soon as a
python-2.4 package depends on one of these other packages, it just
doesn't work because it expects them in /usr/lib/python-2.4/site-packages.

How am I supposed to work around this?

My current workaround is to switch temporarily to python-2.4 and
re-emerge the necessary packages. However, this only works as long as
there are either 2.4-packages which depend on these re-emerged ones or
2.5-packages because portage purges the corresponding byte code in the
now disabled python directory.

Regards,

Florian Philipp


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 261 bytes --]

Re: [gentoo-user] Installing python packages for different versions

[Khanh Nguyen]

Hi Florian.

** (Sat, Jun 13, 2009 at 06:44:27PM +0200) Florian Philipp wrote:
> Hi!
> 
> This is actually a follow-up for my thread "Trouble installing Plone".
> 
> Following scenario: I have packages which run on python-2.4 and other
> packages which work with 2.5. Zope is a prominent example of the 2.4 gang.
> 
> The problem: When I emerge a python package, for example
> dev-python/imaging, it is only installed in the
> /usr/lib/python-2.*/site-packages directory of the python version which
> is currently enabled by eselect.
> 
> Naturally, this is the most recent version: 2.5. However, as soon as a
> python-2.4 package depends on one of these other packages, it just
> doesn't work because it expects them in /usr/lib/python-2.4/site-packages.
> 
> How am I supposed to work around this?

With regards to Zope and Plone, use python2.5 but have python2.4 installed
and use buildout[1] to fetch Zope/Plone.

Zope needs python2.4 with PIL. A quick fix, is to either symlink or
copy /usr/lib/python-2.5/site-packages/PIL* to /usr/lib/python-2.4/site-packages/

You also need dev-libs/libxml2 (with python) and symlink/copy to
/usr/lib/python-2.4/site-packages/{libxml2*,drv_libxml2}  

When Python dependencies are in place, you can use buildout to fetch Zope/Plone
For this you need dev-python/setuptools.


PS! When following the guide[1], remember to bootstrap and buildout with python2.4.
eg: python2.4 bootstrap.py && python2.4 ./bin/buildout

PSS! If you use sudo and have a restrictive umask for your user, you need to lower
it to eg 022 before creating the buildout enviroment.

> Regards,
> 
> Florian Philipp
> 

Good luck :)

[1] http://plone.org/documentation/tutorial/buildout
-- 
Khanh Nguyen

Re: [gentoo-user] Installing python packages for different versions

[Florian Philipp]

[-- Attachment #1: Type: text/plain, Size: 2320 bytes --]

Khanh Nguyen schrieb:
> Hi Florian.
> 
> ** (Sat, Jun 13, 2009 at 06:44:27PM +0200) Florian Philipp wrote:
>> Hi!
>>
>> This is actually a follow-up for my thread "Trouble installing Plone".
>>
>> Following scenario: I have packages which run on python-2.4 and other
>> packages which work with 2.5. Zope is a prominent example of the 2.4 gang.
>>
>> The problem: When I emerge a python package, for example
>> dev-python/imaging, it is only installed in the
>> /usr/lib/python-2.*/site-packages directory of the python version which
>> is currently enabled by eselect.
>>
>> Naturally, this is the most recent version: 2.5. However, as soon as a
>> python-2.4 package depends on one of these other packages, it just
>> doesn't work because it expects them in /usr/lib/python-2.4/site-packages.
>>
>> How am I supposed to work around this?
> 
> With regards to Zope and Plone, use python2.5 but have python2.4 installed
> and use buildout[1] to fetch Zope/Plone.
> 
> Zope needs python2.4 with PIL. A quick fix, is to either symlink or
> copy /usr/lib/python-2.5/site-packages/PIL* to /usr/lib/python-2.4/site-packages/
> 
> You also need dev-libs/libxml2 (with python) and symlink/copy to
> /usr/lib/python-2.4/site-packages/{libxml2*,drv_libxml2}  

Thanks for the hint! Interestingly, zope works without such a
symlink/copy at the moment. Maybe it isn't needed for the tasks I've
done so far?

> 
> When Python dependencies are in place, you can use buildout to fetch Zope/Plone
> For this you need dev-python/setuptools.
> 
> 
> PS! When following the guide[1], remember to bootstrap and buildout with python2.4.
> eg: python2.4 bootstrap.py && python2.4 ./bin/buildout
> 
> PSS! If you use sudo and have a restrictive umask for your user, you need to lower
> it to eg 022 before creating the buildout enviroment.
> 

Thanks for your answer but I have to say, this looks like a really
cumbersome workaround. Wouldn't it be better to make portage and
python-updater aware of this problem?

The update from python-2.4 to 2.5 was a minor one with only a few
incompatible packages. What shall happen when we stabilize 3.0? We'll
run into orders of magnitude more problems than we did till now if we
keep it as it is!

Do you think I should open a bug for this?


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 261 bytes --]

Re: [gentoo-user] Installing python packages for different versions

[Mike Kazantsev]

[-- Attachment #1: Type: text/plain, Size: 1176 bytes --]

On Sun, 14 Jun 2009 18:03:58 +0200
Florian Philipp <lists@f_philipp.fastmail.net> wrote:

...
> 
> Thanks for your answer but I have to say, this looks like a really
> cumbersome workaround. Wouldn't it be better to make portage and
> python-updater aware of this problem?
> 
> The update from python-2.4 to 2.5 was a minor one with only a few
> incompatible packages. What shall happen when we stabilize 3.0? We'll
> run into orders of magnitude more problems than we did till now if we
> keep it as it is!
> 
> Do you think I should open a bug for this?

I guess one way would be to create debian-like split between
packages for one python version and another, so we'll have
"pyxml-0.8.4-py2.4" and "pyxml-0.8.4-py2.5" and zope can request former
as a dependency, which seem to be quite an ugly solution.

Installing every package for each compatible python on system if some
use-flag like "multislot" is enabled (it might also be impossible for
some pkgs, which also sit in share/bin/lib) look better and somewhat
easier - just a eselech switch flip and +x (un)installs.

I wonder, what do you have in mind?

-- 
Mike Kazantsev // fraggod.net

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Installing python packages for different versions

[Florian Philipp]

[-- Attachment #1: Type: text/plain, Size: 1585 bytes --]

Mike Kazantsev schrieb:
> On Sun, 14 Jun 2009 18:03:58 +0200
> Florian Philipp <lists@f_philipp.fastmail.net> wrote:
> 
> ...
>> Thanks for your answer but I have to say, this looks like a really
>> cumbersome workaround. Wouldn't it be better to make portage and
>> python-updater aware of this problem?
>>
>> The update from python-2.4 to 2.5 was a minor one with only a few
>> incompatible packages. What shall happen when we stabilize 3.0? We'll
>> run into orders of magnitude more problems than we did till now if we
>> keep it as it is!
>>
>> Do you think I should open a bug for this?
> 
[...]
> 
> Installing every package for each compatible python on system if some
> use-flag like "multislot" is enabled (it might also be impossible for
> some pkgs, which also sit in share/bin/lib) look better and somewhat
> easier - just a eselech switch flip and +x (un)installs.
> 
> I wonder, what do you have in mind?
> 

I don't know. I'm not a python dev. Therefore I might not understand
every aspect of the problem. I thought about something like this:

eselect maintains a list of all enabled python slots and a primary one,
not just the primary one like now. If nothing else is specified, every
program uses this primary python version (just like now). Portage
installs or symlinks all files which end up in the site-packages
directory in the respective directories for every python slot enabled by
eselect.

python-updater could be augmented to do the necessary rebuilds when a
new version is added or an old one removed from the list.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 261 bytes --]

Re: [gentoo-user] Installing python packages for different versions

[Mike Kazantsev]

[-- Attachment #1: Type: text/plain, Size: 762 bytes --]

On Sun, 14 Jun 2009 19:53:38 +0200
Florian Philipp <lists@f_philipp.fastmail.net> wrote:

> eselect maintains a list of all enabled python slots and a primary one,
> not just the primary one like now. If nothing else is specified, every
> program uses this primary python version (just like now). Portage
> installs or symlinks all files which end up in the site-packages
> directory in the respective directories for every python slot enabled by
> eselect.

Many packages use C libs, linked against specific version of python.so
library, so running it with another python version than it was compiled
with will yield unpredictable results - it might work for 2.4-2.5 but
I'm pretty sure it won't for 2.6-3.1.

-- 
Mike Kazantsev // fraggod.net

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[gentoo-user] Re: Installing python packages for different versions

[Remy Blank]

[-- Attachment #1: Type: text/plain, Size: 553 bytes --]

Florian Philipp wrote:
> How am I supposed to work around this?

I install all Python versions I need from portage, then use
virtualenv[1] to create environments with different versions of Python,
where I install the needed packages with easy_install. Not as simple as
using Portage, but it works.

I'd also be very interested in some kind of "multislot" functionality,
where every Python package installs in all available Python slots. But
that's probably non-trivial to implement.

[1] http://pypi.python.org/pypi/virtualenv

-- Remy


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 187 bytes --]