[gentoo-dev] Python setuptools/eggs

[gentoo-dev] Python setuptools/eggs

[Anders Bruun Olsen]

Hi,

I wanted to try out the new and fabled TurboGears and found that it uses
the new python software distribution tool setuptools and is distributed
as python eggs. I can easily install it with setuptools/easy_setup, but
I wanted to make an ebuild for TurboGears, which is not just so with
setuptools and eggs.
I started making ebuilds for the dependencies, but it seems that
setuptools has a nasty habit of downloading dependencies for a package
if it hasn't been installed by setuptools. So when eg. the new SQLObject
0.7rc1 (which also uses setuptools) depends upon FormEncode (not to
mention depends upon setuptools) it does not care that I have already
emerged setuptools or FormEncode, it just downloads the egg for it and
installs it.

It seems that alot of python projects are starting to use
setuptools/easy_install/eggs, and granted, it is quite neat, but there
needs to figured out a way to handle installing these things through
Portage, or alot of new versions are going to be a pain to have
installed.

How should Portage handle this?

I was thinking that maybe hacking up a Gentoo specific version of
setuptools that would make it work together with Portage or something
like that, but it seems a bit un-gentoo-like.
A different approach could be to grab the .egg files and manually
install them in the ebuilds, which wouldn't be a problem for pure-python
packages, but would suck for packages with c-extensions included, since
some might end bound to a specific subset of archs.

Instead of me just blindly fumbling in the blind for a way to get
ebuilds for TurboGears and it's assorted dependencies, I thought it
would be better to get some official guidelines.

Some useful URLs:
http://peak.telecommunity.com/DevCenter/setuptools
http://peak.telecommunity.com/DevCenter/EasyInstall
http://peak.telecommunity.com/DevCenter/PythonEggs

-- 
Anders
-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GCS/O d--@ s:+ a-- C++ UL+++$ P++ L+++ E- W+ N(+) o K? w O-- M- V
PS+ PE@ Y+ PGP+ t 5 X R+ tv+ b++ DI+++ D+ G e- h !r y?
------END GEEK CODE BLOCK------
PGPKey: http://random.sks.keyserver.penguin.de:11371/pks/lookup?op=get&search=0xD4DEFED0
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Python setuptools/eggs

[Rob Cakebread]

Anders Bruun Olsen wrote:
<snip>
> It seems that alot of python projects are starting to use
> setuptools/easy_install/eggs, and granted, it is quite neat, but there
> needs to figured out a way to handle installing these things through
> Portage, or alot of new versions are going to be a pain to have
> installed.
> 
> How should Portage handle this?
> 

I've been working on an eclass for these but its not ready.
I was hoping it'd be as easy as the Ruby gems eclass, because
they are similar, but eggs don't play nicely in a sandbox yet.

You're probably best off using the straight source for TurboGears
and its dependencies if you want to install via ebuild for now.

Mail me off-list or join #gentoo-python if you're interested
in working on the easy_install eclass, please.



-- 
Rob Cakebread
Gentoo Linux Developer
Public Key: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x96BA679B
Key fingerprint = 5E1A 57A0 0FA6 939D 3258  8369 81C5 A17B 96BA 679B
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Python setuptools/eggs

[Anders Bruun Olsen]

On Sat, Oct 08, 2005 at 03:45:18PM -0700, Rob Cakebread wrote:
> >How should Portage handle this?
> I've been working on an eclass for these but its not ready.
> I was hoping it'd be as easy as the Ruby gems eclass, because
> they are similar, but eggs don't play nicely in a sandbox yet.

But aren't eggs a bit against the Gentoo philosophy? I mean there are
some eggs that contain precompiled C-extensions. Shouldn't it still be
source builds that just somehow work with setuptools?

> You're probably best off using the straight source for TurboGears
> and its dependencies if you want to install via ebuild for now.

The problem is that even straight source does not play nicely in
ebuilds. Take SQLObject 0.7rc1, when you use the distutils eclass, it
will still download setuptools as an egg and use that to do all
dependency checks and resolutions. For SQLObject you can just delete the
ez_setup dir after unpacking it, then it will use ordinary distutils
with no dependency resolution, but with some packages such as
FormEncode, you HAVE to use setuptools, or do everything manually in the
ebuild.

> Mail me off-list or join #gentoo-python if you're interested
> in working on the easy_install eclass, please.

Should we move this off-list?

-- 
Anders
-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GCS/O d--@ s:+ a-- C++ UL+++$ P++ L+++ E- W+ N(+) o K? w O-- M- V
PS+ PE@ Y+ PGP+ t 5 X R+ tv+ b++ DI+++ D+ G e- h !r y?
------END GEEK CODE BLOCK------
PGPKey: http://random.sks.keyserver.penguin.de:11371/pks/lookup?op=get&search=0xD4DEFED0
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Python setuptools/eggs

[Dave Nebinger]

> > I've been working on an eclass for these but its not ready.
> > I was hoping it'd be as easy as the Ruby gems eclass, because
> > they are similar, but eggs don't play nicely in a sandbox yet.
>
> But aren't eggs a bit against the Gentoo philosophy? I mean there are
> some eggs that contain precompiled C-extensions. Shouldn't it still be
> source builds that just somehow work with setuptools?

I may be a bit of-base, but since I don't know much about the ruby gems, I'm 
wondering how close of a situation this is with the perl cpan modules?  
They're integrated to operate using the distribution process of both cpan and 
portage...

Just a suggestion.  Like I said, I could be way off-base...
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Python setuptools/eggs

[Rob Cakebread]

Dave Nebinger wrote:

> I may be a bit of-base, but since I don't know much about the ruby gems, I'm 
> wondering how close of a situation this is with the perl cpan modules?  
> They're integrated to operate using the distribution process of both cpan and 
> portage...
> 

They are similar. There is a project called g-gem [1] that uses the 
gems.eclass to generate ebuilds.

A few of us had a good talk on Eggs on yesterday on irc and came up with
a few proposals. I've setup SVN/Trac[2] so we can get things organized.
Theres a very quickly dished-out eggs.eclass and test ebuild on the
Trac site.

[1] http://rubyforge.org/projects/g-gem/
[2] http://eggs.gentooexperimental.org/


-- 
Rob Cakebread
Gentoo Linux Developer
Public Key: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x96BA679B
Key fingerprint = 5E1A 57A0 0FA6 939D 3258  8369 81C5 A17B 96BA 679B
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Python setuptools/eggs

[Rob Cakebread]

Anders Bruun Olsen wrote:
> 
> But aren't eggs a bit against the Gentoo philosophy? I mean there are
> some eggs that contain precompiled C-extensions. Shouldn't it still be
> source builds that just somehow work with setuptools?
> 


We wouldn't use the precompiled C eggs. The main reason I'm looking
at eggs so soon is because packages may start using them exclusively
and we will have the choice of using them via portage or let people
install them manually via easy_install. This happened with Rails. They
have the source available but comes with no way to install it properly
and they only support gems installations officially, so the gems eclass
was born.


<snip>
>>Mail me off-list or join #gentoo-python if you're interested
>>in working on the easy_install eclass, please.
> 
> 
> Should we move this off-list?
> 

I haven't worked with easy_install for about a month so I haven't
got much to add, but I'm catching up on it tonight. Since nobody else on
the Python team has spoken up, I think they may not be caught up either.
I received a few mails from other interested people, and we're meeting
on irc for now.

Thanks,
Rob


-- 
Rob Cakebread
Gentoo Linux Developer
Public Key: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x96BA679B
Key fingerprint = 5E1A 57A0 0FA6 939D 3258  8369 81C5 A17B 96BA 679B
-- 
gentoo-dev@gentoo.org mailing list