[gentoo-amd64] building and installing 32bit python with amd64 portage without chroot

[gentoo-amd64] building and installing 32bit python with amd64 portage without chroot

[ducasse.isidore]

I'd like to test psyco with some massively-looped python code I use to run. Since psyco is x86-compatible only and depends on python, I assume I have to run a 32bit python.

I'd like to know how feasable it is to build and install such a python with my amd64 portage and multilib, without using a chroot.

After all, us amd64 users use to run 32bit firefox-bin or mplayer-bin without such a chroot. Unfortunately, there is no python-bin ebuild yet. Is it a sorcerer's duty to build it? Or rather, what sorcery skills are involved in doing such a thing?

I'm getting used to portage's neatness and flexibility about those concerns. Would crossdev be of any use?

I know cross-compiling issues are largely covered in many faqs and such, but it seems to me that this question is quite broad and not clearly answered by now. Forgive me if I'm wrong... and tell me!
-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: building and installing 32bit python with amd64 portage without chroot

[Duncan]

"ducasse.isidore@gmail.com" <ducasse.isidore@gmail.com> posted
20070310211905.e52737a1.ducasse.isidore@gmail.com, excerpted below, on 
Sat, 10 Mar 2007 21:19:05 +0100:

> I'd like to test psyco with some massively-looped python code I use to
> run. Since psyco is x86-compatible only and depends on python, I assume
> I have to run a 32bit python.

Without knowing the specifics of the application, if all it does does is 
run python scripts/bytecode, 64-bit python should do.  If it hooks into 
actual native binary machine code (the usual case of that is if it loads 
its own *.so* (so=shared object) libraries into python itself or if it 
loads python *.so* libraries into itself), then yes, you'll need the 32-
bit, because 64-bit shared objects don't work in 32-bit application nor 
do 32-bit shared objects work in 64-bit applications.

> I'd like to know how feasable it is to build and install such a python
> with my amd64 portage and multilib, without using a chroot.

Again, it all depends on shared objects... plus the required 32-bit 
headers from them... python needs to load into itself, therefore, that 
must be available for it to build and then to run.

> After all, us amd64 users use to run 32bit firefox-bin or mplayer-bin
> without such a chroot. Unfortunately, there is no python-bin ebuild yet.
> Is it a sorcerer's duty to build it? Or rather, what sorcery skills are
> involved in doing such a thing?

The firefox-bin and mplayer-bin packages are precompiled binaries 
designed and tested to run on a Gentoo amd64 system with only the 
appropriate emul-linux-x86-* libraries (and glibc, which as a special 
case package is compiled in both 32-bit and 64-bit versions for multilib 
amd64, and the stdc++ from gcc if it's a C++ program, likewise).  Thus, 
their dependencies are known and already included in the appropriate emul-
* library if necessary.  The same /cannot/ be said for $RANDOM_PROGRAM, 
which may or may not need additional libraries not included in the emul-
linux-x86-* stuff.

Thus, what you have to do is figure out what shared object may be needed, 
figure out if they are in the emul-linux-x86-*, and if not, compile them 
first.  Of course, each of those may have its own dependencies with which 
you have to do the same thing.  That is why if you are doing more than a 
couple trivial 32-bit programs, it quickly becomes much easier to create 
a 32-bit chroot and let portage do all that tracking, even if you end up 
compiling more stuff than you might otherwise need -- it quickly becomes 
simpler to do a bit of extra compiling than it is to figure out all those 
dependencies manually!

There are a couple ways to trace those dependencies.  One is to look at 
the dependencies in the ebuild and figure out whether they are actual 
binary (shared object) dependencies, or whether they are called as 
applications. (A dependency on grep or sed, for instance, will likely be 
so they can actually be called, not on a shared object in their package.)

The second and possibly more accurate (because it will account for system 
packages that aren't always specifically listed as dependencies in the 
ebuilds) method is to use "ldd `which python`" on your 64-bit version, so 
see what shared objects it uses, then check to see how many of them you 
already have in 32-bit, for example with "equery belongs libc.so.6", to 
use one shared object listed by ldd here.  As it happens, that's part of 
glibc, which as I already mentioned above is a special case, in that 
multilib profiles compile it for both 32-bit and 64-bit when emerging 
that package on amd64.  Thus, it has two equery belongs hits, one in 
/lib32 and one in /lib64, so that one I'd have.  

If you have 32-bit versions of everything ldd listed, you /may/ be fine 
(a quick look suggests most of those libraries are part of glibc, so no 
problem, but I didn't check them all).  It's possible, however, that 
you'll also need the headers, and I'm not sure they are merged for emul-* 
packages (tho they are for glibc and gcc as part of the toolchain).  Try 
compiling it and see -- don't forget to add the -m32 to get a 32-bit 
version.  You may also need to specify the library and header dir paths 
to search in, as part of the command line you feed to configure.  You can 
take a look in the python ebuild and see what kind of command line it 
uses, and whether it does anything else special you may want to do.

> I'm getting used to portage's neatness and flexibility about those
> concerns. Would crossdev be of any use?

Not really...  Crossdev would be useful on a 32-bit system trying to 
compile for amd64, or on a 64-bit only (no multilib) system trying to 
compile for 32-bit, but it wouldn't avoid the dependency issue I explain 
above, and would be more difficult than simply using -m32 with a multilib 
gcc when compiling the packages manually, since it's there to be used on 
a Gentoo multilib system.

> I know cross-compiling issues are largely covered in many faqs and such,
> but it seems to me that this question is quite broad and not clearly
> answered by now. Forgive me if I'm wrong... and tell me!

I think it's not clearly answered in part because explaining the details, 
and providing support when people get it wrong, can get rather complex.  
It's just easier to provide the details for the chroot method, since 
that's somewhat more automated and close enough to a regular Gentoo 
install that pretty much, one just documents the differences.

Also keep in mind that if one uses a standard x86 stage-3 and the GRP 
packages CD, building the chroot will actually go reasonably fast.  It's 
not like you'll be using it enough to have to worry about full 
optimizations or changing USE flags that much, as you would on a main 
system, so there really isn't a whole lot that /has/ to be recompiled, 
and it can mostly be installed from binary packages, making it go much 
faster than one might at first imagine.  In this case, python would be 
merged with the stage 3 itself, I believe, so one wouldn't even need the 
packages CD.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list