[gentoo-amd64] How to install multilib'ed version of any library (for real multilib for x86 and x86_64 for every libraries)?

[gentoo-amd64] How to install multilib'ed version of any library (for real multilib for x86 and x86_64 for every libraries)?

[Cheng Renquan]

Hello, gentoo amd64 developers,
  I found a big problem with gentoo amd64 recently:

I'm just switching from x86 to amd64 gentoo, several months ago, everything
goes well, but now I have a project that need to compile some 32bit binaries;
since my gcc and glibc both have multilib support, originally I don't think it's
a problem, just like this hellworld.c program:

$ gcc -o hello helloworld.c             # will generate a default
hello executable, 64bit;
$ gcc -m32 -o hello.x86 helloworld.c # specify to generate a 32bit executable

So when I want to compile a separate software source, I just setenv
CC="gcc -m32"
this seems reasonable, but finally it failed with an error,

http://pastebin.org/52915

Now I know it's because of this software need 32bit libbfd, while the
default 64bit
libbfd is provided by sys-devel/binutils, so I need a multilib'ed
binutils library,
but how do I install a 32bit of libbfd in gentoo way?

Furthermore, if compiling any other 32bit program on gentoo-amd64, it may need
other more 32bit of libraries,

Comparing other linux distros like fedora-x86_64 and debian-amd64, I knew there
is simple way to archive this goal, just install both binutils.x86_64
and binutils.i686
packages, but on gentoo-amd64, how can we do this in a similarly simple way?

How about add USE multilib support of every package that contains libraries?

I have asked this problem on #gentoo-amd64 channel, but seemed no people
there knew it, so please someone on the mailing list know how to resolve it?

Thanks very much,

-- 
Cheng Renquan (程任全), from Singapore

[gentoo-amd64] Re: How to install multilib'ed version of any library (for real multilib for x86 and x86_64 for every libraries)?

[Nikos Chantziaras]

On 11/14/2009 01:45 PM, Cheng Renquan wrote:
>[...]
> Furthermore, if compiling any other 32bit program on gentoo-amd64, it may need
> other more 32bit of libraries,
>
> Comparing other linux distros like fedora-x86_64 and debian-amd64, I knew there
> is simple way to archive this goal, just install both binutils.x86_64
> and binutils.i686
> packages, but on gentoo-amd64, how can we do this in a similarly simple way?
>
> How about add USE multilib support of every package that contains libraries?

Gentoo doesn't support this yet.  Work is underway to enable real 
multilib support though.

Re: [gentoo-amd64] Re: How to install multilib'ed version of any library (for real multilib for x86 and x86_64 for every libraries)?

[Cheng Renquan]

On Sat, Nov 14, 2009 at 11:16 PM, Nikos Chantziaras <realnc@arcor.de> wrote:
> On 11/14/2009 01:45 PM, Cheng Renquan wrote:
>> How about add USE multilib support of every package that contains
>> libraries?
>
> Gentoo doesn't support this yet.  Work is underway to enable real multilib
> support though.

Could you give some more clues? Any links to work been done on this
real multilib topic?
Maybe I can participate in working it out,
I have also made some ebuilds contribution to gentoo, the sunrise
overlay, with persistent passion in the hope of becoming a true gentoo
developer some day,

-- 
Dennis, Cheng Renquan (程任全), from Singapore

Re: [gentoo-amd64] Re: How to install multilib'ed version of any library (for real multilib for x86 and x86_64 for every libraries)?

[Justin]

[-- Attachment #1: Type: text/plain, Size: 1095 bytes --]

Cheng Renquan wrote:
> On Sat, Nov 14, 2009 at 11:16 PM, Nikos Chantziaras <realnc@arcor.de> wrote:
>> On 11/14/2009 01:45 PM, Cheng Renquan wrote:
>>> How about add USE multilib support of every package that contains
>>> libraries?
>> Gentoo doesn't support this yet.  Work is underway to enable real multilib
>> support though.
> 
> Could you give some more clues? Any links to work been done on this
> real multilib topic?
> Maybe I can participate in working it out,
> I have also made some ebuilds contribution to gentoo, the sunrise
> overlay, with persistent passion in the hope of becoming a true gentoo
> developer some day,
> 

As you are from sunrise you know Tommy. He assisting some users creating
this overlay:

http://wiki.github.com/sjnewbury/multilib-overlay

You can ask him and/or join #gentoo-multilib-overlay @ freenode.

Here is the official announcement on the dev ml

http://archives.gentoo.org/gentoo-dev/msg_c097f754162fa0432dd0998914df7a30.xml

and a tracker bug in b.g.o

http://bugs.gentoo.org/show_bug.cgi?id=145737

Cheers justin


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[gentoo-amd64] Re: How to install multilib'ed version of any library (for real multilib for x86 and x86_64 for every libraries)?

[Duncan]

Nikos Chantziaras posted on Sat, 14 Nov 2009 17:16:35 +0200 as excerpted:

> On 11/14/2009 01:45 PM, Cheng Renquan wrote:
>>[...]
>> Furthermore, if compiling any other 32bit program on gentoo-amd64, it
>> may need other more 32bit of libraries,
>>
>> Comparing other linux distros like fedora-x86_64 and debian-amd64, I
>> knew there is simple way to archive this goal, just install both
>> binutils.x86_64 and binutils.i686
>> packages, but on gentoo-amd64, how can we do this in a similarly simple
>> way?
>>
>> How about add USE multilib support of every package that contains
>> libraries?
> 
> Gentoo doesn't support this yet.  Work is underway to enable real
> multilib support though.

Nikos is correct... as far as he goes, but there are actually ways around 
it that he didn't describe.

First, for a limited subset of libraries, there are pre-compiled x86_32 
compatibility versions available.  This is the "fast" solution, workable 
for most mainstream stuff, but in addition to being limited to the 
mainstream libraries, it isn't really gentoo-like in some ways as the 
binaries are all precompiled, and for all the reasons that people choose 
Gentoo, that's just not really satisfying to many Gentooers.  Never-the-
less, it suffices for many apps and most 32-bit games.

The operative hint you need here is app-emulation/emul-linux-x86-*.  
Those packages are the various more-or-less mainstream pre-compiled 
x86_32 libraries.   FWIW, there's a limited number of executables, as 
well, normally with the -bin suffix.  mozilla-firefox-bin is one 
example.  These aren't as popular (some would say necessary) as they once 
were, as there are now various 64-bit binary plugins for stuff such as 
flash, but for quite some time 64-bit was out in the cold in that regard, 
so if you chose to run proprietary-ware at all, you generally found the 
32-bit versions helpful as they'd run the proprietary 32-bit-only plugins 
and codecs, etc.

The far less limited (virtually unlimited) and perhaps more Gentoo-ish 
but also MUCH more complicated and MUCH more work second alternative is 
to run a 32-bit chroot.  The problem is that at this point, none of the 
package managers (at least not portage, and I don't believe paludis or 
pkgcore have the feature either) can properly track multiple ABI/bitness 
versions of the same package.  Thus, for amd64, the package manager 
assumes all the packages are 64-bit, and if you were to merge one as 32-
bit, it would replace the 64-bit version of the same package.

The way around this is the 32-bit chroot.  The idea, basically, is to 
setup a second instance of Gentoo in a chroot, so the portage (or other 
PM) running there doesn't know about the main 64-bit system, and the 64-
bit system for the most part ignores the 32-bit chroot because it's not 
in its way.  You mount-bind a few subtrees, typically /tmp, often 
/home, a couple others (I was going to use etc, instead of "a couple 
others", but then realized that was ambiguous, NOT /etc!), into the 
chroot, so they're seen by both systems, start from a normal x86 stage 
tarball, and with some notable exceptions, basically, install an almost 
complete 32-bit system in the chroot.  In theory, you can skip many of 
the services, syslog, cron, etc, because the main system provides them 
for both.  In fact, since you have a nearly complete 32-bit system 
anyway, in many cases it's worth it to just do the whole thing (but maybe 
without the full X, kde/gnome, whatever, or maybe with it, depending on 
your goals for the respective sides), and then be able to dual boot to 
one or the other as desired.

There's more documentation, including a nice Gentoo/AMD64 32-bit chroot 
guide, available.  See the documentation section at the Gentoo/AMD64 
project page.  As you're asking about this, you probably haven't read the 
FAQ yet, so I'd start with that.  You're likely to find a few of your 
other questions answered there.  If after that (and reading the above) 
you're interested in the 32-bit chroot idea, you can read the 32-bit 
chroot guide and then decide if it's worth it or if you want to try the 
emul-linux-x86 stuff first.

Here's the Gentoo/AMD64 project page link.
Again, check the documentation section.

http://amd64.gentoo.org

One more possibility.  If you only have one or two libraries you need 
that aren't available as precompiled binaries in the emul-* packages, 
once you have the 32-bit dependencies installed, you may well be able to 
compile them manually, outside portage, using -m32.  However, once you 
get more than a couple to worry about, the complexity of trying to handle 
all the dependencies manually increases more or less geometrically 
proportional to the number of packages you're trying to handle, so it 
very quickly becomes easier to simply do the chroot and let portage's 
automation handle it.  YMMV.

Finally, a bit more detail on what Nikos hinted at.  There's a couple of 
experimental projects whereby portage is modified to be able to handle 
multiple ABI installations in parallel.  As a matter of practice, I don't 
know if they'll ever get merged, because we've gone this long without it, 
and as I mentioned, the worst need was along about 2006 or so, when a lot 
of folks had switched already but Flash and etc weren't yet available for 
64-bit, and even mainstream FLOSS apps like Open Office hadn't been 
ported.  Since pretty much everything mainstream FLOSS has been ported 
now, and the proprietaryware folks are coming around to 64-bit as well, 
there's far less need for multilib in general than there used to be, and 
the need/demand will be ever weaker with time.  If Gentoo/AMD64 got along 
this long without it, and since we're well past the hump, now, there's 
little real practical reason to worry about it -- for Gentoo/AMD64, at 
least.  But of course there's other archs it would benefit as well, some 
of which have more than two variants, and at least for the devs and arch-
testers, it could be very useful to be able to be able to install, test 
and otherwise work on multiple bitnesses at once.  But AMD64 was the big 
one, and the others too have gone this long without it.  So really, what 
it comes down to is whether enough devs prioritize it high enough to 
continue pushing it until it's not only working well, but until the other 
devs accept it as worth the hassle, an it becomes a part of portage (and/
or the other PMs) and Gentoo in general.  Yes, there's a couple devs 
working on it, but whether their pet project will remain of /enough/ 
interest to them for long enough to be worth pushing into mainline 
Gentoo, is anyone's guess.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

[gentoo-amd64] Re: How to install multilib'ed version of any library (for real multilib for x86 and x86_64 for every libraries)?

[Nikos Chantziaras]

On 11/15/2009 03:25 PM, Duncan wrote:
> There's a couple of experimental projects whereby portage is modified
> to be able to handle multiple ABI installations in parallel.  As a
> matter of practice, I don't know if they'll ever get merged, because
> we've gone this long without it, and as I mentioned, the worst need
> was along about 2006 or so, when a lot of folks had switched already
> but Flash and etc weren't yet available for 64-bit, and even
> mainstream FLOSS apps like Open Office hadn't been ported. Since
> pretty much everything mainstream FLOSS has been ported now, and the
> proprietaryware folks are coming around to 64-bit as well, there's
> far less need for multilib in general than there used to be, and the
> need/demand will be ever weaker with time.

Multilib is still very useful.  64-bit Firefox is slow as molasses 
currently (it uses the legacy javascript engine), and every 32-bit app 
that uses Qt or Gtk looks extremely ugly due to missing 32-bit Qt/Gtk 
styles.  Wine will never be 64-bit and it needs libraries not available 
in emul packages.

Also, lets not forget that many people would prefer a 32-bit userland 
running on a 64-bit kernel.

Bottom line, multilib will be needed (and useful) for many years to come.

Re: [gentoo-amd64] Re: How to install multilib'ed version of any library (for real multilib for x86 and x86_64 for every libraries)?

[Volker Armin Hemmann]

On Sonntag 15 November 2009, Nikos Chantziaras wrote:
> Wine will never be 64-bit and it needs libraries not available
> in emul packages.

oh reaaaally?

win64 (app-emulation/wine):
 Build a  64bit version of Wine (won't run Win32 binaries)

[gentoo-amd64] Re: How to install multilib'ed version of any library (for real multilib for x86 and x86_64 for every libraries)?

[Nikos Chantziaras]

On 11/15/2009 06:13 PM, Volker Armin Hemmann wrote:
> On Sonntag 15 November 2009, Nikos Chantziaras wrote:
>> Wine will never be 64-bit and it needs libraries not available
>> in emul packages.
>
> oh reaaaally?
>
> win64 (app-emulation/wine):
>   Build a  64bit version of Wine (won't run Win32 binaries)

Yarly.  It says right there "won't run Win32 binaries", which is what I 
meant in the first place :P

Re: [gentoo-amd64] How to install multilib'ed version of any library (for real multilib for x86 and x86_64 for every libraries)?

[raxas]

On Saturday 14 November 2009 12:45:28 Cheng Renquan wrote:

> Furthermore, if compiling any other 32bit program on gentoo-amd64, it may
>  need other more 32bit of libraries,

You can use a chroot wrapped with linux32 (sys-apps/util-linux) to build a 
secondary complete 32bit gentoo installation inside your 64bit box, which is 
something I do prefere because it prevents many nasty interferences between 64 
and 32 bit toolchains and libraries or broken platform detection in packages 
themselves.

see: http://www.gentoo.org/proj/en/base/amd64/howtos/index.xml?part=1&chap=2

raxas

Re: [gentoo-amd64] How to install multilib'ed version of any library (for real multilib for x86 and x86_64 for every libraries)?

[Cheng Renquan]

On Tue, Nov 17, 2009 at 10:54 PM, raxas <raxas@pavouk-v-rohu.net> wrote:
>> Furthermore, if compiling any other 32bit program on gentoo-amd64, it may
>>  need other more 32bit of libraries,
>
> You can use a chroot wrapped with linux32 (sys-apps/util-linux) to build a
[...]
Thank you, I know this approach already, but it will duplicate all
static files like etc, share, doc, man, var, ... wasting space;

Just a simple problem:
1) if other linux distros can achieve, why gentoo not?

Re: [gentoo-amd64] How to install multilib'ed version of any library (for real multilib for x86 and x86_64 for every libraries)?

[Homer Parker]

On Mon, 2009-11-30 at 02:14 +0800, Cheng Renquan wrote:
> On Tue, Nov 17, 2009 at 10:54 PM, raxas <raxas@pavouk-v-rohu.net> wrote:
> >> Furthermore, if compiling any other 32bit program on gentoo-amd64, it may
> >>  need other more 32bit of libraries,
> >
> > You can use a chroot wrapped with linux32 (sys-apps/util-linux) to build a
> [...]
> Thank you, I know this approach already, but it will duplicate all
> static files like etc, share, doc, man, var, ... wasting space;

	bind mount /etc and /var to the chroot (*If* everything is compatible),
use FEATUES="nodoc,noman,etc" in the make.conf in the chroot.

> Just a simple problem:
> 1) if other linux distros can achieve, why gentoo not?

	Simple answer, how many of them are installed from source? There is
work being done on true multilib support in gentoo, /join
#gentoo-multilib (or something named that obvious, I'm on a junk
internet connection atm and not sure this reply will make it out
today :P )

-- 
Homer Parker <hparker@gentoo.org>

Re: [gentoo-amd64] How to install multilib'ed version of any library (for real multilib for x86 and x86_64 for every libraries)?

[Barry Schwartz]

Cheng Renquan <crquan@gmail.com> skribis:
> Just a simple problem:
> 1) if other linux distros can achieve, why gentoo not?

This is a question worth thinking about, and here is my hypothesis,
based on my own way of thinking: Gentoo users tend to be people who
like to build their programs from sources, and programs built from
sources usually can be converted to run in 64-bit; so we are more
likely to help convert programs to 64-bit than to work on making
Gentoo multilib fit the standard. Also multilib builds would take
significantly longer, which doesn't matter at all to users of most
other distros, but it does to Gentoo users. For reasons like these, I
myself never have been forceful about multilib.

[gentoo-amd64] Re: How to install multilib'ed version of any library (for real multilib for x86 and x86_64 for every libraries)?

[Nikos Chantziaras]

On 11/29/2009 08:14 PM, Cheng Renquan wrote:
> Just a simple problem:
> 1) if other linux distros can achieve, why gentoo not?

Gentoo *does* provide a way to have full multilib support like in other 
distros; the "emul-linux-x86-*" binary packages.  However, only some 
popular and commonly needed packages are provided.

Since Gentoo source based, the approach to multilib taken by other 
distros doesn't fit well and Gentoo needs to come up with its own way of 
doing multilib.