[gentoo-dev] fhs and multilib question

[gentoo-dev] fhs and multilib question

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 1180 bytes --]

Hi all,

I have been dealing with a bug in openrc which prompted me to look at
our directory structure for libraries on 64 bit systems. The bug will be
referenced below[1]. The problem in the bug isn't the location of
libraries, but the fact that there is a mount point stored under the
library directories.

Here is what I've found in fhs [2].

- /lib should always exist on all architectures.
- /lib64 should only exist on amd64, ppc64, sparc64 and s390x. It
	should hold 64 bit libraries, and /lib should hold 32 bit (or 31 bit
	on s390x) libraries.
- /lib should hold 64 bit libraries on ia64.
- FHS mentions /lib32 but doesn't really define what goes in it, and with
  the definition of when /lib64 is to be used, there doesn't seem to be a
  need for /lib32.
- Also, it seems questionable that /lib is a symlink to /lib64 on
  non-multilib systems. I think we should still have separate /lib and
  /lib64 directories.

Constructive criticism of this idea, thoughts, etc, are welcome.

If there is no opposition, what would it take for us to do this?

Thanks,

William

[1] http://bugs.gentoo.org/show_bug.cgi?id=381783
[2] http://www.pathname.com/fhs/pub/fhs-2.3.html#LIB64

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] fhs and multilib question

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1022 bytes --]

On Sun, 11 Sep 2011 11:50:28 -0500
William Hubbs <williamh@gentoo.org> wrote:

> Hi all,
> 
> I have been dealing with a bug in openrc which prompted me to look at
> our directory structure for libraries on 64 bit systems. The bug will
> be referenced below[1]. The problem in the bug isn't the location of
> libraries, but the fact that there is a mount point stored under the
> library directories.
> 
> Here is what I've found in fhs [2].
> 
> - /lib should always exist on all architectures.
> - /lib64 should only exist on amd64, ppc64, sparc64 and s390x. It
> 	should hold 64 bit libraries, and /lib should hold 32 bit (or
> 31 bit on s390x) libraries.
> - /lib should hold 64 bit libraries on ia64.

That's basically profiles/features/multilib. While amd64 uses
profiles/features/multilib/lib32.

> If there is no opposition, what would it take for us to do this?

Migration seems at least hard for users. Fresh systems should be simple
to deal with.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] fhs and multilib question

[Alexey Shvetsov]

Hi all,

There is a draft of fhs-3.0[1]

Also i think that lib should be symlink to lib64 on 64bit systems its 
at least
will be consistent

My 0.02 $CURRENCY.

[1] http://dev.linuxfoundation.org/~licquia/fhs-3.0-drafts/fhs.html


On Sun, 11 Sep 2011 11:50:28 -0500, William Hubbs wrote:
> Hi all,
>
> I have been dealing with a bug in openrc which prompted me to look at
> our directory structure for libraries on 64 bit systems. The bug will 
> be
> referenced below[1]. The problem in the bug isn't the location of
> libraries, but the fact that there is a mount point stored under the
> library directories.
>
> Here is what I've found in fhs [2].
>
> - /lib should always exist on all architectures.
> - /lib64 should only exist on amd64, ppc64, sparc64 and s390x. It
> 	should hold 64 bit libraries, and /lib should hold 32 bit (or 31 bit
> 	on s390x) libraries.
> - /lib should hold 64 bit libraries on ia64.
> - FHS mentions /lib32 but doesn't really define what goes in it, and 
> with
>   the definition of when /lib64 is to be used, there doesn't seem to 
> be a
>   need for /lib32.
> - Also, it seems questionable that /lib is a symlink to /lib64 on
>   non-multilib systems. I think we should still have separate /lib 
> and
>   /lib64 directories.
>
> Constructive criticism of this idea, thoughts, etc, are welcome.
>
> If there is no opposition, what would it take for us to do this?
>
> Thanks,
>
> William
>
> [1] http://bugs.gentoo.org/show_bug.cgi?id=381783
> [2] http://www.pathname.com/fhs/pub/fhs-2.3.html#LIB64

-- 
Best Regards,
Alexey 'Alexxy' Shvetsov
Petersburg Nuclear Physics Institute, Russia
Department of Molecular and Radiation Biophysics
Gentoo Team Ru
Gentoo Linux Dev
mailto:alexxyum@gmail.com
mailto:alexxy@gentoo.org
mailto:alexxy@omrb.pnpi.spb.ru

Re: [gentoo-dev] fhs and multilib question

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 355 bytes --]

Hi Alexey,

On Sun, Sep 11, 2011 at 08:29:34PM +0300, Alexey Shvetsov wrote:
> Hi all,
> 
> There is a draft of fhs-3.0[1]
> 
> Also i think that lib should be symlink to lib64 on 64bit systems its 
> at least
> will be consistent

I'm not sure how it makes things consistent, but if fhs 3 allows it, I
guess we are ok either way.

William

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] fhs and multilib question

[James Cloos]

>>>>> "WH" == William Hubbs <williamh@gentoo.org> writes:

WH> Here is what I've found in fhs [2].

[ the fhs's nonsense about lib vs lib32 vs lib64 ]

WH> If there is no opposition, what would it take for us to do this?

Please do not break the amd64 status quo; lib -> lib64 is the correct
thing to do for both existing and new installs.

The fhs fails to acknowledge the fact that the 64bit abi on amd64 is the
native abi, and the 32bit is just a legacy compatibility.  In reality
amd64 should be treated exactly like ia64.

The fhs got it wrong only due to an historical accident; (some at) amd
was/were initially concerned that w/o harping on the ia32 compatibility
as amd64's primary feature that they would loose sales.  There might
have been some reason to that argument at first -- at least for
designed-to-be-out-of-date dists like rhel.  But once 64 bit became
mainstream and 64 bit dists hit the streets that old argument lost all
of its value.

Arguably, it should be lib64 pointing to lib, but lib pointing to lib64
works and can make some logic easier, so it should remain as is.

Please ignore the fhs nonsense and leave it as it is.

Thank you.

-JimC
-- 
James Cloos <cloos@jhcloos.com>         OpenPGP: 1024D/ED7DAEA6

[gentoo-dev] multilib setup

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 1166 bytes --]

i dont care what the FHS has to say on the topic, so i havent bothered 
looking.  below is documentation on the current system, and where i plan on 
taking things in the future.

note that this only applies to amd64, ppc64, sparc64, and s390x systems where 
people think in terms of "32bit" and "64bit" ABIs (even though our ABI system 
isn't restricted at all to just these two states).

all current multilib systems use the sub-profile features/multilib/lib32/.  in 
there, we set SYMLINK_LIB=yes.  this causes the 32bit libs to move from their 
normal ABI location in "lib" to "lib32" and then symlink "lib" to "lib64".  
this is the only currently supported setup, but can be considered legacy.

in the future, multilib profiles we'll be moving to SYMLINK_LIB=no and using 
just features/multilib/.  so there will be no "lib32" dir at all, "lib" will 
be for 32bit libs (just like the matching 32bit arch), and "lib64" will be for 
64bit libs.

as for no-multilib systems, "lib64" will be the same, and "lib" will be 
symlinked to "lib64".  this will be easier i think to share files between 
multilib and non-multilib 64bit systems.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] fhs and multilib question

[Joshua Kinard]

[-- Attachment #1: Type: text/plain, Size: 1628 bytes --]

On 09/11/2011 12:50, William Hubbs wrote:

> Here is what I've found in fhs [2].
> 
> - /lib should always exist on all architectures.
> - /lib64 should only exist on amd64, ppc64, sparc64 and s390x. It
> 	should hold 64 bit libraries, and /lib should hold 32 bit (or 31 bit
> 	on s390x) libraries.
> - /lib should hold 64 bit libraries on ia64.
> - FHS mentions /lib32 but doesn't really define what goes in it, and with
>   the definition of when /lib64 is to be used, there doesn't seem to be a
>   need for /lib32.
> - Also, it seems questionable that /lib is a symlink to /lib64 on
>   non-multilib systems. I think we should still have separate /lib and
>   /lib64 directories.


MIPS clarification:
  o32 ABI (32bit) -> /lib
  n32 ABI (32bit w/ full access to 64bit address space/registers) -> /lib32
  n64 ABI (64bit) -> /lib64

In a full-on MIPS multilib, you can have all three folders present and
filled with their respective ABI libs/binaries.  One ABI will dominate the
system, though.  Right now, that's still o32, but not for much longer.  n32
will become the defacto soon, and o32 will be built only if needed (i.e. a
program exhibits issues in n32 until upstream can fix).  n64 is also an
as-needed basis, typically where a program derives a benefit from being true
64bit versus one of the other two.

-- 
Joshua Kinard
Gentoo/MIPS
kumba@gentoo.org
4096R/D25D95E3 2011-03-28

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 834 bytes --]

Re: [gentoo-dev] fhs and multilib question

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 621 bytes --]

On Sun, 11 Sep 2011 17:04:44 -0400
Joshua Kinard <kumba@gentoo.org> wrote:

> MIPS clarification:
>   o32 ABI (32bit) -> /lib
>   n32 ABI (32bit w/ full access to 64bit address space/registers)
> -> /lib32
>   n64 ABI (64bit) -> /lib64
> 
> In a full-on MIPS multilib, you can have all three folders present and
> filled with their respective ABI libs/binaries.  One ABI will
> dominate the system, though.  Right now, that's still o32, but not
> for much longer.

Looks like n32 is less '32' than o32 so a little weird classification
but I guess historical reasons.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

[gentoo-dev] Re: multilib setup

[Duncan]

Mike Frysinger posted on Sun, 11 Sep 2011 15:55:49 -0400 as excerpted:

> all current multilib systems use the sub-profile
> features/multilib/lib32/.
>  in there, we set SYMLINK_LIB=yes.  this causes the 32bit libs to move
> from their normal ABI location in "lib" to "lib32" and then symlink
> "lib" to "lib64". this is the only currently supported setup, but can be
> considered legacy.

As a long-time Gentoo/amd64 user...

It's worth noting that Gentoo was quite out in front on the amd64 thing, 
and this present "legacy" setup was developed pretty much at the same 
time as the FHS spec that deals with the issue.  At the time, lib could 
have gone either way, legacy 32-bit lib with lib64, or new 64-bit lib 
with lib32.

The early amd64/Gentoo implementors apparently decided to follow ia64 and 
make lib64 the 64-bit location, but for transition reasons, made lib a 
symlink to it, with lib32 the 32-bit version.

Then the standard went the other way (apparently they decided the 64-bit 
target needed ported anyway so making it lib64 wasn't a big deal, and 
many 32-bit libraries and packages, with binary-only packages a major 
consideraction, would then simply continue to work without any changes at 
all).

For a couple years, Gentoo/amd64 actively planned to switch, with one of 
the first steps being implementation of portage's
FEATURES=multilib-strict, to catch the "bad" packages so bugs could be 
filed and they could be fixed.  I personally ran this feature for some 
years and reported several bugs based on it.

But somehow, it never happened, and over time, as 64-bit became common 
and pretty much everything (at least everything freedomware) was ported 
and the existing setup worked better and better, the pressure to leave 
what was now working reasonably well, well enough alone, grew.

Thus we arrive at the current situation.  I had actually thought that 
Gentoo/amd64 at least had given up on finally turning off that symlink 
and making lib 32-bit, and decided to leave things as they were and let 
time continue to deprecate 32-bit until it's no longer an issue, but 
Vapier's post indicates otherwise.  Still, it has been "in the future" 
since I switched to Gentoo/amd64 in 2004, and as I said, with the 
pressure for 32-bit gradually going away and the current situation 
established now for quite some time and working reasonably well, 
honestly, is it even worth the upset and bugs it'll cause, any more?

Meanwhile, I personally switched to no-multilib a few years ago now, and 
turned off FEATURES=multilib-strict after filing one last bug on it soon 
thereafter, so I don't know the current state there, but last I used it, 
the feature wasn't often killing merges, indicating that problem pretty 
much sinks into the ordinary bug count background.

Unfortunately, I and others have occasionally tried breaking the symlink 
and having separate lib and lib64 dirs, and even with FEATURES=multilib-
strict guaranteeing I had no breakers of that nature on the system, it 
still proved to be a rather difficult config to try to run, because 
there's apparently a lot of packages installing scripts, etc, to one 
path, but then trying to call them by the other path.  Actually, last I 
tried it, openrc was one of the problem packages so I couldn't even 
complete a normal boot!

Unfortunately, automated tinderbox testing for that isn't as easy as it 
might at first appear, since some packages will legitimately install both 
arch-neutral content to lib, and multilib content to lib64.  But not all 
those will be false-positives either, if they still install something to 
one but look for it in the other.  (AFAIK this was the problem with some 
of the openrc shell scripts, arguably arch-neutral and called from lib, 
but all installed (at least now) to (subdirs of) lib64.  But, last time I 
tried it was before the openrc stabilization push, IIRC, so it's quite 
possible things have changes since then.)

> in the future, multilib profiles we'll be moving to SYMLINK_LIB=no and
> using just features/multilib/.  so there will be no "lib32" dir at all,
> "lib" will be for 32bit libs (just like the matching 32bit arch), and
> "lib64" will be for 64bit libs.
> 
> as for no-multilib systems, "lib64" will be the same, and "lib" will be
> symlinked to "lib64".  this will be easier i think to share files
> between multilib and non-multilib 64bit systems.

Thanks for that update, Mike.  That is likely to be the least problematic 
for no-multilib users, so I at least don't have to worry about it so 
much. =:^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] fhs and multilib question

[Joshua Kinard]

[-- Attachment #1: Type: text/plain, Size: 2939 bytes --]

On 09/11/2011 17:47, Michał Górny wrote:

> Looks like n32 is less '32' than o32 so a little weird classification
> but I guess historical reasons.


In a way, yeah.  The gory details are in the N32 ABI guide here, if your
curiosity is morbid enough:
ftp://ftp.linux-mips.org/pub/linux/mips/doc/ABI2/MIPS-N32-ABI-Handbook.pdf

To quote a small excerpt from that:

-------------
Limitations of the 32-bit ABI

The 32-bit ABI was designed essentially for the R3000. It cannot be extended
to use new performance-related features and instructions of the R4400 and
beyond. For example:

   * 16 of the 32 floating point registers cannot be used.
   * 64-bit arithmetic instructions cannot be used.
   * 64-bit data movement instructions cannot be used.
   * MIPS4/R8000 instructions cannot be used.

Because of this, the performance available from the chip is lost. Floating
point intensive programs are especially hurt by these limitations; indeed
some are 50%-100% slower!


Limitations of the 64-bit ABI

Although the 64-bit ABI exploits many performance-related features of the
MIPS architecture, it also has problems. These include the following:
   * Porting code from the 32-bit ABI to the 64-bit ABI typically
     requires some recoding.
   * When ported from the 32-bit ABI to the 64-bit ABI, some C programs
     get significantly larger.


Motivation for the N32 ABI

Many ISVs and customers are finding it difficult to port to the 64-bit ABI.
An ABI was needed with all of the performance advantages of the 64-bit ABI,
but with the same data type sizes as the 32-bit ABI to allow ease of porting.
-------------

And to add to that, "N32 uses 64-bit registers but restricts addresses to 32
bits."

So we sometimes refer to it as a hybrid ABI, because it's in between the
32-bit and 64-bit worlds.  It actually requires a 64-bit compiler (either an
n32 mips64[el]-* OR n64mips64[el]-* toolchain) to even generate the code.

And it drove the glibc developers, especially Ulrich, absolutely insane.
One of the bigger reasons why non-x86 glibc code was spun off into -ports
several years ago, so they could speed up the glibc release cycle.


To give you an idea of the age of the n32 and o32 ABIs, IRIX 5.x (and IRIX
6.0) was the latest IRIX release to use o32.  All releases under IRIX 6.1
and up were n32.  And IRIX 6.1 came out some time after 1994, probably in
1995 (IRIX 6.5, the last branch, didn't start until 1998).  So Linux support
for n32 has been literal eons in the making.  It is seriously old school
stuff here.  I was just starting middle school when n32 became standard.

-- 
Joshua Kinard
Gentoo/MIPS
kumba@gentoo.org
4096R/D25D95E3 2011-03-28

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 834 bytes --]

Re: [gentoo-dev] Re: multilib setup

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 770 bytes --]

On Sunday, September 11, 2011 19:10:26 Duncan wrote:
> But somehow, it never happened, and over time, as 64-bit became common
> and pretty much everything (at least everything freedomware) was ported
> and the existing setup worked better and better, the pressure to leave
> what was now working reasonably well, well enough alone, grew.

the reason for the slouching is simply man power/interest.  the people 
responsible for the amd64 port and multilib design moved on for various 
reasons, and the multilib support pretty much stayed unchanged.  of late, ive 
been cleaning up and absorbing multilib responsibility as we support more ABIs 
with our various non-amd64 targets.  and with x32 coming, we need this stuff 
to be in place ahead of time.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] multilib setup

[Nathan Phillip Brink]

[-- Attachment #1: Type: text/plain, Size: 394 bytes --]

On Sun, Sep 11, 2011 at 03:55:49PM -0400, Mike Frysinger wrote:
> as for no-multilib systems, "lib64" will be the same, and "lib" will be 
> symlinked to "lib64".  this will be easier i think to share files between 
> multilib and non-multilib 64bit systems.

Isn't this slightly more complicated than it needs to be? Why not just
use /lib and no symlink for no-multilib 64-bit systems?

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] multilib setup

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 596 bytes --]

On Monday, September 12, 2011 15:19:29 Nathan Phillip Brink wrote:
> On Sun, Sep 11, 2011 at 03:55:49PM -0400, Mike Frysinger wrote:
> > as for no-multilib systems, "lib64" will be the same, and "lib" will be
> > symlinked to "lib64".  this will be easier i think to share files between
> > multilib and non-multilib 64bit systems.
> 
> Isn't this slightly more complicated than it needs to be? Why not just
> use /lib and no symlink for no-multilib 64-bit systems?

all the 64bit binaries have /lib64/ld-linux-x86-64.so.2 encoded in them, so 
/lib64/ must exist on the system.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] multilib setup

[Rich Freeman]

[-- Attachment #1: Type: text/plain, Size: 1012 bytes --]

On Mon, Sep 12, 2011 at 3:19 PM, Nathan Phillip Brink <binki@gentoo.org>wrote:

> On Sun, Sep 11, 2011 at 03:55:49PM -0400, Mike Frysinger wrote:
> > as for no-multilib systems, "lib64" will be the same, and "lib" will be
> > symlinked to "lib64".  this will be easier i think to share files between
> > multilib and non-multilib 64bit systems.
>
> Isn't this slightly more complicated than it needs to be? Why not just
> use /lib and no symlink for no-multilib 64-bit systems?
>

Well, the last sentence you quoted is the answer: "this will be easier i
think to share files between multilib and non-multilib 64bit systems."

Whether having to maintain separate binaries for multilib and non-multilib
amd64 systems is worth the tradeoff of having two lib directories is up for
debate I guess.  I'm not sure how many people install precompiled 64-bit
binaries on their systems, but anybody who does maintain such a setup would
probably prefer not to have to build against libraries both in /lib64 and
/lib.

Rich

[-- Attachment #2: Type: text/html, Size: 1397 bytes --]