[gentoo-amd64] file type not allowed in /usr/lib

[gentoo-amd64] file type not allowed in /usr/lib

[Will Briggs]

G'day everyone,

This is a bit of a "help me" but also a question about how things work.

The _"help me"_ relates to a bug I filed almost a month ago 
(http://bugs.gentoo.org/show_bug.cgi?id=98622) that has yet to gain a 
reply of _any sort_ from anyone.  (I'd be happy with a "go away n00b" or 
even a "not our problem, bug here", but nothing).

The issue happens on an emerge of skencil (a dependency of inkscape with 
USE=plugins I believe):

Compiling
/var/tmp/portage/skencil-0.6.16/image///usr/lib/skencil-0.6.16/Script/usersguide.py
...
install: `Examples' is a directory
gzip:
/var/tmp/portage/skencil-0.6.16/image/usr/share/doc/skencil-0.6.16//Examples: 
No
such file or directory
install: `Doc' is a directory
gzip: 
/var/tmp/portage/skencil-0.6.16/image/usr/share/doc/skencil-0.6.16//Doc:
No such file or directory
install: `Misc' is a directory
gzip: 
/var/tmp/portage/skencil-0.6.16/image/usr/share/doc/skencil-0.6.16//Misc:
No such file or directory
man:
prepallstrip:
strip: x86_64-pc-linux-gnu-strip --strip-unneeded
strip: x86_64-pc-linux-gnu-strip --strip-unneeded
    usr/lib/skencil-0.6.16/Lib/paxtkinter.so
    usr/lib/skencil-0.6.16/Lib/paxmodule.so
    usr/lib/skencil-0.6.16/Lib/streamfilter.so
    usr/lib/skencil-0.6.16/Lib/intl.so
    usr/lib/skencil-0.6.16/Sketch/Modules/skreadmodule.so
    usr/lib/skencil-0.6.16/Sketch/Modules/_sketchmodule.so
    usr/lib/skencil-0.6.16/Sketch/Modules/_type1module.so
    usr/lib/skencil-0.6.16/Sketch/Modules/pstokenize.so

!!! ERROR: media-gfx/skencil-0.6.16 failed.
!!! Function dyn_install, Line 1114, Exitcode 0
!!! File
/var/tmp/portage/skencil-0.6.16/image///usr/lib/skencil-0.6.16/Lib/paxtkinter.so
matches a file type that is not allowed in /usr/lib
!!! If you need support, post the topmost build error, NOT this status 
message.

So if anyone can help with a fix, please let me know.

The _question_ relates to the fact that I've seen this sort of error a 
number of times, but by the time I come across it it's been bugged and 
fixed and the next emerge sync fixes it.  Can someone let me know what 
the issue is here - (mainly so I can get a handle on hacking my own 
fixes when bug reports go unanswered)  Why can't a .so file be placed in 
/usr/lib?

I'm assuming there's some AMD64 specificity to this bug.  Because a) it 
hasn't got the attention that x86 bugs get and b) its a /usr/lib issue 
and I'm assuming multilib might be at play somehow.

And besides, I've been subscribed to this list for a while :-)

Point me elsewhere if I'm on another track.

Thanks,

W.



-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] file type not allowed in /usr/lib

[Simon Stelling]

Hi,

Will Briggs wrote:
> !!! ERROR: media-gfx/skencil-0.6.16 failed.
> !!! Function dyn_install, Line 1114, Exitcode 0
> !!! File
> /var/tmp/portage/skencil-0.6.16/image///usr/lib/skencil-0.6.16/Lib/paxtkinter.so 
> matches a file type that is not allowed in /usr/lib
> !!! If you need support, post the topmost build error, NOT this status 
> message.

This means that there are 64bit shared objects going to /usr/lib instead 
of /usr/lib64, which is wrong, but currently not really a bug, since 
/usr/lib is a link to /usr/lib64.

> So if anyone can help with a fix, please let me know.

Just remove 'multilib-strict' from your FEATURES.

> The _question_ relates to the fact that I've seen this sort of error a 
> number of times, but by the time I come across it it's been bugged and 
> fixed and the next emerge sync fixes it.  Can someone let me know what 
> the issue is here - (mainly so I can get a handle on hacking my own 
> fixes when bug reports go unanswered)  Why can't a .so file be placed in 
> /usr/lib?

It can, but it shouldn't. We want 64bit .sos to go to (/usr)/lib64 and 
32bit to (/usr)/lib32, because *taking a deep breath*:

Generally, we want to provide a fully multilib-capable system. That 
means that you will be able to compile every application for 32bit or 
64bit, depending on your mood ;). 32bit applications require 32bit 
libraries, otherwise they can't be linked against each other, same 
(obviously) for 64bit apps/libs. Now imagine you want appfoo (32bit) and 
appbar (64bit), both depending on libfoobar. So you need libfoobar twice 
-- once 64bit and once 32bit.

Now, if libfoobar would get installed to /usr/lib instead /usr/lib32 and 
/usr/lib64, it'd obviously overwrite itself. Depending on what you 
install first, appfoo or appbar will break, as they both require 
installing libfoobar. Also, if you would emerge -C appfoo && emerge 
depclean, appbar wouldn't work anymore, because they both provide 
exactly the same files.

The simple solution to this is making two directories and making sure 
that no 64bit shared object goes anywhere else than to (/usr)/lib64, and 
vice versa.

Regards,

-- 
Simon Stelling
Gentoo/AMD64 Operational Co-Lead
blubb@gentoo.org
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] file type not allowed in /usr/lib

[Will Briggs]

Simon Stelling wrote:
> Hi,
> 
> Will Briggs wrote:
> 
>> !!! ERROR: media-gfx/skencil-0.6.16 failed.
>> !!! Function dyn_install, Line 1114, Exitcode 0
>> !!! File
>> /var/tmp/portage/skencil-0.6.16/image///usr/lib/skencil-0.6.16/Lib/paxtkinter.so 
>> matches a file type that is not allowed in /usr/lib
>> !!! If you need support, post the topmost build error, NOT this status 
>> message.
> 
> 
> This means that there are 64bit shared objects going to /usr/lib instead 
> of /usr/lib64, which is wrong, but currently not really a bug, since 
> /usr/lib is a link to /usr/lib64.
> 
>> So if anyone can help with a fix, please let me know.
> 
> 
> Just remove 'multilib-strict' from your FEATURES.
> 

Excellent.  I learn some more detail just from that.  Thanks.

I've got my head pretty much around what multilib _should_ do 
eventually.  (Duncan's verbosity in the past has helped with that :-) 
especially in the many threads on the 2005.0 changeover).

I'm always keen on knowing the detail of where things are up to though - 
both wrt 2005.1 and dual-bitness of portage.  If Duncan's roadmap (in 
his post) is an invention does that mean that there isn't one?

I'm hanging out for portage becoming dual-bitness aware.  In the 'other 
issue' - the quest to make all ebuilds multilib-friendly - is there a 
standardised process for the amd64 mob to lodge 
"needs-to-be-multilibbed" bug reports?

W.
-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: file type not allowed in /usr/lib

[Duncan]

Will Briggs posted <42F53B6D.1080908@burnieanglican.org.au>, excerpted
below,  on Sun, 07 Aug 2005 08:36:29 +1000:

> !!! ERROR: media-gfx/skencil-0.6.16 failed. !!! Function dyn_install,
> Line 1114, Exitcode 0 !!! File
> /var/tmp/portage/skencil-0.6.16/image///usr/lib/skencil-0.6.16/Lib/paxtkinter.so
> matches a file type that is not allowed in /usr/lib !!! If you need
> support, post the topmost build error, NOT this status message.
> 
> So if anyone can help with a fix, please let me know.
> 
> The _question_ relates to the fact that I've seen this sort of error a
> number of times, but by the time I come across it it's been bugged and
> fixed and the next emerge sync fixes it.  Can someone let me know what
> the issue is here - (mainly so I can get a handle on hacking my own
> fixes when bug reports go unanswered)  Why can't a .so file be placed in
> /usr/lib?
> 
> I'm assuming there's some AMD64 specificity to this bug.  Because a) it
> hasn't got the attention that x86 bugs get and b) its a /usr/lib issue
> and I'm assuming multilib might be at play somehow.

I'm repeating what Simon said, but sometimes it helps to get it said a
couple different ways, and between the two you get the understanding. =8^)
So, here goes...

The immediate user-level fix, as Simon says, is kill the
FEATURES=multilib-strict.  That FEATURE is designed for testing for a
specific situation that works the way things are now, but will cause
serious issues when we go full LSB multilib (Gentoo-amd64 is kindof a
hybrid situation at present).  Those that understand what's going on and
want to test for this issue can set that FEATURE and portage will die
whenever it hits and ebuild or upstream tarball that's making unwarranted
assumptions about the system library locations.  Those who don't want the
hassle should either NOT set FEATURES=multilib-strict, or set
FEATURES=-multilib-strict, and the problem will go away as far as they are
concerned, because it works right now, even if it isn't quite right.

Here's the background.  As Simon explained, in ordered to be able to have
32-bit and 64-bit copies of the same library, they need to be kept in
different directories.  Traditionally, of course, the location has always
been lib, /lib, /usr/lib, etc.  Now, if there are going to be two library
locations, separated by bitness, there are two ways to go. Other archs
previously have gone both ways, some one way, some the other. The old
normal lib location can remain 32-bit, making it easier for legacy 32-bit
programs some of which might be binary-only and therefore not easily
modified, to continue to find their libraries and the like, with 64-bit
libs then put in lib64, if the one way is chosen.  If the other way is
chosen, the old lib location remains the system "main" bitness location,
64-bit on a normal mainly 64-bit distribution, with 32-bit then being
located in lib32.  Therefore, one way, you have a 32-bit lib and also have
lib64, the other, you have a 64-bit lib and also have lib32. One makes it
easier for 32-bit legacy apps that might be difficult to change if they
are binary-only, the other makes it easier (or at least more "natural")
for the system's main 64-bit libraries, so both have their good and bad
points.

As it happens, Gentoo started its amd64 project before there was an agreed
amd64 standard, and originally chose a 64-bit lib, with lib32 as well. In
addition to the "native" lib logic explained above, this is the way ia64
(Intel's Itanium aka "Itanic") handled things, so it did seem totally
logical.

Then they make the standard (LSB/FHS, that's Linux Standard Base, and File
Hierarchy Standard) for amd64, and turn the Gentoo table on its head!

Most of the commercial distributions decide to go with a 32-bit lib and
adding lib64, aiding compatibility with binary-only 32-bit apps, reasoning
that 64-bit is new and will require certain other changes anyway, so it's
easier to just make the change to lib64 at the same time and leave the
legacy location well enough alone, avoiding trouble in 32-bit land.
Additionally, this can be seen as logical as contrasted with ia64, because
with ia64, 64-bit is the ONLY real native bitness, 32-bit being ALWAYS
emulated, and slower, so it only makes sense to make lib the "native"
64-bit libs, with 32-bit libs relegated to the secondary lib32 location.
amd64, OTOH, is actually FULL DUAL BITNESS IN HARDWARE, so whether 64-bit
or 32-bit is "more native" really depends entirely on the approach taken
by the individual OS/distribution platform, NOT the hardware.  Thus, the
legacy issues that amd64 has both the blessing and the curse of being able
to deal with, simply weren't there for ia64, so while lib can be argued
both ways for amd64, the choice was really rather clear for ia64.

So... Gentoo quickly found itself going one way, while pretty much all of
the rest of the Linux world had settled on going the other way, with the
LSB/FHS multilib standard for amd64.  Now, Gentoo is primarily open
source, and there's certainly no law saying we /have/ to do it like
everyone else does, so we /could/ have simply continued doing things the
way we had been doing them.  However, there are two very good reasons to
change, despite the temporary confusion and pain it brings, while we are
in-process.

First, there's the binary compatibility issue, both with legacy apps and
moving forward.  It's simply harder to get binaries from other sources to
work, when they are designed with one set of assumptions about locations,
but installed on a system with a rather different set of assumptions.  As
I said, there's the 32-bit legacy thing, but consider continuing effect
going forward, as well.  If we stayed the way we are, every binary-only
installation a Gentoo-amd64 user ever made, including the new amd64
binaries, when they become common, would be a fight. Despite the temporary
confusion during the switch, therefore, switching will clearly make things
easier for every Gentoo-amd64 user as time moves forward.

Second, there's the upstream compatibility issue.  It has been observed
that despite the freedom to modify that the FLOSS (free, libre, and open
source software) community not only allows but encourages, one of the
major forces keeping the Linux community from splitting into a bunch of
proprietary fiefdoms much as the proprietary UNIX community split, is the
unifying/streamlining effect of open source sharing efficiencies.  While
there are the ever-present forces of differentiation, "my distribution is
better than yours because we have these features you don't" that split the
proprietary Unix community, the balancing force, the reason the same thing
doesn't happen in the open source community, is the fact that for the
mostly duplicated common code, it's ALWAYS more efficient to keep in line
with the rest of the community, so the maintenance and further development
efforts can continue to be shared, so each distribution only carries PART
of the burden, rather than ALL of it, as they do for anything proprietary
or non-standard they offer.  Thus, while there are always differences in
the areas a distribution considers important and valuable enough to be
worth the extra cost to maintain and continue development, the differences
in OTHER "unimportant" areas tend to be rather small, because of the
economics forces favoring sharing the burden, which can only happen if the
differences in the area remain relatively small or non-existent.  It's a
constant balancing act that by natural selection, continuously weeds out
the differences a distribution doesn't consider important enough to
support on its own.

Thus, the upstream compatibility issue resolves to this:  the closer we
are to accepted LSB/FHS standard, the less work we will have to do making
changes so that open source packages which generally target the loose
standards, will work on Gentoo.  Now, consider the fact that there's very
little disagreement that amd64 *IS* the successor to x86.  What
maintenance issues we have now because we choose to do it our own way will
only CONTINUE to increase in the future, as amd64 becomes the default and
VERY widely assumed common platform, against which the changes necessary
for all other platforms are diffed.  There's simply NO good reason to
continue to make all that extra work for ourselves, requiring entirely
unnecessary changes to each new version of each new package, from now,
almost certainly a good decade into the future, very likely a quarter
century into the future, if amd64 has the same run that x86 did, and
assuming of course Gentoo's around that long as well.

Together, these two reasons means the only /possible/ sensible alternative
is to bite that bullet, and exchange some temporary misery now, while the
conversion is in progress, for a /far/ more standards compliant multilib
setup, going forward.

That, then, is the background, how we came to be in the situation we are
in now, and why we are having to change things "in mid stream", as it
were.  The logical next questions are where are we now, and what will it
take to get us where we want to be?

Currently, Gentoo-amd64 has just about finished moving 64-bit libs out of
lib.  We have both a lib32, and a lib64, with lib being a symlink to
lib64, for those remaining packages (such as skencil, apparently) that
have so far fallen thru the cracks, and continue to make the early
Gentoo-amd64 assumption that lib is the 64-bit shared object location.
With 2005.1, I'm expecting FEATURES=multilib-strict will be the default,
to finally weed out as many of the few remaining back packages as
possible.  With 2006.0, it's possible that lib -> lib64 symlink can
finally be broken, and any remaining packages then WILL get bugs filed,
when someone tries to use them and has issues.  With luck, by 2006.1, it
should be safe to start doing basically the same thing with the lib32 ->
lib move, as we will have just got thru doing with the lib -> lib64 move.
First, there will be a symlink between the two, in another release or two,
the main packages will be changed and it'll be time to activate the
multilib-strict test for anything still ending up in lib32 instead of lib.
By 2007.0, therefore, if luck holds, that multilib-strict test can be the
default, to catch all the stragglers possible, and 2007.1 should then
hopefully be able to remove lib32, relegating it to the annuls of
Gentoo-amd64 history.  (Those .1 mentions assume that Gentoo continues
with the twice-yearly releases, thus, they mean second-half.)

However, that's really only half the issue.  The other half is portage,
which currently can't track 32-bit dependencies separate from its 64-bit
dependencies.  We had hoped that portage would have dual-bitness support
by 2005.1, but that's now looking impossible, since the new portage
remains only in CVS, there hasn't yet been even the usual hard masked for
testing alpha snapshot releases, let alone the -preX versions, which would
be the first ones that could even theoretically leave hard mask testing,
for ~arch testing.  Thus, it'll be 2006.0 at the earliest, before portage
will be able to handle dual-bitness tracking, keeping the 32-bit
dependencies separate from the 64-bit dependencies.  Until then,
Gentoo-amd64 32-bit support will remain spotty at best, requiring users
either install from tarball, or use a 32-bit chroot with its own portage
tracking 32-bit dependencies, for pretty much anything beyond the core
system libraries (sandbox, gcc, glibc, now from-source supported, other
32-bit core libraries managed as binary-only packages or from the chroot).

So... back to your error and multilib-strict.  So... suppose you've
decided to run multilib-strict and report the bugs you find.  Since
portage's method of notifying you of a multilib-strict violation is to
die, how do you finish the installation once you've gotten the notice,
filed the bug if necessary, and still want the package, before the bug is
fixed?

Well, as mentioned, the simplest, but /not/ necessarily shortest, method,
would be to simply...

FEATURES=-multilib-strict emerge --oneshot <whatever.trouble.pkg>

(Note that --oneshot.  Packages placing libraries incorrectly and
triggering this error will by definition mostly be libraries, that is,
dependencies, not applications you want listed in your world file.  Thus,
the --oneshot, to emerge it without adding it to your world file.)

HOWEVER, multilib-strict errors will normally be triggered ONLY during the
(fake-)install phase -- when the package is doing the fake-install to its
tempdir image directory.  It's at that point sandbox detects the attempt
to install a shared-object to lib, instead of lib64.  The catch is, by
that time, the major work of the emerge, all the compilation stuff, has
already been done.  The binaries should be perfectly fine, as they are the
ones that would normally be installed, if multilib-strict was off, and
would have been installed the way things worked before that feature was
ever added to portage.  However, emerge just died, so it could report the
problem.  Shouldn't there be a way to tell portage to go ahead and finish
the merge using the existing work, now that we know about the bug and have
presumably reported it if necessary, rather than starting ALL over again
with the package emerge, as the above command will do?

As it turns out, there /is/ such a way.  "ebuild" is the command used to
step thru an emerge of a single package.  You can call it specifically, to
merge a single package, or to step thru one step at a time (normally
fetch, unpack, compile, install, qmerge), and in fact, emerge itself calls
ebuild to handle each individual package, one at a time.  I'm not going to
go into detail on ebuild here, because it's all covered quite well in the
Gentoo Handbook and in the ebuild (1) (5) manpages.  However, that's the
command you use.  HOWEVER, you DO have to tweak one little thing, first,
to tell portage not to worry about that multilib-strict that you've
already dealt with.

Emerge dies with the multilib-strict error, as mentioned, during the
install step.  As mentioned, the compile is already done, the binaries
sitting in the tempdir work tree, ready to be moved over to the image dir,
during the install phase.  Portage will also, by that point, have created
a number of additional files in the package's working tempdir, one of
which contains the FEATURES line, which we have to modify, removing the
"multilib-strict" portion, so ebuild won't trigger once again on it, once
it gets to that point.  The file is
$PORTAGE_TMPDIR/portage/<packagename-version>/temp/environment .  Open
that in your favorite text editor and edit the FEATURES line, removing the
multilib-strict.  Save the file.  NOW you can issue your ebuild <pkg>
merge command, and it should quickly figure out the compile step is done,
and jump back into the install step, completing it and then finishing the
merge.

Whether it's worth the hassle of loading that file in your editor,
removing the multilib-strict, saving, and running ebuild to finish the
merge, against simply issuing the emerge command listed above, but going
thru the whole thing including the compile step again, will probably
depend on how big the package is.  For something taking less than five
minutes to emerge from start to finish, it's not worth the hassle, but it
might be, if that compile step takes say an hour...

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman in
http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html


-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: file type not allowed in /usr/lib

[Simon Stelling]

Hi,

Duncan wrote:
> I'm repeating what Simon said, but sometimes it helps to get it said a
[snip]
> The immediate user-level fix, as Simon says, is kill the
[snip]
> Here's the background.  As Simon explained, in ordered to be able to have

Just a personal note on this: What's the point in repeating everything? 
You only bore people with it. I think we can assume that the original 
poster either gets it the first time, or he'll surely ask again and give 
details about what he didn't understand.

> First, there's the binary compatibility issue, both with legacy apps and
> moving forward.  It's simply harder to get binaries from other sources to
> work, when they are designed with one set of assumptions about locations,
> but installed on a system with a rather different set of assumptions.  As
> I said, there's the 32-bit legacy thing, but consider continuing effect
> going forward, as well.  If we stayed the way we are, every binary-only
> installation a Gentoo-amd64 user ever made, including the new amd64
> binaries, when they become common, would be a fight. Despite the temporary

Why? When the binary gets linked, ldd just looks into various places and 
decides which library to take depending on the bitness. There's no 
fight, there's /etc/ld.so.conf. Sorry, but that paragraph is fubar.

> mostly duplicated common code, it's ALWAYS more efficient to keep in line
> with the rest of the community, so the maintenance and further development

I disagree: There are bad standards, which should be replaced by better 
ones. You can't do that by keeping in line with the rest.

> Thus, the upstream compatibility issue resolves to this:  the closer we
> are to accepted LSB/FHS standard, the less work we will have to do making
> changes so that open source packages which generally target the loose

Wrong. Correct would be: the closer we are to UPSTREAM, the less work we 
will have to do. A nice example: QT

> Together, these two reasons means the only /possible/ sensible alternative
> is to bite that bullet, and exchange some temporary misery now, while the
> conversion is in progress, for a /far/ more standards compliant multilib
> setup, going forward.

What misery?

> Currently, Gentoo-amd64 has just about finished moving 64-bit libs out of
> lib.  We have both a lib32, and a lib64, with lib being a symlink to
> lib64, for those remaining packages (such as skencil, apparently) that
> have so far fallen thru the cracks, and continue to make the early
> Gentoo-amd64 assumption that lib is the 64-bit shared object location.
> With 2005.1, I'm expecting FEATURES=multilib-strict will be the default,
> to finally weed out as many of the few remaining back packages as
> possible.  With 2006.0, it's possible that lib -> lib64 symlink can
> finally be broken, and any remaining packages then WILL get bugs filed,
> when someone tries to use them and has issues.  With luck, by 2006.1, it
> should be safe to start doing basically the same thing with the lib32 ->
> lib move, as we will have just got thru doing with the lib -> lib64 move.
> First, there will be a symlink between the two, in another release or two,
> the main packages will be changed and it'll be time to activate the
> multilib-strict test for anything still ending up in lib32 instead of lib.
> By 2007.0, therefore, if luck holds, that multilib-strict test can be the
> default, to catch all the stragglers possible, and 2007.1 should then
> hopefully be able to remove lib32, relegating it to the annuls of
> Gentoo-amd64 history.  (Those .1 mentions assume that Gentoo continues
> with the twice-yearly releases, thus, they mean second-half.)

Where do you get this information from? We (read: we, the Gentoo/AMD64 
developer team) never published such a roadmap, nor do we have one 
ourselves. Please, don't cook up a story just to enlarge your 
pen^W^W^Wmails.

> However, that's really only half the issue.  The other half is portage,
> which currently can't track 32-bit dependencies separate from its 64-bit
> dependencies.  We had hoped that portage would have dual-bitness support
> by 2005.1, but that's now looking impossible, since the new portage
> remains only in CVS, there hasn't yet been even the usual hard masked for
> testing alpha snapshot releases, let alone the -preX versions, which would
> be the first ones that could even theoretically leave hard mask testing,
> for ~arch testing.  Thus, it'll be 2006.0 at the earliest, before portage
> will be able to handle dual-bitness tracking, keeping the 32-bit
> dependencies separate from the 64-bit dependencies.  Until then,
> Gentoo-amd64 32-bit support will remain spotty at best, requiring users
> either install from tarball, or use a 32-bit chroot with its own portage
> tracking 32-bit dependencies, for pretty much anything beyond the core
> system libraries (sandbox, gcc, glibc, now from-source supported, other
> 32-bit core libraries managed as binary-only packages or from the chroot).

You can take comfort with the fact that even if portage already had this 
functionality, there wouldn't be real multilib now.

Regards,

-- 
Simon Stelling
Gentoo/AMD64 Operational Co-Lead
blubb@gentoo.org
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: file type not allowed in /usr/lib

[Simon Stelling]

I wrote:
>> Currently, Gentoo-amd64 has just about finished moving 64-bit libs out of
>> lib.  We have both a lib32, and a lib64, with lib being a symlink to
>> lib64, for those remaining packages (such as skencil, apparently) that
>> have so far fallen thru the cracks, and continue to make the early
>> Gentoo-amd64 assumption that lib is the 64-bit shared object location.
>> With 2005.1, I'm expecting FEATURES=multilib-strict will be the default,
>> to finally weed out as many of the few remaining back packages as
>> possible.  With 2006.0, it's possible that lib -> lib64 symlink can
>> finally be broken, and any remaining packages then WILL get bugs filed,
>> when someone tries to use them and has issues.  With luck, by 2006.1, it
>> should be safe to start doing basically the same thing with the lib32 ->
>> lib move, as we will have just got thru doing with the lib -> lib64 move.
>> First, there will be a symlink between the two, in another release or 
>> two,
>> the main packages will be changed and it'll be time to activate the
>> multilib-strict test for anything still ending up in lib32 instead of 
>> lib.
>> By 2007.0, therefore, if luck holds, that multilib-strict test can be the
>> default, to catch all the stragglers possible, and 2007.1 should then
>> hopefully be able to remove lib32, relegating it to the annuls of
>> Gentoo-amd64 history.  (Those .1 mentions assume that Gentoo continues
>> with the twice-yearly releases, thus, they mean second-half.)
> 
> 
> Where do you get this information from? We (read: we, the Gentoo/AMD64 
> developer team) never published such a roadmap, nor do we have one 
> ourselves. Please, don't cook up a story just to enlarge your 
> pen^W^W^Wmails.

First of all, it should be one ^W ;)
Second, the default FEATURES in 2005.1 will be the same as in 2005.0:

$ diff -u 2005.0/make.defaults 2005.1/make.defaults
--- 2005.0/make.defaults        2005-07-01 23:14:23.000000000 +0200
+++ 2005.1/make.defaults        2005-06-04 23:20:09.000000000 +0200
@@ -1,6 +1,6 @@
  # Copyright 1999-2005 Gentoo Foundation
  # Distributed under the terms of the GNU General Public License v2
-# $Header: 
/var/cvsroot/gentoo-x86/profiles/default-linux/amd64/2005.0/make.defaults,v 
1.15 2005/07/01 21:14:23 blubb Exp $
+# $Header: 
/var/cvsroot/gentoo-x86/profiles/default-linux/amd64/2005.1/make.defaults,v 
1.1 2005/06/04 21:20:09 eradicator Exp $

  #removed because managers vote requires us to do so
  #("when we go nptl, we all go nptl")

Regards,

-- 
Simon Stelling
Gentoo/AMD64 Operational Co-Lead
blubb@gentoo.org
-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: Re: file type not allowed in /usr/lib

[Duncan]

Simon Stelling posted <42F66B78.1030708@gentoo.org>, excerpted below,  on
Sun, 07 Aug 2005 22:13:44 +0200:

> Hi,
> 
> Duncan wrote:
>> I'm repeating what Simon said, but sometimes it helps to get it said a
> [snip]
> 
> Just a personal note on this: What's the point in repeating everything?
> You only bore people with it. I think we can assume that the original
> poster either gets it the first time, or he'll surely ask again and give
> details about what he didn't understand.

OK, first, the tone here and below seems to indicate I offended you in
some way.  Let me unconditionally state at the outset, that such was in no
way the intent.   It appears you took my post as somehow implying yours
was incomplete or inadequate in some way.  Again, that couldn't have been
farther from my intentions!  I intended to credit you where appropriate,
acknowledging the overlap in ideas, but would have posted whether you had
or not, because I had some of my own ideas to express, in my own reply, as
is customary on public lists and newsgroups.  I evidently came across in a
way I never intended at all, and for that I unconditionally apologize.

The following, therefore, is simply replying to your question, explaining
(again, I /did/ mention it in the original post) why I did repeat some of
what you had already covered.

Not just repeating, but saying it in different words.  As I explained and
as educators well know, reading something said in two different ways
at minimum reinforces the concept, and will in many cases allow someone to
grasp a concept put in different terms, where they couldn't get it as it
was stated the first time.  I'm sure some find your version easier to
understand, and as a dev, it's closer to the horse's mouth in terms of
official policy, certainly.  However, I /know/ from being directly told so
any number of times, that for whatever reason, many find my explanations
of a concept easier to understand than all the explanations they may have
read before.

Anyway, if I'm boring people, there are always the filters, if they
ultimately find my opinion so objectionable that simply skipping over it
isn't enough.  I know others find my posts useful, because they've said as
much.

>> First, there's the binary compatibility issue, both with legacy apps and
>> moving forward.  It's simply harder to get binaries from other sources
>> to work, when they are designed with one set of assumptions about
>> locations, but installed on a system with a rather different set of
>> assumptions.  As I said, there's the 32-bit legacy thing, but consider
>> continuing effect going forward, as well.  If we stayed the way we are,
>> every binary-only installation a Gentoo-amd64 user ever made, including
>> the new amd64 binaries, when they become common, would be a fight.
>> Despite the temporary
> 
> Why? When the binary gets linked, ldd just looks into various places and
> decides which library to take depending on the bitness. There's no fight,
> there's /etc/ld.so.conf. Sorry, but that paragraph is fubar.

Not necessarily.  You are correct about dynamic linking, but what about
binary only installers?  These may be hard coded to a specific
path, certainly by default, tho it's usually possible to change it if
necessary.  Even if it's changeable on the command line, or from a script,
or even if it's open source, making and maintaining that change thru
multiple versions over time is certainly more difficult than never having
to make it in the first place, because our default happens to be the
standard default.  If it's prepackaged as an ebuild, that's work the
Gentoo developers have to do (or not, if the defaults coincide).  If it's
something a Gentoo user is installing on their own, not from an ebuild,
then having to fiddle with installer configs to get it to go to the right
place is "a fight" that /could/ be avoided, if we were using the defaults
in the first place.

>> mostly duplicated common code, it's ALWAYS more efficient to keep in
>> line with the rest of the community, so the maintenance and further
>> development
> 
> I disagree: There are bad standards, which should be replaced by better
> ones. You can't do that by keeping in line with the rest.

That doesn't alter the truth of the point I made.  It's ALWAYS more
efficient (that is, less work) to keep in line with the rest of the
community.  That does NOT always mean it's BETTER, only that it's MORE
EFFICIENT.  The point I made was that this factor serves as a unifier
within the community, always ensuring that distributions diverge only on
the points they consider IMPORTANT, where the extra work is therefore
worth it, as opposed to any old thing not considered so important, in
which case it makes the most sense to go with the community standard,
since that's less work, and the extra work necessary to deviate isn't
considered worth it, since it's "not important".

By definition, if it's a "bad" standard, it's important enough to make a
difference, to be /worth/ the extra work involved in diverging from that
standard.  Thus, I agree with you, in that there ARE valid reasons to
disagree with a standard, to go your (our) own way.  I'm only saying that
doing so unavoidably incurs an additional cost in terms of resources
required to maintain that change.  It doesn't pay to be different just to
be different.  Because there's an unavoidable cost involved with being
different, do it where it's worth the trouble, don't do it where it's not.

Thus, there are dual but opposing forces, normally keeping each other in
balance, the force leading to differentiation from the competition, on the
one hand, with the avoidable cost of that differentiation on the other,
ensuring that it's /only/ done where the benefits are judged to be worth
more than the costs.  This is one of the strengths of open source, and
the most common reason pointed to when someone asks what's keeping us
together, as contrasted with the fragmentation of proprietary Unix. 
Because they were proprietary, each implementation had to bear far more of
its own weight -- they couldn't (chose not to) share implementation and
support details, so the forces of divergence were NOT balanced by the
forces of commonality of implementation and support as expressed in the
open source community.  The proprietary Unix community was therefore
unstable, due to the fragmentation force not being balance by the unifying
force we have in open source, making us more stable, hopefully and so far,
stable enough to avoid their fate.

>> Thus, the upstream compatibility issue resolves to this:  the closer we
>> are to accepted LSB/FHS standard, the less work we will have to do
>> making changes so that open source packages which generally target the
>> loose
> 
> Wrong. Correct would be: the closer we are to UPSTREAM, the less work we
> will have to do. A nice example: QT

Point taken.  The (possibly wrong) assumption on my part being that as
time moves forward and amd64 becomes the common platform that x86 is
today, UPSTREAM (in the plural) will tend to converge on the LSB/FHS in
reference to its spec for multilib on amd64.

As a matter of fact, this does interest me (as should be pretty obvious
by now <g>). Do you have any evidence of an opposing trend, AWAY from the
LSB/FHS standard, or toward no accepted common standard, in this instance?
To date, everything I've read seems to indicate the trend, possibly slow,
but there none-the-less, is /toward/ the LSB/FHS multilib standard, at
least in the case of amd64.  If there's evidence to the contrary, please
do point me to it!  I'd much rather find out and have to admit I'm wrong
now, and have the truth as my guide in the future, than otherwise.  I
simply haven't yet seen anything that would invalidate my previous
assumption, and certainly what the pro-LSB press would have one believe.

>> Together, these two reasons means the only /possible/ sensible
>> alternative is to bite that bullet, and exchange some temporary misery
>> now, while the conversion is in progress, for a /far/ more standards
>> compliant multilib setup, going forward.
> 
> What misery?

That of those who found the 2004.3 -> 2005.0 upgrade rather more painful
than they would have liked?  That of those that ended up formatting and
reinstalling Gentoo, not entirely a pain free process, because they were
stuck with a half-upgraded system that would no longer work right on
/either/ the 2004.3 /or/ the 2005.0 profile?  That of those that chose to
avoid the issue for the time being and are staying on 2004.3, due to the
horror stories as posted here and in the forums, from those faced with the
situation above?

Presumably, the situation will continue to be less stable for
Gentoo-amd64 users than many of them would prefer, until Gentoo has
finished switching, until portage has dual bitness tracking, and the
lib/lib32/lib64 thing is all straightened out.

>> Currently, Gentoo-amd64 has just about finished moving 64-bit libs out
>> of lib.  We have both a lib32, and a lib64, with lib being a symlink to
>> lib64, for those remaining packages (such as skencil, apparently) that
>> have so far fallen thru the cracks, and continue to make the early
>> Gentoo-amd64 assumption that lib is the 64-bit shared object location.
>> With 2005.1, I'm expecting FEATURES=multilib-strict will be the default,
>> to finally weed out as many of the few remaining back packages as
>> possible.  With 2006.0, it's possible that lib -> lib64 symlink can
>> finally be broken, and any remaining packages then WILL get bugs filed,
>> when someone tries to use them and has issues.  With luck, by 2006.1, it
>> should be safe to start doing basically the same thing with the lib32 ->
>> lib move, as we will have just got thru doing with the lib -> lib64
>> move. First, there will be a symlink between the two, in another release
>> or two, the main packages will be changed and it'll be time to activate
>> the multilib-strict test for anything still ending up in lib32 instead
>> of lib. By 2007.0, therefore, if luck holds, that multilib-strict test
>> can be the default, to catch all the stragglers possible, and 2007.1
>> should then hopefully be able to remove lib32, relegating it to the
>> annuls of Gentoo-amd64 history.  (Those .1 mentions assume that Gentoo
>> continues with the twice-yearly releases, thus, they mean second-half.)
> 
> Where do you get this information from? We (read: we, the Gentoo/AMD64
> developer team) never published such a roadmap, nor do we have one
> ourselves. Please, don't cook up a story just to enlarge your
> pen^W^W^Wmails.

OK, it's quite apparent you have a personal problem with my style, and
possibly with the fact that I paralleled your post, tho I really don't
know why that should be a problem at all.  Perhaps that part should be
taken to private mail, if you wish to continue the discussion.

To answer your question, however, do you see the "I'm expecting" in what
you quoted?  What about the "it's possible that"?  It should be fairly
clear from those and the rest of the context that this part was simply my
speculations, based both on what I know to have happened in the past, and
on the various blogs, meeting logs, and dev and amd64 list comments by
devs, then filling in the gaps and projecting the trend forward.

The key is, it's my speculation, and the phrasing I chose /should/ have
made that quite apparent.

I'm resisting the urge to make stronger snide comments, returning yours in
kind, because I don't see how doing so would be conducive to a more
informed list.  I'll accept your disagreement.  That's fine.  If there's a
different plan and my incorrect post ends up bringing it out, GREAT!  I
and other list participants will be the better informed, and therefore,
better prepared to do our jobs as system admins, aka Gentoo users, because
we know better what to expect ahead.  I don't believe it's necessary,
however, to take things personally, or to make personally cutting comments
like the above, and honestly try to refrain from doing so, myself, tho I
certainly can't and don't claim perfection in that or any other area.

Back to the question...  Can we agree that it's a fact that we (meaning
the devs moving it, and the rest of us continuing our life as Gentoo-amd64
users amidst the mess of "construction", as it were) are most of the way
done with the task of ensuring that 64-bit libs go to lib64, not lib?

I know the reasons given for going to all the trouble, based on previous
dev comments.  Are you saying that has changed?  If so, than certainly,
what I clearly marked as my expectations, are going to be incorrect, as
they obviously are if I got something wrong along the way.  The fact
remains, however, I was writing what "I expect", what I consider
"possible", or even likely, and labeled it as exactly that.  I certainly
did **NOT** label it as any sort of official road map, quite the contrary!

Assuming that we agree on the current status, and that nothing has changed
from previous plans (and of course that I understood said plans
correctly), what's the next reasonable step in the process? Before we
break that lib -> lib64 link, isn't it reasonable to go stable with the
multilib-strict warning, to flush anything out that was missed by those
deliberately testing?  I know I'd rather file a bug about a warning and
failed emerge as a result, with my previous version continuing to function
normally, than I would file a bug because the link was broken and I now
have currently merged programs that no longer work!

The rest of the steps follow, one at a time.  Unless I'm seriously
mistaken, or unless things have changed, we'll follow the general outline,
tho the timing could easily be off, and the individual steps may be
somewhat different.  In any case, even if I AM seriously mistaken, it was
labeled "my expectation", not "official Gentoo roadmap", and who am I,
that someone should trust my predictions over that of Joe Blow?  It's not
as if I'm posting with an @gentoo.org email address or something.  Just
like the speculations of any other Joe Blow, if they find my speculations
reasonable and useful, great, if not, they can take someone else's.  My
opinion was freely given, and I really couldn't force someone to respect
it even if I /wanted/ to, which I don't.  If they want to value it at what
they paid for it, no skin off my nose!  It's their life and their opinion,
which they are certainly as free to have as I am to have mine! <g>

>> However, that's really only half the issue.  The other half is portage,
>> which currently can't track 32-bit dependencies separate from its 64-bit
>> dependencies.
> 
> You can take comfort with the fact that even if portage already had this
> functionality, there wouldn't be real multilib now.

Somewhat of an enigmatic comment.  There are a number of ways it could be
taken.

If "now" is taken to mean "as opposed to previous plans", it /could/ mean
that Gentoo in general, and Gentoo-amd64 in particular, has put on hold
further advances toward multilib, and may in fact reverse course.  Luckily
for me, the only 32-bit stuff I run is the stuff in the 2005.0 normal
profile (along with LILO), since I believe in FLOSS strongly enough to be
uninterested in closed source, even 32-bit only proprietary codecs and
plugins, and open source means 64-bit portable and for my purposes, that
means pretty much already ported.  Thus, personally, 32-bit and therefore
multilib, is more a hassle than anything, meaning double the time
compiling glibc, for instance.  However, I certainly empathize with those
who have 32-bit stuff they still want/need to run.  (If this is the way
"now" is to be read, the tone of the comment could also imply a bit of
bitterness, that this decision was made, as well.  That's not a given, as
it could imply satisfaction at the result, instead, or simply nothing. 
Again, enigmantic...)

OTOH, "now" could mean "currently", in which case, I'd generally agree. 
Gentoo-amd64 is not in the position, currently, to do "real" multilib,
even if portage supported multi-track dependency resolution.  Then of
course, there's the whole question of what defines "real" multilib...

Or it could mean something else.  In any case, I'd welcome any
enlightenment you might be able to provide, on the subject, or for that
matter, on anything else list related.  

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman in
http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html


-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: Re: file type not allowed in /usr/lib

[Simon Stelling]

Hi,

Duncan wrote:
>>Just a personal note on this: What's the point in repeating everything?
>>You only bore people with it. I think we can assume that the original
>>poster either gets it the first time, or he'll surely ask again and give
>>details about what he didn't understand.
> 
> 
> OK, first, the tone here and below seems to indicate I offended you in
> some way.  Let me unconditionally state at the outset, that such was in no
> way the intent.   It appears you took my post as somehow implying yours
> was incomplete or inadequate in some way.  Again, that couldn't have been
> farther from my intentions!  I intended to credit you where appropriate,
> acknowledging the overlap in ideas, but would have posted whether you had
> or not, because I had some of my own ideas to express, in my own reply, as
> is customary on public lists and newsgroups.  I evidently came across in a
> way I never intended at all, and for that I unconditionally apologize.

First, I didn't really want to offend you, although I have to admit that 
a few statements were rather harsh. I didn't feel offended, and I know 
you didn't want to say my answer was incompetent. However, if I *were* 
the questioner, I'd feel offended, since I don't need everything 
explained twice, and if I don't get it, I ask further questions. 
Generally explaining it a second time somehow implies that I didn't get 
it the first time, but perhaps that's just me.

> Not necessarily.  You are correct about dynamic linking, but what about
> binary only installers?  These may be hard coded to a specific
> path, certainly by default, tho it's usually possible to change it if
> necessary.  Even if it's changeable on the command line, or from a script,
> or even if it's open source, making and maintaining that change thru
> multiple versions over time is certainly more difficult than never having
> to make it in the first place, because our default happens to be the
> standard default.  If it's prepackaged as an ebuild, that's work the
> Gentoo developers have to do (or not, if the defaults coincide).  If it's
> something a Gentoo user is installing on their own, not from an ebuild,
> then having to fiddle with installer configs to get it to go to the right
> place is "a fight" that /could/ be avoided, if we were using the defaults
> in the first place.

Judging from my experience, the opposite is the case: Most of the 
hardcoded programs just install to /usr/lib without bothering about 
their bitness, so it'd be better to set lib = 64bit as a 
longterm-solution, assuming that amd64 will (at least partly) replace 
x86. Also, most of the "hardcoded programs" aren't closed source but 
open source apps. Also, once these hardcoded paths are made suitable, 
it's quite easy to do the same with the next version: Just bump the ebuild.
There are *very* few configure scripts which try to find out the correct 
place for libraries theirselves, and in most of these very few cases, 
they're broken anyway, as the just grep uname -a for Athlon64 or 
Opteron, which then would break 32bit.

>>>mostly duplicated common code, it's ALWAYS more efficient to keep in
>>>line with the rest of the community, so the maintenance and further
>>>development
>>
>>I disagree: There are bad standards, which should be replaced by better
>>ones. You can't do that by keeping in line with the rest.
> 
> 
> That doesn't alter the truth of the point I made.  It's ALWAYS more
> efficient (that is, less work) to keep in line with the rest of the
> community.  That does NOT always mean it's BETTER, only that it's MORE
> EFFICIENT.  The point I made was that this factor serves as a unifier
> within the community, always ensuring that distributions diverge only on
> the points they consider IMPORTANT, where the extra work is therefore
> worth it, as opposed to any old thing not considered so important, in
> which case it makes the most sense to go with the community standard,
> since that's less work, and the extra work necessary to deviate isn't
> considered worth it, since it's "not important".

'more efficient' is a synonym to 'better', for me, as I consider 
efficiency a 'good' thing. If you mean it's more comfortable, you're 
right, but then this questions pops up: Do we want to make good 
applications, or comfortable ones? (Comfortable to the developer, not 
the user of course.)

> required to maintain that change.  It doesn't pay to be different just to
> be different.  Because there's an unavoidable cost involved with being
> different, do it where it's worth the trouble, don't do it where it's not.

Sure, if that's the point you made, I absolutely agree, I just don't 
think such a self-evident statement is even worth to mention, so I 
thought you wanted to state something different.

> As a matter of fact, this does interest me (as should be pretty obvious
> by now <g>). Do you have any evidence of an opposing trend, AWAY from the
> LSB/FHS standard, or toward no accepted common standard, in this instance?
> To date, everything I've read seems to indicate the trend, possibly slow,
> but there none-the-less, is /toward/ the LSB/FHS multilib standard, at
> least in the case of amd64.  If there's evidence to the contrary, please
> do point me to it!  I'd much rather find out and have to admit I'm wrong
> now, and have the truth as my guide in the future, than otherwise.  I
> simply haven't yet seen anything that would invalidate my previous
> assumption, and certainly what the pro-LSB press would have one believe.

I can't see neither a movement toward LSB/FHS nor away from it. I think 
such standards are generally usefull as they make programming a lot 
easier, but in the end, each user should receive the power to decide 
where which type of file goes. So the ultimate goal shouldn't be to 
adore a certain standard but to make the tool as flexible as possible, 
e.g. by providing Makefiles where you just can reset LIBDIR if you don't 
the default. If every tool would be that flexible, we wouldn't need to 
bother about FH standards at all.

>>What misery?
> 
> 
> That of those who found the 2004.3 -> 2005.0 upgrade rather more painful
> than they would have liked?  That of those that ended up formatting and
> reinstalling Gentoo, not entirely a pain free process, because they were
> stuck with a half-upgraded system that would no longer work right on
> /either/ the 2004.3 /or/ the 2005.0 profile?  That of those that chose to
> avoid the issue for the time being and are staying on 2004.3, due to the
> horror stories as posted here and in the forums, from those faced with the
> situation above?

That's a misery indeed, however, that's not related to the 
lib/lib64/lib32 change. As you can see in the new development guide [1],
the situation is basically the same in 2004.3 and 2005.0. The major 
change which causes problems for many users is that in 2005.0, multilib 
is enabled by default, while on 2004.3 it's not, and to get a 
multilib-capable system, you need a multilib-capable toolchain (well, 
not the whole toolchain, otherwise it'd be impossible without using 
binpkgs). Anyway, the issues caused with the upgrade to 2005.0 would be 
the same if the lib/lib64/lib32 scheme would be the same as in 2004.3.

>>>Currently, Gentoo-amd64 has just about finished moving 64-bit libs out
>>>of lib.  We have both a lib32, and a lib64, with lib being a symlink to
>>>lib64, for those remaining packages (such as skencil, apparently) that
>>>have so far fallen thru the cracks, and continue to make the early
>>>Gentoo-amd64 assumption that lib is the 64-bit shared object location.
>>>With 2005.1, I'm expecting FEATURES=multilib-strict will be the default,
>>>to finally weed out as many of the few remaining back packages as
>>>possible.  With 2006.0, it's possible that lib -> lib64 symlink can
>>>finally be broken, and any remaining packages then WILL get bugs filed,
>>>when someone tries to use them and has issues.  With luck, by 2006.1, it
>>>should be safe to start doing basically the same thing with the lib32 ->
>>>lib move, as we will have just got thru doing with the lib -> lib64
>>>move. First, there will be a symlink between the two, in another release
>>>or two, the main packages will be changed and it'll be time to activate
>>>the multilib-strict test for anything still ending up in lib32 instead
>>>of lib. By 2007.0, therefore, if luck holds, that multilib-strict test
>>>can be the default, to catch all the stragglers possible, and 2007.1
>>>should then hopefully be able to remove lib32, relegating it to the
>>>annuls of Gentoo-amd64 history.  (Those .1 mentions assume that Gentoo
>>>continues with the twice-yearly releases, thus, they mean second-half.)
>>
>>Where do you get this information from? We (read: we, the Gentoo/AMD64
>>developer team) never published such a roadmap, nor do we have one
>>ourselves. Please, don't cook up a story just to enlarge your
>>pen^W^W^Wmails.
> 
> 
> OK, it's quite apparent you have a personal problem with my style, and
> possibly with the fact that I paralleled your post, tho I really don't
> know why that should be a problem at all.  Perhaps that part should be
> taken to private mail, if you wish to continue the discussion.

Not at all. The "enlarge your pen^W" shouldn't offend you, it was meant 
as a joke, critisizing the massive amount of spam we all get with this 
topic, not you :D Sorry if it looked like a diparaging remark, that 
really wasn't my intention.

> The key is, it's my speculation, and the phrasing I chose /should/ have
> made that quite apparent.
[snip]
> The fact
> remains, however, I was writing what "I expect", what I consider
> "possible", or even likely, and labeled it as exactly that.  I certainly
> did **NOT** label it as any sort of official road map, quite the contrary!

It rather looked like an official roadmap coming from a dev to me, but 
in that case, I just got it completely wrong. I apologize.

> Assuming that we agree on the current status, and that nothing has changed
> from previous plans (and of course that I understood said plans
> correctly), what's the next reasonable step in the process? Before we
> break that lib -> lib64 link, isn't it reasonable to go stable with the
> multilib-strict warning, to flush anything out that was missed by those
> deliberately testing?  I know I'd rather file a bug about a warning and
> failed emerge as a result, with my previous version continuing to function
> normally, than I would file a bug because the link was broken and I now
> have currently merged programs that no longer work!

What you stated above was correct, but given the fact that many users 
think much of you, i find it risky to issue a complete roadmap with more 
or less precise dates for each step. The main problem I saw here were 
users coming to us in 2007 and asking why the hell we didn't finish the 
whole thing.

> The rest of the steps follow, one at a time.  Unless I'm seriously
> mistaken, or unless things have changed, we'll follow the general outline,
> tho the timing could easily be off, and the individual steps may be
> somewhat different.  In any case, even if I AM seriously mistaken, it was
> labeled "my expectation", not "official Gentoo roadmap", and who am I,
> that someone should trust my predictions over that of Joe Blow?  It's not
> as if I'm posting with an @gentoo.org email address or something.  Just
> like the speculations of any other Joe Blow, if they find my speculations

Well, your not just a random guy like Joe Blow, you're a much 
appreciated user with a lot of knowledge, and that gives you power as 
well as responsibility.

>>>However, that's really only half the issue.  The other half is portage,
>>>which currently can't track 32-bit dependencies separate from its 64-bit
>>>dependencies.
>>
>>You can take comfort with the fact that even if portage already had this
>>functionality, there wouldn't be real multilib now.
> 
> Somewhat of an enigmatic comment.  There are a number of ways it could be
> taken.
> 
> If "now" is taken to mean "as opposed to previous plans", it /could/ mean
> that Gentoo in general, and Gentoo-amd64 in particular, has put on hold
> further advances toward multilib, and may in fact reverse course.  Luckily
> for me, the only 32-bit stuff I run is the stuff in the 2005.0 normal
> profile (along with LILO), since I believe in FLOSS strongly enough to be
> uninterested in closed source, even 32-bit only proprietary codecs and
> plugins, and open source means 64-bit portable and for my purposes, that
> means pretty much already ported.  Thus, personally, 32-bit and therefore
> multilib, is more a hassle than anything, meaning double the time
> compiling glibc, for instance.  However, I certainly empathize with those
> who have 32-bit stuff they still want/need to run.  (If this is the way
> "now" is to be read, the tone of the comment could also imply a bit of
> bitterness, that this decision was made, as well.  That's not a given, as
> it could imply satisfaction at the result, instead, or simply nothing. 
> Again, enigmantic...)

Heh, that really made me laugh. We're not going back, everything is 
going on as planned. The problem I had in mind when writing the above 
statement is following:

Let's say we have appfoo (yes, again :D), which needs various 
X11-libraries, as it has a GUI. We want appfoo to be compiled 32bit, but 
  the huge majority is 64bit, and of course xorg-x11 is 64bit too. Now, 
where we want to install appfoo as a 32bit binary with the portage able 
to track 32bit dependencies, it will install xorg-x11 32bit, because we 
need the libraries in that packages 32bit. But then, the install routine 
would overwrite the xorg binary and we'd end up with a 32bit xorg, which 
we don't want. To solve this issue, we have the following options:

	a) Split up libraries and binaries into seperate packages. This
	is what currently happens to X, thanks to spyderous. See
	eradicator's thoughts on this [2].

	b) Implement a filter to just install the shared objects. This
	has the advantage that we don't have to split up a huge amount
	of packages and keep closer to upstream, which is generally a
	good idea. However, there are still unresolved issues: Portage
	would need another addition, something like a 'only libs' flag,
	and personally, I don't like that idea so much, as it looks a
	bit untidy. Also, you need e.g. headers the first time you
	install a certain library, so things are getting complicated.

In addition, there is still the unresolved issue about the -config 
scripts in /usr/bin, which would get overwritten everytime you install 
the same lib with another bitness. Unfortunately we can't just copy a 
solution from another (binary-based) distro, as they don't have this 
problem. Also, the universally beloved FHS doesn't give us an answer to 
this problem. A really easy solution would be to just create /usr/bin64, 
similiar to /usr/lib and /usr/lib64, but we all know this would lead to 
a massive amount of time-consuming work, and as you said in your 
original mail, it's simply not worth the effort. Of course it wouldn't 
be a problem at all if all applications would have a more flexible build 
system, as I stated above.

All in all, I'm sure we will have a fully multilib-capable system in the 
end, but I'm rather pessimistic about the time schedule.

I'll better finish now, this mail got much longer than I wanted it to 
be, and I hope I don't bore the readers ;)

Kindly regards,

Simon

[1] http://dev.gentoo.org/~plasmaroo/devmanual/archs/amd64/#libdir-links
[2] 
http://ramblings.hudscabin.com/blogs/index.php/2005/05/02/multilib_and_toolchain_thoughts
-- 
Simon Stelling
Gentoo/AMD64 Operational Co-Lead
blubb@gentoo.org
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: Re: file type not allowed in /usr/lib

[Matt Randolph]

Simon Stelling wrote:

>
> First, I didn't really want to offend you, although I have to admit 
> that a few statements were rather harsh. I didn't feel offended, and I 
> know you didn't want to say my answer was incompetent. However, if I 
> *were* the questioner, I'd feel offended, since I don't need 
> everything explained twice, and if I don't get it, I ask further 
> questions. Generally explaining it a second time somehow implies that 
> I didn't get it the first time, but perhaps that's just me.


Google "Active Listening."  Basically, active listening means that you 
repeat everything another person says but you put it in your own words.  
This shows the other person that you're actually paying attention, and 
it helps to reveal any miscommunication the instant it occurs.  It's a 
management technique and a tool used in conflict resolution/avoidance.  
I'm NOT saying that's what happened here, but your words suggest to me 
that you _might_ be unprepared if someone used active listening on you.

-- 
"Pluralitas non est ponenda sine necessitate" - W. of O.

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: Re: file type not allowed in /usr/lib

[Matt Randolph]

Matt Randolph wrote:

> Simon Stelling wrote:
>
>>
>> First, I didn't really want to offend you, although I have to admit 
>> that a few statements were rather harsh. I didn't feel offended, and 
>> I know you didn't want to say my answer was incompetent. However, if 
>> I *were* the questioner, I'd feel offended, since I don't need 
>> everything explained twice, and if I don't get it, I ask further 
>> questions. Generally explaining it a second time somehow implies that 
>> I didn't get it the first time, but perhaps that's just me.
>
>
Simon Stelling wrote:

>
> Just a personal note on this: What's the point in repeating 
> everything? You only bore people with it. I think we can assume that 
> the original poster either gets it the first time, or he'll surely ask 
> again and give details about what he didn't understand.
>
Oops!  I meant to quote you when you said it the first time.


-- 
"Pluralitas non est ponenda sine necessitate" - W. of O.

-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: Re: Re: file type not allowed in /usr/lib

[Duncan]

Matt Randolph posted <42F7E516.2030103@erols.com>, excerpted below,  on
Mon, 08 Aug 2005 19:04:54 -0400:

> Simon Stelling wrote:
> 
>> First, I didn't really want to offend you, although I have to admit that
>> a few statements were rather harsh. I didn't feel offended, and I know
>> you didn't want to say my answer was incompetent. However, if I *were*
>> the questioner, I'd feel offended, since I don't need everything
>> explained twice, and if I don't get it, I ask further questions.
>> Generally explaining it a second time somehow implies that I didn't get
>> it the first time, but perhaps that's just me.
> 
> 
> Google "Active Listening."  Basically, active listening means that you
> repeat everything another person says but you put it in your own words.
> This shows the other person that you're actually paying attention, and it
> helps to reveal any miscommunication the instant it occurs.  It's a
> management technique and a tool used in conflict resolution/avoidance. I'm
> NOT saying that's what happened here, but your words suggest to me that
> you _might_ be unprepared if someone used active listening on you.

Hmm...  I've seen "active listening" described before, and even
participated in exercises where we were supposed to /do/ it.  However,
that always seemed frustratingly artificial, to me, and I had difficulty
moving it out of the book/classroom/exercise and into the real world, in
part because the idea seemed natural on the one hand, but, I'm guessing
because of the artificial emphasis on the single technique, something that
really doesn't happen in the real world, entirely artificial in all the
examples and exercises.

Thus, it's very instructive to see someone bring it up in a "real world"
(OK, Gentoo list <g>) situation I was involved in, in post-analysis, so I
wasn't trying to focus on it at the time, only later.

You are correct, it wasn't /exactly/ what was happening here, because it
was a three-party exchange with a fairly large known audience in the form
of the rest of the list, but are also correct in pointing out the
parallels, as, looking back in post-analysis myself, I can see that I was
instinctively using some of the same principles.  While not direct active
listening because it was my own separate response, I did repeat and
acknowledge some of the same concepts, with the implicit understanding
that not only was I reinforcing them as I earlier claimed, but that,
should I get something wrong, I'd be called out on it.  

Actually, it's a bit more than that, because at the same time, there's
also the check on the effectiveness of my delivery, that I wouldn't
necessarily get, if I wasn't paralleling someone else.  Every once in
awhile, something happens to make me aware that the received message from
one of my posts isn't quite what I intended.  In this case, that's
rather an understatement of what happened <g>.  However, in any case,
while the experience isn't usually all that pleasant (another
understatement <g>), the result is a better awareness of how my
communications are received, ultimately resulting in changes to what I
send.  It may not be pleasant, but the end result, therefore, is, I
believe/hope, better and more effective communication from my end.

Anyway... what's really interesting to me about your comment, is how I was
doing this and wasn't really aware of it, nor could I have been, because I
had never been able to successfully translate the book/classroom knowledge
of "active listening" into something approaching reality.  Thus, your
comment provoked a personal Ah-ha! "lightbulb goes on" type moment, for
me.  I was finally able to make that real-world connection, and hopefully,
can now benefit from it, as the consequences of that linking sink in.

I don't know if the above really makes sense to anyone else or not, but
what it all means is that I got something unexpected out of your comment,
that should be of help to me in the future.  So... THANKS! =8^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman in
http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html


-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: Re: Re: file type not allowed in /usr/lib

[Duncan]

Simon Stelling posted <42F7B071.7060603@gentoo.org>, excerpted below,  on
Mon, 08 Aug 2005 21:20:17 +0200:

> Duncan wrote:
>>>Just a personal note on this: What's the point in repeating everything?
>>>You only bore people with it. I think we can assume that the original
>>>poster either gets it the first time, or he'll surely ask again and
>>>give details about what he didn't understand.
>> 
>> OK, first, the tone here and below seems to indicate I offended you in
>> some way.  Let me unconditionally state at the outset, that such was in
>> no way the intent.
> 
> First, I didn't really want to offend you, although I have to admit that
> a few statements were rather harsh. I didn't feel offended, and I know
> you didn't want to say my answer was incompetent. However, if I *were*
> the questioner, I'd feel offended, since I don't need everything
> explained twice, and if I don't get it, I ask further questions.
> Generally explaining it a second time somehow implies that I didn't get
> it the first time, but perhaps that's just me.

Hmm...  Maybe it's because I've been quite active on USENET (and public
mailing lists like this one, now) for some time, but to me, that's just
the nature of the medium.  Two or sometimes five or ten replies all
stating the same thing or sometimes wildly different things <g>, just
comes with the territory.  Sometimes folks build on what others have said,
sometimes they disagree and say why, sometimes the posts don't have any
relation but say essentially the same thing, because neither poster saw
the other's post before they posted.

IMO, it's one of the strengths of the medium that each poster says it his
own way, most rather briefly, some (like myself) taking hundreds of lines
to explain (their view of) why things work as they do, the implications,
and some of the exceptions to watch out for and what to do to avoid or
work around them, all to the Nth degree! <g>  Basically, it allows the
original poster (aka OP) to pick and choose the replies that meet their
needs.  If they want a quick solution, the five line "do this" is perfect!
If they want to know the background and what to do to prevent the issue
again, and/or how to fix similar but not identical issues, without having
to post another query, the next time, then the 200-liners <g> come in
handy.

So... yes, occasionally someone gets offended.  However, once they
understand the medium, they generally don't any more, and, as always with
the medium, for those seriously offended, there's always the kill
file/filter option, the exercise of which does not require notice or
reason be given.

>> [W]hat about binary only installers?
>
> Judging from my experience, the opposite is the case: Most of the
> hardcoded programs just install to /usr/lib without bothering about
> their bitness, so it'd be better to set lib = 64bit as a
> longterm-solution, assuming that amd64 will (at least partly) replace
> x86. Also, most of the "hardcoded programs" aren't closed source but
> open source apps. Also, once these hardcoded paths are made suitable,
> it's quite easy to do the same with the next version: Just bump the
> ebuild. There are *very* few configure scripts which try to find out the
> correct place for libraries theirselves, and in most of these very few
> cases, they're broken anyway, as the just grep uname -a for Athlon64 or
> Opteron, which then would break 32bit.

Hmm.. surprised me on the hardcoding being mostly open source...  I guess
it makes sense, tho, as the OS folks figure it's open, so folks can change
it if need be, while the closed source folks don't /want/ folks examining
their stuff, so have reason to make it configurable.

However, the real point is dealt with below...

>>>>mostly duplicated common code, it's ALWAYS more efficient to keep in
>>>>line with the rest of the community, so the maintenance and further
>>>>development
> 
> 'more efficient' is a synonym to 'better', for me, as I consider
> efficiency a 'good' thing. If you mean it's more comfortable, you're
> right, but then this questions pops up: Do we want to make good
> applications, or comfortable ones? (Comfortable to the developer, not
> the user of course.)

Ah...  I see your viewpoint, now!  It never occurred to me...

I was using "efficient" as in "getting as far as possible using the least
amount of work and resources possible".  Thus, "efficiency" is usually
thought of as having a good effect, but is actually neutral, in that
something can be "ruthlessly efficient", or "an efficient killing
machine", or the like, in which case the effect can be bad, depending of
course on one's viewpoint, whether one is doing the killing or being
killed (or watching it occur, with approval or disapproval), in the case
of the "efficient killing machine", for instance.

Thus, here, I was saying it's "less work and requires less resources" to
go with the community common standard.  That's ALWAYS the case.  Whether
it's actually worth spending the extra work and resources, therefore,
depends on just what value is assigned to the proposed difference from
standard, in question.

>> Because there's an unavoidable cost involved with being different, do
>> it where it's worth the trouble, don't do it where it's not.
> 
> Sure, if that's the point you made, I absolutely agree, I just don't
> think such a self-evident statement is even worth to mention, so I
> thought you wanted to state something different.

The trouble is... what's self-evident to one person, isn't always so
evident to another!  I've unfortunately had to find this out the hard way
on more than /one/ occasion!  <g>

Anyway, here, the context was in how that forms the unifying force that
keeps (GNU/)Linux from fragmenting into incompatibility, as proprietary
Unix did, and how that forms a counter-balance to the forces of
fragmentation.  The argument is quite familiar to those who've been around
open source for some time, certainly, but evidently, it isn't quite
self-evident to everyone, as the question of fragmentation does continue
to come up.

> So the ultimate goal shouldn't be to adore a certain standard but to
> make the tool as flexible as possible, e.g. by providing Makefiles where
> you just can reset LIBDIR if you don't the default. If every tool would
> be that flexible, we wouldn't need to bother about FH standards at all.

It /would/ be nice...

[snipping a section I entirely agree with, post was getting too long...]

>>>Where do you get this information from? We (read: we, the Gentoo/AMD64
>>>developer team) never published such a roadmap, nor do we have one
>>>ourselves. Please, don't cook up a story just to enlarge your
>>>pen^W^W^Wmails.
>> 
>> OK, it's quite apparent you have a personal problem with my style, and
>> possibly with the fact that I paralleled your post, tho I really don't
>> know why that should be a problem at all.  Perhaps that part should be
>> taken to private mail, if you wish to continue the discussion.
> 
> Not at all. The "enlarge your pen^W" shouldn't offend you, it was meant
> as a joke, critisizing the massive amount of spam we all get with this
> topic, not you :D Sorry if it looked like a diparaging remark, that
> really wasn't my intention.

OK, got it!  The case of the missing smiley chalked up another victim,
here, I must admit!  (See next for content comment.)

>> The key is, it's my speculation, and the phrasing I chose /should/ have
>> made that quite apparent. [snip]
> 
> It rather looked like an official roadmap coming from a dev to me, but
> in that case, I just got it completely wrong. I apologize.
> 
>> Assuming that we agree on the current status, and that nothing has
>> changed from previous plans (and of course that I understood said plans
>> correctly), what's the next reasonable step in the process? [snip]
> 
> What you stated above was correct, but given the fact that many users
> think much of you, i find it risky to issue a complete roadmap with more
> or less precise dates for each step. The main problem I saw here were
> users coming to us in 2007 and asking why the hell we didn't finish the
> whole thing.

OK, it's clearer now what you were referring to, and indeed, I agree,
I could have been clearer in specifically stating it was NOT a devs
roadmap.

I'll plead the "I'm still learning" excuse, here. You make a
decent point. I've been personally in the process of coming to terms with
what it might mean for me to be a Gentoo AT, and the implications that
should/will have for my posting. I've been a user, only a user, if perhaps
a somewhat influential one, in 100% of my USENET and mailing list rolls,
for as long as I've had such rolls. As such, I've not had the
responsibility of representing someone else, and could speak my mind,
because it was /only/ me I was representing.  I take seriously the advice
in the AT guide, and have been considering the implications that will have
on my posting style, as well as whether I'll choose to post with my Gentoo
identity (once I get it, let me be absolutely clear lest anyone be in any
way confused, I don't have it nor do I claim to have it yet) as part of my
regular sig, only some of the time, or if I'll only use it with the devs
and on bugzilla.

Quite apart from AT, however, it just happens to fit in there too, you are
correct in that it would have been possible, indeed, preferable, that I
made it a bit more clear that the "we" I mentioned several times was NOT
"we" as in Gentoo-amd64 devs, but "we" as in the commonality of
Gentoo-amd64 users AND devs -- and that I was speaking as the "user" side
of that "we".  Specifically in this "roadmap", while I DID mention "I
expect", and "possible" and other such qualifiers, I COULD and SHOULD have
also specifically stated this was NOT a devs provided roadmap.  Again, I'm
still learning, but you've persuaded me.  =8^)

> Well, your not just a random guy like Joe Blow, you're a much
> appreciated user with a lot of knowledge, and that gives you power as
> well as responsibility.

Aye.  Sometimes I'd rather be just a Joe Blow!  <g>

>>>>However, that's really only half the issue.  The other half is
>>>>portage, which currently can't track 32-bit dependencies separate from
>>>>its 64-bit dependencies.
>>>
>>>You can take comfort with the fact that even if portage already had
>>>this functionality, there wouldn't be real multilib now.
>> 
>> Somewhat of an enigmatic comment.  There are a number of ways it could
>> be taken.
>> 
[snip speculation that proved /not/ to be the case, thankfully! <g>]
> 
> Heh, that really made me laugh. We're not going back, everything is
> going on as planned. The problem I had in mind when writing the above
> statement is following:
> 
> Let's say we have appfoo (yes, again :D), which needs various
> X11-libraries, as it has a GUI. We want appfoo to be compiled 32bit, but
> the huge majority is 64bit, and of course xorg-x11 is 64bit too. Now,
> where we want to install appfoo as a 32bit binary with the portage able
> to track 32bit dependencies, it will install xorg-x11 32bit, because we
> need the libraries in that packages 32bit. But then, the install routine
> would overwrite the xorg binary and we'd end up with a 32bit xorg, which
> we don't want. To solve this issue, we have the following options:
> 
> 	a) Split up libraries and binaries into seperate packages. This is what
> 	currently happens to X, thanks to spyderous. See eradicator's thoughts
> 	on this [2].
> 
> 	b) Implement a filter to just install the shared objects. This has the
> 	advantage that we don't have to split up a huge amount of packages and
> 	keep closer to upstream, which is generally a good idea. However, there
> 	are still unresolved issues: Portage would need another addition,
> 	something like a 'only libs' flag, and personally, I don't like that
> 	idea so much, as it looks a bit untidy. Also, you need e.g. headers the
> 	first time you install a certain library, so things are getting
> 	complicated.
> 
> In addition, there is still the unresolved issue about the -config
> scripts in /usr/bin, which would get overwritten everytime you install
> the same lib with another bitness. Unfortunately we can't just copy a
> solution from another (binary-based) distro, as they don't have this
> problem. Also, the universally beloved FHS doesn't give us an answer to
> this problem. A really easy solution would be to just create /usr/bin64,
> similiar to /usr/lib and /usr/lib64, but we all know this would lead to
> a massive amount of time-consuming work, and as you said in your
> original mail, it's simply not worth the effort. Of course it wouldn't
> be a problem at all if all applications would have a more flexible build
> system, as I stated above.
> 
> All in all, I'm sure we will have a fully multilib-capable system in the
> end, but I'm rather pessimistic about the time schedule.

Very, /very/ good point.  It's one I'd thought of, but not really in this
context, and certainly not with all the implications, so there's some
rather new info for me here.  That's what I was hoping for!  =8^)

So, rather than the two aspects I had outlined, there are /three/, and
when I said that was only half, it was only a third.  MORE challenges to
work thru!

As an aside, I'd often wondered at the implications of having a
"multibin", similar to "multilib", and why, with all the work going into
multilib, nobody seemed to be doing anything with multibin.  I had
wondered why having a 64-bit mplayer for most stuff, and a 32-bit mplayer,
for 32-bit-only binary-only codecs, wasn't as reasonable a solution as
it seemed to be for libraries.  Your comments just touch on that, but
it's the first time I've seen /any/ sort of comments on the subject, from
anyone who'd be in a position to know the issues involved, from any of the
distributions.


> [1] http://dev.gentoo.org/~plasmaroo/devmanual/archs/amd64/#libdir-links
> [2]
> http://ramblings.hudscabin.com/blogs/index.php/2005/05/02/multilib_and_toolchain_thoughts

I'll probably go over these tonight, after work.  (I /did/ see and
read with interest the thread on dev about modular xorg, tho I
haven't gotten to the that link either, to see how it's comming
along.)  Thanks for the additional links to read thru!

...  Anyway, /very/ glad to get this whole thing straightened out, again. 
I must admit, I refreshed the group this morning with a bit of
trepidation.  Feels good to be at peace with my world, once again! =8^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman in
http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html


-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: Re: Re: file type not allowed in /usr/lib

[Simon Stelling]

Duncan wrote:
> As an aside, I'd often wondered at the implications of having a
> "multibin", similar to "multilib", and why, with all the work going into
> multilib, nobody seemed to be doing anything with multibin.  I had
> wondered why having a 64-bit mplayer for most stuff, and a 32-bit mplayer,
> for 32-bit-only binary-only codecs, wasn't as reasonable a solution as
> it seemed to be for libraries.  Your comments just touch on that, but
> it's the first time I've seen /any/ sort of comments on the subject, from
> anyone who'd be in a position to know the issues involved, from any of the
> distributions.

Well, the problem is, even if you have two different directories for 
each bitness, the binaries will still collide, because of $PATH: How do 
you decide whether you want the 32bit or 64bit player? There's no real 
solution other than having suffixes like e.g. mplayer32 and mplayer64. 
You could reset PATH each time you launch a binary of other bitness, but 
that's just an ugly workaround and it may not always do what you want. 
So you have to use suffixes. But then, you don't need two directories, 
as the binaries won't collide anymore... Additionally, what do you do 
about scripts? PATH doesn't work there, and so you'd have to first edit 
the #!-line before executing it.

On the other hand, multilib will never be a permanent solution, I see it 
as a nice way to keep compatibility with old stuff, but as soon as 
companies are starting to provide 64bit binaries, it will slowly 
disappear from amd64's sight.

So the final question is: Is it really worth the effort?

Regards,

-- 
Simon Stelling
Gentoo/AMD64 Operational Co-Lead
blubb@gentoo.org
-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: Re: Re: Re: file type not allowed in /usr/lib

[Duncan]

Simon Stelling posted <42F9FD31.4050307@gentoo.org>, excerpted below,  on
Wed, 10 Aug 2005 15:12:17 +0200:

> Well, the problem is, even if you have two different directories for each
> bitness, the binaries will still collide, because of $PATH: How do you
> decide whether you want the 32bit or 64bit player? There's no real
> solution other than having suffixes like e.g. mplayer32 and mplayer64. You
> could reset PATH each time you launch a binary of other bitness, but
> that's just an ugly workaround and it may not always do what you want. So
> you have to use suffixes. But then, you don't need two directories, as the
> binaries won't collide anymore... Additionally, what do you do about
> scripts? PATH doesn't work there, and so you'd have to first edit the
> #!-line before executing it.

As I've imagined it here, altho I admit there are likely huge holes in it
that I'm not aware of given I haven't had to be concerned with the actual
implementation process...

$PATH would include $PREFDIR:$DEFAULTBIN:$SECONDARYBIN.  $DEFAULTBIN would
be the normal system default, presumably bin64.  $PREFDIR would contain
symlinks to individual executables in $SECONDARYBIN, where they are
preferred over those in $DEFAULTBIN, allowing a system default, with
individual application defaults differing from the system default, as
necessary.

In addition, $PREFDIR could contain suffixed or otherwise identified
symlinks to both bitness versions specifically.  Thus, applications could
be referred to by general name to get the locally preferred default in
$PREFDIR, suffixed name to get the specific bitness version, or by
absolute path, getting the specific bitness version or the locally
preferred default as specified in $PREFDIR, as desired/appropriate.

There'd be a similar config dir setup, with a system default config, and a
secondary config dir for those apps that are configured differently
between the bitnesses.  Again, a general symlink dir could be used to
coordinate things, or compliant apps could do the usual mult-dir search
until they found a config, searching first their specific bitness
config-dir then the system default config-dir.

Presumably, the suffixed symlinks would be created in $PREFDIR at
installation.  There's be a helper script similar to *-config that could
be used to select between them, or the sysadmin could set the symlinks
manually if desired.

Each distribution could set up its own system, but again, LSB/FHS or the
like standardization would be a pretty big benefit, here, such that at
least the standard dir locations to search were agreed.  Some
distributions would then go with the more or less simple symlink
infrastructure as outlined above, others may replace the symlinks with
more complex wrapper scripts that would "do the right thing" for that
specific distribution.

I don't see scripts as being a huge problem, particularly if the dirs were
standardized.  Those scripts that needed specific bitness versions would
call them, presumably by suffix; those that didn't would call the
unsuffixed version, which would automatically find the appropriate
default, either picking it up from $PREFDIR, or falling thru to the global
system default in $DEFAULTBIN if a prefdir symlink or wrapper wasn't
defined, presuming of course, that path was set appropriately.  Where such
path presumptions couldn't be relied upon, for security or other reasons,
absolute paths could be used, using the already common in scripts
if-executable type tests except in the case of the #! lines, which would
still need tweaked by distributions or sysadmins in some instances.

Like I said, there's probably huge holes in all that, due to the fact that
I've never been in the position of someone having to worry about such
implementation details at a distribution level...


> On the other hand, multilib will never be a permanent solution, I see it
> as a nice way to keep compatibility with old stuff, but as soon as
> companies are starting to provide 64bit binaries, it will slowly
> disappear from amd64's sight.
> 
> So the final question is: Is it really worth the effort?

Well...  I suppose it depends on just how "slow" reality defines "slowly"
to be.

We have the precedent of the 16 -> 32-bit transition.  I'm not familiar
with *ix from back then, but know at least on the proprietaryware aka MS
side, the practical transition took a decade or longer, and there are
still 16-bit binaries in use today, altho very often run under software,
if not hardware, emulation.  Think about how long it was before 32-bit
virtual 386 mode became common.  Then consider how much more society
depends on 32-bit x86 today, as compared to 16-bit x86 back then, and the
fact that 32-bit is far closer to "good enough" for most folks using it
today than 16-bit was back then -- the improvements are more incremental
now, because the percentage improvement on the established base is
smaller, even if the improvement in the absolute sense is the same or
even greater.

Given that, I think a decade is a realistic estimate.  Admittedly, there
are quite a few different factors this time around, the far higher degree
of computer use in society, and the "good enough" factor, on the one hand,
against the possible paradigm shift to FLOSS, and what that might mean to
the whole debate, on the other.  Still, a decade seems reasonable, and
that's a /very/ long time, in computer evolution terms.

Then there's the whole question as to where to start counting.  Where is
day zero in that decade?  I'd contend that it's in the past, certainly,
but do we call the introduction of the Opteron day zero, or do we call the
introduction on the desktop, the Athlon 64, day zero, or do we call
Intel's introduction day zero?  Are we already a couple years into that
decade, or only a few months?  If it's a couple years, you are correct, by
the time we get multilib fully implemented, we may have already passed the
half-way mark, and the actual need for multilib, let alone multibin, will
be on the decline.  If day zero is taken to be when the Intel version
actually became available on the streets, it's still fairly early in the
game, yet, and it may well be worthwhile considering multibin, even now,
particularly since the bulk switch from x86(32) definitely remains ahead,
and if the decade timeline proves optimistic.  Also throw into that the
whole proprietary/open thing.  Certainly, those with more freedom to
modify the software they run, aren't going to be needing such drastic
compatibility measures as those without.  (It's for that reason that the
entire debate is mostly an intellectual exercise, for me, as I personally
come down far closer to the freedom end of things, than, say, those
willing to run even proprietary video drivers and games, let alone entire
proprietary OSs, and most of the apps on them.)

So... I'm /definitely/ of the opinion that multilib is worthwhile.  I
think multibin /could///would/ have been as well, were the momentum for it
at the same level as multilib is today.  However, I think most will agree
that it's definitely rather late to start worrying about multibin at this
point, however nice it /might/ have been to have it.

IOW, as a practical matter, I don't see multilib disappearing from sight
any time soon.  If anything, its importance is likely to increase
DRAMATICALLY over the next few years (three at least), as the mainline x86
computing public switches over, bringing with them all that 32-bit
binary-only cruft they'll invariably have.  That is, after all, one of the
big reasons x86_64 has become the success it has, even in the face of
Intel's pushing ia64 as the "preferred" alternative.  By 2008,
multilib importance may begin to decrease, tho I don't see it
dropping to something that can really be ignored (unless deliberately so),
until 2010 at the earliest.

Then again, what's this "Joe Blow" know? =8^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman in
http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html


-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: file type not allowed in /usr/lib

[Jeremy Huddleston]

[-- Attachment #1: Type: text/plain, Size: 2960 bytes --]

> With 2005.1, I'm expecting FEATURES=multilib-strict will be the default,
> to finally weed out as many of the few remaining back packages as
> possible.

I don't suspect that will be "default" as I'd rather not burden users
with those problems that should be caught by developers.  Before we make
the switch to eliminating the symlink for good, we'll need to QA every
amd64 marked package to make sure it doesn't break under
multilib-strict... I'm not looking forward to that, but hopefully we'll
be able to break up the burden among the devs, ATs and community.

> With 2006.0, it's possible that lib -> lib64 symlink can
> finally be broken, and any remaining packages then WILL get bugs filed,
> when someone tries to use them and has issues.

You can actually do that now with the 2005.0/no-symlinks profile, but I
haven't done much QA on it yet, but PLEASE file bugs that you come
across with it.

> With luck, by 2006.1, it
> should be safe to start doing basically the same thing with the lib32 ->
> lib move, as we will have just got thru doing with the lib -> lib64 move.

Again, 2005.0/no-symlinks/no-lib32 profile... uh... don't expect this
one to work though ;)  BUG reports welcome... also developer muscle
welcome.  I want to get gcc-config 2.0 done first (see my blog).

> First, there will be a symlink between the two, in another release or two,
> the main packages will be changed and it'll be time to activate the
> multilib-strict test for anything still ending up in lib32 instead of lib.
> By 2007.0, therefore, if luck holds, that multilib-strict test can be the
> default, to catch all the stragglers possible, and 2007.1 should then
> hopefully be able to remove lib32, relegating it to the annuls of
> Gentoo-amd64 history.  (Those .1 mentions assume that Gentoo continues
> with the twice-yearly releases, thus, they mean second-half.)

So yeah, in like many years down the road, this should all be worked out
assuming I get off my ass and get things rolling...

> However, that's really only half the issue.  The other half is portage,
> which currently can't track 32-bit dependencies separate from its 64-bit
> dependencies.  We had hoped that portage would have dual-bitness support
> by 2005.1, but that's now looking impossible, 

Uhm... yeah... did I mention that I don't like python... so that means
this probably won't happen soon or quickly unless someone else steps up
to the task to help out (or someone convinces me to enjoy python)...
the design is pretty much there, but it's not gonna get implemented by
me alone.  If anyone is interested in helping out with this project,
this is a great opportunity to contribute a much needed feature.

> Thus, it'll be 2006.0 at the earliest, before portage
> will be able to handle dual-bitness tracking, keeping the 32-bit
> dependencies separate from the 64-bit dependencies.

And that's very optimistic.

--Jeremy


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]