From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from lists.gentoo.org ([140.105.134.102] helo=robin.gentoo.org) by nuthatch.gentoo.org with esmtp (Exim 4.43) id 1E1l6E-0007GJ-A3 for garchives@archives.gentoo.org; Sun, 07 Aug 2005 13:21:59 +0000 Received: from robin.gentoo.org (localhost [127.0.0.1]) by robin.gentoo.org (8.13.4/8.13.4) with SMTP id j77DKWQF026840; Sun, 7 Aug 2005 13:20:32 GMT Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by robin.gentoo.org (8.13.4/8.13.4) with ESMTP id j77DKWRu011790 for ; Sun, 7 Aug 2005 13:20:32 GMT Received: from list by ciao.gmane.org with local (Exim 4.43) id 1E1l3p-0001Bq-U0 for gentoo-amd64@lists.gentoo.org; Sun, 07 Aug 2005 15:19:29 +0200 Received: from ip68-230-97-182.ph.ph.cox.net ([68.230.97.182]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 07 Aug 2005 15:19:29 +0200 Received: from 1i5t5.duncan by ip68-230-97-182.ph.ph.cox.net with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 07 Aug 2005 15:19:29 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: gentoo-amd64@lists.gentoo.org From: Duncan <1i5t5.duncan@cox.net> Subject: [gentoo-amd64] Re: file type not allowed in /usr/lib Date: Sun, 07 Aug 2005 06:18:16 -0700 Organization: Sometimes Message-ID: References: <42F53B6D.1080908@burnieanglican.org.au> Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-amd64@gentoo.org Reply-to: gentoo-amd64@lists.gentoo.org Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-Complaints-To: usenet@sea.gmane.org X-Gmane-NNTP-Posting-Host: ip68-230-97-182.ph.ph.cox.net User-Agent: Pan/0.14.2.91 (As She Crawled Across the Table) Sender: news X-Archives-Salt: a25c94a7-5918-490d-bfc2-14a61ef4b3d8 X-Archives-Hash: 06073844c0830dee0353ae2ebd6d1cc1 Will Briggs posted <42F53B6D.1080908@burnieanglican.org.au>, excerpted below, on Sun, 07 Aug 2005 08:36:29 +1000: > !!! ERROR: media-gfx/skencil-0.6.16 failed. !!! Function dyn_install, > Line 1114, Exitcode 0 !!! File > /var/tmp/portage/skencil-0.6.16/image///usr/lib/skencil-0.6.16/Lib/paxtkinter.so > matches a file type that is not allowed in /usr/lib !!! If you need > support, post the topmost build error, NOT this status message. > > So if anyone can help with a fix, please let me know. > > The _question_ relates to the fact that I've seen this sort of error a > number of times, but by the time I come across it it's been bugged and > fixed and the next emerge sync fixes it. Can someone let me know what > the issue is here - (mainly so I can get a handle on hacking my own > fixes when bug reports go unanswered) Why can't a .so file be placed in > /usr/lib? > > I'm assuming there's some AMD64 specificity to this bug. Because a) it > hasn't got the attention that x86 bugs get and b) its a /usr/lib issue > and I'm assuming multilib might be at play somehow. I'm repeating what Simon said, but sometimes it helps to get it said a couple different ways, and between the two you get the understanding. =8^) So, here goes... The immediate user-level fix, as Simon says, is kill the FEATURES=multilib-strict. That FEATURE is designed for testing for a specific situation that works the way things are now, but will cause serious issues when we go full LSB multilib (Gentoo-amd64 is kindof a hybrid situation at present). Those that understand what's going on and want to test for this issue can set that FEATURE and portage will die whenever it hits and ebuild or upstream tarball that's making unwarranted assumptions about the system library locations. Those who don't want the hassle should either NOT set FEATURES=multilib-strict, or set FEATURES=-multilib-strict, and the problem will go away as far as they are concerned, because it works right now, even if it isn't quite right. Here's the background. As Simon explained, in ordered to be able to have 32-bit and 64-bit copies of the same library, they need to be kept in different directories. Traditionally, of course, the location has always been lib, /lib, /usr/lib, etc. Now, if there are going to be two library locations, separated by bitness, there are two ways to go. Other archs previously have gone both ways, some one way, some the other. The old normal lib location can remain 32-bit, making it easier for legacy 32-bit programs some of which might be binary-only and therefore not easily modified, to continue to find their libraries and the like, with 64-bit libs then put in lib64, if the one way is chosen. If the other way is chosen, the old lib location remains the system "main" bitness location, 64-bit on a normal mainly 64-bit distribution, with 32-bit then being located in lib32. Therefore, one way, you have a 32-bit lib and also have lib64, the other, you have a 64-bit lib and also have lib32. One makes it easier for 32-bit legacy apps that might be difficult to change if they are binary-only, the other makes it easier (or at least more "natural") for the system's main 64-bit libraries, so both have their good and bad points. As it happens, Gentoo started its amd64 project before there was an agreed amd64 standard, and originally chose a 64-bit lib, with lib32 as well. In addition to the "native" lib logic explained above, this is the way ia64 (Intel's Itanium aka "Itanic") handled things, so it did seem totally logical. Then they make the standard (LSB/FHS, that's Linux Standard Base, and File Hierarchy Standard) for amd64, and turn the Gentoo table on its head! Most of the commercial distributions decide to go with a 32-bit lib and adding lib64, aiding compatibility with binary-only 32-bit apps, reasoning that 64-bit is new and will require certain other changes anyway, so it's easier to just make the change to lib64 at the same time and leave the legacy location well enough alone, avoiding trouble in 32-bit land. Additionally, this can be seen as logical as contrasted with ia64, because with ia64, 64-bit is the ONLY real native bitness, 32-bit being ALWAYS emulated, and slower, so it only makes sense to make lib the "native" 64-bit libs, with 32-bit libs relegated to the secondary lib32 location. amd64, OTOH, is actually FULL DUAL BITNESS IN HARDWARE, so whether 64-bit or 32-bit is "more native" really depends entirely on the approach taken by the individual OS/distribution platform, NOT the hardware. Thus, the legacy issues that amd64 has both the blessing and the curse of being able to deal with, simply weren't there for ia64, so while lib can be argued both ways for amd64, the choice was really rather clear for ia64. So... Gentoo quickly found itself going one way, while pretty much all of the rest of the Linux world had settled on going the other way, with the LSB/FHS multilib standard for amd64. Now, Gentoo is primarily open source, and there's certainly no law saying we /have/ to do it like everyone else does, so we /could/ have simply continued doing things the way we had been doing them. However, there are two very good reasons to change, despite the temporary confusion and pain it brings, while we are in-process. First, there's the binary compatibility issue, both with legacy apps and moving forward. It's simply harder to get binaries from other sources to work, when they are designed with one set of assumptions about locations, but installed on a system with a rather different set of assumptions. As I said, there's the 32-bit legacy thing, but consider continuing effect going forward, as well. If we stayed the way we are, every binary-only installation a Gentoo-amd64 user ever made, including the new amd64 binaries, when they become common, would be a fight. Despite the temporary confusion during the switch, therefore, switching will clearly make things easier for every Gentoo-amd64 user as time moves forward. Second, there's the upstream compatibility issue. It has been observed that despite the freedom to modify that the FLOSS (free, libre, and open source software) community not only allows but encourages, one of the major forces keeping the Linux community from splitting into a bunch of proprietary fiefdoms much as the proprietary UNIX community split, is the unifying/streamlining effect of open source sharing efficiencies. While there are the ever-present forces of differentiation, "my distribution is better than yours because we have these features you don't" that split the proprietary Unix community, the balancing force, the reason the same thing doesn't happen in the open source community, is the fact that for the mostly duplicated common code, it's ALWAYS more efficient to keep in line with the rest of the community, so the maintenance and further development efforts can continue to be shared, so each distribution only carries PART of the burden, rather than ALL of it, as they do for anything proprietary or non-standard they offer. Thus, while there are always differences in the areas a distribution considers important and valuable enough to be worth the extra cost to maintain and continue development, the differences in OTHER "unimportant" areas tend to be rather small, because of the economics forces favoring sharing the burden, which can only happen if the differences in the area remain relatively small or non-existent. It's a constant balancing act that by natural selection, continuously weeds out the differences a distribution doesn't consider important enough to support on its own. Thus, the upstream compatibility issue resolves to this: the closer we are to accepted LSB/FHS standard, the less work we will have to do making changes so that open source packages which generally target the loose standards, will work on Gentoo. Now, consider the fact that there's very little disagreement that amd64 *IS* the successor to x86. What maintenance issues we have now because we choose to do it our own way will only CONTINUE to increase in the future, as amd64 becomes the default and VERY widely assumed common platform, against which the changes necessary for all other platforms are diffed. There's simply NO good reason to continue to make all that extra work for ourselves, requiring entirely unnecessary changes to each new version of each new package, from now, almost certainly a good decade into the future, very likely a quarter century into the future, if amd64 has the same run that x86 did, and assuming of course Gentoo's around that long as well. Together, these two reasons means the only /possible/ sensible alternative is to bite that bullet, and exchange some temporary misery now, while the conversion is in progress, for a /far/ more standards compliant multilib setup, going forward. That, then, is the background, how we came to be in the situation we are in now, and why we are having to change things "in mid stream", as it were. The logical next questions are where are we now, and what will it take to get us where we want to be? Currently, Gentoo-amd64 has just about finished moving 64-bit libs out of lib. We have both a lib32, and a lib64, with lib being a symlink to lib64, for those remaining packages (such as skencil, apparently) that have so far fallen thru the cracks, and continue to make the early Gentoo-amd64 assumption that lib is the 64-bit shared object location. With 2005.1, I'm expecting FEATURES=multilib-strict will be the default, to finally weed out as many of the few remaining back packages as possible. With 2006.0, it's possible that lib -> lib64 symlink can finally be broken, and any remaining packages then WILL get bugs filed, when someone tries to use them and has issues. With luck, by 2006.1, it should be safe to start doing basically the same thing with the lib32 -> lib move, as we will have just got thru doing with the lib -> lib64 move. First, there will be a symlink between the two, in another release or two, the main packages will be changed and it'll be time to activate the multilib-strict test for anything still ending up in lib32 instead of lib. By 2007.0, therefore, if luck holds, that multilib-strict test can be the default, to catch all the stragglers possible, and 2007.1 should then hopefully be able to remove lib32, relegating it to the annuls of Gentoo-amd64 history. (Those .1 mentions assume that Gentoo continues with the twice-yearly releases, thus, they mean second-half.) However, that's really only half the issue. The other half is portage, which currently can't track 32-bit dependencies separate from its 64-bit dependencies. We had hoped that portage would have dual-bitness support by 2005.1, but that's now looking impossible, since the new portage remains only in CVS, there hasn't yet been even the usual hard masked for testing alpha snapshot releases, let alone the -preX versions, which would be the first ones that could even theoretically leave hard mask testing, for ~arch testing. Thus, it'll be 2006.0 at the earliest, before portage will be able to handle dual-bitness tracking, keeping the 32-bit dependencies separate from the 64-bit dependencies. Until then, Gentoo-amd64 32-bit support will remain spotty at best, requiring users either install from tarball, or use a 32-bit chroot with its own portage tracking 32-bit dependencies, for pretty much anything beyond the core system libraries (sandbox, gcc, glibc, now from-source supported, other 32-bit core libraries managed as binary-only packages or from the chroot). So... back to your error and multilib-strict. So... suppose you've decided to run multilib-strict and report the bugs you find. Since portage's method of notifying you of a multilib-strict violation is to die, how do you finish the installation once you've gotten the notice, filed the bug if necessary, and still want the package, before the bug is fixed? Well, as mentioned, the simplest, but /not/ necessarily shortest, method, would be to simply... FEATURES=-multilib-strict emerge --oneshot (Note that --oneshot. Packages placing libraries incorrectly and triggering this error will by definition mostly be libraries, that is, dependencies, not applications you want listed in your world file. Thus, the --oneshot, to emerge it without adding it to your world file.) HOWEVER, multilib-strict errors will normally be triggered ONLY during the (fake-)install phase -- when the package is doing the fake-install to its tempdir image directory. It's at that point sandbox detects the attempt to install a shared-object to lib, instead of lib64. The catch is, by that time, the major work of the emerge, all the compilation stuff, has already been done. The binaries should be perfectly fine, as they are the ones that would normally be installed, if multilib-strict was off, and would have been installed the way things worked before that feature was ever added to portage. However, emerge just died, so it could report the problem. Shouldn't there be a way to tell portage to go ahead and finish the merge using the existing work, now that we know about the bug and have presumably reported it if necessary, rather than starting ALL over again with the package emerge, as the above command will do? As it turns out, there /is/ such a way. "ebuild" is the command used to step thru an emerge of a single package. You can call it specifically, to merge a single package, or to step thru one step at a time (normally fetch, unpack, compile, install, qmerge), and in fact, emerge itself calls ebuild to handle each individual package, one at a time. I'm not going to go into detail on ebuild here, because it's all covered quite well in the Gentoo Handbook and in the ebuild (1) (5) manpages. However, that's the command you use. HOWEVER, you DO have to tweak one little thing, first, to tell portage not to worry about that multilib-strict that you've already dealt with. Emerge dies with the multilib-strict error, as mentioned, during the install step. As mentioned, the compile is already done, the binaries sitting in the tempdir work tree, ready to be moved over to the image dir, during the install phase. Portage will also, by that point, have created a number of additional files in the package's working tempdir, one of which contains the FEATURES line, which we have to modify, removing the "multilib-strict" portion, so ebuild won't trigger once again on it, once it gets to that point. The file is $PORTAGE_TMPDIR/portage//temp/environment . Open that in your favorite text editor and edit the FEATURES line, removing the multilib-strict. Save the file. NOW you can issue your ebuild merge command, and it should quickly figure out the compile step is done, and jump back into the install step, completing it and then finishing the merge. Whether it's worth the hassle of loading that file in your editor, removing the multilib-strict, saving, and running ebuild to finish the merge, against simply issuing the emerge command listed above, but going thru the whole thing including the compile step again, will probably depend on how big the package is. For something taking less than five minutes to emerge from start to finish, it's not worth the hassle, but it might be, if that compile step takes say an hour... -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman in http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html -- gentoo-amd64@gentoo.org mailing list