[gentoo-amd64] problems with gmp and portage

[gentoo-amd64] problems with gmp and portage

[Kyle Liddell]

(Disclaimer:  This is pretty much a copy of a bug report (http://bugs.gentoo.org/show_bug.cgi?id=138231), but I'm hoping somebody here might have any ideas on what to try.)

I've been unable to use emerge to update gmp from 4.2 to 4.2.1 for a couple weeks now.  The compile will go on for a while, and then fail with "recompile with -fPIC".  However, I can build gmp by hand with no problems.  I can apply all the patches that portage throws on, and it still works.

Does anyone have any ideas?

Thanks,
Kyle Liddell

-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: problems with gmp and portage

[Duncan]

Kyle Liddell <kyle@foobox.homelinux.net> posted
20060705133740.GA2652@athlon.localdomain, excerpted below, on  Wed, 05 Jul
2006 08:37:40 -0500:

> (Disclaimer:  This is pretty much a copy of a bug report
> (http://bugs.gentoo.org/show_bug.cgi?id=138231), but I'm hoping somebody
> here might have any ideas on what to try.)
> 
> I've been unable to use emerge to update gmp from 4.2 to 4.2.1 for a
> couple weeks now.  The compile will go on for a while, and then fail with
> "recompile with -fPIC".  However, I can build gmp by hand with no
> problems.  I can apply all the patches that portage throws on, and it
> still works.
> 
> Does anyone have any ideas?

Interesting.  Take a look at the kdm (hijacked) thread too.  It's the same
-fPIC R_X86_64_32 type error there, but a bit different, and as no bug had
been filed, all the system details from emerge --info are missing.

In both cases (kdm and gmp), no issues at all merging them here.  Same
~amd64 keyworded system, altho I /was/ running the masked 4.0 and 4.1 for
some time before 4.1.1 was released and unmasked to ~.

My first thought was a corrupted ccache or confcache that kept getting
used thru portage, and I was going to suggest disabling that, but I see
from the bug neither of you have confcache merged and while one of you had
ccache, it was disabled, so that can't be the issue.

Have you run fix_libtool_files.sh recently?  I'm asking because I see the
*.la in the trace and both kdm and gmp appear to be C++.  If some of those
are still pointing at your old gcc-3.4.x install and not the new
gcc-4.1.1 it could create issues, tho I can't say it would create /this/
one.

Also, revdep-rebuild.  Have you run it recently?  (Run it with -p first,
of course.)  What about emerge --depclean (again, with -p first)?

Do you routinely run your emerge --update with the --deep (-D) switch?

How much of your system has been recompiled with gcc-4.1.x?

Of course, all of these questions boil down to one thing, how good your
general system hygiene and cleanliness is.  I can't say for sure that it'll
help, but certainly, having a bunch of old cruft hanging around gathering
bitrot will complicate things a bit.  In particular, it's possible
something compiled with the old gcc isn't quite compatible with the new
gcc.  As I was running the masked 4.0.x and 4.1.0 for some time, and I did
an emerge --emptytree world somewhere in the process, there's very little
of my system that's still compiled with gcc-3.4.x.  If you haven't done an
emerge --emptytree world since gcc-4.1.1 was unmasked and weren't running
the masked 4.x versions as I was, perhaps that's the difference between it
working fine here and not there?

Another possible difference... I run very little 32-bit at all, and in
fact have considered switching to the no-multilib profiles.  (About the
only thing besides the 32-bit toolchain, sandbox/gcc/glibc/binutils, that
I need multilib for, is grub, and I /could/ merge the grub-static binary
if I were to decide to go no-multilib.)  I have a rather intense aversion
to slaveryware, so don't need 32-bit for that, and don't need OOo, so
don't need it for that or any other "obscure" freedomware stuff that's
still 32-bit only, so other than grub and the multilib toolchain, I'm 100%
64-bit.  The R_x86_64_32 name hints that it /might/ be 32-bit related.  Do
both you guys have lots of 32-bit stuff installed or are you like me,
nearly all 64-bit?  Maybe it's another problem like the opengl problem,
with the compile unsuccessfully of course trying to use the 32-bit libs
for some reason.  Since I have very few of them on my system, I wouldn't
have that problem, just as I don't have that particular opengl problem. 
If you have a lot of 32-bit libraries, try temporarily moving those dirs
out of the way to somewhere the compile can't find them, and see if the
problem "magically" disappears.  It's worth a shot, anyway.

The system cleanliness thing is a good thing to check anyway, even if it
doesn't fix this particular issue, so that's good to try first.  The
32-bit thing is simple enough to try, so that's next.  Beyond that, given
that it compiles fine outside portage, the issue must be one of the
differing environment between the portage compile and doing it on your
own.  

Since you've checked the patches, you've probably checked this too, but you
didn't mention it so I will.  Did you try exactly duplicating the ebuild's
configure (econf) switches?  If you are aren't giving it the same
configure input, that could be the culprit.  In fact, I've seen it happen
more than once in my own troubleshooting, so be sure and check that.

Are you familiar with the ebuild command?  If not, man ebuild and bone up
on the various stages (fetch/unpack/compile/install/qmerge, in that order)
that emerge automates, then try doing them by hand.  In particular,
compare ebuild compile, with ebuild unpack, then doing the configure
(using the ebuild switches) and make manually, with unpack and doing the
configure and make manually but without the ebuild switches.  Also note
the "elibtoolize" at the end of the unpack step.  Perhaps that's the issue.

Kind of shots in the dark -- unfortunately I don't understand the issues
well enough to do better than that -- but maybe they'll be of help...

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: problems with gmp and portage

[Kyle Liddell]

On Wed, Jul 05, 2006 at 05:04:00PM +0000, Duncan wrote:
> Have you run fix_libtool_files.sh recently?  I'm asking because I see the
> 
I always forget to run that, but I did, and still no gmp.

> Also, revdep-rebuild.  Have you run it recently?  (Run it with -p first,
Yep.  No help.

> of course.)  What about emerge --depclean (again, with -p first)?
Nothing looked like it was related in any way.

> Do you routinely run your emerge --update with the --deep (-D) switch?
Always.

> How much of your system has been recompiled with gcc-4.1.x?
Umm...not much.  If I could ever get gmp compiled, then I could get around to compiling a good chunk of the rest...

However, I have built/rebuilt/rebuilt again gcc, glibc, and binutils with gcc 4.1.1.

> of my system that's still compiled with gcc-3.4.x.  If you haven't done an
> emerge --emptytree world since gcc-4.1.1 was unmasked and weren't running
> the masked 4.x versions as I was, perhaps that's the difference between it
> working fine here and not there?
> 

Ick, that's (almost) the same as a reinstall...but I think I will end up doing that.

> Another possible difference... I run very little 32-bit at all, and in
> fact have considered switching to the no-multilib profiles.  (About the
	...
> out of the way to somewhere the compile can't find them, and see if the
> problem "magically" disappears.  It's worth a shot, anyway.

And oh yeah, I tried emerge'ing with -sandbox.  Same deal.

About all I have 32bit-wise is enough to get a flash plugin for firefox-bin.  I have thought that multilib was a bit messy though, and I might try dumping it since most of my 32bit stuff only works in a chroot'd system (ISE, wine, etc).  But as for this exact problem, I don't really see anything that is safe to delete/move/hide that is definitely 32bit only that could be making problems.

> The system cleanliness thing is a good thing to check anyway, even if it
> doesn't fix this particular issue, so that's good to try first.  The
	...
> configure input, that could be the culprit.  In fact, I've seen it happen
> more than once in my own troubleshooting, so be sure and check that.
> 
Building by hand works.

> Are you familiar with the ebuild command?  If not, man ebuild and bone up
> on the various stages (fetch/unpack/compile/install/qmerge, in that order)
	...
> configure and make manually but without the ebuild switches.  Also note
> the "elibtoolize" at the end of the unpack step.  Perhaps that's the issue.
> 
Doing it in stages with ebuild fails during compile at the same place.  However, I can do ebuild unpack, and then run configure the same way as ebuild does, and then run make by hand and it also works.

I suppose it'd be good to get everything compiled/broken with gcc 4.1.1, so I'm just going to give up and start an emerge -ep world tonight and see what happens...

Thanks for the help though.

Kyle

-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: Re: problems with gmp and portage

[Duncan]

Kyle Liddell <kyle@foobox.homelinux.net> posted
20060706042949.GA24867@athlon.localdomain, excerpted below, on  Wed, 05
Jul 2006 23:29:49 -0500:

> On Wed, Jul 05, 2006 at 05:04:00PM +0000, Duncan wrote:
>> Have you run fix_libtool_files.sh recently?  I'm asking because I see
>> the
>> 
> I always forget to run that, but I did, and still no gmp.

I didn't really expect it to, as those errors are usually rather
different, but thought it was worth a shot.

>> Do you routinely run your emerge --update with the --deep (-D) switch?
> Always.

Someone who thinks like me! =8^)  I'd rather have them updated one at a
time than find I need to update a whole bunch when say a new KDE comes out
and I have 100-ish just KDE packages to update!  While it does
occasionally produce headaches, I'd rather have those headaches than
trying to trace something that isn't working just because it's too old.

So anyway, that's eliminated too...
 
>> How much of your system has been recompiled with gcc-4.1.x?

[reordered quote below a bit to make sense after snippage]
> Ick, that's (almost) the same as a reinstall...
> Umm...not much.  If I could ever get gmp compiled, then I could get
> around to compiling a good chunk of the rest...
> 
> However, I have built/rebuilt/rebuilt again gcc, glibc, and binutils
> with gcc 4.1.1.

The gcc upgrade guide (http://www.gentoo.org/doc/en/gcc-upgrading.xml)
suggests an emerge -e system followed by emerge -e world, which would
actually recompile the entire system twice, once for each command. 
Certainly, I can see that for at minimum gcc, since the first time it's
built it will be with the old version, so a second gcc rebuild is necessary
to have /everything/ compiled with it.  However, for the non-toolchain
parts of system IMO that's a bit overboard.  Still, since there's no
"tool-chain" target, that does keep the instructions simpler than having
to list each bit of the toolchain separately.

Personally, I think it's a bit overboard to run an emerge --emptytree
world anyway, in most cases for ~arch anyway, since it upgrades more
frequently.  There'd be the occasional exception (like the special-cases
in the upgrade guide), but I don't think it needs done every time.  Stable
is a bit more iffy, since they don't get the benefit of the intermediate
upgrades that ~arch usually does.  I guess I can see the argument being
made for them, which of course means it's made for everyone since stable
is intended to be the arch default.

That said, I'd not be /too/ worried about it in general, but certainly,
when strange errors start occurring shortly after a gcc upgrade, it's a
good troubleshooting step if the process was previously skipped.

Again, I've personally been running far more intermediate steps, unmasking
and making system default 4.0.0, 4.0.1, 4.0.2, 4.0.3, and 4.1.0 in turn as
they came available.  It /is/ quite a big jump from 3.4.6 to 4.1.1, but I
didn't do it all at once as those running ~arch that didn't unmask along
the way will have done.  Actually, that emerge --emptytree that I did
along the way wasn't necessary AFAIK.  I just did it after upgrading to 8
gig of memory and setting $PORTAGE_TMPDIR to a tmpfs, mainly to test my
hardware, but also to see if all the previous issues with gcc-4.x on
specific packages had been fixed, as well as to just do an emerge
--emptytree to say I'd done it, since I hadn't before that and I wanted to
try it.

Anyway, like I said, I'd not go as far as the upgrade guide and do it
routinely for every gcc upgrade, but certainly, after moving from 3.4.x to
4.1.x, if I experienced weird errors, I'd do it just to eliminate that as
a possibility, which is what I'm saying here.


>> Another possible difference... I run very little 32-bit at all
> 
> And oh yeah, I tried emerge'ing with -sandbox.  Same deal.

Oh yeah, I was going to mention that but forgot.  You're ahead of me on
that!  =8^)
 
> About all I have 32bit-wise is enough to get a flash plugin for
> firefox-bin.  I have thought that multilib was a bit messy though, and I
> might try dumping it since most of my 32bit stuff only works in a
> chroot'd system (ISE, wine, etc).  But as for this exact problem, I
> don't really see anything that is safe to delete/move/hide that is
> definitely 32bit only that could be making problems.

The possible problem would be the configure for some reason pointing at
the 32-bit lib.  Since this isn't a 32-bit compile, as long as you don't
run any 32-bit programs while doing it, you should have no issues
temporarily moving any 32-bit libdirs out of the way.  It's a long shot,
but I can't see any harm in it as long as you aren't running anything
32-bit when you move it, and if it has chance at working...

> Building by hand works.

>> configure and make manually but without the ebuild switches.  Also note
>> the "elibtoolize" at the end of the unpack step.  Perhaps that's the
>> issue.
>> 
> Doing it in stages with ebuild fails during compile at the same place.
> However, I can do ebuild unpack, and then run configure the same way as
> ebuild does, and then run make by hand and it also works.

Well, that eliminates the elibtoolize at the end of the unpack as the
culprit.

> I suppose it'd be good to get everything compiled/broken with gcc 4.1.1,
> so I'm just going to give up and start an emerge -ep world tonight and
> see what happens...

One bonus you will likely see, particularly with glibc, kde if you have it
merged, and all the parts of modular-X, is that the gcc-4.1.1
compiled versions should be noticeably faster than 3.4.x versions were.
=8^)  gcc 4.1 is the first gcc to be truly optimized for amd64 --
previous to 4.0 amd64 support was really just bolted onto the side as an
afterthought and 4.0 was just the rewrite attempting not to have any big
regressions, so 4.1 is the first truly optimized 4.x series so the first
to really shine in terms of amd64 optimizations.  Of course, CFLAGS and
what you have merged as well as your specific hardware will play very big
parts in all that as well, but I DEFINITELY noticed the difference here,
and was duly impressed! =8^)

So... even if the --emptytree doesn't solve this particular gmp issue, it
won't be all for naught, as with any luck, you'll definitely notice the
speed improvement!

---

As for troubleshooting beyond the previous suggestions...

Try a diff between the config.log (or is it configure.log, IDR) produced by
your by-hand compile (that succeeds, and the one produced by the failing
ebuild.  I don't know why I didn't think of this earlier, but by rights
that /ought/ to spotlight the problem!

Of course, while that will likely spotlight the problem, you (or someone
on the bug) still have to figure out what's triggering the different
behavior.  Again, all I can figure is something in the environment that
differs between you doing it manually and portage's run, but what?  Maybe
the bug will end up being one in portage!

(I think I'll cc myself to the bug, as stuff like this intrigues me, and
I'd love to know what the problem ultimately turns out to be!)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: problems with gmp and portage

[Kyle Liddell]

So, I decided to give up and try doing an emerge -ep world to rebuild everything.  However, I get very similar errors when trying to build gettext.  Just for fun, I changed my profile to 2006.0/no-multilib, and I got the same errors.  Finally, in the spirit of really breaking things, I ran the bootstrap.sh script, but it died in the same way.

How do I fix my system now?

Thanks,
Kyle
-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: Re: problems with gmp and portage

[Duncan]

Kyle Liddell <kyle@foobox.homelinux.net> posted
20060706141513.GA14608@athlon.localdomain, excerpted below, on  Thu, 06
Jul 2006 09:15:13 -0500:

> So, I decided to give up and try doing an emerge -ep world to rebuild
> everything.  However, I get very similar errors when trying to build
> gettext.  Just for fun, I changed my profile to 2006.0/no-multilib, and I
> got the same errors.  Finally, in the spirit of really breaking things, I
> ran the bootstrap.sh script, but it died in the same way.
> 
> How do I fix my system now?

Well, it's at least consistent! <g>

I was hoping that the rebuild would correct the problem before it got to
anything affected by it, but it seems not.

Did you try a diff on the configure.log (config.log?) from the successful
non-portage run and the unsuccessful portage run?  As I said in a previous
response, with any luck at all, that should reveal the difference, and
from there it should be possible to work backward to what environmental
difference caused it.

For kicks, if you go back to stable portage (looks like 2.1-r1, from my
sync about 18 hours ago), does that fix it? What about going back to 2.0.5x
portage (2.0.54-r2, or 2.0.51.22-r3)?  Note that you should do an emerge
--metadata after reverting to 2.0.5x.



-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list