[gentoo-amd64] Problems configuring gentoo

[gentoo-amd64] Problems configuring gentoo

[Greg]

Thanks greatly for the responses. I have found them informative, but am sad to report the same result.

I downloaded and installed the stage3-amd64-2008.0 tarvol.
after copying over a few configuration files to save myself time (mainly /etc/make.conf, /etc/locale.gen, /etc/passwd, 
/etc/shadow, /etc/resolv.conf, and my ssh hostkeys)
I then performed emerge -e world

I ended up with the same kind of message:


                           
 * Messages for package sys-devel/gcc-4.1.2:
                                                                                                                                
 * If you have issues with packages unable to locate libstdc++.la,
 * then try running 'fix_libtool_files.sh' on the old gcc versions.
                                                                                                                                
 * Messages for package sys-libs/glibc-2.6.1:
                                                                                                                                
 *
 * ERROR: sys-libs/glibc-2.6.1 failed.
 * Call stack:
 * ebuild.sh, line 49:  Called src_compile
 * environment, line 3431:  Called eblit-run 'src_compile'
 * environment, line 1110:  Called eblit-glibc-src_compile
 * src_compile.eblit, line 173:  Called src_compile
 * environment, line 3431:  Called eblit-run 'src_compile'
 * environment, line 1110:  Called eblit-glibc-src_compile
 * src_compile.eblit, line 181:  Called toolchain-glibc_src_compile
 * src_compile.eblit, line 122:  Called die
 * The specific snippet of code:
 * make PARALLELMFLAGS="${MAKEOPTS}" || die "make for ${ABI} failed"
 * The die message:
 * make for x86 failed
 *
 * If you need support, post the topmost build error, and the call stack if relevant.
 * A complete build log is located at '/var/log/portage/sys-libs:glibc-2.6.1:20081003-103750.log'.
 * The ebuild environment file is located at '/var/build/portage/sys-libs/glibc-2.6.1/temp/environment'.
 *          

Here is the copy of my make.conf file that I'm trying to use. I kept make.profile as 
/usr/portage/profiles/default/linux/amd64/2008.0
I always do env-update and source /etc/profile, so these haven't been the issues.

I knew stage1s were no longer supported, but I came into using gentoo back in 2004 and learned how to use it through 
setting up stage1, making stage2, etc. I've never had major problems doing that with the various versions until now. I 
wasn't sure if emerge -e system or emerge -e world etc would do the same tasks.

Anyways, here is my make.conf -- and again, thanks for all the help and suggestions.

CFLAGS="-march=nocona -O2 -pipe"
CXXFLAGS="${CFLAGS} -fpermissive"
CHOST="x86_64-pc-linux-gnu"
USE="3dnow mmx sse sse2 threads threadpool -multilib"
#ACCEPT_KEYWORDS="~amd64"
PORTAGE_TMPDIR=/var/build
PORTDIR=/usr/portage
PKGDIR=${PORTDIR}/packages
PORT_LOGDIR=/var/log/portage
GENTOO_MIRRORS="http://gentoo.osuosl.org/ http://adelie.polymtl.ca/ http://distfiles.gentoo.org http://www.ibiblio.org/pub/Linux/distributions/gentoo"

SYNC="rsync://rsync.namerica.gentoo.org/gentoo-portage"
PORTAGE_RSYNC_RETRIES="3"
#EMERGE_DEFAULT_OPTS=""
MAKEOPTS="-j3"
#PORTAGE_NICENESS=10
AUTOCLEAN="yes"
PORTAGE_TMPFS="/tmp"
FEATURES="sandbox userpriv usersandbox notitles fixpackages loadpolicy unmerge-orphans"

Re: [gentoo-amd64] Problems configuring gentoo

[Barry Walsh]

Nocona isn't the right architecture for your CPU, that's for the Intel 
EMT64 CPU's. k8 would be more appropriate.



Greg wrote:
> Thanks greatly for the responses. I have found them informative, but am sad to report the same result.
>
> I downloaded and installed the stage3-amd64-2008.0 tarvol.
> after copying over a few configuration files to save myself time (mainly /etc/make.conf, /etc/locale.gen, /etc/passwd, 
> /etc/shadow, /etc/resolv.conf, and my ssh hostkeys)
> I then performed emerge -e world
>
> I ended up with the same kind of message:
>
>
>                            
>  * Messages for package sys-devel/gcc-4.1.2:
>                                                                                                                                 
>  * If you have issues with packages unable to locate libstdc++.la,
>  * then try running 'fix_libtool_files.sh' on the old gcc versions.
>                                                                                                                                 
>  * Messages for package sys-libs/glibc-2.6.1:
>                                                                                                                                 
>  *
>  * ERROR: sys-libs/glibc-2.6.1 failed.
>  * Call stack:
>  * ebuild.sh, line 49:  Called src_compile
>  * environment, line 3431:  Called eblit-run 'src_compile'
>  * environment, line 1110:  Called eblit-glibc-src_compile
>  * src_compile.eblit, line 173:  Called src_compile
>  * environment, line 3431:  Called eblit-run 'src_compile'
>  * environment, line 1110:  Called eblit-glibc-src_compile
>  * src_compile.eblit, line 181:  Called toolchain-glibc_src_compile
>  * src_compile.eblit, line 122:  Called die
>  * The specific snippet of code:
>  * make PARALLELMFLAGS="${MAKEOPTS}" || die "make for ${ABI} failed"
>  * The die message:
>  * make for x86 failed
>  *
>  * If you need support, post the topmost build error, and the call stack if relevant.
>  * A complete build log is located at '/var/log/portage/sys-libs:glibc-2.6.1:20081003-103750.log'.
>  * The ebuild environment file is located at '/var/build/portage/sys-libs/glibc-2.6.1/temp/environment'.
>  *          
>
> Here is the copy of my make.conf file that I'm trying to use. I kept make.profile as 
> /usr/portage/profiles/default/linux/amd64/2008.0
> I always do env-update and source /etc/profile, so these haven't been the issues.
>
> I knew stage1s were no longer supported, but I came into using gentoo back in 2004 and learned how to use it through 
> setting up stage1, making stage2, etc. I've never had major problems doing that with the various versions until now. I 
> wasn't sure if emerge -e system or emerge -e world etc would do the same tasks.
>
> Anyways, here is my make.conf -- and again, thanks for all the help and suggestions.
>
> CFLAGS="-march=nocona -O2 -pipe"
> CXXFLAGS="${CFLAGS} -fpermissive"
> CHOST="x86_64-pc-linux-gnu"
> USE="3dnow mmx sse sse2 threads threadpool -multilib"
> #ACCEPT_KEYWORDS="~amd64"
> PORTAGE_TMPDIR=/var/build
> PORTDIR=/usr/portage
> PKGDIR=${PORTDIR}/packages
> PORT_LOGDIR=/var/log/portage
> GENTOO_MIRRORS="http://gentoo.osuosl.org/ http://adelie.polymtl.ca/ http://distfiles.gentoo.org http://www.ibiblio.org/pub/Linux/distributions/gentoo"
>
> SYNC="rsync://rsync.namerica.gentoo.org/gentoo-portage"
> PORTAGE_RSYNC_RETRIES="3"
> #EMERGE_DEFAULT_OPTS=""
> MAKEOPTS="-j3"
> #PORTAGE_NICENESS=10
> AUTOCLEAN="yes"
> PORTAGE_TMPFS="/tmp"
> FEATURES="sandbox userpriv usersandbox notitles fixpackages loadpolicy unmerge-orphans"
>
>
> .
>
>   

Re: [gentoo-amd64] Problems configuring gentoo

[Beso]

2008/10/3 Greg <journey@raven.ontheside.net>:
> Thanks greatly for the responses. I have found them informative, but am sad to report the same result.
>
> I downloaded and installed the stage3-amd64-2008.0 tarvol.
> after copying over a few configuration files to save myself time (mainly /etc/make.conf, /etc/locale.gen, /etc/passwd,
> /etc/shadow, /etc/resolv.conf, and my ssh hostkeys)
> I then performed emerge -e world
>
> I ended up with the same kind of message:
>
>
>
>  * Messages for package sys-devel/gcc-4.1.2:
>
>  * If you have issues with packages unable to locate libstdc++.la,
>  * then try running 'fix_libtool_files.sh' on the old gcc versions.
>
>  * Messages for package sys-libs/glibc-2.6.1:
>
>  *
>  * ERROR: sys-libs/glibc-2.6.1 failed.
>  * Call stack:
>  * ebuild.sh, line 49:  Called src_compile
>  * environment, line 3431:  Called eblit-run 'src_compile'
>  * environment, line 1110:  Called eblit-glibc-src_compile
>  * src_compile.eblit, line 173:  Called src_compile
>  * environment, line 3431:  Called eblit-run 'src_compile'
>  * environment, line 1110:  Called eblit-glibc-src_compile
>  * src_compile.eblit, line 181:  Called toolchain-glibc_src_compile
>  * src_compile.eblit, line 122:  Called die
>  * The specific snippet of code:
>  * make PARALLELMFLAGS="${MAKEOPTS}" || die "make for ${ABI} failed"
>  * The die message:
>  * make for x86 failed
>  *
>  * If you need support, post the topmost build error, and the call stack if relevant.
>  * A complete build log is located at '/var/log/portage/sys-libs:glibc-2.6.1:20081003-103750.log'.
>  * The ebuild environment file is located at '/var/build/portage/sys-libs/glibc-2.6.1/temp/environment'.
>  *
>
> Here is the copy of my make.conf file that I'm trying to use. I kept make.profile as
> /usr/portage/profiles/default/linux/amd64/2008.0
> I always do env-update and source /etc/profile, so these haven't been the issues.
>
> I knew stage1s were no longer supported, but I came into using gentoo back in 2004 and learned how to use it through
> setting up stage1, making stage2, etc. I've never had major problems doing that with the various versions until now. I
> wasn't sure if emerge -e system or emerge -e world etc would do the same tasks.
>
> Anyways, here is my make.conf -- and again, thanks for all the help and suggestions.
>
> CFLAGS="-march=nocona -O2 -pipe"

first of all: nocona breaks your build. nocona isn't an amd processor
but an emt64 (intel). for your processor you need to use march=k8.
also, i'd use -Os instead of -O2 since it has some speed and size
optimizations and it's stable. if you've recompiled gcc like that you
probably won't be able to compile other packages after that and need
to restart the stage-3. for fresh stage 3 go to www.funtoo.org and
download the one you want. don't delete the distfiles since you'll
probably need them again.

> CXXFLAGS="${CFLAGS} -fpermissive"

remove -fpermissive and set it as per package if it's needed.

> CHOST="x86_64-pc-linux-gnu"
> USE="3dnow mmx sse sse2 threads threadpool -multilib"

sse isn't supported on athlons, so remove it


> #ACCEPT_KEYWORDS="~amd64"

this should accept both amd64 and ~amd64

> PORTAGE_TMPDIR=/var/build
> PORTDIR=/usr/portage
> PKGDIR=${PORTDIR}/packages
> PORT_LOGDIR=/var/log/portage
> GENTOO_MIRRORS="http://gentoo.osuosl.org/ http://adelie.polymtl.ca/ http://distfiles.gentoo.org http://www.ibiblio.org/pub/Linux/distributions/gentoo"
>
> SYNC="rsync://rsync.namerica.gentoo.org/gentoo-portage"
> PORTAGE_RSYNC_RETRIES="3"
> #EMERGE_DEFAULT_OPTS=""
> MAKEOPTS="-j3"

you could set this to -j5 without any issue. if you point the
/var/build directory to a /tmpfs (if you have some ram (about 3-4gb is
better, but if you don't have it, with at least 2gb you could try it
out)) you could also go with -j (unlimited jobs) or something near 8-9
jobs without killing the system.

> #PORTAGE_NICENESS=10
> AUTOCLEAN="yes"
> PORTAGE_TMPFS="/tmp"
> FEATURES="sandbox userpriv usersandbox notitles fixpackages loadpolicy unmerge-orphans"

remove unmerge-orphans from the features while compiling for the first time.

-- 
dott. ing. beso

Re: [gentoo-amd64] Problems configuring gentoo

[Greg]

Hmm. Okay, I thought sure I'd read that -march=k8 was for older amd processors and the -nocona was for newer 64 ones... I 
also know I compiled from stage1 up with nocona before... but given my history of problems with this machine and the 64bit 
gentoo compiles, I definitely appreciate the feedback. I'm sure that is a great deal of the problem.

Again, many, many, many thanks.

Re: [gentoo-amd64] Problems configuring gentoo

[Sascha Moeller]

Greg schrieb:
> Hmm. Okay, I thought sure I'd read that -march=k8 was for older amd processors and the -nocona was for newer 64 ones... I 
> also know I compiled from stage1 up with nocona before... but given my history of problems with this machine and the 64bit 
> gentoo compiles, I definitely appreciate the feedback. I'm sure that is a great deal of the problem.
>
> Again, many, many, many thanks.
>
>
>   
you could also try -march=athlon64  when you have the Athlon 64 X2 CPU 
:) for more informations see

http://gentoo-wiki.com/Safe_Cflags#Athlon_64_X2_.28AMD.29

Re: [gentoo-amd64] Problems configuring gentoo

[Ben de Groot]

[-- Attachment #1: Type: text/plain, Size: 1393 bytes --]

Beso wrote:
>> CFLAGS="-march=nocona -O2 -pipe"
> 
> first of all: nocona breaks your build. nocona isn't an amd processor
> but an emt64 (intel). for your processor you need to use march=k8.
> also, i'd use -Os instead of -O2 since it has some speed and size
> optimizations and it's stable. 

-Os optimizes for size, while -O2 optimizes for speed. There is no need
at all to use -Os on a modern desktop machine, and it will run
comparatively slower than -O2 optimized code, which is probably not what
you want.

>> CHOST="x86_64-pc-linux-gnu"
>> USE="3dnow mmx sse sse2 threads threadpool -multilib"
> 
> sse isn't supported on athlons, so remove it

As we're talking about athlon64 here, it definitely is. The Athlon64 X2
supports mmx, mmxext, 3dnow, 3dnowext, sse and sse2. And the AM2 models
also support sse3, but not ssse3 or sse4.

And as for -march, -march=athlon64 equals -march=k8 (see the gcc
manpage). So you can choose either one and get the exact same optimizations.

I hope this clears a few things up.

Cheers,
-- 
Ben de Groot
Gentoo Linux developer (lxde, media, desktop-misc)
Gentoo Linux Release Engineering PR liaison
__________________________________________________

yngwin@gentoo.org
http://ben.liveforge.org/
irc://chat.freenode.net/#gentoo-media
irc://irc.oftc.net/#lxde
__________________________________________________



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: [gentoo-amd64] Problems configuring gentoo

[Richard Freeman]

Ben de Groot wrote:
> 
> -Os optimizes for size, while -O2 optimizes for speed. There is no need
> at all to use -Os on a modern desktop machine, and it will run
> comparatively slower than -O2 optimized code, which is probably not what
> you want.
> 

There are a couple of schools of thought on that, and I think 
performance can depend a great deal on what program you're talking about.

On any machine, memory is a limited resource.  Oh sure, you could just 
"spend a little more on decent RAM", but you could also "spend a little 
more on a decent CPU" or whatever.  For a given amount of money you can 
only buy so much hardware, so any dollar spent on RAM is a dollar not 
spent on something else.

So, if you reduce the memory footprint of processes, then you increase 
the amount of memory available for buffers/cache.  That cache is many 
orders of magnitude faster than even the fastest hard drive.  You also 
reduce swapping which obviously greatly slows things down.

On the other hand, if you have a smaller program that does 
highly-CPU-intensive tasks like compression/transcoding/etc then speed 
optimization makes sense just about all the time (generally -O2 only - 
-O3 sometimes does worse due to L2 cache misses).

So, there are trade-offs.  To make things even more complicated the 
practical results can be impacted by your CPU model - different CPUs 
have a different cost for looping/jumping/branching vs cache misses. 
And the compiler makes a big difference - many of these observations 
date back to the gcc 3.4 days.  I've heard that newer GCC versions are 
more aggressive with size optimization at the expense of speed, which 
could tip the balance.

A while back there were some posts from folks who had done some actual 
benchmarking.  I don't think it has been repeated.  Note that I wouldn't 
expect -Os to have much benefit unless you compile EVERYTHING on the 
system that way - since the freeing up of memory is cumulative.  The 
rather high level of pain associated with rebuilding your system with 
-Os vs -O2 for some benchmarking and subjective evaluation is probably 
the reason why everybody has an opinion but there isn't much hard data.

Right now I'm using -Os across the board, and -O2 on an exception basis. 
  Maybe I should give that some more thought...

Re: [gentoo-amd64] Problems configuring gentoo

[Beso]

2008/10/4 Richard Freeman <rich0@gentoo.org>:
> Ben de Groot wrote:
>>
>> -Os optimizes for size, while -O2 optimizes for speed. There is no need
>> at all to use -Os on a modern desktop machine, and it will run
>> comparatively slower than -O2 optimized code, which is probably not what
>> you want.
>>
>
> There are a couple of schools of thought on that, and I think performance
> can depend a great deal on what program you're talking about.
>
> On any machine, memory is a limited resource.  Oh sure, you could just
> "spend a little more on decent RAM", but you could also "spend a little more
> on a decent CPU" or whatever.  For a given amount of money you can only buy
> so much hardware, so any dollar spent on RAM is a dollar not spent on
> something else.
>
> So, if you reduce the memory footprint of processes, then you increase the
> amount of memory available for buffers/cache.  That cache is many orders of
> magnitude faster than even the fastest hard drive.  You also reduce swapping
> which obviously greatly slows things down.
>
> On the other hand, if you have a smaller program that does
> highly-CPU-intensive tasks like compression/transcoding/etc then speed
> optimization makes sense just about all the time (generally -O2 only - -O3
> sometimes does worse due to L2 cache misses).
>
> So, there are trade-offs.  To make things even more complicated the
> practical results can be impacted by your CPU model - different CPUs have a
> different cost for looping/jumping/branching vs cache misses. And the
> compiler makes a big difference - many of these observations date back to
> the gcc 3.4 days.  I've heard that newer GCC versions are more aggressive
> with size optimization at the expense of speed, which could tip the balance.
>
> A while back there were some posts from folks who had done some actual
> benchmarking.  I don't think it has been repeated.  Note that I wouldn't
> expect -Os to have much benefit unless you compile EVERYTHING on the system
> that way - since the freeing up of memory is cumulative.  The rather high
> level of pain associated with rebuilding your system with -Os vs -O2 for
> some benchmarking and subjective evaluation is probably the reason why
> everybody has an opinion but there isn't much hard data.
>
> Right now I'm using -Os across the board, and -O2 on an exception basis.
>  Maybe I should give that some more thought...
>
>

well, Os not only does O2 but does also some O3 safe flags and some
size optimizations. if you look at the -Os vs O2 and vs O3
optimization you'll see that most of the O3 flags that you want over
O2 are insinde Os. i've actually tried out O2, O3 and Os and found out
that O3 is sometimes faster (not everytime and especially with system
packages is even slower than O2) while Os is almost always faster than
O2. this leads me to think that if something is faster and also
smaller then it really worths trying it out. i've made a comparison on
gcc 4.1.0 with all the system and world built with a single option (of
course some packages have harcoded ebuild optimization and won't be
affected). when i'll have some spare time i'll look out for the
reports and post them.

-- 
dott. ing. beso

[gentoo-amd64] Re: Problems configuring gentoo

[Duncan]

Richard Freeman <rich0@gentoo.org> posted 48E759DB.6080905@gentoo.org,
excerpted below, on  Sat, 04 Oct 2008 07:56:11 -0400:

> Ben de Groot wrote:
>> 
>> -Os optimizes for size, while -O2 optimizes for speed. There is no need
>> at all to use -Os on a modern desktop machine, and it will run
>> comparatively slower than -O2 optimized code, which is probably not
>> what you want.
>> 
>> 
> There are a couple of schools of thought on that, and I think
> performance can depend a great deal on what program you're talking
> about.
> 
> On any machine, memory is a limited resource.  Oh sure, you could just
> "spend a little more on decent RAM", but you could also "spend a little
> more on a decent CPU" or whatever.  For a given amount of money you can
> only buy so much hardware, so any dollar spent on RAM is a dollar not
> spent on something else.

I agree, but stress the limits of the L1/L2(/L3?) caches more than 
general system memory.  You can generally buy more system memory fairly 
easily, but Lx cache sizes aren't so easy to chance, and are likely to 
remain quite limited resources for some time.

So I generally see the benefits of -Os over -O2, with a few exceptions.  
-freorder-blocks-and-partition (only in CFLAGS, it doesn't work on C++/
CXXFLAGS) can increase the raw size but manages cache better because it 
separates code into hot and cold blocks, giving the hot code a better 
chance at staying in-cache (according to the gcc manpage).  There's a 
couple other similar flags.

Of course, to /really/ get performance, one would need to compile with 
code profiling instrumentation turned on, run the program as you would 
normally (but profiled) for awhile to generate some profiling history, 
then recompile using that history to help optimize things.  This BTW is 
one of the reasons I wonder about -ftracer when I see it in someone's 
CFLAGS.  The gcc manpage says it helps other optimization, but then links 
it to -fprofile-use.  How much help it is without the profiling isn't 
covered, but given the increase in size and the effect of that on caches, 
it's likely not worth it without the profiling.  How many people compile 
first for profiling, run the program to generate profiles, then recompile 
using the profile data?  Right, not so many, at least for most apps.  In 
that case, why do they have -ftracer in their general CFLAGS?

That said, I recently switched to -O2 from my long time -Os.  Much of the 
difference in gcc-3 was due to -funit-at-a-time and similar 
optimizations, enabled by default early on for -Os, but not for -O2 until 
gcc-4.something, I believe.  Modern gcc is more cache-usage-performance 
aware than gcc-3 was, and I think most of the remaining differences are 
like the -freorder-blocks-and-partition thing, they affect CPU cache 
usage negatively enough that you don't want them enabled except for old 
machines, embedded, and perhaps the now popular netbook/atom type 
applications.

Talking about which... I just got my Acer Aspire One (32-bit Atom n270 
CPU), and intend to do a 32-bit chroot on my main machine and create 
binpkgs to merge to the AA1.  Any idea what sort of CFLAGS to use on it?  
I know it doesn't have all that fancy branch prediction and prefetch 
stuff of a normal modern x86_(32/64) CPU.  One suggestion I've seen is 
-march=686, and I'll probably do -Os for it, but what about stuff like 
-fweb -frename-registers, etc?  It does have thru SSE3 at least, so I can 
enable that too.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman