[gentoo-amd64] gcc 4.1 upgrade - bad desktop interactivity anyone?

[gentoo-amd64] gcc 4.1 upgrade - bad desktop interactivity anyone?

[Mark Knecht]

Hi,
   I'm just curious whether anyone besides me is noticing their
machine feeling somewhat sluggish since doing the gcc-4.1 upgrade?
Mine seems ot be using a lot of memory. Alt-tabbing between windows
seems slow. Ethernet traffic in my browser is causing pretty
noticeable interruptions in things like MythTV.

   I noticed that the rtirq scripts were not setting one of my sound
cards priorities correctly after the upgrade so I changed the config
file and fixed that, but I don't think it should have changed at all.

   The machine is still quite usable, but it doesn't feel as snappy as
it did last week.

   I made no changes in /etc/make.conf for the upgrade. Everything is
pretty basic as far as I can tell:

CFLAGS="-march=k8 -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
USE="dri radeon mmx mmxext sse sse2 3dnow 3dnowext gnome kde -esd
-arts ladspa nptl nptlonly audiofile gimp gimpprint ppds usb alsa cdr
dvd dvdr dvdread jack jack-tmpfs fluidsynth tcltk sndfile v4l v4l2
mysql flac xscreensaver -samba i8x0 mythtv apache2 -lirc mjpeg xvid
real cjk unicode vorbis ogg truetype java -eds"
CXXFLAGS="${CFLAGS}"
MAKEOPTS="-j2"

   3GHz AMD64 with 1GB & NVidia chipset/SATA drives.

Mark
-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: gcc 4.1 upgrade - bad desktop interactivity anyone?

[Duncan]

"Mark Knecht" <markknecht@gmail.com> posted
5bdc1c8b0609140715t2c135c9q50bf7ff8cbf1d82e@mail.gmail.com, excerpted
below, on  Thu, 14 Sep 2006 07:15:42 -0700:

> I'm just curious whether anyone besides me is noticing their machine
> feeling somewhat sluggish since doing the gcc-4.1 upgrade? Mine seems ot
> be using a lot of memory. Alt-tabbing between windows seems slow.
> Ethernet traffic in my browser is causing pretty noticeable
> interruptions in things like MythTV.

> The machine is still quite usable, but it doesn't feel as snappy as it
> did last week.
> 
> I made no changes in /etc/make.conf for the upgrade. Everything is
> pretty basic as far as I can tell:
> 
> CFLAGS="-march=k8 -O2 -pipe"

> CXXFLAGS="${CFLAGS}"

I've noticed rather the opposite, here.  gcc-4.1.1 compiled binaries are
/dramatically/ faster and more efficient than 3.x.  However, I'm using a
rather more elaborate CFLAGS/CXXFLAGS, and it's my conviction that gcc-4.1
does better at optimizing exactly the way you've told it to.  That is, if
you've given it inefficient optimizations, I'm convinced it makes a bad
thing worse, while if you've chosen your optimizations well, it makes a
good thing dramatically better.

Here's my CFLAGS/CXXFLAGS:

CFLAGS="-march=k8 -Os -pipe -frename-registers -fweb -freorder-blocks
-freorder-blocks-and-partition -combine -funit-at-a-time -ftree-pre
-fgcse-sm -fgcse-las -fgcse-after-reload -fmerge-all-constants"

CXXFLAGS="-march=k8 -Os -pipe -frename-registers -fweb -freorder-blocks
-funit-at-a-time -ftree-pre -fgcse-sm -fgcse-las -fgcse-after-reload
-fmerge-all-constants"

The general strategy here is to take advantage of size optimization -- on
modern compilers, L1 and L2 cache are FAR FAR faster than main memory, and
raw CPU cycles runs circles around even cache speeds.  Thus, optimizing
for CPU speed at the expense of size makes little sense, because all those
saved cycles and more are likely to be spent waiting for memory to return
code that /would/ have fit in the cache were it size optimized.

Thus, for example, where traditional optimizations unroll loops into
flat code where possible, to avoid the expense of the jump back to the top
of the loop, that spreads out the loop to several times its original code
size, thus taking far more room in fast cache and forcing the CPU to wait
far more often for code to be fetched from main memory.  I prefer to keep
the loops, making the code smaller and thus allowing more of it to fit in
faster cache.  I believe that for most code, this technique will result in
faster execution in the real world, despite the theoretical loss of a CPU
cycle here or there due to jumping back to the top of the loop.

The -freorder-blocks-and-partition, OTOH, can make code slightly larger,
but the effect is the same as the above, increasing execution speed.  What
this optimization does is separate code that is used often from that which
is seldom used, so the "hot" code is smaller and fits better in high speed
cache, while the "cold" code ends up in slower main memory most of the
time.  While a lower percentage of the code may be in cache due to the
larger size, cache will be used far more effectively, as more "hot" code
will be retained therein, with the cold code that's not used so often
allowed to drop out of cache into main memory.  This particular
optimization doesn't work well with C++, however, so it's in my CFLAGS but
not my CXXFLAGS.

Likewise with -combine, which allows the compiler to optimize across
multiple source files at a time.  It's only implemented for C at this time
(according to the gcc manpage), so it's in my CFLAGS but omitted from my
CXXFLAGS.

The other strategy here is to make as full a use of the extra registers
available to amd64 in 64-bit mode (as opposed to 32-bit x86 mode) as
possible.  Registers operate at the speed of the CPU, no wait at all, as
there is for even L1 cache, so it pays to use them as efficiently as
possible.  Several of the flags (-frename-registers of course, -fweb, etc)
in my CFLAGS are therefore designed to encourage gcc to do this.

All the flags I've not mentioned specifically are designed to further the
three common goals mentioned above, making as efficient a use as possible
of the speed of (1) registers and (2) cache memory, by allowing gcc to
optimize over as wide a scope (3, whole units with unit-at-a-time, or
even multiple units with -combine) as possible.  Of course, see the gcc
manpage for additional details.

As I said, with the above, there's a /dramatic/ improvement in
performance between gcc-3.x and gcc-4.1.x.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: gcc 4.1 upgrade - bad desktop interactivity anyone?

[Richard Freeman]

[-- Attachment #1: Type: text/plain, Size: 475 bytes --]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Duncan wrote:


Hmm - no -ftree-vectorize?  Care to comment on that?  I hear that it can
be buggy with a few packages, but I'm guessing it is worth having in
there in general.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFFCemPG4/rWKZmVWkRArAVAJ9e6zKeeHuNvEa6PlJm3iqzgVmJ8gCgm1rG
P+lazfNdJNmNaaoMMlBBmPw=
=fJaA
-----END PGP SIGNATURE-----

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/x-pkcs7-signature, Size: 3875 bytes --]

Re: [gentoo-amd64] Re: gcc 4.1 upgrade - bad desktop interactivity anyone?

[Mark Knecht]

On 9/14/06, Duncan <1i5t5.duncan@cox.net> wrote:
> "Mark Knecht" <markknecht@gmail.com> posted
> 5bdc1c8b0609140715t2c135c9q50bf7ff8cbf1d82e@mail.gmail.com, excerpted
> below, on  Thu, 14 Sep 2006 07:15:42 -0700:
>
> > I'm just curious whether anyone besides me is noticing their machine
> > feeling somewhat sluggish since doing the gcc-4.1 upgrade? Mine seems ot
> > be using a lot of memory. Alt-tabbing between windows seems slow.
> > Ethernet traffic in my browser is causing pretty noticeable
> > interruptions in things like MythTV.
>
> > The machine is still quite usable, but it doesn't feel as snappy as it
> > did last week.
> >
> > I made no changes in /etc/make.conf for the upgrade. Everything is
> > pretty basic as far as I can tell:
> >
> > CFLAGS="-march=k8 -O2 -pipe"
>
> > CXXFLAGS="${CFLAGS}"
>
> I've noticed rather the opposite, here.  gcc-4.1.1 compiled binaries are
> /dramatically/ faster and more efficient than 3.x.  However, I'm using a
> rather more elaborate CFLAGS/CXXFLAGS, and it's my conviction that gcc-4.1
> does better at optimizing exactly the way you've told it to.  That is, if
> you've given it inefficient optimizations, I'm convinced it makes a bad
> thing worse, while if you've chosen your optimizations well, it makes a
> good thing dramatically better.
>
> Here's my CFLAGS/CXXFLAGS:
>
> CFLAGS="-march=k8 -Os -pipe -frename-registers -fweb -freorder-blocks
> -freorder-blocks-and-partition -combine -funit-at-a-time -ftree-pre
> -fgcse-sm -fgcse-las -fgcse-after-reload -fmerge-all-constants"
>
> CXXFLAGS="-march=k8 -Os -pipe -frename-registers -fweb -freorder-blocks
> -funit-at-a-time -ftree-pre -fgcse-sm -fgcse-las -fgcse-after-reload
> -fmerge-all-constants"
>
<SNIP>
>
> As I said, with the above, there's a /dramatic/ improvement in
> performance between gcc-3.x and gcc-4.1.x.
>
> --
> Duncan - List replies preferred.   No HTML msgs.

Hi Duncan,
   As always, very deep thanks for the answer. Very informative and interesting.

   Now, you are very adept at this. You're explanations make sense to
the level I've considered them. (Not very far right now...) Main
questions:

1) What can be done to test this out at my end without making a 2-day
commitment to rebuild the complete machine. Is it possibly to rebuild
only portions of the machine using a different set of flags or is it a
system wide commitment requiring that I rebuild 575 packages as I did
last weekend?

2) What about building the kernel? How do the standard

make && make modules_install

command make any use of the flags in /etc/make.conf?

   This machine is a fairly standard desktop running Xorg-7, Gnome and
just a few apps most of the time. However I am an audio oriented
person so my kernel is rt-sources from the proaudio overlay. (Ingo
Molnar's patches to the kernel.org kernels and not a Gentoo kernel.) I
need to ensure that the audio stuff (Jack, Ardour, Aqualung, 1394 hard
drives) continue to work well.

   Your ideas are most welcome.

Thanks,
Mark
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: gcc 4.1 upgrade - bad desktop interactivity anyone?

[Greg Bur]

[-- Attachment #1: Type: text/plain, Size: 1570 bytes --]

On 9/14/06, Mark Knecht <markknecht@gmail.com> wrote:
>
> On 9/14/06, Duncan <1i5t5.duncan@cox.net> wrote:
> > "Mark Knecht" <markknecht@gmail.com> posted
> > 5bdc1c8b0609140715t2c135c9q50bf7ff8cbf1d82e@mail.gmail.com, excerpted
> > below, on  Thu, 14 Sep 2006 07:15:42 -0700:
>
> 2) What about building the kernel? How do the standard
>
> make && make modules_install
>
> command make any use of the flags in /etc/make.conf?


I believe you have to modify the Makefile in /usr/src/linux to enable
additional optimizations.  I have noticed in recent kernels that there is an
option to compile using -Os as well however I have not used that yet simply
because I try to play it safe, especially with my kernels.  I would be
interested in hearing feedback as to which "safe" optimizations can be used
when building a kernel.

   This machine is a fairly standard desktop running Xorg-7, Gnome and
> just a few apps most of the time. However I am an audio oriented


I share your concern here as well.  One app in particular that comes to mind
is lyx which in the past has not gotten along well with heavy optimization,
at least for me.  Granted this was nearly two years ago when I, like many
newcomers to Gentoo, got a bit ridiculous with the CFLAGS.  Duncan's post
was very educational and has made me reconsider trying additional
optimizations again.  I am definitely interested in hearing recommendations
for proceeding even though like Mark I recently completed a rebuild of my
system recently. I am looking forward to the continuation of this thread.

Regards,

Greg

[-- Attachment #2: Type: text/html, Size: 2322 bytes --]

[gentoo-amd64] gcc4 CFLAGS Was: gcc 4.1 upgrade - bad desktop interactivity anyone?

[Duncan]

Richard Freeman <rich@thefreemanclan.net> posted
4509E998.7020503@thefreemanclan.net, excerpted below, on  Thu, 14 Sep 2006
19:45:28 -0400:

> Duncan wrote: [snip]
> 
> Hmm - no -ftree-vectorize?  Care to comment on that?  I hear that it can
> be buggy with a few packages, but I'm guessing it is worth having in
> there in general.

The gcc manpage is a bit sparse (understatement) on vectorize, but the next
entry has a bit more info.

<quote, reformatted for posting>

  -ftree-vectorize
    Perform loop vectorization on trees.

  -ftree-vect-loop-version
    Perform loop versioning when doing loop vectorization on trees.  When a
    loop appears to be vector-izable except that data alignment or data
    dependence cannot be determined at compile time then vectorized and
    non-vectorized versions of the loop are generated along with runtime
    checks for alignment or dependence to control which version is
    executed.  This option is enabled by default except at level -Os where
    it is disabled.

</quote>

I'm unclear as to what "vectorization" means as used here.  My
understanding of "vector" is as a synonym for "line", thus implying loop
unrolling of some form or another, which will increase size.  As I
explained in the grandparent, I believe such optimizations to be
counterproductive on modern processors due to the extreme cost of cache
misses as opposed to slight cycle inefficiencies.

I am however aware that vectorization has a somewhat different meaning in
programming terms than the above, but am not sufficiently educated on the
topic to make an informed choice, so I've simply left gcc to go with its
default choice given my overall stated intention of -Os.

If you can sufficiently explain the concept to me such that I
understand enough about it to feel comfortable going with other than the
default (which means I can explain why I chose it and why it won't
interfere with my overall strategy as outlined in the grandparent, or is
worth it even if it does), I'd be very grateful! =8^)

BTW, I'm also looking for a good reference on LDFLAGS.  I'm using one ATM
(LDFLAGS="-Wl,-z,now", which I naturally can explain if asked but will
skip for the moment), but have seen mention of a couple others that look
interesting, but haven't come across anything detailed enough on them to
justify further divergence from the default at this time.  man gcc just
doesn't do it, in this case. =8^(

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] gcc4 CFLAGS Was: gcc 4.1 upgrade - bad desktop interactivity anyone?

[felix]

On Fri, Sep 15, 2006 at 04:47:14PM +0000, Duncan wrote:

> I'm unclear as to what "vectorization" means as used here.  My
> understanding of "vector" is as a synonym for "line", thus implying loop
> unrolling of some form or another, which will increase size.  As I
> explained in the grandparent, I believe such optimizations to be
> counterproductive on modern processors due to the extreme cost of cache
> misses as opposed to slight cycle inefficiencies.
> 
> I am however aware that vectorization has a somewhat different meaning in
> programming terms than the above, but am not sufficiently educated on the
> topic to make an informed choice, so I've simply left gcc to go with its
> default choice given my overall stated intention of -Os.
> 
> If you can sufficiently explain the concept to me such that I
> understand enough about it to feel comfortable going with other than the
> default (which means I can explain why I chose it and why it won't
> interfere with my overall strategy as outlined in the grandparent, or is
> worth it even if it does), I'd be very grateful! =8^)

Back in the day, vectorization was, I believe, a supercomputer SIMD
(single instruction multiple data) concept, where instruction operands
were pointers to data, so it would, for instance, add two arrays of
numbers to produce a third array.  Isn't this what the Altivec
instructions do?

I do not claim to know this is what gcc means, or even if the current
concept of vectorization has any relation to Yee Olde Concepte.

-- 
            ... _._. ._ ._. . _._. ._. ___ .__ ._. . .__. ._ .. ._.
     Felix Finch: scarecrow repairman & rocket surgeon / felix@crowfix.com
  GPG = E987 4493 C860 246C 3B1E  6477 7838 76E9 182E 8151 ITAR license #4933
I've found a solution to Fermat's Last Theorem but I see I've run out of room o
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] gcc4 CFLAGS Was: gcc 4.1 upgrade - bad desktop interactivity anyone?

[Olivier Crete]

On Fri, 2006-15-09 at 10:08 -0700, felix@crowfix.com wrote:
> On Fri, Sep 15, 2006 at 04:47:14PM +0000, Duncan wrote:
> 
> > I'm unclear as to what "vectorization" means as used here.  My
> > understanding of "vector" is as a synonym for "line", thus implying loop
> > unrolling of some form or another, which will increase size.  As I
> > explained in the grandparent, I believe such optimizations to be
> > counterproductive on modern processors due to the extreme cost of cache
> > misses as opposed to slight cycle inefficiencies.
> > 
> > I am however aware that vectorization has a somewhat different meaning in
> > programming terms than the above, but am not sufficiently educated on the
> > topic to make an informed choice, so I've simply left gcc to go with its
> > default choice given my overall stated intention of -Os.
> > 
> > If you can sufficiently explain the concept to me such that I
> > understand enough about it to feel comfortable going with other than the
> > default (which means I can explain why I chose it and why it won't
> > interfere with my overall strategy as outlined in the grandparent, or is
> > worth it even if it does), I'd be very grateful! =8^)
> 
> Back in the day, vectorization was, I believe, a supercomputer SIMD
> (single instruction multiple data) concept, where instruction operands
> were pointers to data, so it would, for instance, add two arrays of
> numbers to produce a third array.  Isn't this what the Altivec
> instructions do?

That's exactly what it means. On x86/amd64 some MMX, SSE/SSE2 and 3dnow
operations are SIMD operations. Vectorizing a loop means that if you try
to add two tables of lets say 12000 elements, instead of doing the loops
12000 times for 1 element each time, it will do the loop lets say 3000
times with 4 element each time. Which should be faster... (but isn't
always depending if the vector ops have been implemented properly).

-- 
Olivier Crête
tester@gentoo.org
Gentoo Developer


-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: gcc 4.1 upgrade - bad desktop interactivity anyone?

[Mark Knecht]

On 9/14/06, Mark Knecht <markknecht@gmail.com> wrote:
> Hi,
>    I'm just curious whether anyone besides me is noticing their
> machine feeling somewhat sluggish since doing the gcc-4.1 upgrade?

I noticed this morning that MythTV's frontend program is often using
>90% CPU when viewed in top.

It never used more than 10% before the upgrade to gcc-4.

Clearly this is at least part of the problem here.

I'm interested in Duncan's flags and how to convert the machine
successfully. Is it a complete rebuild?

- Mark
-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: gcc 4.1 upgrade - bad desktop interactivity anyone?

[Duncan]

"Mark Knecht" <markknecht@gmail.com> posted
5bdc1c8b0609141743y7c5f6042ncd0a596eb290c8ba@mail.gmail.com, excerpted
below, on  Thu, 14 Sep 2006 17:43:19 -0700:

>    Now, you are very adept at this. You're explanations make sense to
> the level I've considered them. (Not very far right now...) Main
> questions:

Adept, perhaps, but don't take my observations as being from God or
anything! =8^)  I try to be fairly cautious with my CFLAGS, but if
anything quits working, I know how to undo them and try with a more
generic set, and in fact do so from time to time on individual packages,
before filing bugs on them.  Sometimes it's my CFLAGS, tho usually my
config doesn't matter a whit to the bug, as I've been reasonably cautious
in my choices to begin with and don't tend to enable stuff like the unsafe
floating-point math options that give folks problems from time to time.

In particular, as you can see from the -ftree-vectorize subthread, I tend
to stay with the defaults when I can't explain with some degree of
confidence exactly what the effect of a flag might be and why I might or
might not want it.  I don't know enough about that area to do that, so
I've stayed well away from it in my CFLAGS.

> 1) What can be done to test this out at my end without making a 2-day
> commitment to rebuild the complete machine. Is it possibly to rebuild
> only portions of the machine using a different set of flags or is it a
> system wide commitment requiring that I rebuild 575 packages as I did
> last weekend?

In general, you /can/ rebuild only a part of your system and test that,
before making further changes.  However, it's important to use a bit of
(un?)common sense when doing so, or your results won't be worth much. 
Basically, in ordered to see how an optimization affects something, you
must have some awareness of the shared libraries it uses and to what
extent it uses them, recompiling enough of the heavily used dependencies
that the critical parts of your test applications (including the libraries
they load) are using the new optimizations.

One lib that all applications make some use of is glibc, so it can be worth
recompiling.  It's a big recompile on its own, but of course nowhere near
as big as recompiling the entire system. =8^)  However, glibc is a special
case in some aspects for a number of reasons. The glibc ebuild is pretty
conservative with the flags it allows, and actually replaces -Os with -O2,
due to problems -Os had mainly on x86, back in the gcc-3.2 and 3.3 era. 
Since the system is pretty horribly broken if glibc breaks, to the point
you are likely to have to boot to a backup or liveCD to fix it, this isn't
an unreasonable policy at all.

None-the-less, after making doubly sure I had tested-working backups, I
decided to see just what the effect of taking out that -Os -> -O2 replace
in the glibc ebuild might be.  For awhile I actually ran a glibc I had
built after having removed that replace.  The system continued to work
just fine with a -Os compiled glibc, it didn't break or anything, but it
didn't seem to be much better either and in some cases seemed worse.  It
turns out that glibc is built in a much more modular fashion than many
libraries, so an app will only load the parts of it it needs, not the
parts it doesn't, and that -Os doesn't work so well with this rather
extreme (compared to most libs) modularization.  As well, as I said, glibc
is used by everything on the system, which meant that having bypassed one
of the safeties in the glibc ebuild, I could never be sure whether a bug I
was experiencing was due to my strange glibc, or to some problem with the
package the bug was showing up in or one of its other dependencies.  I
concluded that it simply wasn't worth bypassing the safeties in the
ebuild, and since then, have left them there.

Thus, with glibc anyway, simply switching to -Os in your CFLAGS won't make
any difference, since the ebuild replaces that with -O2 anyway.  The
/other/ CFLAGS might make a difference, but -Os it self won't, unless you
bypass the replace in the ebuild, and as my experimentation demonstrated
well enough for me, that's really not worth the trouble.  As I said, the
other CFLAGS may make a bit of difference tho, so you might consider it
anyway, if you decide to try them.

For X users, another library that's going to be commonly used is libX11. 
You'll probably want to recompile xorg-server (assuming modular-X) as
well, plus whatever xf86-video-* driver you use, and libXcomposite if you
use the composite extension (transparent windows and the like).  Together,
those will be pretty critical for performance of any X app.  For OpenGL
accelerated apps, mesa is likely to be critical to performance as well,
for any functions not handled by hardware.

For anything written in C++, almost anything KDE among other packages, gcc
libstdc++, a part of gcc, will be critical.  Other than for C++
apps/libraries, recompiling gcc with new CFLAGS shouldn't make that much
difference in how the app runs, tho it might make some difference in how
fast compiles the app.  (Of course, note that compile speed can be
dramatically affected by the optimizations being compiled, since many of
them cause additional passes in the process, to catch that little bit of
extra optimization you are telling gcc to enable.  Tell gcc to do more
work and of course it'll take longer doing it!)

You mentioned gnome.  For anything gnome related, you'll want to recompile
glib, gtk, libgnome, possibly orbit, and maybe others (your window
manager). I don't know enough about gnome internals to go further with it.
With KDE, it'd be qt and kdelibs (plus as I mentioned gcc, for libstdc++),
kwin, probably kicker and konqueror, etc, plus the packages behind various
kparts as used in whatever app you are testing, as well as the app itself.

> 2) What about building the kernel? How do the standard
> 
> make && make modules_install
> 
> command make any use of the flags in /etc/make.conf?

They normally don't.  However, as is commonly the case with manually
compiled packages, the kernel build process makes use of CFLAGS if it
finds that environment variable set.  Thus, you can source make.conf,
export the desired variables, and go from there.  In fact, I have a small
utility script installed as /usr/local/bin/buildflags.sh that contains the
following:

# Source this previous to doing a manual build to import portage buildflags

[ -z "$CFLAGS" ] && CFLAGS="`. /etc/make.conf 2>/dev/null; echo $CFLAGS`"
[ -z "$CXXFLAGS" ] && CXXFLAGS="`. /etc/make.conf 2>/dev/null; echo $CXXFLAGS`"
[ -z "$LDFLAGS" ] && LDFLAGS="`. /etc/make.conf 2>/dev/null; echo $LDFLAGS`"
export CFLAGS CXXFLAGS LDFLAGS

As you can see, that gathers and exports the three build variables CFLAGS,
CXXFLAGS, and LDFLAGS.  If I source that into my running shell before I
build anything (including the kernel) manually, the build should see my
existing environment and compile accordingly.

As it happens, I use a script to compile and install the kernel, as well,
and it sources buildflags.sh before it does the kernel compile.

/usr/local/sbin/ki (for kernel install:

#!/bin/bash
# kernel compile and install
echo
echo sourcing buildflags
ki_buildflagfile=/usr/local/bin/buildflags.sh
if [ -f $ki_buildflagfile ] ; then
    . $ki_buildflagfile
else
    pause 10 x \\a$ki_buildflagfile doesn\'t exist
fi
unset buildflagfile

echo
echo cd-ing to /usr/src/linux
cd /usr/src/linux
echo Next: make
sleep 1
echo
make || exit 1
echo
echo Next: make modules_install
sleep 1
echo
make modules_install || exit 2
echo
echo Next: mount /boot
sleep 1
echo
mount /boot &>/dev/null
echo
echo Next: make install
sleep 1
echo
make install || exit 3
echo
echo Next: umount /boot
sleep 1
echo
umount /boot &>/dev/null
echo Done.
echo

In addition to using buildflags.sh, that script also uses a utility script
I've written called pause, installed as /usr/local/bin/pause (and also as
/bin/pause here, so I can use it to debug initscripts before /usr/local is
mounted).

#!/bin/bash

# putting everything in a function to allow local scoped vars
function doit {
echo
local PROMPT="Pause for char"
case $# in
        0)      read -n1 -p "$PROMPT:";;
        1)      if [ $1 == "--help" -o $1 == "-h" -o $1 == "-?" ]
                then
                        echo $PROMPT script utility.
                        echo
                        echo Usage: pause [timeoutsec [defaultchar [promptstring]]]
                        echo \ \ \ \ \ \ \ pause \(--help\|-h\|-?\)
                        echo
                        echo Source pause to have \$REPLY set to the returned char.
                        echo
                        echo A timeoutsec of 0 indicates no timeout.  Otherwise,
                        echo on timeout, defaultchar if provided is returned in \$REPLY.
                        echo
                        echo With timeout and defaultchar, promptstring defaults to:
                        echo $PROMPT \(default defaultchar in timeoutsec\):
                        echo With just a timeout, prompt default is:
                        echo $PROMPT \(timeout timeoutsec\):
                        echo Without a timeout, the default is:
                        echo $PROMPT:
                        echo
                        echo
                elif [ $1 == 0 ]
                then read -n1 -p "$PROMPT:"
                else read -n1 -t$1 -p "$PROMPT (timeout $1):"; echo
                fi;;
        2)      if [ $1 == 0 ]
                then read -n1 -p "$PROMPT:"
                else
                        read -n1 -t$1 -p "$PROMPT (default $2 in $1):"; echo
                        REPLY=${REPLY:-$2}
                fi;;
        *)      if [ $1 == 0 ]
                then
                        shift 2
                        echo -e $*
                        read -n1
                else
                        local TIMEOUT=$1 DEFAULTCHAR=$2; shift 2
                        echo -e $*
                        read -n1 -t$TIMEOUT; echo
                        REPLY=${REPLY:-$DEFAULTCHAR}
                fi;;
esac
echo
}

doit $*

Obviously, I've taken a bit more time to properly document pause, so you
can run pause -h or --help to get the usual usage summary.  The other two
scripts aren't so nicely --help equipped; you have to look at the source
to see what they are doing.  I've found pause to be very useful over the
years. =8^)

As you can see, I routinely compile the kernel with my chosen CFLAGS, and
it works fine here.  Of course, your kernel config will be different and I
can't vouch for every single driver, but the concept is sane and my CFLAGS
are demonstrated to work with the core kernel and the drivers I use,
anyway.

As Greg mentions, there's also a compile for size option in the kernel
config, and I have it enabled as well.  It enables -Os, but I think it
does a few other kernel specific things as well.  I've never looked into
it in detail, but I know I've had it enabled every since its introduction
some kernels ago, with the accompanying coverage in LWN.  That coverage in
fact was what got me started on the whole -Os thing, before I'd even
switched to Gentoo with 2004.1 and was still on Mandrake IIRC, so it has
been a kernel option and I've been using it for over two years now.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: gcc4 CFLAGS Was: gcc 4.1 upgrade - bad desktop interactivity anyone?

[Duncan]

Olivier Crete <tester@gentoo.org> posted
1158340730.14248.61.camel@cocagne.max-t.internal, excerpted below, on Fri,
15 Sep 2006 13:18:49 -0400:

> On Fri, 2006-15-09 at 10:08 -0700, felix@crowfix.com wrote:
>> On Fri, Sep 15, 2006 at 04:47:14PM +0000, Duncan wrote:
>> 
>> > I'm unclear as to what "vectorization" means as used here.  My
>> > understanding of "vector" is as a synonym for "line", thus implying
>> > loop unrolling of some form or another, which will increase size.
>> > 
>> > I am however aware that vectorization has a somewhat different
>> > meaning in programming terms than the above, but am not sufficiently
>> > educated on the topic to make an informed choice[.]
>> > 
>> > If you can sufficiently explain the concept to me such that I
>> > understand enough about it to feel comfortable going with other than
>> > the default (which means I can explain why I chose it and why it
>> > won't interfere with my overall strategy as outlined in the
>> > grandparent, or is worth it even if it does), I'd be very grateful!
>> 
>> Back in the day, vectorization was, I believe, a supercomputer SIMD
>> (single instruction multiple data) concept, where instruction operands
>> were pointers to data, so it would, for instance, add two arrays of
>> numbers to produce a third array.  Isn't this what the Altivec
>> instructions do?
> 
> That's exactly what it means. On x86/amd64 some MMX, SSE/SSE2 and 3dnow
> operations are SIMD operations. Vectorizing a loop means that if you try
> to add two tables of lets say 12000 elements, instead of doing the loops
> 12000 times for 1 element each time, it will do the loop lets say 3000
> times with 4 element each time. Which should be faster... (but isn't
> always depending if the vector ops have been implemented properly).

I was somewhat aware of that, but hadn't considered the effect on loops,
and don't understand it enough to be able explain it as you did, nor enough
to grok why if it's so much more efficient, gcc doesn't do it by default
at least on archs sufficiently specified to know the instructions are
there and that it makes sense. (amd64 being new enough not to have all the
different generations of mmx/sse/sse2/etc/etc it should make sense, as it
would on x86 with -march=pentium4 or whatever, so it knows what
vectorization levels are available as opposed to plain pentium, which was
pre-mmx, let alone the later vectorization functions.)

IOW, that explains why it should be more efficient, but not why gcc isn't
already doing it on amd64, or maybe it is, and specifying the flag would
be redundant?  This is precisely what I mean when I say I don't have
enough information to make a defensible decision, so I've chosen to stick
with the safe defaults.  If it's not being done by default, there's likely
a good reason somewhere, and lacking enough information to make an
informed decision, the defaults are the safe way to go.

This is also one of those places where the manpage is frustratingly
uninformative.  The explanation on -ftree-vect-loop-version explains that
it's enabled by default, that both vectorized and unvectorized versions of
loops are created where compile-time can't tell for sure that vectorizing
is possible, /except/ for -Os.  Since this flag forces double-code in some
cases, disabling it for -Os makes perfect sense so no problem there.  The
problem is that this implies that where it /can/ tell vectorization is
possible, it should be doing that by default as well -- only it never
/says/ it does it by default, neither under the regular -ftree-vectorize
description, nor under the lists of what gets enabled by default at the
various -OX levels.  The documentation therefore leaves the answer to the
question of whether it's enabled by default very much up in the air,
implying it is in the description of something else, but nowhere stating
explicitly one way or the other.

Another example of unclearly specifying the default is -ftree-pre.  It's
certainly the default for -O2 and -O3, and the section on -Os doesn't say
it's disabled there while saying all the -O2 except where that would
increase the size, and there's no direct indication this increases size,
but the description for -ftree-pre specifies -O2 and -O3 specifically
only, so one is left wondering what side of -O2 except where that would
increase size it falls on, and why.  As you can see, I've chosen to
include it in my CFLAGS because it seems like it should be of benefit
(compare -ftree-fre, enabled at -O and higher including -Os) and shouldn't
increase size /too/ much, just in case it's /not/ default for -Os for some
reason.  

With -ftree-pre I can be pretty sure it's safe to include since -O2 is
known to include it, but -ftree-vectorize is different as there's nothing
saying /where/ it's the default (if anywhere), tho as I explained it's
implied as the default by the description for the -ftree-vect-loop-version
entry.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: gcc 4.1 upgrade - bad desktop interactivity anyone?

[Duncan]

"Mark Knecht" <markknecht@gmail.com> posted
5bdc1c8b0609151106u52bcf8f6s3c98497cf41bc2e8@mail.gmail.com, excerpted
below, on  Fri, 15 Sep 2006 11:06:47 -0700:

> On 9/14/06, Mark Knecht <markknecht@gmail.com> wrote:
>> Hi,
>>    I'm just curious whether anyone besides me is noticing their
>> machine feeling somewhat sluggish since doing the gcc-4.1 upgrade?
> 
> I noticed this morning that MythTV's frontend program is often using
> >90% CPU when viewed in top.
> 
> It never used more than 10% before the upgrade to gcc-4.
> 
> Clearly this is at least part of the problem here.

Indeed, that would explain your observations.  Perhaps either the
front-end or some library it loads is one of the few programs that just
doesn't work quite right with gcc-4.1 yet.  Good detective work!

So it would appear you have to try recompiling it with gcc-3.x again, and
see if that eliminates the problem.  If not, you'll have to check its
dependency tree and try recompiling it.  Get that 90% off the CPU and
maybe you'll see the better general efficiency of gcc-4.1, regardless of
whether you try my cflags or not.  In fact, that's what I'd recommend you
do, before trying my cflags.  You'd then have a better base on which to
measure whether my cflags made a difference for you or not, as opposed to
what gcc-4.1.x itself did.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: gcc4 CFLAGS Was: gcc 4.1 upgrade - bad desktop interactivity anyone?

[Richard Freeman]

[-- Attachment #1: Type: text/plain, Size: 1842 bytes --]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Duncan wrote:
> 
> I was somewhat aware of that, but hadn't considered the effect on loops,
> and don't understand it enough to be able explain it as you did, nor enough
> to grok why if it's so much more efficient, gcc doesn't do it by default
> at least on archs sufficiently specified to know the instructions are
> there and that it makes sense. 

- From my reading tree-vectorize attempts to use SIMD wherever possible
for parallel computation of arrays/etc.  In theory that should almost
always be a net-benefit with few drawbacks.

The problem I understand is that it is sometimes a bit buggy - ie it
sometimes creates broken code.  I think these issues have mostly been
fixed, but that would explain why it is not applied by default.

There is also an -ftree-vectorize-verbose=# parameter which generates
informational messages about why particular loops were or were not
vectorized.  In theory this can help you develop more-easily-optimized code.

MMX/etc can tremendously improve program speed.  Anytime you can do 4
operations per cycle vs 1 you're going to improve throughput.

Otherwise, I agree with you as far as reducing memory footprint goes -
I've been running -Os for ages and I'm very happy with this.  My RAM is
better applied to disk caching than storing unrolled loops in almost all
cases.  I'm sure in niche cases the opposite is true, but the same
applies to -ffast-math and other dangerous optimizations.  They should
probably be applied on a per-file basis by the developer, and not across
an entire build/system.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFFCzL2G4/rWKZmVWkRAkzOAKC1jKik3Q+JdWvpH3qkmMfvWZ823gCeP5km
odH1v8qKb4xrDL5YPLeC62o=
=NnSA
-----END PGP SIGNATURE-----

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/x-pkcs7-signature, Size: 3875 bytes --]

[gentoo-amd64] vectorization (was: gcc4 CFLAGS)

[John S. Yates, Jr.]

On Fri, 15 Sep 2006 16:47:14 +0000 (UTC), you wrote:

> I am however aware that vectorization has a somewhat different meaning in
> programming terms than the above, but am not sufficiently educated on the
> topic to make an informed choice, so I've simply left gcc to go with its
> default choice given my overall stated intention of -Os.

Older super-computers, especially those designed or inspired by Seymour Cray
Cray, included "vector registers".  These were multiple registers (typically
a small power of 2, say 32, 64, or 128) that could be manipulated as a unit.
This was an earlier form of SIMD.  By issuing a single instruction -- such as
a vector load or vector add -- you repeated the same operation on a sequence
of operands.  The crucial difference was that these vector operations had some
start-up overhead and then ran autonomously delivering one result every clock
tick for the length of the vector register.

While some compilers added proprietary language extension to support vector
values as actual data type, most numeric code was written in scalar form.
To make such super-computers useful it was crucial that compilers be able to
recognize when a scalar loop could be implemented using the machine's vector
facilities.  Fundamentally this came down to figuring out when successive
loop iterations were independent and hence could execute in parallel.  Since
the compiler was attempting to re-express scalar loops as loops using vector
primitives the optimization became know as "vectorization".

In essence a multi-media SIMD mechanism is very similar.  A 64 bit register
containing 4 16-bit operands is essentially a length 4 "vector register".
Finding opportunities to use such SIMD instructions in scalar code requires
exactly the same forms of analysis and optimization.

/john


-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] vectorization (was: gcc4 CFLAGS)

[Peter Humphrey]

On Tuesday 19 September 2006 01:24, John S. Yates, Jr. wrote:

> Older super-computers, especially those designed or inspired by Seymour
> Cray Cray, included "vector registers".

[...]

What a fine description. Thank you.

-- 
Rgds
Peter
-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: vectorization (was: gcc4 CFLAGS)

[Duncan]

Peter Humphrey <prh@gotadsl.co.uk> posted
200609191234.24711.prh@gotadsl.co.uk, excerpted below, on  Tue, 19 Sep
2006 12:34:24 +0000:

> On Tuesday 19 September 2006 01:24, John S. Yates, Jr. wrote:
> 
>> Older super-computers, especially those designed or inspired by Seymour
>> Cray Cray, included "vector registers".
> 
> [...]
> 
> What a fine description. Thank you.

Jumping back in here since this happens to be the only message on the
thread I hadn't read yet, so showing.

I decided to try -ftree-vectorize and see how it goes.  Haven't merged
much of anything with it yet, but...

Meanwhile, I stumbled across my first failure with -combine.  The latest
(~amd64) logrotate ebuild (logrotate-3.7.2) fails with it, in the compile
phase of course, with an error to the effect of too many files passed to
cc1 or some such.  I don't know enough about gcc to know what that means
in terms of gcc module, but I do know removing -combine from CFLAGS allowed
it to merge just fine.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: vectorization (was: gcc4 CFLAGS)

[Peter Humphrey]

On Tuesday 19 September 2006 14:13, Duncan wrote:

> Meanwhile, I stumbled across my first failure with -combine.  The latest
> (~amd64) logrotate ebuild (logrotate-3.7.2) fails with it, in the compile
> phase of course, with an error to the effect of too many files passed to
> cc1 or some such.  I don't know enough about gcc to know what that means
> in terms of gcc module, but I do know removing -combine from CFLAGS
> allowed it to merge just fine.

Well, on this 
box, /var/log/portage/app-admin:logrotate-3.7.2:20060919-130412.log has the 
following first three lines:

 * Filtering out invalid CFLAG "-combine"
 * Filtering out invalid CFLAG "-ftree-pre"
 * Filtering out invalid CXXFLAG "-ftree-pre"

So it seems the system is working just nicely here.

-- 
Rgds
Peter
-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: vectorization (was: gcc4 CFLAGS)

[Duncan]

Peter Humphrey <prh@gotadsl.co.uk> posted
200609201126.35314.prh@gotadsl.co.uk, excerpted below, on  Wed, 20 Sep
2006 11:26:35 +0000:

> Well, on this 
> box, /var/log/portage/app-admin:logrotate-3.7.2:20060919-130412.log has the 
> following first three lines:
> 
>  * Filtering out invalid CFLAG "-combine"
>  * Filtering out invalid CFLAG "-ftree-pre"
>  * Filtering out invalid CXXFLAG "-ftree-pre"
> 
> So it seems the system is working just nicely here.

Hmm...  That's the amd64 profile bashrc, I think.  It filters flags that
aren't valid for the gcc version you are pointing at at the time.  Those
flags are all new to gcc4 IIRC.  You sure your eselect compiler or
gcc-config isn't still pointing at gcc3?

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: vectorization (was: gcc4 CFLAGS)

[Peter Humphrey]

On Wednesday 20 September 2006 13:42, Duncan wrote:

> That's the amd64 profile bashrc, I think.  It filters flags that
> aren't valid for the gcc version you are pointing at at the time.  Those
> flags are all new to gcc4 IIRC.  You sure your eselect compiler or
> gcc-config isn't still pointing at gcc3?

You're right, it is. I'd noticed 4.1.1 had been installed, but I haven't yet 
switched to it. I'll go and look at the upgrade guide.

Actually, that may have to wait, as important things are happening around 
here over the next few weeks.

-- 
Rgds
Peter
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: gcc 4.1 upgrade - bad desktop interactivity anyone?

[Peter Humphrey]

On Thursday 14 September 2006 20:08, Duncan wrote:

> Here's my CFLAGS/CXXFLAGS:

...etc.

Which model of Opteron are your CPUs? I have a feeling they differ from my 
246s, and I've been wondering how I ought to tune your helpfully explained 
flags to suit my box.

-- 
Rgds
Peter
-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: gcc 4.1 upgrade - bad desktop interactivity anyone?

[Duncan]

Peter Humphrey <prh@gotadsl.co.uk> posted
200609231439.11739.prh@gotadsl.co.uk, excerpted below, on  Sat, 23 Sep
2006 14:39:11 +0000:

> Which model of Opteron are your CPUs? I have a feeling they differ from my
> 246s, and I've been wondering how I ought to tune your helpfully explained
> flags to suit my box.

I'm running 242s at present, so they should be fairly similar.

I plan on upgrading to dual-cores later this year or early next, when the
prices seem to be down to a reasonable level as the new socket format
takes over, and will run that for another couple years before I even think
of upgrading mobo/cpu/memory again, at which point I'll have been running
the same base mobo and platform for over five years(!!), and expect to
upgrade to a single socket 8-core model as mid-grade. (Of course by then
AMD's multi-socket co-processor model or a variation thereof may have
taken the market by storm, and I might as a result be buying a two or more
socket mobo with one for CPU and the other for GPU, or some such.)

However, the only difference (CFLAGS wise) that I'm aware of for the AMD
dual-cores is that they now incorporate SSE3, while my old 242s and I
presume your 246s don't.  The other changes I'll be making at the upgrade
will be in terms of kernel config.  Naturally, with dual Opterons, I'm
already running SMP, but I have it set for two max, and with the
dual-cores, that will of course change to four.  Additionally, there's
only one level of CPU/core zoning ATM, while there will be two levels
then, as the pair of cores on the same CPU will cooperate even closer than
the two in separate sockets but connected by hypertransport bus do.

The big difference in CFLAGS at this point is between Intel and AMD
products, and since we are both running AMD, that's not an issue.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: gcc 4.1 upgrade - bad desktop interactivity anyone?

[Peter Humphrey]

On Saturday 23 September 2006 19:52, Duncan wrote:

> However, the only difference (CFLAGS wise) that I'm aware of for the AMD
> dual-cores is that they now incorporate SSE3, while my old 242s and I
> presume your 246s don't.

Nope. SSE and SSE2, but not SSE3. According to /proc/cpuinfo, that is.

-- 
Rgds
Peter
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: vectorization (was: gcc4 CFLAGS)

[Peter Humphrey]

On Tuesday 19 September 2006 14:13, Duncan wrote:

> I stumbled across my first failure with -combine.  The latest
> (~amd64) logrotate ebuild (logrotate-3.7.2) fails with it, in the compile
> phase of course, with an error to the effect of too many files passed to
> cc1 or some such.  I don't know enough about gcc to know what that means
> in terms of gcc module, but I do know removing -combine from CFLAGS
> allowed it to merge just fine.

And I've found that module-init-tools won't compile with -combine either. It 
complains about too many modules being passed. I did the same as you and 
removed -combine while merging module-init-tools.

I did have another package fail to compile during the first emerge -e 
system, but I don't remember the details. On repeating emerge -e world the 
problem didn't recur (after emerge --resume --skipfirst to allow the first 
invocation to finish), so I stopped thinking about it.

-- 
Rgds
Peter
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: gcc 4.1 upgrade - bad desktop interactivity anyone?

[Jason Booth]

On Saturday 23 September 2006 16:21, Peter Humphrey wrote:
> Nope. SSE and SSE2, but not SSE3. According to /proc/cpuinfo, that is.

The flag in cpuinfo is pni for "Prescott New Instructions".

Cheers,

Jason
-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: vectorization (was: gcc4 CFLAGS)

[Duncan]

Peter Humphrey <prh@gotadsl.co.uk> posted
200609232230.36532.prh@gotadsl.co.uk, excerpted below, on  Sat, 23 Sep
2006 22:30:36 +0000:

> And I've found that module-init-tools won't compile with -combine either.
> It complains about too many modules being passed. I did the same as you
> and removed -combine while merging module-init-tools.

Yeah.  I've not done a full emerge --emptytree world with -combine in my
CFLAGS, so while I know it doesn't kill a majority of packages, it
apparently does kill a few.  Still, for the additional optimization it
enables, I consider it worth it on the others.

> I did have another package fail to compile during the first emerge -e
> system, but I don't remember the details. On repeating emerge -e world the
> problem didn't recur (after emerge --resume --skipfirst to allow the first
> invocation to finish), so I stopped thinking about it.

The latest ~arch portage fixed a few circular dependency and other
dependency resolution issues.  It wasn't uncommon to have a few packages
fail on emerge --emptytree, the first go round, due to these issues.  The
fixes now in portage are said to fix the most common such issues, but may
not fix them all.  What that all means here is that a few instances of
having to invoke --skipfirst isn't unexpected, and likely weren't due to
anything in the new CFLAGS or whatever.  Hopefully, as a portage with the
new changes eventually moves into stable, there will be fewer of those
--skipfirsts necessary.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: gcc 4.1 upgrade - bad desktop interactivity anyone?

[Boyd Stephen Smith Jr.]

[-- Attachment #1: Type: text/plain, Size: 1326 bytes --]

On Saturday 23 September 2006 17:21, Peter Humphrey <prh@gotadsl.co.uk> 
wrote about 'Re: [gentoo-amd64]  Re: gcc 4.1 upgrade - bad desktop 
interactivity anyone?':
> On Saturday 23 September 2006 19:52, Duncan wrote:
> > However, the only difference (CFLAGS wise) that I'm aware of for the
> > AMD dual-cores is that they now incorporate SSE3, while my old 242s
> > and I presume your 246s don't.
>
> Nope. SSE and SSE2, but not SSE3. According to /proc/cpuinfo, that is.

I can verify that the 275s do support SSE3 (flag: pni) from 
my /proc/cpuinfo:
vendor_id       : AuthenticAMD
cpu family      : 15
model           : 33
model name      : Dual Core AMD Opteron(tm) Processor 275
stepping        : 2
cpu MHz         : 2200.000
fpu             : yes
fpu_exception   : yes
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca 
cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 
3dnowext 3dnow pni lahf_lm cmp_legacy
address sizes   : 40 bits physical, 48 bits virtual
power management: ts fid vid ttp


-- 
"If there's one thing we've established over the years,
it's that the vast majority of our users don't have the slightest
clue what's best for them in terms of package stability."
-- Gentoo Developer Ciaran McCreesh

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-amd64] Re: vectorization (was: gcc4 CFLAGS)

[Peter Humphrey]

On Saturday 23 September 2006 22:30, I wrote:

> I did have another package fail to compile during the first emerge -e
> system, but I don't remember the details. On repeating emerge -e world

I meant emerge -e system, of course.

> the problem didn't recur (after emerge --resume --skipfirst to allow the
> first invocation to finish), so I stopped thinking about it.

-- 
Rgds
Peter
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: vectorization (was: gcc4 CFLAGS)

[Peter Humphrey]

On Saturday 23 September 2006 23:50, Duncan wrote:

> I've not done a full emerge --emptytree world with -combine in my
> CFLAGS, so while I know it doesn't kill a majority of packages, it
> apparently does kill a few.  Still, for the additional optimization it
> enables, I consider it worth it on the others.

Is there a way to specify CFLAGS on the command line in such a way as to 
negate (unset) those that are set in /etc/make.conf? I have -combine 
and -ftree-pre in make.conf, and while emerge -e world is running it fails 
some packages by not recognising those two. When that happens I want to 
emerge --resume --skipfirst so as not to lose my place in the world emerge, 
and in another terminal emerge the ones that have failed.

I can think of an expensive way to obtain the same effect, namely to remove 
the two offending flags from make.conf, run emerge -e world, replace the 
flags and emerge -e world again, using --skipfirst every time that one 
stops.

Or I could write a script that keeps resuming a failed emerge until it 
finishes. Not sure how I'd go about that: how would the script distinguish 
the end of the emerge from in interruption of it?

The last option would be to create a bug for each package that stops on 
unrecognised flags, but I'd rather continue to be able to talk to 
people :-(

Apologies if I seem to be thinking on my feet here.

-- 
Rgds
Peter
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: vectorization (was: gcc4 CFLAGS)

[Kevin F. Quinn]

[-- Attachment #1: Type: text/plain, Size: 1110 bytes --]

On Sun, 24 Sep 2006 11:47:21 +0000
Peter Humphrey <prh@gotadsl.co.uk> wrote:

> Is there a way to specify CFLAGS on the command line in such a way as
> to negate (unset) those that are set in /etc/make.conf? I have
> -combine and -ftree-pre in make.conf, and while emerge -e world is
> running it fails some packages by not recognising those two. When
> that happens I want to emerge --resume --skipfirst so as not to lose
> my place in the world emerge, and in another terminal emerge the ones
> that have failed.

CFLAGS=<set of flags you want> emerge --oneshot <package>

ought to do the trick - it should override the make.conf setting.  You
can do the same with FEATURES.


Aside from that, what you're really after is per-package CFLAGS (or
more generally, per-package FEATURES) which is a long-standing portage
feature request.  You can do it via a bashrc script
(in /etc/portage/bashrc) although it's not 100% - bashrc scripts are not
sourced in all phases of emerge (in particular, fetch), although they
are for the ones you're most likely to care about.

-- 
Kevin F. Quinn

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-amd64] Re: vectorization

[Simon Stelling]

Kevin F. Quinn wrote:
> Aside from that, what you're really after is per-package CFLAGS (or
> more generally, per-package FEATURES) which is a long-standing portage
> feature request.  You can do it via a bashrc script
> (in /etc/portage/bashrc) although it's not 100% - bashrc scripts are not
> sourced in all phases of emerge (in particular, fetch), although they
> are for the ones you're most likely to care about.

per package env vars are supported through the base profile.bashrc. You
don't have to clutter /etc/portage/bashrc, you can just do e.g.

cat > /etc/portage/env/app-foo/bar << EOF
CFLAGS="-O -march=k8 -pipe"
CXXFLAGS="${CFLAGS}"
EOF

And it will set C(XX)FLAGS for app-foo/bar based on that.
/etc/portage/env/app-foo/bar-1.0 and /etc/portage/env/app-foo/bar-1.0-r1
would also work.

That being said, "fetch" is not a phase, that's why there are no hooks
for it.

-- 
Kind Regards,

Simon Stelling
Gentoo/AMD64 developer
-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: vectorization (was: gcc4 CFLAGS)

[Duncan]

Peter Humphrey <prh@gotadsl.co.uk> posted
200609241147.22022.prh@gotadsl.co.uk, excerpted below, on  Sun, 24 Sep
2006 11:47:21 +0000:

> Is there a way to specify CFLAGS on the command line in such a way as to
> negate (unset) those that are set in /etc/make.conf? I have -combine and
> -ftree-pre in make.conf, and while emerge -e world is running it fails
> some packages by not recognising those two. When that happens I want to
> emerge --resume --skipfirst so as not to lose my place in the world
> emerge, and in another terminal emerge the ones that have failed.

Kevin answers your direct question.  However, based on the other thread, I
believe your main problem is that you are still using gcc 3.x.  Those
flags I use are for gcc >= 4.1, and *WILL* cause complications on gcc 3.x,
because 3.x simply doesn't recognize them.  As it doesn't recognize them,
it throws warnings, which causes parts of configure to believe a test
failed and that something (like the -fPIC flag that must be added for
libraries on amd64, don't put it in cflags yourself tho) isn't supported,
so they remove it, causing merges to fail.

IOW, don't try to use my CFLAGS on gcc 3.x!  It WILL cause problems! 
Either upgrade (and switch to using gcc-config or eselect compiler) to gcc
4.1.1, or go back to your regular CFLAGS on gcc 3.x and rebuild to ensure
you have a clean gcc 3.x system without any contamination from gcc 4.x
only CFLAGS.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: vectorization

[Duncan]

Simon Stelling <blubb@gentoo.org> posted 45170C6C.7010104@gentoo.org,
excerpted below, on  Mon, 25 Sep 2006 00:53:32 +0200:

> per package env vars are supported through the base profile.bashrc. You
> don't have to clutter /etc/portage/bashrc, you can just do e.g.
> 
> cat > /etc/portage/env/app-foo/bar << EOF
> CFLAGS="-O -march=k8 -pipe"
> CXXFLAGS="${CFLAGS}"
> EOF
> 
> And it will set C(XX)FLAGS for app-foo/bar based on that.
> /etc/portage/env/app-foo/bar-1.0 and /etc/portage/env/app-foo/bar-1.0-r1
> would also work.

Wow!  I knew that had been talked about, but had no idea it had been
actually deployed in the tree. =8^)  A new toy to play with! =8^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: vectorization

[Peter Humphrey]

On Sunday 24 September 2006 22:53, Simon Stelling wrote:

> per package env vars are supported through the base profile.bashrc. You
> don't have to clutter /etc/portage/bashrc, you can just do e.g.
>
> cat > /etc/portage/env/app-foo/bar << EOF
> CFLAGS="-O -march=k8 -pipe"
> CXXFLAGS="${CFLAGS}"
> EOF
>
> And it will set C(XX)FLAGS for app-foo/bar based on that.
> /etc/portage/env/app-foo/bar-1.0 and /etc/portage/env/app-foo/bar-1.0-r1
> would also work.

Impressive! One question: will the flags set here override those in 
portage's environment, or supplement them? In other words, can I switch 
flags off this way, or only on?

-- 
Rgds
Peter
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: vectorization

[Simon Stelling]

Peter Humphrey wrote:
> In other words, can I switch 
> flags off this way, or only on?

It replaces the value from make.conf, so yes, you can disable flags this
way.

-- 
Kind Regards,

Simon Stelling
Gentoo/AMD64 developer
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: vectorization

[Peter Humphrey]

On Monday 25 September 2006 13:02, Simon Stelling wrote:
> Peter Humphrey wrote:
> > In other words, can I switch
> > flags off this way, or only on?
>
> It replaces the value from make.conf, so yes, you can disable flags this
> way.

Good news - thanks Simon.

-- 
Rgds
Peter
-- 
gentoo-amd64@gentoo.org mailing list