[gentoo-amd64] Old topic comes round again: -Ox optimizations.

[gentoo-amd64] Old topic comes round again: -Ox optimizations.

[Duncan]

The topic of the usefulness of various -Ox optimizations comes up every so
often.  This will likely be old news to Gentoo old hands, but it
should be useful for newbies, at least.  Anyway, I found this commentary
by Linus interesting:

http://permalink.gmane.org/gmane.linux.kernel/344440

The thread wanders quite a bit, but this subthread started with someone
displaying a listing of their kernels ordered by size back thru 2.2.x,
noting that the size had increased from ~ half a MB back then to ~ 1.5 MB
today.

Somebody then asked (among other things) if the compiler used was the
same, listing the size of the /same/ kernel compiled with gcc-2.95.x
against the gcc-4.1.0-snapshot they are running.  The 4.1 compiled kernel
was MUCH larger.

Someone replied that the comparison wasn't fair -- if you don't ask gcc to
optimize for size, don't blame it for not doing so -- and proceeded to
compare kernels compiled with 4.1 and normal options, against
those compiled using the kernel embedded option, and then choosing
appropriately the answers to the output of "make oldconfig" after having
changed to "embedded". Among other things, he added -Os to the kernel's
gcc command line, and he probably dropped symbols as well.  What else he
chose from the options he didn't say, but the resulting core kernel
(allnoconfig) ended up /very/ similar in size to the old 2.5 kernel he
compared against (I believe the first one to have allnoconfig as an option).

A few posts down, Linus then posted this comment, from the bottom of the
linked post above:

<quote>

And we should probably make -Os the default. Apparently Fedora 
already does that by just forcibly hacking the Kconfig files.

With modern CPU's, instructions are almost "free". The real cost is in 
cache misses, and that tends to be doubly true of system software that 
tends to have a lot more cache misses than "normal" programs (because 
people try hard to batch up system calls like write etc, so by the time 
the kernel is called, the L1 cache is mostly flushed already - possibly 
the L2 too. And interrupts may be in the "fast path", but they'd sure as 
hell better not happen so often that they stay cached very well etc etc).

So -Os probably performs better in real life, and likely only performs 
worse on micro-benchmarks. Sadly, micro-benchmarks are often very 
instructive in many other ways.

</quote>

(Dave Jones later confirms that Fedora does indeed normally run -Os.)

(If you want to view the entire thread, use the "Go to the topic" link
near the top left of the page.)

So... certainly for kernel and probably for glibc stuff (tho I believe
Gentoo kills -Os on glibc compiles unless you hack out that portion of the
ebuild, in your own overlay or whatever), -Os is likely to be the best
choice.  For most of userland, -Os may be best, but to a smaller degree,
and performance should be similar with -O2, trading size for medium speed
optimizations in what amounts to a wash. The exceptions would likely be
media encoders and the like, where the working set is large and in a data
streaming environment, and -O3 may make sense.  In the general case,
however, -O3 likely does NOT make sense, because it's almost always so
expensive in size that the gains in speed over -O2 are far outweighed.

IMO (and for the kernel anyway, Linus's as well)...

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman in
http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html


-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Old topic comes round again: -Ox optimizations.

[Hemmann, Volker Armin]

On Thursday 03 November 2005 09:47, Duncan wrote:


>
> So... certainly for kernel and probably for glibc stuff (tho I believe
> Gentoo kills -Os on glibc compiles unless you hack out that portion of the
> ebuild, in your own overlay or whatever), -Os is likely to be the best
> choice.  For most of userland, -Os may be best, but to a smaller degree,
> and performance should be similar with -O2, trading size for medium speed
> optimizations in what amounts to a wash. The exceptions would likely be
> media encoders and the like, where the working set is large and in a data
> streaming environment, and -O3 may make sense.  In the general case,
> however, -O3 likely does NOT make sense, because it's almost always so
> expensive in size that the gains in speed over -O2 are far outweighed.
>

AFAIR and so on.

Os is 'forbidden' for glibc, because there were some gcc bugs, that 
miscompiled glibc with Os.

But with gentoo 1.2/1.4, I went for some time with Os in CFLAGS - and my 
system was not faster than with O2. But it was a little faster with Os or O2 
than O3 ;)
-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: Old topic comes round again: -Ox optimizations.

[Duncan]

Hemmann, Volker Armin posted
<200511031222.04016.volker.armin.hemmann@tu-clausthal.de>, excerpted
below,  on Thu, 03 Nov 2005 12:22:03 +0100:

> Os is 'forbidden' for glibc, because there were some gcc bugs, that 
> miscompiled glibc with Os.

I've actually run an -Os compiled glibc here (hacked the ebuild to
remove the filterflags/replaceflags/stripflags calls) for some time, no
noticed issues (tho I haven't tried it with gcc-4.x yet).

However, it's unsupported, and because glibc is linked by virtually
everything, I couldn't be sure whether problems I might have were due to
it or not.  The thing that switched me back was attempting to trace down
why xorg had quite responding to CTRL-ALT-Fx VT-switch commands.  I later
found out it was a known issue with that version on some configs, fixed in
a later version, but glibc is too big a compile (expecially when it's
compiled twice, once for 32-bit and once for 64-bit, up to four times for
those poor folks running nptl without nptl-only, just the two compiles
here, with a dual Opteron, is long, can't imagine four compiles on a
single CPU...) to be worth fighting with all the time, I decided.

Note that for something as universally used as glibc, I'd definitely
recommend having a backup root partition with an untouched copy, if you
start fooling with removing the strip-flags type stuff.

Anyway, I expect the -Os problems for glibc were on x86, not amd64. 
That's apparently been the case in most cases where flags are simply
removed without regard for arch, assuming everything else is as broken as
32-bit x86, when that's not necessarily the case.  Not saying that
problems won't occur on amd64, only that I actually expected issues from
it and didn't see any.

As I said, however, I'm done with my -Os fiddling for glibc, for the time
being, if only because the hacking to get it and constant wondering if
that's the problem any time something strange happens, isn't worth the
trouble, for me.

If folks wish to try it, however, and have a well tested backup boot
method in case glibc fails /entirely/, making the existing install
entirely unbootable, understand the risks, and have the time to do the
compiles, I'd say if you're the type that won't be satisfied until you at
least try it and get it out of your blood, so to speak, you might as well
do so.  It's not as fatal as the dire warnings seem to make it out to be.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman in
http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html


-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: Old topic comes round again: -Ox optimizations.

[Hemmann, Volker Armin]

On Thursday 03 November 2005 16:27, Duncan wrote:
> Hemmann, Volker Armin posted
> <200511031222.04016.volker.armin.hemmann@tu-clausthal.de>, excerpted
>
> below,  on Thu, 03 Nov 2005 12:22:03 +0100:
> > Os is 'forbidden' for glibc, because there were some gcc bugs, that
> > miscompiled glibc with Os.
>
> I've actually run an -Os compiled glibc here (hacked the ebuild to
> remove the filterflags/replaceflags/stripflags calls) for some time, no
> noticed issues (tho I haven't tried it with gcc-4.x yet).
>
> However, it's unsupported, and because glibc is linked by virtually
> everything, I couldn't be sure whether problems I might have were due to
> it or not.  The thing that switched me back was attempting to trace down
> why xorg had quite responding to CTRL-ALT-Fx VT-switch commands.  I later
> found out it was a known issue with that version on some configs, fixed in
> a later version, but glibc is too big a compile (expecially when it's
> compiled twice, once for 32-bit and once for 64-bit, up to four times for
> those poor folks running nptl without nptl-only, just the two compiles
> here, with a dual Opteron, is long, can't imagine four compiles on a
> single CPU...

it is not so bad with a co and without ntplonly
genlop -t glibc
 * sys-libs/glibc

     Sat Sep 24 07:51:46 2005 >>> sys-libs/glibc-2.3.5-r1
       merge time: 1 hour, 14 minutes and 49 seconds.

     Tue Oct  4 23:12:51 2005 >>> sys-libs/glibc-2.3.5-r1
       merge time: 1 hour, 18 minutes and 56 seconds.

     Fri Oct  7 04:43:40 2005 >>> sys-libs/glibc-2.3.5-r2
       merge time: 1 hour, 53 minutes and 50 seconds.

     Fri Oct 28 05:52:35 2005 >>> sys-libs/glibc-2.3.5-r3
       merge time: 1 hour, 36 minutes and 1 second.

-- 
gentoo-amd64@gentoo.org mailing list