Re: [gentoo-dev] "Distro Day (Measuring the benefits of the Gentoo approach)"

[William Kenworthy <billk@iinet.net.au>]

I posted this to the gentoo-user list by mistake last night (it was
after midnight and ...) - wondered why I didnt get too many
flames/replies!

Apologies to those who get two copies, but it was originally meant to go
with this thread to stop the mis-information:

______posted to gentoo-user_______________________

I'll stick my hand up and say I was the person who installed gentoo for
this test.  For those who made the previous posts (mostly crap, and who
dont seem to have read the article very well - though it could have been
more informative), perhaps a few facts may help:

1. was fully bootstrapped and compiled as stage 1/2/3 on the machine -
not a binary install
2. gentoo-sources 2.4.20 was used - Mandrake came with a newer kernel
than gentoo's reccomended one (still does), debian was a dogs breakfast
because stable is so old.  We actually tried to put the gentoo kernel on
mandrake/debian when tracking down the ide cable prob, but got too hard
- not the way some posts tried to imply)
3. optimisations were EXACTLY as recommended by both the make.conf
entries, which were supported by the cflags from the forum for this cpu:
a 2G celery (P4 based core)  I am not sure now, but I believe I ran
prelink as well (to match mandrake) - need to find and check the notes.
4. Gnumerics problems have been identified and come down to the
particular version - is fixed in the upcoming stable release even before
this was found, but the project was unaware that what they believed was
a slightly slower mod in this version, could be so bad on particular
data sets - i.e., 30 odd mins in 1.0.13, but is less that 30s on 1.0.19
on my laptop

There seems to be quite a few myths about this test and people upset
that months were not spent tuning gentoo and every effort made to
cripple the competition! (one person even suggested the faulty ide cable
should have been left in the debian box, as that was the way it was
delivered!)  Read the article, and if you need extra information to
reproduce it, email me or or the author (Indy).  It is reproducable - if
you can obtain the same hardware - I would be very interested if someone
has this and the time to really go into the why these results occurred
in more detail than I had the chance to.

and why was this the result?  Daniel Robbins suggested on this list that
gentoo-sources may be the problem, but tests on another machine (we had
the trial machines for only a couple of days, all of which time was used
to build gentoo right up until I ctrl-c'd the OO build so we could do
the tests before handing the hardware back) showed that turning off
pre-empt and low-latency had zero effect, but changing to an open-mosix
kernel 2.4.20  was ~10% slower (no thread export).  It seemed to come
down to the fact we used -O3 instead of -O2 (think spider might have
suggested this ?)- in effect over-optimised, and we didnt have a chance
to correct. From my perspective, most of the "he should have used ...
may actually have made performance even worse! And besides the time
issue, these were supposedly the safe, reccomended flags so we went with
them.  Please note that even Mandrake made only a slight gain on debian,
so 386.586/686 does not make a lot of difference in real world tasks
(the original aim of the tests) - the tests did tasks that particular
people used linux for in their day-to-day work - no special tests, so no
special bias.  Yes, I could choose tests that make gentoo shine, or
debian, or windowsXP.  But I dont do those tests every day, whilst that
spreadsheet was/is used as part of my normal work.  And its the same
with the other tests.

So how many gentoo systems out there have every possible optimisation in
the book, and are actually running slower than ideal?  This is a real
problem, and I will be interested in how the cflags projects around
handle this, as most seem to aim at setting the maximum possible flags:
not actually tune the system for the ones that work best/most stably.  A
live benchmark test might be more appropriate.

Most posts on irc and lists have settled down to "he doesnt know what
he's doing" (I do), or the tests were unfair to gentoo (they werent, but
then the same criteria were met by all 3 systems, but with some question
marks over debian because of its mix - some packages had to be compiled
locally, not binary) - but the thrust of the article was not that gentoo
was a dud, but that this was the result within the criteria and time we
were given, not what we expected, so we need to find out why.  Also note
that this was not intentionally a debian/mandrake/gentto distro test.

We may be getting a P4 hyperthreaded system to play with, but under
different rules, where I get to do a bit of tuning first.  I have
already built the core system on another machine using gcc-3.2.3,
"-march=pentium4 -O3 -pipe -fomit-frame-pointer"  I note that the
pentium4 warning still appears in make.conf, though I believe it no
longer applies to this gcc.

A while ago I emailed this list and asked for information on tests and
settings for HT P4's, without a reply.  So again, has anyone done any
tests on a HT P4 and is willing to support the flags they chose as being
"the best"?  In particular, does -ffast-math give a measurable gain? 
Most of my machines have been built as scientific stations, so accuracy
is more important than ultimate speed, so this is one I have never
tested.  I am not interested in the -O9 -max-everything kiddies who have
been so vocal, but reasoned choices.

If you want to flame, go ahead - but support your statements!

:)

BillK



On Thu, 2003-08-14 at 05:15, FRLinux wrote:
> On Wed, 2003-08-13 at 15:20, Philippe Lafoucrière wrote:
> > totally agree ! Btw, gnumeric speed is related to version apparently,
> > and they didn't use the official gentoo (patched) kernel ("The same
> > 2.4.21 source was copied to all machines"). This sux !!
> 
> Well i don't think this sucks, keeping at least consistency on the
> kernel between the 3 distributions is a good idea. Now, that being said,
> they seriously fsck'd up the rest of the test and optimisations.
> 
> I don't use specific gentoo kernels on my boxes, this is a personal
> choice made about a year ago and actually since 2.6, i don't use 2.4
> anymore on personal machines (my laptop and my workstation).
> 
> Steph
-- 
William Kenworthy <billk@iinet.net.au>


--
gentoo-dev@gentoo.org mailing list