[gentoo-amd64] First Impressions

[gentoo-amd64] First Impressions

[Daniel Iliev]

Hello, everyone!

It's my first mail to this list. I hope the community here is as
friendly as the one at the general "gentoo-user" list ;-)
Please, forgive me if open some threads about already discussed issues
until I catch up with rhythm of the list.

So let me start a with 2 newbie questions caused by my first impressions
from the x86_64 world:

1) I use CFLAGS="-march=athlon64 -mfpmath=sse -msse -msse2 -msse3
-m3dnow -mmmx -O3 -fomit-frame-pointer -pipe -fpic". Portage complains
with *red letters* about the fpic flag. Every time I emerge something it
says that "fpic breaks things", but I haven't met a single breakage so
far. Is that a bug? Actually there was an ebuild which could not be
compiled if mysql was compiled w/o "fpic". I'm not 100% sure but AFAIR
it was dev-perl/DBD-mysql.

2) I see too many flags that are disabled by the profile - the kind with
the parenthesis around them, like "(-3dnow)". Why? As I mentioned above
I enable some of these through my CFLAGS - e.g. (-mmx), (-mmxext),
(-sse) and (-sse2) and everything works perfect.



-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] First Impressions

[Barry.SCHWARTZ]

[-- Attachment #1: Type: text/plain, Size: 1255 bytes --]

Daniel Iliev <danny@ilievnet.com> skribis:
> 1) I use CFLAGS="-march=athlon64 -mfpmath=sse -msse -msse2 -msse3
> -m3dnow -mmmx -O3 -fomit-frame-pointer -pipe -fpic". Portage complains
> with *red letters* about the fpic flag. Every time I emerge something it
> says that "fpic breaks things", but I haven't met a single breakage so
> far. Is that a bug? Actually there was an ebuild which could not be
> compiled if mysql was compiled w/o "fpic". I'm not 100% sure but AFAIR
> it was dev-perl/DBD-mysql.

I guess the best question might be ‘Why do you want -fpic?’

> 2) I see too many flags that are disabled by the profile - the kind with
> the parenthesis around them, like "(-3dnow)". Why? As I mentioned above
> I enable some of these through my CFLAGS - e.g. (-mmx), (-mmxext),
> (-sse) and (-sse2) and everything works perfect.

Parentheses indicate that you can’t change those USE flags; they are
immutably set in the profile, for whatever reason.

-- 
Barry.SCHWARTZ at chemoelectric.org   http://chemoelectric.org
        Free stuff / Senpagaj varoj:  http://crudfactory.com (PDF)
'Democracies don't war; democracies are peaceful countries.' - Bush
(http://www.whitehouse.gov/news/releases/2005/12/20051219-2.html)

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-amd64] First Impressions

[Dice R. Random]

On 9/26/06, Daniel Iliev <danny@ilievnet.com> wrote:
> Hello, everyone!

Hi :)

> 1) I use CFLAGS="-march=athlon64 -mfpmath=sse -msse -msse2 -msse3
> -m3dnow -mmmx -O3 -fomit-frame-pointer -pipe -fpic". Portage complains
> with *red letters* about the fpic flag. Every time I emerge something it
> says that "fpic breaks things", but I haven't met a single breakage so
> far. Is that a bug? Actually there was an ebuild which could not be
> compiled if mysql was compiled w/o "fpic". I'm not 100% sure but AFAIR
> it was dev-perl/DBD-mysql.

It is not necessary to tell gcc to use special instruction sets for
your processor.  When you specify -march gcc automatically knows
exactly what instructions your processor is capable of utilizing most
effectively.

I am not familiar with PIC, but if portage gives a big red warning
about explicitly enabling it in your CFLAGS I'm sure that there have
been problems with it.  If a package requires the flag then it should
enable it in its own local CFLAGS.

> 2) I see too many flags that are disabled by the profile - the kind with
> the parenthesis around them, like "(-3dnow)". Why? As I mentioned above
> I enable some of these through my CFLAGS - e.g. (-mmx), (-mmxext),
> (-sse) and (-sse2) and everything works perfect.

A 3dnow instruction path isn't always the best optimization for every
user and every package.  Also, AMD's Athlon and Opteron processors are
not the only ones supported under the amd64 arch, there are also
Intel's x86-64 processors to consider.  While I imagine that Intel
processors can execute 3dnow instructions I also imagine that SSE
would be faster.
-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: First Impressions

[Duncan]

Daniel Iliev <danny@ilievnet.com> posted 451A110B.2080401@ilievnet.com,
excerpted below, on  Wed, 27 Sep 2006 08:50:03 +0300:

> So let me start a with 2 newbie questions caused by my first impressions
> from the x86_64 world:
> 
> 1) I use CFLAGS="-march=athlon64 -mfpmath=sse -msse -msse2 -msse3
> -m3dnow -mmmx -O3 -fomit-frame-pointer -pipe -fpic". Portage complains
> with *red letters* about the fpic flag. Every time I emerge something it
> says that "fpic breaks things", but I haven't met a single breakage so
> far. Is that a bug? Actually there was an ebuild which could not be
> compiled if mysql was compiled w/o "fpic". I'm not 100% sure but AFAIR
> it was dev-perl/DBD-mysql.
> 
> 2) I see too many flags that are disabled by the profile - the kind with
> the parenthesis around them, like "(-3dnow)". Why? As I mentioned above
> I enable some of these through my CFLAGS - e.g. (-mmx), (-mmxext),
> (-sse) and (-sse2) and everything works perfect.

It seems that you missed some of the Gentoo/AMD64 documentation. 
Many/most of your questions are answered there.  Unfortunately, I'm not
aware of a simple easy to use list of everything in one spot, so it's
reading a bit of documentation here, a bit more there, etc.

The main Gentoo/AMD64 project page.  (This would be the logical place for
such a list, but it's more the project page, tho it links some of the
docs, it's just not as easy to find those links as it could be.)
http://amd64.gentoo.org

Gentoo/AMD64 FAQ:
http://www.gentoo.org/doc/en/gentoo-amd64-faq.xml

Gentoo/AMD64 HOWTOs.  (There's one on -fPIC here, tho the explanation is
a bit developer-centric.)
http://www.gentoo.org/proj/en/base/amd64/howtos/index.xml

A brief direct answer to your questions follows:

*  The sse etc CFLAGS are arch dependent.  Unlike x86 where the
mmx/sse/other-extensions instructions were added as the arch matured, on
amd64, they are part of the definition of the arch itself.  All x86_64
(amd64) CPUs will have mmx/sse/sse2, etc.  Thus, -march=athlon64 already
tells gcc these are available to use where it wants/needs to.  The others
don't therefore provide gcc any more information than what it already has.

* -fomit-frame-pointer isn't needed on 64-bit amd64 either, as it's turned
on for all -O levels on archs (including amd64) where doing so doesn't
interfere with debugging.  (See the gcc manpage, under -O optimization.)
You may wish to continue to specify it for stuff that's compiled for
32-bit, however, including parts of gcc, a version of glibc, a version of
the (portage) sandbox library, etc.

* Generally speaking, -fPIC is required on amd64 for ALL LIBRARIES but the
ebuilds normally take care of it.  Under certain circumstances (like
unsupported CFLAGS), the configure scripts will turn it off by mistake, see
the above mentioned -fPIC HOWTO link for details, but the solution isn't
to add it to your CFLAGS, as that means it will be used for executable
applications as well as libraries, and /some/ applications /do/ break with
it.  Not many, but some, and if it's in your CFLAGS, you WILL have bugs
you file closed as INVALID or the like, due to CFLAG abuse.  If there's
something not working without it, then THAT'S a bug and should be filed as
such (unless it's due to use of CFLAGS gcc doesn't support and warns
about, thus triggering the configure script detection problem discussed
above and in the HOWTO).

* The profile "disabled" USE flags are simply hard-locked either on or
off by the profile, so aren't a USE flag option.  It does NOT mean whatever
the USE flag controls is actually disabled.  Sometimes, as with the
multilib USE flag, it can mean it's /enabled/.  It just means that the
profile is set up to control it, generally for a pretty good reason.  In
the particular cases you mention, the way Gentoo uses the SSE and similar
USE flags is 32-bit specific, enabling 32-bit specific assembler code in
the ebuild, for instance.   As already mentioned, the AMD64 arch by
definition already has these features activated, so no 64-bit USE flags
are necessary, and enabling the 32-bit USE flags will cause breakage since
it activates 32-bit specific code in many instances.  Thus the amd64
profiles have a /very/ good reason to hard-lock these USE flags "off".  An
example where a USE flag is hard-locked ON by a profile would be multilib.
The normal AMD64 profiles are all multilib and thus lock this flag ON (tho
it's still shown as disabled), while 64-bit-only profiles lock it OFF.

A couple of other notes:

Portage now supports per-package CFLAGS and certain other variables as
controlled by the environment (as long as they are used in an ebuild.sh
phase, not the python phase, since execution is via a bashrc hook). 
Create /etc/portage/env/<category> as a directory, populated with package
or package-version files.  The contents of these files will be sourced
into the ebuild.sh execution environment for every phase that uses
ebuild.sh.  CFLAGS and similar variables as found in these files REPLACE
(that is, they don't add to, they replace entirely) the default make.conf
CFLAGS.  You can use this mechanism to specify specific CFLAGS for
specific packages, and could thus set -fomit-frame-pointer and other
32-bit x86 specific CFLAGS here if desired, avoiding them in your regular
make.conf.

You may wish to read a bit of the archives for this list, in particular,
the recent threads on gcc 4.1.1 CFLAGS, where I discuss mine. 
Specifically, it's likely -O3 is actually /worse/ performing in many
instances than -O2 or even -Os (my choice).  The reasoning is this:  CPU
cycles are fairly cheap in a modern processor, while the expense of
waiting on main memory in the case of a cache miss is MUCH HIGHER, due to
the fact that main memory is clocked so much slower than cache.  Smaller
code fits in cache better and is thus often faster than larger code, even
when the smaller code isn't as theoretically CPU cycle efficient.  While
there will certainly be certain applications where -O3 is beneficial, I
believe if you do actual comparisons, you will find -O2 or -Os faster on a
system-wide basis.  Of course, it's up to you and much virtual ink has
been spilled discussing this issue, but that's just my take on things.  If
you've actually done speed comparisons on AMD64 or can point to some, I'd
certainly be interested, as I've honestly not cared enough about it to do
my own, but that's my general take in the absence of specific hard data to
the contrary.  Rather than optimizing for CPU cycles (-O3), I choose to
optimize for better register usage (registers being at full CPU speed,
therefore faster even than L1 cache, -frename-registers and etc) size
(-Os, disabling loop unrolling), whole and multiple unit optimization
(-funit-at-a-time, -combine) and hot/cold partitioning
(-freorder-blocks-and-partition, tho it can't be used on C++ code, etc).  A
few of my flags fail on a very few specific packages, another use for the
package specific CFLAGS stuff above.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] First Impressions

[Daniel Iliev]

Barry.SCHWARTZ@chemoelectric.org wrote:

> I guess the best question might be ‘Why do you want -fpic?’
>   


I use prelink [1] which requires PIC in order to work. At least that is
true for X86. That is the only reason - I suppose if it works on x86_64
it would have the same requirements.



[1] http://www.gentoo.org/doc/en/prelink-howto.xml

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] First Impressions

[Daniel Iliev]

Dice R. Random wrote:
>
> It is not necessary to tell gcc to use special instruction sets for
> your processor.  When you specify -march gcc automatically knows
> exactly what instructions your processor is capable of utilizing most
> effectively.
I'm not sure about this. Tonight I'll make a test: I'll compile a
package with and without these flags and watch the output and the outcome.

>
> I am not familiar with PIC, but if portage gives a big red warning
> about explicitly enabling it in your CFLAGS I'm sure that there have
> been problems with it.  If a package requires the flag then it should
> enable it in its own local CFLAGS.
>
Already answered about Position Independent Code and why I use it.
Shortly: http://www.gentoo.org/doc/en/prelink-howto.xml
> A 3dnow instruction path isn't always the best optimization for every
> user and every package.  Also, AMD's Athlon and Opteron processors are
> not the only ones supported under the amd64 arch, there are also
> Intel's x86-64 processors to consider.  While I imagine that Intel
> processors can execute 3dnow instructions I also imagine that SSE
> would be faster.
I'm really not sure here. I cant tell which is better -3dnow(ext)  or
sse{1..3}, but isn't it best if one enables all kinds of optimizations
supported by the CPU?

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: First Impressions

[Daniel Iliev]

Duncan wrote:
> Daniel Iliev <danny@ilievnet.com> posted 451A110B.2080401@ilievnet.com,
> excerpted below, on  Wed, 27 Sep 2006 08:50:03 +0300:
>
>   
>> So let me start a with 2 newbie questions caused by my first impressions
>> from the x86_64 world:
>>
>> 1) I use CFLAGS="-march=athlon64 -mfpmath=sse -msse -msse2 -msse3
>> -m3dnow -mmmx -O3 -fomit-frame-pointer -pipe -fpic". Portage complains
>> with *red letters* about the fpic flag. Every time I emerge something it
>> says that "fpic breaks things", but I haven't met a single breakage so
>> far. Is that a bug? Actually there was an ebuild which could not be
>> compiled if mysql was compiled w/o "fpic". I'm not 100% sure but AFAIR
>> it was dev-perl/DBD-mysql.
>>
>> 2) I see too many flags that are disabled by the profile - the kind with
>> the parenthesis around them, like "(-3dnow)". Why? As I mentioned above
>> I enable some of these through my CFLAGS - e.g. (-mmx), (-mmxext),
>> (-sse) and (-sse2) and everything works perfect.
>>     
>
> It seems that you missed some of the Gentoo/AMD64 documentation. 
> Many/most of your questions are answered there.  Unfortunately, I'm not
> aware of a simple easy to use list of everything in one spot, so it's
> reading a bit of documentation here, a bit more there, etc.
>
> The main Gentoo/AMD64 project page.  (This would be the logical place for
> such a list, but it's more the project page, tho it links some of the
> docs, it's just not as easy to find those links as it could be.)
> http://amd64.gentoo.org
>
> Gentoo/AMD64 FAQ:
> http://www.gentoo.org/doc/en/gentoo-amd64-faq.xml
>
> Gentoo/AMD64 HOWTOs.  (There's one on -fPIC here, tho the explanation is
> a bit developer-centric.)
> http://www.gentoo.org/proj/en/base/amd64/howtos/index.xml
>
> A brief direct answer to your questions follows:
>
> *  The sse etc CFLAGS are arch dependent.  Unlike x86 where the
> mmx/sse/other-extensions instructions were added as the arch matured, on
> amd64, they are part of the definition of the arch itself.  All x86_64
> (amd64) CPUs will have mmx/sse/sse2, etc.  Thus, -march=athlon64 already
> tells gcc these are available to use where it wants/needs to.  The others
> don't therefore provide gcc any more information than what it already has.
>
> * -fomit-frame-pointer isn't needed on 64-bit amd64 either, as it's turned
> on for all -O levels on archs (including amd64) where doing so doesn't
> interfere with debugging.  (See the gcc manpage, under -O optimization.)
> You may wish to continue to specify it for stuff that's compiled for
> 32-bit, however, including parts of gcc, a version of glibc, a version of
> the (portage) sandbox library, etc.
>
> * Generally speaking, -fPIC is required on amd64 for ALL LIBRARIES but the
> ebuilds normally take care of it.  Under certain circumstances (like
> unsupported CFLAGS), the configure scripts will turn it off by mistake, see
> the above mentioned -fPIC HOWTO link for details, but the solution isn't
> to add it to your CFLAGS, as that means it will be used for executable
> applications as well as libraries, and /some/ applications /do/ break with
> it.  Not many, but some, and if it's in your CFLAGS, you WILL have bugs
> you file closed as INVALID or the like, due to CFLAG abuse.  If there's
> something not working without it, then THAT'S a bug and should be filed as
> such (unless it's due to use of CFLAGS gcc doesn't support and warns
> about, thus triggering the configure script detection problem discussed
> above and in the HOWTO).
>
> * The profile "disabled" USE flags are simply hard-locked either on or
> off by the profile, so aren't a USE flag option.  It does NOT mean whatever
> the USE flag controls is actually disabled.  Sometimes, as with the
> multilib USE flag, it can mean it's /enabled/.  It just means that the
> profile is set up to control it, generally for a pretty good reason.  In
> the particular cases you mention, the way Gentoo uses the SSE and similar
> USE flags is 32-bit specific, enabling 32-bit specific assembler code in
> the ebuild, for instance.   As already mentioned, the AMD64 arch by
> definition already has these features activated, so no 64-bit USE flags
> are necessary, and enabling the 32-bit USE flags will cause breakage since
> it activates 32-bit specific code in many instances.  Thus the amd64
> profiles have a /very/ good reason to hard-lock these USE flags "off".  An
> example where a USE flag is hard-locked ON by a profile would be multilib.
> The normal AMD64 profiles are all multilib and thus lock this flag ON (tho
> it's still shown as disabled), while 64-bit-only profiles lock it OFF.
>
> A couple of other notes:
>
> Portage now supports per-package CFLAGS and certain other variables as
> controlled by the environment (as long as they are used in an ebuild.sh
> phase, not the python phase, since execution is via a bashrc hook). 
> Create /etc/portage/env/<category> as a directory, populated with package
> or package-version files.  The contents of these files will be sourced
> into the ebuild.sh execution environment for every phase that uses
> ebuild.sh.  CFLAGS and similar variables as found in these files REPLACE
> (that is, they don't add to, they replace entirely) the default make.conf
> CFLAGS.  You can use this mechanism to specify specific CFLAGS for
> specific packages, and could thus set -fomit-frame-pointer and other
> 32-bit x86 specific CFLAGS here if desired, avoiding them in your regular
> make.conf.
>
> You may wish to read a bit of the archives for this list, in particular,
> the recent threads on gcc 4.1.1 CFLAGS, where I discuss mine. 
> Specifically, it's likely -O3 is actually /worse/ performing in many
> instances than -O2 or even -Os (my choice).  The reasoning is this:  CPU
> cycles are fairly cheap in a modern processor, while the expense of
> waiting on main memory in the case of a cache miss is MUCH HIGHER, due to
> the fact that main memory is clocked so much slower than cache.  Smaller
> code fits in cache better and is thus often faster than larger code, even
> when the smaller code isn't as theoretically CPU cycle efficient.  While
> there will certainly be certain applications where -O3 is beneficial, I
> believe if you do actual comparisons, you will find -O2 or -Os faster on a
> system-wide basis.  Of course, it's up to you and much virtual ink has
> been spilled discussing this issue, but that's just my take on things.  If
> you've actually done speed comparisons on AMD64 or can point to some, I'd
> certainly be interested, as I've honestly not cared enough about it to do
> my own, but that's my general take in the absence of specific hard data to
> the contrary.  Rather than optimizing for CPU cycles (-O3), I choose to
> optimize for better register usage (registers being at full CPU speed,
> therefore faster even than L1 cache, -frename-registers and etc) size
> (-Os, disabling loop unrolling), whole and multiple unit optimization
> (-funit-at-a-time, -combine) and hot/cold partitioning
> (-freorder-blocks-and-partition, tho it can't be used on C++ code, etc).  A
> few of my flags fail on a very few specific packages, another use for the
> package specific CFLAGS stuff above.
>
>   
Very detailed answer! Thank you!

Yes, you are right. I have missed the "AMD64 HowTo" documentation. I
found only the FAQ via Google.
It was the easiest (fastest) way to get some answers. ;-)

Thank you all.

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] First Impressions

[Barry.SCHWARTZ]

[-- Attachment #1: Type: text/plain, Size: 1039 bytes --]

Daniel Iliev <danny@ilievnet.com> skribis:
> Barry.SCHWARTZ@chemoelectric.org wrote:
> 
> > I guess the best question might be ‘Why do you want -fpic?’
> >   
> 
> 
> I use prelink [1] which requires PIC in order to work. At least that is
> true for X86. That is the only reason - I suppose if it works on x86_64
> it would have the same requirements.

Alright. Another reason would be to use PaX for security. I’ve used
PaX and what I did for that, to get PIC code, was to use the
‘hardened’ 3.x compiler. The ‘hardened no SSP’ setting would be the
one for prelink, I think.

When I really want a program to start quickly, though, I use the
sticky bit, so the program stays in memory. I think this is the
default for GNU Emacs.


-- 
Barry.SCHWARTZ at chemoelectric.org   http://chemoelectric.org
        Free stuff / Senpagaj varoj:  http://crudfactory.com (PDF)
'Democracies don't war; democracies are peaceful countries.' - Bush
(http://www.whitehouse.gov/news/releases/2005/12/20051219-2.html)

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

[gentoo-amd64] Re: First Impressions

[Duncan]

Barry.SCHWARTZ@chemoelectric.org posted
20060927104629.GA31464@crud.crud.mn.org, excerpted below, on  Wed, 27 Sep
2006 05:46:29 -0500:

> Alright. Another reason would be to use PaX for security. I’ve used
> PaX and what I did for that, to get PIC code, was to use the
> ‘hardened’ 3.x compiler. The ‘hardened no SSP’ setting would be the
> one for prelink, I think.

Right on the security thing in general, tho I'm not sure of the specific
specs files specifics.  (Wow, that sounds weird! <g>)

> When I really want a program to start quickly, though, I use the
> sticky bit, so the program stays in memory. I think this is the
> default for GNU Emacs.

Eh?  That's a rather dated use of that bit, AFAIK.  See the wikipedia (stub
entry) here:

http://en.wikipedia.org/wiki/Sticky_bit

While keeping a program in memory was the original use of that bit, it
hasn't been used for that for a long time, as modern swapping and caching
methods tend to be decently efficient at that on their own.  In modern
system use, the sticky bit has significance only for directories, where it
affects deletion/rename permissions in multiple user access situations,
such as the /tmp dir, which is commonly world writable but set sticky so
only a file's owner or root can delete files.  Of course, you can use the
sticky bit on non-dirs for other things if you wish, but there's no system
significance on executables as there once was.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] First Impressions

[Christoph Mende]

One thing that I think wasn't mentioned yet, while -fPIC is needed for
libraries, it must be disabled for binaries (don't know if that's true
for prelink), as portage says, it might break things and your binaries
are most likely becoming slower when you compile them with -fPIC.

2006/9/27, Daniel Iliev <danny@ilievnet.com>:
> Hello, everyone!
>
> It's my first mail to this list. I hope the community here is as
> friendly as the one at the general "gentoo-user" list ;-)
> Please, forgive me if open some threads about already discussed issues
> until I catch up with rhythm of the list.
>
> So let me start a with 2 newbie questions caused by my first impressions
> from the x86_64 world:
>
> 1) I use CFLAGS="-march=athlon64 -mfpmath=sse -msse -msse2 -msse3
> -m3dnow -mmmx -O3 -fomit-frame-pointer -pipe -fpic". Portage complains
> with *red letters* about the fpic flag. Every time I emerge something it
> says that "fpic breaks things", but I haven't met a single breakage so
> far. Is that a bug? Actually there was an ebuild which could not be
> compiled if mysql was compiled w/o "fpic". I'm not 100% sure but AFAIR
> it was dev-perl/DBD-mysql.
>
> 2) I see too many flags that are disabled by the profile - the kind with
> the parenthesis around them, like "(-3dnow)". Why? As I mentioned above
> I enable some of these through my CFLAGS - e.g. (-mmx), (-mmxext),
> (-sse) and (-sse2) and everything works perfect.
>
>
>
> --
> gentoo-amd64@gentoo.org mailing list
>
>
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: First Impressions

[Barry.SCHWARTZ]

[-- Attachment #1: Type: text/plain, Size: 1042 bytes --]

Duncan <1i5t5.duncan@cox.net> skribis:
> > When I really want a program to start quickly, though, I use the
> > sticky bit, so the program stays in memory. I think this is the
> > default for GNU Emacs.
> 
> Eh?  That's a rather dated use of that bit, AFAIK.

Thus it gets very little use. :)

In my experience programs that start up slowly often have bugs causing
it. For example, Opera slows down dramatically if you have a lot of
fonts, which I do; because of this, I run Opera under a separate
fontconfig. Programs that use Pango, such as Inkscape, can be slow to
start up if you have a lot of fonts, though they won’t slow down, once
started, as dramatically as does Opera. GIMP has its own code for
listing your fonts and starts up quickly.


-- 
Barry.SCHWARTZ at chemoelectric.org   http://chemoelectric.org
        Free stuff / Senpagaj varoj:  http://crudfactory.com (PDF)
'Democracies don't war; democracies are peaceful countries.' - Bush
(http://www.whitehouse.gov/news/releases/2005/12/20051219-2.html)

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-amd64] First Impressions

[Daniel Iliev]

Christoph Mende wrote:
> One thing that I think wasn't mentioned yet, while -fPIC is needed for
> libraries, it must be disabled for binaries (don't know if that's true
> for prelink), as portage says, it might break things and your binaries
> are most likely becoming slower when you compile them with -fPIC.
>

Unfortunately I am not a C programmer and I can't be sure about how
exactly PIC and prelink work.
As far as I understood the mechanism (please, correct me if I'm wrong)
prelink scans the executables to find which libs they load. Then it
makes cache and when a program is started it uses already pre-loaded
libs. So the program is ready for action faster.

My only way to judge about the result is to test. Taking this in mind
you can understand that the part about "becoming slower" sounds very
disturbing to me. ;-)
Can you, please, explain how it comes that PIC is good for libs and bad
for execs? I'm confused because the gcc man page says:
-fpic
           Generate position-independent code (PIC) --snip--
           If the GOT size for the linked executable exceeds a
           machine-specific maximum size, you get an error message
           from the linker indicating that -fpic does not work; in that
case,
           recompile with -fPIC instead. -snip- *The 386 has no such limit*

-fPIC
           If supported for the target machine, emit position-independent
           code, suitable for dynamic linking and avoiding any limit on the
           size of the global offset table.  *This option makes a
difference on
           the m68k, PowerPC and SPARC.*

So, both flags let gcc produce PIC for libs and programs and none of
them lets gcc produce PIC for libs only? The next flag described in the
man page is -fpie, which makes PIC for programs only. Something is wrong
and don't know how to find the right explanation.

-- 
Best regards,
Daniel


-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] First Impressions

[Pawel Kraszewski]

Dnia środa, 27 września 2006 07:50, Daniel Iliev napisał:


> 1) I use CFLAGS="-march=athlon64 -mfpmath=sse -msse -msse2 -msse3
> -m3dnow -mmmx -O3 -fomit-frame-pointer -pipe -fpic". Portage complains
> with *red letters* about the fpic flag. Every time I emerge something it
> says that "fpic breaks things", but I haven't met a single breakage so
> far. Is that a bug? Actually there was an ebuild which could not be
> compiled if mysql was compiled w/o "fpic". I'm not 100% sure but AFAIR
> it was dev-perl/DBD-mysql.

That's what 'pic' USE flag is for...

$ euse -i pic
global use flags (searching: pic)
************************************************************

[+ C  ] pic - Build Position Independent Code.  Do not utilize this flag 
unless you know what you're doing.


"fpic breaks things". Trust Gentoo developers. They don't spread FUD. If they 
say it breaks, it surely does. You just haven't been struck by it. Yet.


-- 
 Pawel Kraszewski
 www.kraszewscy.net

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] First Impressions

[Simon Stelling]

Daniel Iliev wrote:
> So, both flags let gcc produce PIC for libs and programs and none of
> them lets gcc produce PIC for libs only? The next flag described in the
> man page is -fpie, which makes PIC for programs only. Something is wrong
> and don't know how to find the right explanation.

The problem is that -fPIC is a code-generating flag, i.e. it applies to 
the .o object files built. The compiler can't work out whether that 
object is used in a program or a shared library, so it can't know 
whether using PIC is a good idea or not.

-- 
Kind Regards,

Simon Stelling
Gentoo/AMD64 developer
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] First Impressions

[Hemmann, Volker Armin]

On Wednesday 27 September 2006 07:50, Daniel Iliev wrote:

>
> 1) I use CFLAGS="-march=athlon64 -mfpmath=sse -msse -msse2 -msse3
> -m3dnow -mmmx -O3 -fomit-frame-pointer -pipe -fpic". P

-mfpmath=sse not needed. amd64 uses sse for math by default. remove that flag
-msse not needed, see above
-msse2 not needed, see above
-msse3 can be set, but only if your CPU supports it. Not all CPUs do. 
Check /proc/cpuinfo for 'pni'
-m3dnow just no. sse supersedes it. Also it is a '32bit flag'
-mmmx see above
-O3 don't do it. O2 is much, much safer and not really slower. It will prevent 
a lot of breakage.
-fomit-frame-pointer not needed
-pipe a good flag
-fpic NO! Really, really really no! Don't do it! It is just wrong. Stop it 
now! This flag is not needed! Even for prelink on 32bit it was never needed 
(because everything that could compiled with that flag set it anyway). 
Please, read the documentation and remove that flag ASAP.

so, march=k8 -O2 -pipe
would be the result, if you remove all superflous and dangerous flags.


>
> 2) I see too many flags that are disabled by the profile - the kind with
> the parenthesis around them, like "(-3dnow)". Why? As I mentioned above
> I enable some of these through my CFLAGS - e.g. (-mmx), (-mmxext),
> (-sse) and (-sse2) and everything works perfect.

because 3dnow, mmx&co are useflags for 32bit code that was optimized for that 
extensions. If you use that code in 64bit apps, certain breakage is the 
result. And if something does not break, it will be A LOT slower.

Don't use that flags. They are bad for amd64. Trust the devs, they know better 
than you or me.
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] First Impressions

[Barry.SCHWARTZ]

[-- Attachment #1: Type: text/plain, Size: 1109 bytes --]

Daniel Iliev <danny@ilievnet.com> skribis:
> Unfortunately I am not a C programmer and I can't be sure about how
> exactly PIC and prelink work.
> As far as I understood the mechanism (please, correct me if I'm wrong)
> prelink scans the executables to find which libs they load. Then it
> makes cache and when a program is started it uses already pre-loaded
> libs. So the program is ready for action faster.

What you are interested in, probably, is PIE: position-independent
executables. These look like shared libraries but are runnable
programs. These are easy to get using the version 3.4.x compiler and
‘hardenednossp’ specs. I suspect you will have more fun and fewer
headaches by using the 4.1.x compiler and no prelink. There are
already enough extra headaches in amd64 Gentoo, compared to x86
Gentoo! :)

-- 
Barry.SCHWARTZ at chemoelectric.org   http://chemoelectric.org
        Free stuff / Senpagaj varoj:  http://crudfactory.com (PDF)
'Democracies don't war; democracies are peaceful countries.' - Bush
(http://www.whitehouse.gov/news/releases/2005/12/20051219-2.html)

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-amd64] First Impressions

[Daniel Iliev]

Barry.SCHWARTZ@chemoelectric.org wrote:
> Daniel Iliev <danny@ilievnet.com> skribis:
>   
>> Unfortunately I am not a C programmer and I can't be sure about how
>> exactly PIC and prelink work.
>> As far as I understood the mechanism (please, correct me if I'm wrong)
>> prelink scans the executables to find which libs they load. Then it
>> makes cache and when a program is started it uses already pre-loaded
>> libs. So the program is ready for action faster.
>>     
>
> What you are interested in, probably, is PIE: position-independent
> executables. These look like shared libraries but are runnable
> programs. These are easy to get using the version 3.4.x compiler and
> ‘hardenednossp’ specs. I suspect you will have more fun and fewer
> headaches by using the 4.1.x compiler and no prelink. There are
> already enough extra headaches in amd64 Gentoo, compared to x86
> Gentoo! :)
>
>   
Yep! gcc -v says: Target: x86_64-pc-linux-gnu.....gcc version 4.1.1
(Gentoo 4.1.1)

Are you proposing me to put -fpie in the CFLAGS?

-- 
Best regards,
Daniel


-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] First Impressions

[Daniel Iliev]

Simon Stelling wrote:
> Daniel Iliev wrote:
>> So, both flags let gcc produce PIC for libs and programs and none of
>> them lets gcc produce PIC for libs only? The next flag described in the
>> man page is -fpie, which makes PIC for programs only. Something is wrong
>> and don't know how to find the right explanation.
>
> The problem is that -fPIC is a code-generating flag, i.e. it applies
> to the .o object files built. The compiler can't work out whether that
> object is used in a program or a shared library, so it can't know
> whether using PIC is a good idea or not.
>
Well, then I have to simplify more (because of my lack of knowledge) and
ask what are the pros and cons to use PIC when building a program and a
share library?

-- 
Best regards,
Daniel


-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] First Impressions

[Daniel Iliev]

Pawel Kraszewski wrote:
> That's what 'pic' USE flag is for...
> $ euse -i pic
> global use flags (searching: pic)
> ************************************************************
>
> [+ C  ] pic - Build Position Independent Code.  Do not utilize this flag 
> unless you know what you're doing.
>
>
> "fpic breaks things". Trust Gentoo developers. They don't spread FUD. If they 
> say it breaks, it surely does. You just haven't been struck by it. Yet.
>
>
>   
Thanks. I know about this USE flag and it is (of course) turned on here. ;-)

-- 
Best regards,
Daniel


-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] First Impressions

[Simon Stelling]

Daniel Iliev wrote:
> Well, then I have to simplify more (because of my lack of knowledge) and
> ask what are the pros and cons to use PIC when building a program and a
> share library?

Pro is your able to run it on an amd64. Con is it can drastically slow 
down code (sometimes), and just plain doesn't make sense.

-- 
Kind Regards,

Simon Stelling
Gentoo/AMD64 developer
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] First Impressions

[Simon Stelling]

Daniel Iliev wrote:
> Are you proposing me to put -fpie in the CFLAGS?

Dear God, no! It's automatically appended where needed (read: hardened).

-- 
Kind Regards,

Simon Stelling
Gentoo/AMD64 developer
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] First Impressions

[Simon Stelling]

Simon Stelling wrote:
> Pro is your able to run it on an amd64. Con is it can drastically slow 
> down code (sometimes), and just plain doesn't make sense.

Err, this is missing a bit:

Pro of applying it to objects used in shared libs: You can run it.

Con of applying it to objects used in executables: It can slow down things.

-- 
Kind Regards,

Simon Stelling
Gentoo/AMD64 developer
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] First Impressions

[Barry.SCHWARTZ]

[-- Attachment #1: Type: text/plain, Size: 540 bytes --]

"Hemmann, Volker Armin" <volker.armin.hemmann@tu-clausthal.de> skribis:
> so, march=k8 -O2 -pipe
> would be the result, if you remove all superflous and dangerous
> flags.

I endorse this setting. IMO as a programmer, other flags are for
programmers. :)



-- 
Barry.SCHWARTZ at chemoelectric.org   http://chemoelectric.org
        Free stuff / Senpagaj varoj:  http://crudfactory.com (PDF)
'Democracies don't war; democracies are peaceful countries.' - Bush
(http://www.whitehouse.gov/news/releases/2005/12/20051219-2.html)

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-amd64] First Impressions

[Daniel Iliev]

Hemmann, Volker Armin wrote:
> On Wednesday 27 September 2006 07:50, Daniel Iliev wrote:
>
>   
>> 1) I use CFLAGS="-march=athlon64 -mfpmath=sse -msse -msse2 -msse3
>> -m3dnow -mmmx -O3 -fomit-frame-pointer -pipe -fpic". P
>>     
>
> -mfpmath=sse not needed. amd64 uses sse for math by default. remove that flag
> -msse not needed, see above
> -msse2 not needed, see above
> -msse3 can be set, but only if your CPU supports it. Not all CPUs do. 
> Check /proc/cpuinfo for 'pni'
> -m3dnow just no. sse supersedes it. Also it is a '32bit flag'
> -mmmx see above
> -O3 don't do it. O2 is much, much safer and not really slower. It will prevent 
> a lot of breakage.
> -fomit-frame-pointer not needed
> -pipe a good flag
> -fpic NO! Really, really really no! Don't do it! It is just wrong. Stop it 
> now! This flag is not needed! Even for prelink on 32bit it was never needed 
> (because everything that could compiled with that flag set it anyway). 
> Please, read the documentation and remove that flag ASAP.
>
> so, march=k8 -O2 -pipe
> would be the result, if you remove all superflous and dangerous flags.
>
>
>   
>> 2) I see too many flags that are disabled by the profile - the kind with
>> the parenthesis around them, like "(-3dnow)". Why? As I mentioned above
>> I enable some of these through my CFLAGS - e.g. (-mmx), (-mmxext),
>> (-sse) and (-sse2) and everything works perfect.
>>     
>
> because 3dnow, mmx&co are useflags for 32bit code that was optimized for that 
> extensions. If you use that code in 64bit apps, certain breakage is the 
> result. And if something does not break, it will be A LOT slower.
>
> Don't use that flags. They are bad for amd64. Trust the devs, they know better 
> than you or me.
>   
Thank you very much for this reply. I'll follow your advise and remove
the redundant flags.
About trusting the devs - well, it depends. I trust them 100% if I need
a rock stable system. In that case I would not dare to divert from any
of the official instructions.
BUT. Devs always tent to advertise the safest ways - this brings users
no/less headaches and therefore less bug-reports ;-)
My case however is slightly different - I'm talking about my home
desktop which is dedicated for experiments and fun. So I'm not afraid to
break the system here and rebuild it again. In this particular moment my
purpose is to get the most out of the hardware no matter the stability.
So I'll leave -O3 ;-)

BTW, Everyone,
I'm observing something very interesting:
I was told not to go gentoo-amd64 for it was not stable. I was told not
to migrate because there were still many important programs pending to
be ported. I read almost everywhere about headaches and breakages.
Reading your replies in this thread also suggests strictly following the
official way otherwise - problems.
It is very strange - I was ready to meet tons of major problems but I
haven't met a single one yet. It is my opinion that the possibility of
problems on gentoo-amd64 is highly overrated. I installed it with no
problems, I obviously have tweaked it a lot beyond normal and what I see
is a perfectly working system. It appears that gentoo-amd64 team along
with the GNU, linux-kernel and all other nice guys who provide free/open
source software have done a great work and we owe them BIG THANKS. I
just wonder how come that so many people talk about some non-existing
problems.
How come that still in my first try I have bootstrapped from stage3,
made "emerge -e system", installed xfce4, gnome, firefox, thunderbird,
and a bunch of other packages along with all their dependencies, then
made "emerge -e world" and after all this compiling I had to do "emerge
--resume" only once when some package wanted mysql build with -fpic
flag. I'm I lucky or what? ;-)


-- 
Best regards,
Daniel


-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] First Impressions

[Daniel Iliev]

Simon Stelling wrote:
> Daniel Iliev wrote:
>> Are you proposing me to put -fpie in the CFLAGS?
>
> Dear God, no! It's automatically appended where needed (read: hardened).
>
After a definitive answer like this (especially when it comes from
@gentoo.org) there's nothing left for me but to erase the word "-fpie"
from my vocabulary!

Thanks! ;-)

-- 
Best regards,
Daniel


-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] First Impressions

[Daniel Iliev]

Simon Stelling wrote:
> Simon Stelling wrote:
>> Pro is your able to run it on an amd64. Con is it can drastically
>> slow down code (sometimes), and just plain doesn't make sense.
>
> Err, this is missing a bit:
>
> Pro of applying it to objects used in shared libs: You can run it.
>
> Con of applying it to objects used in executables: It can slow down
> things.
>
Thanks again!

OK, you convinced me: As my purpose is to let portage build the fastest
code possible, "-fpic" goes out of my CFLAGS

-- 
Best regards,
Daniel


-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] First Impressions

[Barry.SCHWARTZ]

[-- Attachment #1: Type: text/plain, Size: 1131 bytes --]

Daniel Iliev <danny@ilievnet.com> skribis:
> Well, then I have to simplify more (because of my lack of knowledge) and
> ask what are the pros and cons to use PIC when building a program and a
> share library?

When a program runs it needs to know the addresses of variables,
subroutines, etc. A shared library can’t know those addresses until
run time, when it gets linked into the program. Therefore its code
needs to have fancy things done to it so it can be taught where to
find things at run time. The PIC stuff is largely overhead. A static
library is linked in at program build time, and so doesn’t have this
overhead.

My wild guess is that prelink would need position independent
executables because the build-time linker (/usr/bin/ld) doesn’t
know how to link a program to a shared library as if it were a static
library.


-- 
Barry.SCHWARTZ at chemoelectric.org   http://chemoelectric.org
        Free stuff / Senpagaj varoj:  http://crudfactory.com (PDF)
'Democracies don't war; democracies are peaceful countries.' - Bush
(http://www.whitehouse.gov/news/releases/2005/12/20051219-2.html)

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-amd64] First Impressions

[Barry.SCHWARTZ]

[-- Attachment #1: Type: text/plain, Size: 1305 bytes --]

Daniel Iliev <danny@ilievnet.com> skribis:
> How come that still in my first try I have bootstrapped from stage3,
> made "emerge -e system", installed xfce4, gnome, firefox, thunderbird,
> and a bunch of other packages along with all their dependencies, then
> made "emerge -e world" and after all this compiling I had to do "emerge
> --resume" only once when some package wanted mysql build with -fpic
> flag. I'm I lucky or what? ;-)

Probably. :)

I’m also a home user and have broken my system repeatedly. If I’m not
breaking my system, it’s because I’m busy with some other fun. Back
when I was playing with PaX and Gentoo was less mature, I did
sometimes ‘have to’ put flags of that sort in to get a program built
with PIE, until I switched to the hardened compiler, anyway. I got
tired of that, eventually, and now use the plain vanilla 4.1 compiler
and standard executables.

(As a programmer who likes handwritten parsers, I like the 4.1
compiler because it has a handwritten parser. :) )

-- 
Barry.SCHWARTZ at chemoelectric.org   http://chemoelectric.org
        Free stuff / Senpagaj varoj:  http://crudfactory.com (PDF)
'Democracies don't war; democracies are peaceful countries.' - Bush
(http://www.whitehouse.gov/news/releases/2005/12/20051219-2.html)

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-amd64] First Impressions

[Mike Bonar]

On Wednesday 27 September 2006 12:35, Daniel Iliev wrote:
...snip
> BTW, Everyone,
> I'm observing something very interesting:
> I was told not to go gentoo-amd64 for it was not stable. I was told not
> to migrate because there were still many important programs pending to
> be ported. I read almost everywhere about headaches and breakages.
> Reading your replies in this thread also suggests strictly following the
> official way otherwise - problems.
> It is very strange - I was ready to meet tons of major problems but I
> haven't met a single one yet. It is my opinion that the possibility of
> problems on gentoo-amd64 is highly overrated. I installed it with no
> problems, I obviously have tweaked it a lot beyond normal and what I see
> is a perfectly working system. It appears that gentoo-amd64 team along
> with the GNU, linux-kernel and all other nice guys who provide free/open
> source software have done a great work and we owe them BIG THANKS. I
> just wonder how come that so many people talk about some non-existing
> problems.
> How come that still in my first try I have bootstrapped from stage3,
> made "emerge -e system", installed xfce4, gnome, firefox, thunderbird,
> and a bunch of other packages along with all their dependencies, then
> made "emerge -e world" and after all this compiling I had to do "emerge
> --resume" only once when some package wanted mysql build with -fpic
> flag. I'm I lucky or what? ;-)
>
>
> --
> Best regards,
> Daniel
Somebody misinformed you.  Gentoo-amd64 rulez, the rest drule! ;-)  I run the 
unstable (~amd64) branch, and even that is rock solid.  Sure it breaks now 
and then, but that's why they call it unstable.  Your success with your first 
build is a testament to the great community we have here.  Welcome aboard!

glide
-- 
Mike Bonar
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] First Impressions

[Hemmann, Volker Armin]

hi,

On Wednesday 27 September 2006 19:35, Daniel Iliev wrote:

> >
> > Don't use that flags. They are bad for amd64. Trust the devs, they know
> > better than you or me.
>
> Thank you very much for this reply. I'll follow your advise and remove
> the redundant flags.
> About trusting the devs - well, it depends. I trust them 100% if I need
> a rock stable system. In that case I would not dare to divert from any
> of the official instructions.
> BUT. Devs always tent to advertise the safest ways - this brings users
> no/less headaches and therefore less bug-reports ;-)
> My case however is slightly different - I'm talking about my home
> desktop which is dedicated for experiments and fun. So I'm not afraid to
> break the system here and rebuild it again. In this particular moment my
> purpose is to get the most out of the hardware no matter the stability.
> So I'll leave -O3 ;-)

if you really want to be adventurous, try frename-registers, fpeel-loops, fweb 
or ftree-vectorize. Or ftracer. But don't complain, if something does not 
built (I have a set of save flags, just to be able to build freeciv. With my 
stupid standard flags, gcc will ICE)
>
> BTW, Everyone,
> I'm observing something very interesting:
> I was told not to go gentoo-amd64 for it was not stable. I was told not
> to migrate because there were still many important programs pending to
> be ported. I read almost everywhere about headaches and breakages.
> Reading your replies in this thread also suggests strictly following the
> official way otherwise - problems.
> It is very strange - I was ready to meet tons of major problems but I
> haven't met a single one yet. It is my opinion that the possibility of
> problems on gentoo-amd64 is highly overrated. I installed it with no
> problems, I obviously have tweaked it a lot beyond normal and what I see
> is a perfectly working system. It appears that gentoo-amd64 team along
> with the GNU, linux-kernel and all other nice guys who provide free/open
> source software have done a great work and we owe them BIG THANKS. I
> just wonder how come that so many people talk about some non-existing
> problems.
> How come that still in my first try I have bootstrapped from stage3,
> made "emerge -e system", installed xfce4, gnome, firefox, thunderbird,
> and a bunch of other packages along with all their dependencies, then
> made "emerge -e world" and after all this compiling I had to do "emerge
> --resume" only once when some package wanted mysql build with -fpic
> flag. I'm I lucky or what? ;-)
>

maybe. When I switched from gcc3 to gcc4 I had to -e world.. and from the 
hundreds of packages 14 broke. 10 of them package.masked.

The problem: people with problems are very vocal, while the users without 
probs are silent.

So will see always lots of complaints, but almost never success stories.

I have not any more problems with my ~amd64 system than I had with my ~x86 
system. I would even postulate, that I have seen less breakage on ~amd64.
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] First Impressions

[Boyd Stephen Smith Jr.]

[-- Attachment #1: Type: text/plain, Size: 2191 bytes --]

On Wednesday 27 September 2006 11:11, "Hemmann, Volker Armin" 
<volker.armin.hemmann@tu-clausthal.de> wrote about 'Re: [gentoo-amd64] 
First Impressions':
> -O3 don't do it. O2 is much, much safer and not really slower. It will
> prevent a lot of breakage.

Bah!

-O3 breaking just doesn't happen anymore.  I'm been running it for years.  
In fact -O3 has never generated invalid code [1] on x86 or amd64 
platforms.  It did for a period of time on ppc, but I think that was like 
10 years ago.

Also, some applications break with just -O2.  I count 8 ebuilds that 
replace this cflag specifically, there could be others that filter it or 
filter/replace optimization options entirely.  My methodology is available 
on request.

Now, it is an open question whether -O3 is significantly faster than -O2 
and depending on a number of factors it may actually be slower.  -O3 will 
enlarge code to make execution paths shorter, but a cache miss is probably 
goes to blow away any advantage gained.  Conversely, -Os will make 
execution paths longer to shrink code, and a cache hits may make up for 
the longer path. [2]

So, which is best depends on the program and on the CPU used (not only 
instruction set, but also cache sizes).  Therefore, making a sweeping 
recommendations is not recommended, but that doesn't stop anyone.  My 
recommendation is -O3. :)

Naturally, Gentoo wins over binary distros on this point, since you can use 
CFLAGS that are appropriate for your specific processor (including cflags 
which control the optimization tradeoff parameters) as well as tuning for 
the specific application.

-- 
"If there's one thing we've established over the years,
it's that the vast majority of our users don't have the slightest
clue what's best for them in terms of package stability."
-- Gentoo Developer Ciaran McCreesh

[1] That doesn't mean there weren't programs that crashed when compiled 
under -O3, just that the deficiency was found to be with the program not 
the compiler.

[2] But -Os may also break programs, and I've heard more problems on -Os 
than -O3 recently.  -Os had also had times when it generates invalid code.

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-amd64] First Impressions

[Boyd Stephen Smith Jr.]

[-- Attachment #1: Type: text/plain, Size: 753 bytes --]

On Wednesday 27 September 2006 13:52, "Hemmann, Volker Armin" 
<volker.armin.hemmann@tu-clausthal.de> wrote about 'Re: [gentoo-amd64] 
First Impressions':
> frename-registers, 

I have it, it's good especially for 64-bit code where we have a lot of 
registers.

> fweb

Ditto.

> ftracer.

I use it and...

> But don't complain, if   
> something does not built

I'm having no (compiler-related) problems so far, even on ~amd64.

I also like:
funit-at-a-time, funswitch-loops, and fprefetch-loop-arrays

-- 
"If there's one thing we've established over the years,
it's that the vast majority of our users don't have the slightest
clue what's best for them in terms of package stability."
-- Gentoo Developer Ciaran McCreesh

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-amd64] First Impressions

[Hemmann, Volker Armin]

On Thursday 28 September 2006 00:03, Boyd Stephen Smith Jr. wrote:
> On Wednesday 27 September 2006 13:52, "Hemmann, Volker Armin"
> <volker.armin.hemmann@tu-clausthal.de> wrote about 'Re: [gentoo-amd64]
>
> First Impressions':
> > frename-registers,
>
> I have it, it's good especially for 64-bit code where we have a lot of
> registers.
>
> > fweb
>
> Ditto.
>
> > ftracer.
>
> I use it and...

I too, but it is known to make trouble ;)

>
> > But don't complain, if
> > something does not built
>
> I'm having no (compiler-related) problems so far, even on ~amd64.

try freeciv ;)

>
> I also like:
> funit-at-a-time, funswitch-loops, and fprefetch-loop-arrays

not with 512kb cache ;)

Maybe, if I had more...
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] First Impressions

[Barry.SCHWARTZ]

[-- Attachment #1: Type: text/plain, Size: 525 bytes --]

"Boyd Stephen Smith Jr." <bss03@volumehost.net> skribis:
> I also like:
> funit-at-a-time, funswitch-loops, and fprefetch-loop-arrays

unit-at-a-time and (I believe) prefetch-loop-arrays are enabled by -O2
unswitch-loops is included in -O3



-- 
Barry.SCHWARTZ at chemoelectric.org   http://chemoelectric.org
        Free stuff / Senpagaj varoj:  http://crudfactory.com (PDF)
'Democracies don't war; democracies are peaceful countries.' - Bush
(http://www.whitehouse.gov/news/releases/2005/12/20051219-2.html)

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-amd64] First Impressions

[Boyd Stephen Smith Jr.]

[-- Attachment #1: Type: text/plain, Size: 668 bytes --]

On Wednesday 27 September 2006 17:40, "Hemmann, Volker Armin" 
<volker.armin.hemmann@tu-clausthal.de> wrote about 'Re: [gentoo-amd64] 
First Impressions':
> On Thursday 28 September 2006 00:03, Boyd Stephen Smith Jr. wrote:
> > I also like:
> > funit-at-a-time, funswitch-loops, and fprefetch-loop-arrays
>
> not with 512kb cache ;)

Do you not have separate D and I caches?  Cause fprefetch-loop-arrays won't 
hurt your I cache.

-- 
"If there's one thing we've established over the years,
it's that the vast majority of our users don't have the slightest
clue what's best for them in terms of package stability."
-- Gentoo Developer Ciaran McCreesh

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-amd64] First Impressions

[Boyd Stephen Smith Jr.]

[-- Attachment #1: Type: text/plain, Size: 563 bytes --]

On Wednesday 27 September 2006 18:25, Barry.SCHWARTZ@chemoelectric.org 
wrote about 'Re: [gentoo-amd64] First Impressions':
> unit-at-a-time and (I believe) prefetch-loop-arrays are enabled by -O2
> unswitch-loops is included in -O3

Well, I actually include them for when my -O3 is changed (by the ebuild) 
to -O2 or lower. >:)

-- 
"If there's one thing we've established over the years,
it's that the vast majority of our users don't have the slightest
clue what's best for them in terms of package stability."
-- Gentoo Developer Ciaran McCreesh

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-amd64] First Impressions

[Daniel Iliev]

Boyd Stephen Smith Jr. wrote:
> On Wednesday 27 September 2006 11:11, "Hemmann, Volker Armin" 
> <volker.armin.hemmann@tu-clausthal.de> wrote about 'Re: [gentoo-amd64] 
> First Impressions':
>   
>> -O3 don't do it. O2 is much, much safer and not really slower. It will
>> prevent a lot of breakage.
>>     
>
> Bah!
>
> -O3 breaking just doesn't happen anymore.  I'm been running it for years.  
> In fact -O3 has never generated invalid code [1] on x86 or amd64 
> platforms.  It did for a period of time on ppc, but I think that was like 
> 10 years ago.
>
> Also, some applications break with just -O2.  I count 8 ebuilds that 
> replace this cflag specifically, there could be others that filter it or 
> filter/replace optimization options entirely.  My methodology is available 
> on request.
>
> Now, it is an open question whether -O3 is significantly faster than -O2 
> and depending on a number of factors it may actually be slower.  -O3 will 
> enlarge code to make execution paths shorter, but a cache miss is probably 
> goes to blow away any advantage gained.  Conversely, -Os will make 
> execution paths longer to shrink code, and a cache hits may make up for 
> the longer path. [2]
>
> So, which is best depends on the program and on the CPU used (not only 
> instruction set, but also cache sizes).  Therefore, making a sweeping 
> recommendations is not recommended, but that doesn't stop anyone.  My 
> recommendation is -O3. :)
>
> Naturally, Gentoo wins over binary distros on this point, since you can use 
> CFLAGS that are appropriate for your specific processor (including cflags 
> which control the optimization tradeoff parameters) as well as tuning for 
> the specific application.
>
>   
I've always used "-O3". The only exception is busybox, when I compile it
for initrd/initramfs. In those cases I prefer "-Os". I've never seen a
package which explodes with "-O3" and compiles with "-O2". That's the
reason I hate to find that "replace-flags -O? -O2" stuff in the ebuilds.
I think I've read somewhere in *.gentoo.org that It should be reported
if a "testing" version works fine so it could get in the stable tree
faster. Should this "replace-flags" stuff be reported when the package
compiles successfully with "-O3"?


-- 
Best regards,
Daniel


-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] First Impressions

[Barry.SCHWARTZ]

[-- Attachment #1: Type: text/plain, Size: 851 bytes --]

"Boyd Stephen Smith Jr." <bss03@volumehost.net> skribis:
> Well, I actually include them for when my -O3 is changed (by the ebuild) 
> to -O2 or lower. >:)

That works for me.

I personally don’t worry much about optimizations unless I am working
on something that’s essentially assembly code, where I need the
compiler to generate a specific sort of code. Then I play with
optimizations to see what code gets generated. This has happened once
with amd64, when I wrote coexpression support for amd64 in the Unicon
language. (It’s a very tiny bit of code.)


-- 
Barry.SCHWARTZ at chemoelectric.org   http://chemoelectric.org
        Free stuff / Senpagaj varoj:  http://crudfactory.com (PDF)
'Democracies don't war; democracies are peaceful countries.' - Bush
(http://www.whitehouse.gov/news/releases/2005/12/20051219-2.html)

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-amd64] First Impressions

[Boyd Stephen Smith Jr.]

[-- Attachment #1: Type: text/plain, Size: 2243 bytes --]

On Wednesday 27 September 2006 19:56, Daniel Iliev <danny@ilievnet.com> 
wrote about 'Re: [gentoo-amd64] First Impressions':
> Boyd Stephen Smith Jr. wrote:
> > On Wednesday 27 September 2006 11:11, "Hemmann, Volker Armin"
> > <volker.armin.hemmann@tu-clausthal.de> wrote about 'Re: [gentoo-amd64]
> >
> > First Impressions':
> >> -O3 don't do it. O2 is much, much safer and not really slower. It
> >> will prevent a lot of breakage.
> > -O3 breaking just doesn't happen anymore.  
> > Also, some applications break with just -O2.
> > Now, it is an open question whether -O3 is significantly faster than 
> > -O2.  My
> > recommendation is -O3. :)
> I've always used "-O3".
> I think I've read somewhere in *.gentoo.org that It should be reported
> if a "testing" version works fine so it could get in the stable tree
> faster.

Please only file a stabilization bug if (a) there are no open bug reports 
on the version you want stabilized (b) all dependencies of the that 
version can be satisfied with stable packages on the ARCH you want 
stabilized AND (c) at least 30 days have passed since the ebuild for that 
version has been modified.

There has to be some time between an ebuild being available and it becoming 
stable to test on setups that aren't exactly like yours.  I know (some) 
people want things declared stable ASAP, but declaring a package stable 
doesn't make it so and there are (many, many) people that want stable 
Gentoo to actually be stable.  E.g. they do real work on the system. :P

> Should this "replace-flags" stuff be reported when the package 
> compiles successfully with "-O3"?

I fear you may find such bugs closed as WONTFIX/INVALID fairly quickly, 
especially if you started reporting en masse.  That said, if you have a 
compelling reason to use -O3 instead of what it is replaced with (not 
just "it works for me") for a specific package, I'd say file the bug, but 
be sure to include the reason(s) the replacement doesn't work for you.

-- 
"If there's one thing we've established over the years,
it's that the vast majority of our users don't have the slightest
clue what's best for them in terms of package stability."
-- Gentoo Developer Ciaran McCreesh

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-amd64] First Impressions

[Dan Pasanen]

[-- Attachment #1: Type: text/plain, Size: 2563 bytes --]

Ok, im not too sure about this, but i was wondering, i have dual core
turion64's should that affect if im running -O2 or -O3? currently im running
-O2.  and if i did change it to -O3, what, if anything should i recompile?

On 9/27/06, Boyd Stephen Smith Jr. <bss03@volumehost.net> wrote:
>
> On Wednesday 27 September 2006 19:56, Daniel Iliev <danny@ilievnet.com>
> wrote about 'Re: [gentoo-amd64] First Impressions':
> > Boyd Stephen Smith Jr. wrote:
> > > On Wednesday 27 September 2006 11:11, "Hemmann, Volker Armin"
> > > <volker.armin.hemmann@tu-clausthal.de> wrote about 'Re: [gentoo-amd64]
> > >
> > > First Impressions':
> > >> -O3 don't do it. O2 is much, much safer and not really slower. It
> > >> will prevent a lot of breakage.
> > > -O3 breaking just doesn't happen anymore.
> > > Also, some applications break with just -O2.
> > > Now, it is an open question whether -O3 is significantly faster than
> > > -O2.  My
> > > recommendation is -O3. :)
> > I've always used "-O3".
> > I think I've read somewhere in *.gentoo.org that It should be reported
> > if a "testing" version works fine so it could get in the stable tree
> > faster.
>
> Please only file a stabilization bug if (a) there are no open bug reports
> on the version you want stabilized (b) all dependencies of the that
> version can be satisfied with stable packages on the ARCH you want
> stabilized AND (c) at least 30 days have passed since the ebuild for that
> version has been modified.
>
> There has to be some time between an ebuild being available and it
> becoming
> stable to test on setups that aren't exactly like yours.  I know (some)
> people want things declared stable ASAP, but declaring a package stable
> doesn't make it so and there are (many, many) people that want stable
> Gentoo to actually be stable.  E.g. they do real work on the system. :P
>
> > Should this "replace-flags" stuff be reported when the package
> > compiles successfully with "-O3"?
>
> I fear you may find such bugs closed as WONTFIX/INVALID fairly quickly,
> especially if you started reporting en masse.  That said, if you have a
> compelling reason to use -O3 instead of what it is replaced with (not
> just "it works for me") for a specific package, I'd say file the bug, but
> be sure to include the reason(s) the replacement doesn't work for you.
>
> --
> "If there's one thing we've established over the years,
> it's that the vast majority of our users don't have the slightest
> clue what's best for them in terms of package stability."
> -- Gentoo Developer Ciaran McCreesh
>
>
>

[-- Attachment #2: Type: text/html, Size: 3217 bytes --]

Re: [gentoo-amd64] First Impressions

[Daniel Iliev]

Dan Pasanen wrote:
> Ok, im not too sure about this, but i was wondering, i have dual core
> turion64's should that affect if im running -O2 or -O3? currently im
> running -O2.  and if i did change it to -O3, what, if anything should
> i recompile?

>From gcc man page:

  -O3 Optimize yet more.  -O3 turns on all optimizations specified by -O2
           and also turns on the -finline-functions, -funswitch-loops and
           -fgcse-after-reload options.


So "-O3" is adding 3 more optimization flags to the bunch of flags "-O2"
brings. I thing it is reasonable to doubt that a program compiled with
"-O3" would greatly outperform the same program compiled with "-O2". The
same logic is applicable about stability. I have to admit that my logic
here is a plain arithmetic of type: "1 lemon - 10 cents, 10 lemons - 100
cents". There are programmers here who's opinion should be respected if
it differs from this logic.

-- 
Best regards,
Daniel


-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] First Impressions

[Daniel Iliev]

Boyd Stephen Smith Jr. wrote:
> On Wednesday 27 September 2006 19:56, Daniel Iliev <danny@ilievnet.com> 
> wrote about 'Re: [gentoo-amd64] First Impressions':
>   
>> Boyd Stephen Smith Jr. wrote:
>>     
>>> On Wednesday 27 September 2006 11:11, "Hemmann, Volker Armin"
>>> <volker.armin.hemmann@tu-clausthal.de> wrote about 'Re: [gentoo-amd64]
>>>
>>> First Impressions':
>>>       
>>>> -O3 don't do it. O2 is much, much safer and not really slower. It
>>>> will prevent a lot of breakage.
>>>>         
>>> -O3 breaking just doesn't happen anymore.  
>>> Also, some applications break with just -O2.
>>> Now, it is an open question whether -O3 is significantly faster than 
>>> -O2.  My
>>> recommendation is -O3. :)
>>>       
>> I've always used "-O3".
>> I think I've read somewhere in *.gentoo.org that It should be reported
>> if a "testing" version works fine so it could get in the stable tree
>> faster.
>>     
>
> Please only file a stabilization bug if (a) there are no open bug reports 
> on the version you want stabilized (b) all dependencies of the that 
> version can be satisfied with stable packages on the ARCH you want 
> stabilized AND (c) at least 30 days have passed since the ebuild for that 
> version has been modified.
>
> There has to be some time between an ebuild being available and it becoming 
> stable to test on setups that aren't exactly like yours.  I know (some) 
> people want things declared stable ASAP, but declaring a package stable 
> doesn't make it so and there are (many, many) people that want stable 
> Gentoo to actually be stable.  E.g. they do real work on the system. :P
>
>   
>> Should this "replace-flags" stuff be reported when the package 
>> compiles successfully with "-O3"?
>>     
>
> I fear you may find such bugs closed as WONTFIX/INVALID fairly quickly, 
> especially if you started reporting en masse.  That said, if you have a 
> compelling reason to use -O3 instead of what it is replaced with (not 
> just "it works for me") for a specific package, I'd say file the bug, but 
> be sure to include the reason(s) the replacement doesn't work for you.
>
>   

But of course! ;-)
I was just thinking if it would be theoretically right to classify the
use of "replace-flags" as a bug if it is not needed (and also not harming).
I don't think someone would really file a bug report for this It's true
that my OCD about optimizing becomes painful when I see my flags
overruled but I (still) can control myself ;-)
The thing I like most in Gentoo is that it gives many choices while its
maintenance still remains automated. If I wanted to choose and manually
change everything and everywhere I'd rather build myself an LFS [1] system.

[1] http://www.linuxfromscratch.org/

-- 
Best regards,
Daniel


-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] First Impressions

[Dice R. Random]

On 9/27/06, Daniel Iliev <danny@ilievnet.com> wrote:
> I don't think someone would really file a bug report for this It's true
> that my OCD about optimizing becomes painful when I see my flags
> overruled but I (still) can control myself ;-)

Your cflags really aren't going to change much.  It's better to stick
with something that's been tested by a large number of people and is
known to be safe.  For instance, consider the 3rd post in this forum
thread [1].  Even with an application like povray which is highly math
and CPU driven tossing in all kinds of crazy cflags only results in a
maximum of 0.05s (1.7%) improvement over the baseline -O2 -march
settings, and that sort of difference in times is almost certainly
within the margin of error.  Most of the cflags (including -O3)
actually decreased performance.

[1] http://forums.gentoo.org/viewtopic.php?p=613970
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] First Impressions

[Hemmann, Volker Armin]

On Thursday 28 September 2006 02:26, Boyd Stephen Smith Jr. wrote:
> On Wednesday 27 September 2006 17:40, "Hemmann, Volker Armin"
> <volker.armin.hemmann@tu-clausthal.de> wrote about 'Re: [gentoo-amd64]
>
> First Impressions':
> > On Thursday 28 September 2006 00:03, Boyd Stephen Smith Jr. wrote:
> > > I also like:
> > > funit-at-a-time, funswitch-loops, and fprefetch-loop-arrays
> >
> > not with 512kb cache ;)
>
> Do you not have separate D and I caches?  Cause fprefetch-loop-arrays won't
> hurt your I cache.

I am talking about L2 cache.
L1 has that destinction, L2 not ;)
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] First Impressions

[Boyd Stephen Smith Jr.]

[-- Attachment #1: Type: text/plain, Size: 2183 bytes --]

On Wednesday 27 September 2006 22:59, Daniel Iliev <danny@ilievnet.com> 
wrote about 'Re: [gentoo-amd64] First Impressions':
> Dan Pasanen wrote:
> > Ok, im not too sure about this, but i was wondering, i have dual core
> > turion64's should that affect if im running -O2 or -O3? currently im
> > running -O2.  and if i did change it to -O3, what, if anything should
> > i recompile?

Dual core or not won't affect your CFLAGS much.  Worry mostly about the 
size of your L1 cache, compared to processors of the same architecture 
being sold today.  If yours is fairly large, go with -O3; If undersized, 
go with -Os; Otherwise, -O2 will serve you well.

> I thing it is reasonable to doubt that a program compiled with
> "-O3" would greatly outperform the same program compiled with "-O2".

This is a reasonable doubt, for reasons already mentioned.

> The 
> same logic is applicable about stability. I have to admit that my logic
> here is a plain arithmetic of type: "1 lemon - 10 cents, 10 lemons - 100
> cents".

Well, optimization techniques are quite a bit more varied than lemons.

In particular, if you could only choose one optimization technique from the 
gcc info pages there is probably a "correct" choice that 90% of the gcc 
developers would agree on.  And guess what?  They probably put that one 
in -O1. :)

By the time you get to -O3, you are really just instructing the compiler on 
what tradeoffs to make. finline-functions and funswitch-loops are code 
size vs. execution path length tradeoffs and fgcse-after-reload is a [lots 
of] compilation time vs. [a little] (code size AND execution path length) 
tradeoff.

Again, the performance gained (or lost) from a given compiler optimization 
option can vary based on program and processor.  If you don't feel you can 
make a good decision based on reading the gcc info pages, don't include 
the option in your CFLAGS and just go with -O2 or -O3.

-- 
"If there's one thing we've established over the years,
it's that the vast majority of our users don't have the slightest
clue what's best for them in terms of package stability."
-- Gentoo Developer Ciaran McCreesh

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-amd64] First Impressions

[Paul de Vrieze]

Boyd Stephen Smith Jr. wrote:
> On Wednesday 27 September 2006 11:11, "Hemmann, Volker Armin" 
> <volker.armin.hemmann@tu-clausthal.de> wrote about 'Re: [gentoo-amd64] 
> First Impressions':
>> -O3 don't do it. O2 is much, much safer and not really slower. It will
>> prevent a lot of breakage.
> 
> Bah!
> 
> -O3 breaking just doesn't happen anymore.  I'm been running it for years.  
> In fact -O3 has never generated invalid code [1] on x86 or amd64 
> platforms.  It did for a period of time on ppc, but I think that was like 
> 10 years ago.

The thing is that besides compiler errors, there are many program errors 
out there that are not caused by optimization levels, just exposed. From 
the point of view of a user however it doesn't matter where the error 
is. It is an error, and thus should be fixed.

> Now, it is an open question whether -O3 is significantly faster than -O2 
> and depending on a number of factors it may actually be slower.  -O3 will 
> enlarge code to make execution paths shorter, but a cache miss is probably 
> goes to blow away any advantage gained.  Conversely, -Os will make 
> execution paths longer to shrink code, and a cache hits may make up for 
> the longer path. [2]

The main thing is that -O2 is the default at most places, and as such 
gets the most testing. That means that things generally work for -O2 
where -O3 or -Os exposes bugs in the software.

Paul
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] First Impressions

[Boyd Stephen Smith Jr.]

[-- Attachment #1: Type: text/plain, Size: 2228 bytes --]

On Thursday 28 September 2006 03:35, Paul de Vrieze <pauldv@gentoo.org> 
wrote about 'Re: [gentoo-amd64] First Impressions':
> Boyd Stephen Smith Jr. wrote:
> > On Wednesday 27 September 2006 11:11, "Hemmann, Volker Armin"
> > <volker.armin.hemmann@tu-clausthal.de> wrote about 'Re: [gentoo-amd64]
> >
> > First Impressions':
> >> -O3 don't do it.
> > -O3 breaking just doesn't happen anymore.
> The thing is that besides compiler errors, there are many program errors
> out there that are not caused by optimization levels, just exposed. From
> the point of view of a user however it doesn't matter where the error
> is. It is an error, and thus should be fixed.

Agreed, but the fix is not "don't use useful compiler features", 
it's "change the program source to not be erroneous".  Of course, there's 
also the short-term solution of "filter/replace flags for the few programs 
known to have issues".

It's wrong-headed to deride or discourage users for using the features of 
their compiler when those functions are not erroneous.  Instead, you 
should be leaning on the developers to fix the erroneous code.

> The main thing is that -O2 is the default at most places, and as such
> gets the most testing. That means that things generally work for -O2
> where -O3 or -Os exposes bugs in the software.

Again, there's no completely safe setting.  -O2 breaks stuff; -O3 breaks 
stuff; -Os breaks stuff.  None of them are due to a bad compiler.  I 
therefore urge people to use whatever optimization level they think is 
best and then file bugs if the package is broken.  (It is the package 
that's broken, not the compiler or the choice of CFLAGS.)

Also, surveying existing ebuilds, there are evidently quite a few more than 
I originally thought that force an optimization lower than -O2... around 
40.  There's actually about 14 that *force* -Os.  Well, drilling down I 
see that is just every postfix ebuild -- that could certainly be 
accumulated cruft.

-- 
"If there's one thing we've established over the years,
it's that the vast majority of our users don't have the slightest
clue what's best for them in terms of package stability."
-- Gentoo Developer Ciaran McCreesh

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-amd64] First Impressions

[Barry.SCHWARTZ]

[-- Attachment #1: Type: text/plain, Size: 744 bytes --]

"Boyd Stephen Smith Jr." <bss03@volumehost.net> skribis:
> It's wrong-headed to deride or discourage users for using the features of 
> their compiler when those functions are not erroneous.  Instead, you 
> should be leaning on the developers to fix the erroneous code.

This looks like a false dichotomy to me.

IMO people ought to use -O2 unless they are using a different setting
for some specific purpose, including but not limited to ‘having fun’.


-- 
Barry.SCHWARTZ at chemoelectric.org   http://chemoelectric.org
        Free stuff / Senpagaj varoj:  http://crudfactory.com (PDF)
'Democracies don't war; democracies are peaceful countries.' - Bush
(http://www.whitehouse.gov/news/releases/2005/12/20051219-2.html)

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-amd64] First Impressions

[Boyd Stephen Smith Jr.]

[-- Attachment #1: Type: text/plain, Size: 1491 bytes --]

On Thursday 28 September 2006 14:16, Barry.SCHWARTZ@chemoelectric.org wrote 
about 'Re: [gentoo-amd64] First Impressions':
> "Boyd Stephen Smith Jr." <bss03@volumehost.net> skribis:
> > It's wrong-headed to deride or discourage users for using the features
> > of their compiler when those functions are not erroneous.  Instead,
> > you should be leaning on the developers to fix the erroneous code.
>
> This looks like a false dichotomy to me.

But, it's not.  A piece of code it either conforms to a C/C++ standard the 
compiler implements or not.  This may be hard to determine but, it is a 
statement that is either true or false, absolutely.

If the code does not conform to the standard, the output of the compiler is 
undefined.  In particular, it is acceptable for the compiler produce 
errors or produce a binary that crashes.  The fault is with the code.

If the code does conform to the standard, the behavior of the output of the 
compiler is specified.  In particular, subject to resource limitations and 
hardware failure, the compile job must succeed and produce a binary that, 
again subject to resources and hardware, does not crash and performs the 
operations described in the source code.  The fault is with the compiler.

-- 
"If there's one thing we've established over the years,
it's that the vast majority of our users don't have the slightest
clue what's best for them in terms of package stability."
-- Gentoo Developer Ciaran McCreesh

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

[gentoo-amd64] Re: First Impressions

[Duncan]

Barry.SCHWARTZ@chemoelectric.org posted
20060928191638.GA5413@crud.crud.mn.org, excerpted below, on  Thu, 28 Sep
2006 14:16:38 -0500:

> "Boyd Stephen Smith Jr." <bss03@volumehost.net> skribis:
>> It's wrong-headed to deride or discourage users for using the features
>> of their compiler when those functions are not erroneous.  Instead, you
>> should be leaning on the developers to fix the erroneous code.
> 
> This looks like a false dichotomy to me.
> 
> IMO people ought to use -O2 unless they are using a different setting for
> some specific purpose, including but not limited to ‘having fun’.

Well stated.  One could compare -O2 to stable.  Fancier CFLAGS then are
the logical parallel of running ~arch or even masked packages.  If you
have a good reason to do so, including fun, and have the knowledge, time,
and interest to fix the occasional problems when they do occur, there's
nothing wrong with it, but those that just want something that works with
the least hassle are  better off with stable, and vanilla CFLAGS such as 
-march=k8 -pipe -O2 (only).

Happily, I'm in the group that likes the fun and has the
time/interest/developing-knowledge to run the fun stuff, and Gentoo makes
it easy, as it exposes all that power to my itchy little tweaking fingers!
=8^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: First Impressions

[Duncan]

"Boyd Stephen Smith Jr." <bss03@volumehost.net> posted
200609281452.44538.bss03@volumehost.net, excerpted below, on  Thu, 28 Sep
2006 14:52:40 -0500:

> If the code does not conform to the standard, the output of the compiler
> is undefined.  In particular, it is acceptable for the compiler produce
> errors or produce a binary that crashes.  The fault is with the code.
> 
> If the code does conform to the standard, the behavior of the output of
> the compiler is specified.  In particular, subject to resource limitations
> and hardware failure, the compile job must succeed and produce a binary
> that, again subject to resources and hardware, does not crash and performs
> the operations described in the source code.  The fault is with the
> compiler.

See, that's the problem, we don't live in a perfect world, and neither the
code nor the compiler are faultless.  Additionally, it's well known that
gcc is getting stricter (insisting the code hew closer to that standard
you mentioned), so stuff that once worked fine now doesn't, and fancy new
optimizations that cause code that again once worked fine to fail continue
to appear.  Since -O2 on a specific arch, with no further optimizations,
tends to be the most commonly tested, and few if any "new" optimizations
get invoked by it without at least a minor version span's testing first,
that's the most solid choice for those who prefer to let others deal with
all the "surprises".

Fortunately, some of us like the challenge of surprises, however, and run
these new optimizations, or it'd no longer be possible to widely test them
and get the worst kinks worked out of both the code and the compiler
optimizations before they could eventually be added to -O2 (or
-O<whatever> as appropriate).  There's nothing wrong with that, as long
as one realizes the risks involved and is willing to put in the work
necessary to overcome the obstacles when they popup.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] First Impressions

[Barry.SCHWARTZ]

[-- Attachment #1: Type: text/plain, Size: 1432 bytes --]

"Boyd Stephen Smith Jr." <bss03@volumehost.net> skribis:
> On Thursday 28 September 2006 14:16, Barry.SCHWARTZ@chemoelectric.org wrote 
> about 'Re: [gentoo-amd64] First Impressions':
> > "Boyd Stephen Smith Jr." <bss03@volumehost.net> skribis:
> > > It's wrong-headed to deride or discourage users for using the features
> > > of their compiler when those functions are not erroneous.  Instead,
> > > you should be leaning on the developers to fix the erroneous code.
> >
> > This looks like a false dichotomy to me.
> 
> But, it's not.  A piece of code it either conforms to a C/C++ standard the 
> compiler implements or not.  This may be hard to determine but, it is a 
> statement that is either true or false, absolutely.

Not that dichotomy, but the one that says it is wrongheaded to
discourage users (who may have no interest in encountering or fixing
bugs) from doing things that expose bugs, because developers should be
bothered to fix their code. You can do both.

Indeed, there aren’t many fields of ‘production’ in which end users
ought to be encouraged to try things that expose flaws in design or
manufacture.


-- 
Barry.SCHWARTZ at chemoelectric.org   http://chemoelectric.org
        Free stuff / Senpagaj varoj:  http://crudfactory.com (PDF)
'Democracies don't war; democracies are peaceful countries.' - Bush
(http://www.whitehouse.gov/news/releases/2005/12/20051219-2.html)

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Optimization flags (was Re: [gentoo-amd64] Re: First Impressions)

[Barry.SCHWARTZ]

[-- Attachment #1: Type: text/plain, Size: 807 bytes --]

Duncan <1i5t5.duncan@cox.net> skribis:
> Well stated.  One could compare -O2 to stable.

Which to me is a little weird, because only -O is something you can
count on from compiler to compiler. :)

There is one optimization level that, from my viewpoint, guarantees
bad code, and that’s no optimization, which breaks tail recursion. Of
course I realize that tail recursion optimization isn’t guaranteed by
C standards, but maybe tail recursion optimization should be a default
that you have to turn off explicitly. :)

-- 
Barry.SCHWARTZ at chemoelectric.org   http://chemoelectric.org
        Free stuff / Senpagaj varoj:  http://crudfactory.com (PDF)
'Democracies don't war; democracies are peaceful countries.' - Bush
(http://www.whitehouse.gov/news/releases/2005/12/20051219-2.html)

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-amd64] First Impressions

[Boyd Stephen Smith Jr.]

[-- Attachment #1: Type: text/plain, Size: 2467 bytes --]

On Thursday 28 September 2006 15:17, Barry.SCHWARTZ@chemoelectric.org wrote 
about 'Re: [gentoo-amd64] First Impressions':
> "Boyd Stephen Smith Jr." <bss03@volumehost.net> skribis:
> > On Thursday 28 September 2006 14:16, Barry.SCHWARTZ@chemoelectric.org
> > wrote
> >
> > about 'Re: [gentoo-amd64] First Impressions':
> > > "Boyd Stephen Smith Jr." <bss03@volumehost.net> skribis:
> > > > It's wrong-headed to deride or discourage users for using the
> > > > features of their compiler when those functions are not erroneous.
> > > >  Instead, you should be leaning on the developers to fix the
> > > > erroneous code.
> > > This looks like a false dichotomy to me.
> > A piece of code it either conforms to a C/C++ standard
> > the compiler implements or not.
> Not that dichotomy, but the one that says it is wrongheaded to
> discourage users (who may have no interest in encountering or fixing
> bugs) from doing things that expose bugs, because developers should be
> bothered to fix their code. You can do both.

Ooooh, sorry, yeah.  That's true.  Although, rather than saying "don't 
use -O3" to the user, I think it's better just to replace -O3 with -O2 in 
the ebuild until upstream fixes things.  I just don't think the user 
should ever feel breakage is their "fault" because they used advertised 
features of their compiler and headed the associated warnings.

Now, I have to problem with developers closing some bug opened by a ricer 
out there that's added -ffast-math to their CFLAGS as WONTFIX/INVALID 
since it's not really a problem.  All the parts provided by Gentoo 
(ebuild, source, compiler, portage, etc.) all behaving properly.  It's 
just that, just like the warning says, programs WILL break 
under -ffast-math.

> Indeed, there aren’t many fields of ‘production’ in which end users
> ought to be encouraged to try things that expose flaws in design or
> manufacture.

I disagree.  I want to know about those flaws, and I want the manufactures 
to fix them -- which is why I like Consumer Reports so much. :)

I encourage people to test the tensile strength of their Windows CDs and 
let me know at what point they break.  I'll collect and publish 
statistics. ;)

-- 
"If there's one thing we've established over the years,
it's that the vast majority of our users don't have the slightest
clue what's best for them in terms of package stability."
-- Gentoo Developer Ciaran McCreesh

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Optimizations (was Re: [gentoo-amd64] First Impressions)

[Barry.SCHWARTZ]

[-- Attachment #1: Type: text/plain, Size: 1606 bytes --]

"Boyd Stephen Smith Jr." <bss03@volumehost.net> skribis:
> Now, I have to problem with developers closing some bug opened by a ricer 
> out there that's added -ffast-math to their CFLAGS as WONTFIX/INVALID 
> since it's not really a problem.  All the parts provided by Gentoo 
> (ebuild, source, compiler, portage, etc.) all behaving properly.  It's 
> just that, just like the warning says, programs WILL break 
> under -ffast-math.

I have to disagree with this strongly, because -ffast-math means you
want code that violates the IEEE floating point standard.

> > Indeed, there aren’t many fields of ‘production’ in which end users
> > ought to be encouraged to try things that expose flaws in design or
> > manufacture.
> 
> I disagree.  I want to know about those flaws, and I want the manufactures 
> to fix them -- which is why I like Consumer Reports so much. :)

It’s one thing to read Consumer Reports and have it say your SUV rolls
over easily, and another to drive in a manner likely to roll your SUV
over. It’s even worse to sort of hold SUV drivers hostage by
encouraging them to roll over their SUVs unless and until the
manufacturer fixes the SUV. People should be encouraged to drive their
dangerous vehicles safely, and so should users be encouraged to set
their CFLAGS safely.

-- 
Barry.SCHWARTZ at chemoelectric.org   http://chemoelectric.org
        Free stuff / Senpagaj varoj:  http://crudfactory.com (PDF)
'Democracies don't war; democracies are peaceful countries.' - Bush
(http://www.whitehouse.gov/news/releases/2005/12/20051219-2.html)

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Optimizations (was Re: [gentoo-amd64] First Impressions)

[Boyd Stephen Smith Jr.]

[-- Attachment #1: Type: text/plain, Size: 2730 bytes --]

On Thursday 28 September 2006 16:17, Barry.SCHWARTZ@chemoelectric.org wrote 
about 'Optimizations (was Re: [gentoo-amd64] First Impressions)':
> "Boyd Stephen Smith Jr." <bss03@volumehost.net> skribis:
> > Now, I have to problem with developers closing some bug opened by a
> > ricer out there that's added -ffast-math to their CFLAGS as
> > WONTFIX/INVALID since it's not really a problem.  All the parts
> > provided by Gentoo (ebuild, source, compiler, portage, etc.) all
> > behaving properly.  It's just that, just like the warning says,
> > programs WILL break
> > under -ffast-math.
>
> I have to disagree with this strongly, because -ffast-math means you
> want code that violates the IEEE floating point standard.

Sorry, I mistyped "no" as "to" in the first sentence which completely 
changed my meaning.  I agree that -ffast-math ricers could have their bugs 
closed with prejudice.  -ffast-math is basically something that should 
only be allowed on a per-package basis, after consulting with upstream.

> > > Indeed, there aren’t many fields of ‘production’ in which end users
> > > ought to be encouraged to try things that expose flaws in design or
> > > manufacture.
> > I disagree.  I want to know about those flaws, and I want the
> > manufactures to fix them -- which is why I like Consumer Reports so
> > much. :)
> It’s one thing to read Consumer Reports and have it say your SUV rolls
> over easily, and another to drive in a manner likely to roll your SUV
> over. It’s even worse to sort of hold SUV drivers hostage by
> encouraging them to roll over their SUVs unless and until the
> manufacturer fixes the SUV. People should be encouraged to drive their
> dangerous vehicles safely, and so should users be encouraged to set
> their CFLAGS safely.

I encourage users to try to roll their SUVs safely. :)

If you are running a nuclear power plant off of Gentoo, then I hope you are 
going quite a bit further than just not using -O3.

However, using -O3 is usually dangerous to only the user themselves (at 
most) so I see no problem encouraging it.

That said, once somebody figures out (and publishes) that when you do X, Y, 
and Z to brand A SUV it rolls over.  I'd discourage doing X, Y, and Z to 
brand A SUVs (it'll cost you an SUV and really doesn't provide much 
benefit).  I'd encourage to X, Y, and Z to brand B, C, and D SUVs 
(*safely*) so that we can determine if the manufacturing flaw affects them 
as well.

-- 
"If there's one thing we've established over the years,
it's that the vast majority of our users don't have the slightest
clue what's best for them in terms of package stability."
-- Gentoo Developer Ciaran McCreesh

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]