RE: [gentoo-user] New computer and Gentoo

RE: [gentoo-user] New computer and Gentoo

[Pandu Poluan]

-original message-
Subject: Re: [gentoo-user] New computer and Gentoo
From: Bill Kenworthy <billk@iinet.net.au>
Date: 2011-07-21 12:54

>On Thu, 2011-07-21 at 06:26 +0100, Mick wrote:
>> On Thursday 21 Jul 2011 03:29:17 Adam Carter wrote:
>> > amd64 means any x86 64bit platform, so Intel too.
>> > 
>> > march=native is good if you're not using distcc, or if you're only
>> > using distcc on core2 boxes. Otherwise be specific.
>> 
>> I recommend using the "64 bit profile (amd64) for >= GCC 4.3" which shows -
>> march=core2.  This is what I use here with multilib and had no problems.
>
>Ive just stumbled on something weird with march=native:
>
>At some point I had march=prescott on a core2 E4600 running 32bit -
>worked well.  Changed to march=native and did some upgrades with a few
>odd things like asterisk segfaulting in a glibc library afterwards, and
>some things not building.  Then to add confusion, I changed to an
>pentium Duo E6600 (flies!) and added another stick of ram.  More odd
>things happening such as reiserfs oopsing on shutdown.
>
>Last night the penny dropped and I looked the new processor up and
>changed to march=core2 and have mostly corrected (recompiled) the
>damage.
>
>So not sure about march=native now as it is only what was built with
>native thats been problematic.  With 20-20 hindsight it was perhaps
>predictable ...

IMO you're not supposed to compile part of the system with -march=<something> and the rest with -march=native. The instructions (and optimizations) emitted by -march=native might not be compatible with your previous -march.

I had deployed more than 10 Gentoo servers using -march=native without any problem.

CMIIW

Rgds,
--
FdS Pandu E Poluan
~ IT Optimizer ~

Sent from Nokia E72-1

Re: [gentoo-user] New computer and Gentoo

[Florian Philipp]

[-- Attachment #1: Type: text/plain, Size: 2146 bytes --]

Am 21.07.2011 10:57, schrieb Pandu Poluan:
> -original message-
> Subject: Re: [gentoo-user] New computer and Gentoo
> From: Bill Kenworthy <billk@iinet.net.au>
> Date: 2011-07-21 12:54
> 
>> On Thu, 2011-07-21 at 06:26 +0100, Mick wrote:
[...]
>> Ive just stumbled on something weird with march=native:
>>
>> At some point I had march=prescott on a core2 E4600 running 32bit -
>> worked well.  Changed to march=native and did some upgrades with a few
>> odd things like asterisk segfaulting in a glibc library afterwards, and
>> some things not building.  Then to add confusion, I changed to an
>> pentium Duo E6600 (flies!) and added another stick of ram.  More odd
>> things happening such as reiserfs oopsing on shutdown.
>>
>> Last night the penny dropped and I looked the new processor up and
>> changed to march=core2 and have mostly corrected (recompiled) the
>> damage.
>>
>> So not sure about march=native now as it is only what was built with
>> native thats been problematic.  With 20-20 hindsight it was perhaps
>> predictable ...
> 
> IMO you're not supposed to compile part of the system with -march=<something>
> and the rest with -march=native. The instructions (and optimizations)
> emitted by -march=native might not be compatible with your previous
> -march.

I'd like to see a reference for this claim. -march=native doesn't do
more than set -march=core2 and some other optimizations for cache size
etc. This should be no more troublesome than mixing code compiled with
different specific -march settings. When you look at binary
distributions (and especially precompiled packages from the developer
instead of the distribution), this is pretty much normal.

The compiler is not allowed to change the external interfaces of
functions for optimization purposes (see [1]). Besides this, I can only
think of alignment problems ([2]) but even this should be handled
correctly by the compiler. Everything else is a compiler bug that should
be reported.

[1] http://en.wikipedia.org/wiki/Calling_convention
[2] http://en.wikipedia.org/wiki/Data_alignment

Regards,
Florian Philipp


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: [gentoo-user] New computer and Gentoo

[Pandu Poluan]

On Thu, Jul 21, 2011 at 16:21, Florian Philipp <lists@binarywings.net> wrote:
>
> Am 21.07.2011 10:57, schrieb Pandu Poluan:
> > -original message-
> > Subject: Re: [gentoo-user] New computer and Gentoo
> > From: Bill Kenworthy <billk@iinet.net.au>
> > Date: 2011-07-21 12:54
> >
> >> On Thu, 2011-07-21 at 06:26 +0100, Mick wrote:
> [...]
> >> Ive just stumbled on something weird with march=native:
> >>
> >> At some point I had march=prescott on a core2 E4600 running 32bit -
> >> worked well.  Changed to march=native and did some upgrades with a few
> >> odd things like asterisk segfaulting in a glibc library afterwards, and
> >> some things not building.  Then to add confusion, I changed to an
> >> pentium Duo E6600 (flies!) and added another stick of ram.  More odd
> >> things happening such as reiserfs oopsing on shutdown.
> >>
> >> Last night the penny dropped and I looked the new processor up and
> >> changed to march=core2 and have mostly corrected (recompiled) the
> >> damage.
> >>
> >> So not sure about march=native now as it is only what was built with
> >> native thats been problematic.  With 20-20 hindsight it was perhaps
> >> predictable ...
> >
> > IMO you're not supposed to compile part of the system with -march=<something>
> > and the rest with -march=native. The instructions (and optimizations)
> > emitted by -march=native might not be compatible with your previous
> > -march.
>
> I'd like to see a reference for this claim. -march=native doesn't do
> more than set -march=core2 and some other optimizations for cache size
> etc. This should be no more troublesome than mixing code compiled with
> different specific -march settings. When you look at binary
> distributions (and especially precompiled packages from the developer
> instead of the distribution), this is pretty much normal.
>
> The compiler is not allowed to change the external interfaces of
> functions for optimization purposes (see [1]). Besides this, I can only
> think of alignment problems ([2]) but even this should be handled
> correctly by the compiler. Everything else is a compiler bug that should
> be reported.
>
> [1] http://en.wikipedia.org/wiki/Calling_convention
> [2] http://en.wikipedia.org/wiki/Data_alignment
>

Hmmm... You do have a point. So, gcc's "-march" option *should*
maintain ABI compatibility.

There *are* other switches that can impact compatibility [1]. What I
can't be sure of, whether the "-march" option (improperly) activates
one or more of those switches when set to "-march=native"

[1] http://gcc.gnu.org/onlinedocs/gcc/i386-and-x86_002d64-Options.html

Rgds,
--
Pandu E Poluan
~ IT Optimizer ~

Blog : http://pepoluan.tumblr.com • Linked-In :
http://id.linkedin.com/in/pepoluan

Re: [gentoo-user] New computer and Gentoo

[Florian Philipp]

[-- Attachment #1: Type: text/plain, Size: 2816 bytes --]

Am 21.07.2011 11:21, schrieb Florian Philipp:
> Am 21.07.2011 10:57, schrieb Pandu Poluan:
>> -original message-
>> Subject: Re: [gentoo-user] New computer and Gentoo
>> From: Bill Kenworthy <billk@iinet.net.au>
>> Date: 2011-07-21 12:54
>>
>>> On Thu, 2011-07-21 at 06:26 +0100, Mick wrote:
> [...]
>>> Ive just stumbled on something weird with march=native:
>>>
>>> At some point I had march=prescott on a core2 E4600 running 32bit -
>>> worked well.  Changed to march=native and did some upgrades with a few
>>> odd things like asterisk segfaulting in a glibc library afterwards, and
>>> some things not building.  Then to add confusion, I changed to an
>>> pentium Duo E6600 (flies!) and added another stick of ram.  More odd
>>> things happening such as reiserfs oopsing on shutdown.
>>>
>>> Last night the penny dropped and I looked the new processor up and
>>> changed to march=core2 and have mostly corrected (recompiled) the
>>> damage.
>>>
>>> So not sure about march=native now as it is only what was built with
>>> native thats been problematic.  With 20-20 hindsight it was perhaps
>>> predictable ...
>>
>> IMO you're not supposed to compile part of the system with -march=<something>
>> and the rest with -march=native. The instructions (and optimizations)
>> emitted by -march=native might not be compatible with your previous
>> -march.
> 
> I'd like to see a reference for this claim. -march=native doesn't do
> more than set -march=core2 and some other optimizations for cache size
> etc. This should be no more troublesome than mixing code compiled with
> different specific -march settings. When you look at binary
> distributions (and especially precompiled packages from the developer
> instead of the distribution), this is pretty much normal.
> 
> The compiler is not allowed to change the external interfaces of
> functions for optimization purposes (see [1]). Besides this, I can only
> think of alignment problems ([2]) but even this should be handled
> correctly by the compiler. Everything else is a compiler bug that should
> be reported.
> 
> [1] http://en.wikipedia.org/wiki/Calling_convention
> [2] http://en.wikipedia.org/wiki/Data_alignment
> 
> Regards,
> Florian Philipp
> 

I've just checked it by comparing the output of `gcc -v -Q -O2
-march=native test.c` with `gcc -v -Q -O2 -march=core2 test.c`:

-march=native is resolved to:
-march=core2 -mcx16 -msahf -mpopcnt -msse4.2 --param l1-cache-size=32
--param l1-cache-line-size=64 --param l2-cache-size=256 -mtune=core2

I also see that the -march=core2 version has the following options
enabled: -mno-sse4
whereas -march=native enabled: -mpopcnt -msse4 -msse4.1 -msse4.2

-mssse3 and lower SSE versions were enabled in both cases.

Regards,
Florian Philipp


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: [gentoo-user] New computer and Gentoo

[William Kenworthy]

On Thu, 2011-07-21 at 11:21 +0200, Florian Philipp wrote:
> Am 21.07.2011 10:57, schrieb Pandu Poluan:
> > -original message-
> > Subject: Re: [gentoo-user] New computer and Gentoo
> > From: Bill Kenworthy <billk@iinet.net.au>
> > Date: 2011-07-21 12:54
> > 
> >> On Thu, 2011-07-21 at 06:26 +0100, Mick wrote:
> [...]
> >> Ive just stumbled on something weird with march=native:
> >>

> 
> I'd like to see a reference for this claim. -march=native doesn't do
> more than set -march=core2 and some other optimizations for cache size
> etc. This should be no more troublesome than mixing code compiled with

unfortunately this is now my main machine so I cant fiddle too much with
it!  It would mean going back to the E4600 and comparing march=prescott,
march=native then fitting the E6600 and checking march=native and
march=core2.  What I cant find is a reference to how it works out what
native is? - lookup-table, checking the flags in /proc/cpuinfo or what?

Ive now rolled back (that is recompiled) the majority of packages so now
I can keep working while it does an emerge -e world.

I would have thought the two intel processors would be close enough that
it would be just a performance hit and not segfaults, but the machine is
now working reliably so thats proof enough for me.  What I am having
difficulty with is that packages compiled with native should have been a
closer match to the cpu so why was it those packages (asterisk, glibc
and some random others cause problems.

BillK

Re: [gentoo-user] New computer and Gentoo

[Florian Philipp]

[-- Attachment #1: Type: text/plain, Size: 1725 bytes --]

Am 21.07.2011 15:10, schrieb William Kenworthy:
> On Thu, 2011-07-21 at 11:21 +0200, Florian Philipp wrote:
>> Am 21.07.2011 10:57, schrieb Pandu Poluan:
>>> -original message-
>>> Subject: Re: [gentoo-user] New computer and Gentoo
>>> From: Bill Kenworthy <billk@iinet.net.au>
>>> Date: 2011-07-21 12:54
>>>
>>>> On Thu, 2011-07-21 at 06:26 +0100, Mick wrote:
>> [...]
>>>> Ive just stumbled on something weird with march=native:
>>>>
> 
>>
>> I'd like to see a reference for this claim. -march=native doesn't do
>> more than set -march=core2 and some other optimizations for cache size
>> etc. This should be no more troublesome than mixing code compiled with
> 
> unfortunately this is now my main machine so I cant fiddle too much with
> it!  It would mean going back to the E4600 and comparing march=prescott,
> march=native then fitting the E6600 and checking march=native and
> march=core2.  What I cant find is a reference to how it works out what
> native is? - lookup-table, checking the flags in /proc/cpuinfo or what?
> 
[...]

Use the source, Luke. Take a look at the gcc sources in your distfiles
directory. It is in the gcc package, file
"./gcc/config/i386/driver-i386.c" (found via grep for "\<native" and
then grep for "host_detect_local_cpu"). It is surprisingly understandable.

Most of the detection is done by introspecting CPUID. Most importantly,
the CPU family is deducted from the found capabilities, not vice versa.
Therefore there is no chance that a CPU that claims to be, let's say, a
Core2, but lacks some capabilities, ends up being classified as a Core2
(unless, of course, if the GCC guys get their switch-case logic wrong).

Regards,
Florian Philipp


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: [gentoo-user] New computer and Gentoo

[Stroller]

On 22 July 2011, at 07:45, Florian Philipp wrote:
> Every "native" setting is just resolved by gcc at compile time to some
> concrete setting like "core2". 

Perfectly correct.

On 21 July 2011, at 15:05, Florian Philipp wrote:
> ...
> (unless, of course, if the GCC guys get their switch-case logic wrong).

Unfortunately, this has been known to happen. 

See GCC bug 48743.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48743

Admittedly the affected CPU in that case is somewhat older, dated and unfashionable.
Hopefully such errors will be less common on newer CPUs, but we shouldn't rule it out.

Stroller.

[gentoo-user] New computer and Gentoo

[CJoeB]

Hi everyone,

Today, I ordered a new desktop from Dell (offer too good to pass up!). 
The system is a Dell XPS 8300 with an Intel Core i7 processor.  I was
reading the Gentoo wiki about safe CFLAGS and it said that march=native
is recommended if I use gcc >= 4.2.3.

I looked at processor specific CFLAGS and if I am understanding this
correctly, for an Intel i7, I would use march=prescott for a 32 bit OS. 
It also mentions march=core2 if using gcc > 4.3 for a 64-bit OS. 
However, it has amd64 in brackets.  So would this be for an amd system?

Should I stick with march="native"?

Advice would be appreciated. 

Regards,

Colleen


-- 

Registered Linux User #411143 with the Linux Counter, http://counter.li.org

Re: [gentoo-user] New computer and Gentoo

[Adam Carter]

amd64 means any x86 64bit platform, so Intel too.

march=native is good if you're not using distcc, or if you're only
using distcc on core2 boxes. Otherwise be specific.

Re: [gentoo-user] New computer and Gentoo

[Mick]

[-- Attachment #1: Type: Text/Plain, Size: 409 bytes --]

On Thursday 21 Jul 2011 03:29:17 Adam Carter wrote:
> amd64 means any x86 64bit platform, so Intel too.
> 
> march=native is good if you're not using distcc, or if you're only
> using distcc on core2 boxes. Otherwise be specific.

I recommend using the "64 bit profile (amd64) for >= GCC 4.3" which shows -
march=core2.  This is what I use here with multilib and had no problems.
-- 
Regards,
Mick

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] New computer and Gentoo

[Bill Kenworthy]

On Thu, 2011-07-21 at 06:26 +0100, Mick wrote:
> On Thursday 21 Jul 2011 03:29:17 Adam Carter wrote:
> > amd64 means any x86 64bit platform, so Intel too.
> > 
> > march=native is good if you're not using distcc, or if you're only
> > using distcc on core2 boxes. Otherwise be specific.
> 
> I recommend using the "64 bit profile (amd64) for >= GCC 4.3" which shows -
> march=core2.  This is what I use here with multilib and had no problems.

Ive just stumbled on something weird with march=native:

At some point I had march=prescott on a core2 E4600 running 32bit -
worked well.  Changed to march=native and did some upgrades with a few
odd things like asterisk segfaulting in a glibc library afterwards, and
some things not building.  Then to add confusion, I changed to an
pentium Duo E6600 (flies!) and added another stick of ram.  More odd
things happening such as reiserfs oopsing on shutdown.

Last night the penny dropped and I looked the new processor up and
changed to march=core2 and have mostly corrected (recompiled) the
damage.

So not sure about march=native now as it is only what was built with
native thats been problematic.  With 20-20 hindsight it was perhaps
predictable ...

BillK