[gentoo-user] gcc fails and then succeeds - definitely a problem?

[gentoo-user] gcc fails and then succeeds - definitely a problem?

[Grant]

The gcc update just failed to compile on one of my systems with a
segfault, but then succeeded after trying again even though I didn't
change anything.  Does that indicate a hardware problem for sure?
Should I run memtester?  Any other tests to run?  Nothing in dmesg.

- Grant

Re: [gentoo-user] gcc fails and then succeeds - definitely a problem?

[Michael Mol]

On Thu, Feb 23, 2012 at 2:17 PM, Grant <emailgrant@gmail.com> wrote:
> The gcc update just failed to compile on one of my systems with a
> segfault, but then succeeded after trying again even though I didn't
> change anything.  Does that indicate a hardware problem for sure?
> Should I run memtester?  Any other tests to run?  Nothing in dmesg.

Not definitively anything; it could have been a race condition.

Memtest if you like. prime95 is designed for CPU and memory burning,
too, and wouldn't require you to shutdown your system.

-- 
:wq

Re: [gentoo-user] gcc fails and then succeeds - definitely a problem?

[Grant]

>> The gcc update just failed to compile on one of my systems with a
>> segfault, but then succeeded after trying again even though I didn't
>> change anything.  Does that indicate a hardware problem for sure?
>> Should I run memtester?  Any other tests to run?  Nothing in dmesg.
>
> Not definitively anything; it could have been a race condition.
>
> Memtest if you like. prime95 is designed for CPU and memory burning,
> too, and wouldn't require you to shutdown your system.

Thanks everyone.  I ran memtester for a little bit and it came up with
this before I killed it:

# memtester 14000
memtester version 4.0.8 (64-bit)
Copyright (C) 2007 Charles Cazabon.
Licensed under the GNU General Public License version 2 (only).

pagesize is 4096
pagesizemask is 0xfffffffffffff000
want 14000MB (14680064000 bytes)
got  14000MB (14680064000 bytes), trying mlock ...locked.
Loop 1:
  Stuck Address       : ok
  Random Value        : ok
FAILURE: 0x524e8edb0512f3a7 != 0x524ecedb0512f3a7 at offset 0x04bd5130.
FAILURE: 0x224c0b76048d37c0 != 0x224c4b76048d37c0 at offset 0x0de17970.
FAILURE: 0x207dad0b8c3aced0 != 0x207ded0b8c3aced0 at offset 0x0de36970.
FAILURE: 0x847e610e840fb84e != 0x847e210e840fb84e at offset 0x1dc7922f.
FAILURE: 0x3f69916b940c7907 != 0x3f69d16b940c7907 at offset 0x1ed37770.
  Compare XOR         : FAILURE: 0x13664bb2c7a58ca3 !=
0x13668bb2c7a58ca3 at offset 0x04bd5130.
FAILURE: 0x61bcd9d27eba2967 != 0x61bd19d27eba2967 at offset 0x0686b930.
FAILURE: 0xe363c84dc71fd0bc != 0xe364084dc71fd0bc at offset 0x0de17970.
FAILURE: 0xe19569e34ecd67cc != 0xe195a9e34ecd67cc at offset 0x0de36970.
FAILURE: 0x7b844f40969fc496 != 0x7b848f40969fc496 at offset 0x0de94930.
FAILURE: 0x45961de646a2514a != 0x4595dde646a2514a at offset 0x1dc7922f.
FAILURE: 0x67e4594142a19ffa != 0x67e4994142a19ffa at offset 0x1ea14730.
FAILURE: 0x8341dc6542a103ab != 0x83421c6542a103ab at offset 0x1ecd4730.
FAILURE: 0x814e43569f1203 != 0x818e43569f1203 at offset 0x1ed37770.
  Compare SUB         : FAILURE: 0x1082d4779192eec4 !=
0xefbfd4779192eec4 at offset 0x02d10930.
FAILURE: 0xad2dd70ca745ff5c != 0x8c6ad70ca745ff5c at offset 0x04bd5130.
FAILURE: 0x189f6452fe165a2c != 0xf7dc6452fe165a2c at offset 0x0686b930.
FAILURE: 0xc9ac41a7eab20330 != 0xa8e941a7eab20330 at offset 0x0de17970.
FAILURE: 0x1b9b05b99a41be70 != 0xfad805b99a41be70 at offset 0x0de36970.
FAILURE: 0x300cb2e02ea06f8 != 0xe23dcb2e02ea06f8 at offset 0x0de94930.
FAILURE: 0xb29086ae7fdf2d4 != 0xea66086ae7fdf2d4 at offset 0x0e1c5970.
FAILURE: 0x89126e3b0ccb5288 != 0xa9d56e3b0ccb5288 at offset 0x1dc7922f.
FAILURE: 0x4d7afcf6378f9248 != 0x2cb7fcf6378f9248 at offset 0x1ea14730.
FAILURE: 0x5a9034aa259352fc != 0x39cd34aa259352fc at offset 0x1ecd4730.
FAILURE: 0x7b1c0d3184539edc != 0x5a590d3184539edc at offset 0x1ed37770.
  Compare MUL         : FAILURE: 0x00000000 != 0x00000001 at offset 0x0686b930.
FAILURE: 0x00000000 != 0x00000001 at offset 0x0de36970.
  Compare DIV         :   Compare OR          : ok
  Compare AND         : ok
  Sequential Increment: ok
  Solid Bits          : testing  29

Now I've emerged gimps and I'm running the mprime "Blend" stress test
so we'll see what that turns up.

- Grant

Re: [gentoo-user] gcc fails and then succeeds - definitely a problem?

[Grant]

>>> The gcc update just failed to compile on one of my systems with a
>>> segfault, but then succeeded after trying again even though I didn't
>>> change anything.  Does that indicate a hardware problem for sure?
>>> Should I run memtester?  Any other tests to run?  Nothing in dmesg.
>>
>> Not definitively anything; it could have been a race condition.
>>
>> Memtest if you like. prime95 is designed for CPU and memory burning,
>> too, and wouldn't require you to shutdown your system.
>
> Thanks everyone.  I ran memtester for a little bit and it came up with
> this before I killed it:
>
> # memtester 14000
> memtester version 4.0.8 (64-bit)
> Copyright (C) 2007 Charles Cazabon.
> Licensed under the GNU General Public License version 2 (only).
>
> pagesize is 4096
> pagesizemask is 0xfffffffffffff000
> want 14000MB (14680064000 bytes)
> got  14000MB (14680064000 bytes), trying mlock ...locked.
> Loop 1:
>  Stuck Address       : ok
>  Random Value        : ok
> FAILURE: 0x524e8edb0512f3a7 != 0x524ecedb0512f3a7 at offset 0x04bd5130.
> FAILURE: 0x224c0b76048d37c0 != 0x224c4b76048d37c0 at offset 0x0de17970.
> FAILURE: 0x207dad0b8c3aced0 != 0x207ded0b8c3aced0 at offset 0x0de36970.
> FAILURE: 0x847e610e840fb84e != 0x847e210e840fb84e at offset 0x1dc7922f.
> FAILURE: 0x3f69916b940c7907 != 0x3f69d16b940c7907 at offset 0x1ed37770.
>  Compare XOR         : FAILURE: 0x13664bb2c7a58ca3 !=
> 0x13668bb2c7a58ca3 at offset 0x04bd5130.
> FAILURE: 0x61bcd9d27eba2967 != 0x61bd19d27eba2967 at offset 0x0686b930.
> FAILURE: 0xe363c84dc71fd0bc != 0xe364084dc71fd0bc at offset 0x0de17970.
> FAILURE: 0xe19569e34ecd67cc != 0xe195a9e34ecd67cc at offset 0x0de36970.
> FAILURE: 0x7b844f40969fc496 != 0x7b848f40969fc496 at offset 0x0de94930.
> FAILURE: 0x45961de646a2514a != 0x4595dde646a2514a at offset 0x1dc7922f.
> FAILURE: 0x67e4594142a19ffa != 0x67e4994142a19ffa at offset 0x1ea14730.
> FAILURE: 0x8341dc6542a103ab != 0x83421c6542a103ab at offset 0x1ecd4730.
> FAILURE: 0x814e43569f1203 != 0x818e43569f1203 at offset 0x1ed37770.
>  Compare SUB         : FAILURE: 0x1082d4779192eec4 !=
> 0xefbfd4779192eec4 at offset 0x02d10930.
> FAILURE: 0xad2dd70ca745ff5c != 0x8c6ad70ca745ff5c at offset 0x04bd5130.
> FAILURE: 0x189f6452fe165a2c != 0xf7dc6452fe165a2c at offset 0x0686b930.
> FAILURE: 0xc9ac41a7eab20330 != 0xa8e941a7eab20330 at offset 0x0de17970.
> FAILURE: 0x1b9b05b99a41be70 != 0xfad805b99a41be70 at offset 0x0de36970.
> FAILURE: 0x300cb2e02ea06f8 != 0xe23dcb2e02ea06f8 at offset 0x0de94930.
> FAILURE: 0xb29086ae7fdf2d4 != 0xea66086ae7fdf2d4 at offset 0x0e1c5970.
> FAILURE: 0x89126e3b0ccb5288 != 0xa9d56e3b0ccb5288 at offset 0x1dc7922f.
> FAILURE: 0x4d7afcf6378f9248 != 0x2cb7fcf6378f9248 at offset 0x1ea14730.
> FAILURE: 0x5a9034aa259352fc != 0x39cd34aa259352fc at offset 0x1ecd4730.
> FAILURE: 0x7b1c0d3184539edc != 0x5a590d3184539edc at offset 0x1ed37770.
>  Compare MUL         : FAILURE: 0x00000000 != 0x00000001 at offset 0x0686b930.
> FAILURE: 0x00000000 != 0x00000001 at offset 0x0de36970.
>  Compare DIV         :   Compare OR          : ok
>  Compare AND         : ok
>  Sequential Increment: ok
>  Solid Bits          : testing  29
>
> Now I've emerged gimps and I'm running the mprime "Blend" stress test
> so we'll see what that turns up.
>
> - Grant

mprime ran for about 1.5 hours until it found this:

[Work thread Feb 23 13:04] FATAL ERROR: Rounding was 0.5, expected less than 0.4
[Work thread Feb 23 13:04] Hardware failure detected, consult stress.txt file.
[Work thread Feb 23 13:04] Torture Test completed 85 tests in 1 hour,
33 minutes - 1 errors, 0 warnings.
[Work thread Feb 23 13:04] Worker stopped.
[Main thread Feb 23 13:04] Execution halted.

I have a 1200 watt Corsair power supply and my temps are very low even
during the stress test so I'm thinking bad (Corsair) RAM.  I should
remove modules one at a time and re-test to narrow it down?

- Grant

Re: [gentoo-user] gcc fails and then succeeds - definitely a problem?

[Mark Knecht]

On Thu, Feb 23, 2012 at 1:11 PM, Grant <emailgrant@gmail.com> wrote:
<SNIP>
>
> mprime ran for about 1.5 hours until it found this:
>
> [Work thread Feb 23 13:04] FATAL ERROR: Rounding was 0.5, expected less than 0.4
> [Work thread Feb 23 13:04] Hardware failure detected, consult stress.txt file.
> [Work thread Feb 23 13:04] Torture Test completed 85 tests in 1 hour,
> 33 minutes - 1 errors, 0 warnings.
> [Work thread Feb 23 13:04] Worker stopped.
> [Main thread Feb 23 13:04] Execution halted.
>
> I have a 1200 watt Corsair power supply and my temps are very low even
> during the stress test so I'm thinking bad (Corsair) RAM.  I should
> remove modules one at a time and re-test to narrow it down?
>
> - Grant
>

If it's a modern machine then most of the memory channels are 2 or
even 3 DIMM's wide. Consult you manual as to whether you can run less
than pairs or DIMMS.

If you have 4 DIMM's installed then I'd consider removing two, testing
two, testing the other two, and then if you see a problem testing all
the combinations until you figure out which one is causing the error.

Good luck,
Mark

Re: [gentoo-user] gcc fails and then succeeds - definitely a problem?

[Alex Schuster]

Grant writes:

> I have a 1200 watt Corsair power supply and my temps are very low even
> during the stress test so I'm thinking bad (Corsair) RAM.  I should
> remove modules one at a time and re-test to narrow it down?

This sounds just like the right thing to do. Well, if you have four RAM
chips, remove two at once to speed up the diagnosis.

	Wonko

[gentoo-user] Re: gcc fails and then succeeds - definitely a problem?

[Heorhi Valakhanovich]

On Thu, 23 Feb 2012 11:17:54 -0800
Grant <emailgrant@gmail.com> wrote:

> The gcc update just failed to compile on one of my systems with a
> segfault, but then succeeded after trying again even though I didn't
> change anything.  Does that indicate a hardware problem for sure?
> Should I run memtester?  Any other tests to run?  Nothing in dmesg.
> 
> - Grant
> 
> 

Building gcc usually requires large amount of memory. May be you
haven't enough first time.

Re: [gentoo-user] gcc fails and then succeeds - definitely a problem?

[Mark Knecht]

On Thu, Feb 23, 2012 at 11:17 AM, Grant <emailgrant@gmail.com> wrote:
> The gcc update just failed to compile on one of my systems with a
> segfault, but then succeeded after trying again even though I didn't
> change anything.  Does that indicate a hardware problem for sure?
> Should I run memtester?  Any other tests to run?  Nothing in dmesg.
>
> - Grant
>

Might be.....might be nothing. Maybe a stray neutrino hit your
processor at just the wrong instant. ;-)

More likely i my mind is some little corner condition in the software
running on your system. I've had the same thing happen many times
actually, and actually a few more times since I started playing with
your /etc/make.conf -j/-l values which push the system a little
harder. I wouldn't personally file a bug or even worry about it much
unless it becomes a common occurrence.

HTH,
Mark

Re: [gentoo-user] gcc fails and then succeeds - definitely a problem?

[Michael Mol]

On Thu, Feb 23, 2012 at 2:28 PM, Mark Knecht <markknecht@gmail.com> wrote:
> On Thu, Feb 23, 2012 at 11:17 AM, Grant <emailgrant@gmail.com> wrote:
>> The gcc update just failed to compile on one of my systems with a
>> segfault, but then succeeded after trying again even though I didn't
>> change anything.  Does that indicate a hardware problem for sure?
>> Should I run memtester?  Any other tests to run?  Nothing in dmesg.
>>
>> - Grant
>>
>
> Might be.....might be nothing. Maybe a stray neutrino hit your
> processor at just the wrong instant. ;-)
>
> More likely i my mind is some little corner condition in the software
> running on your system. I've had the same thing happen many times
> actually, and actually a few more times since I started playing with
> your /etc/make.conf -j/-l values which push the system a little
> harder.

Whenever I get build failures with the load-adaptive MAKEOPTS and
EMERGE_DEFAULT_OPTS, I check the build log to see if it's relatively
obvious that something was depended upon before it was built. If so, I
file a bug.

Happens every month or so, for me.

-- 
:wq

Re: [gentoo-user] gcc fails and then succeeds - definitely a problem?

[Mark Knecht]

On Thu, Feb 23, 2012 at 11:36 AM, Michael Mol <mikemol@gmail.com> wrote:
> On Thu, Feb 23, 2012 at 2:28 PM, Mark Knecht <markknecht@gmail.com> wrote:
>> On Thu, Feb 23, 2012 at 11:17 AM, Grant <emailgrant@gmail.com> wrote:
>>> The gcc update just failed to compile on one of my systems with a
>>> segfault, but then succeeded after trying again even though I didn't
>>> change anything.  Does that indicate a hardware problem for sure?
>>> Should I run memtester?  Any other tests to run?  Nothing in dmesg.
>>>
>>> - Grant
>>>
>>
>> Might be.....might be nothing. Maybe a stray neutrino hit your
>> processor at just the wrong instant. ;-)
>>
>> More likely i my mind is some little corner condition in the software
>> running on your system. I've had the same thing happen many times
>> actually, and actually a few more times since I started playing with
>> your /etc/make.conf -j/-l values which push the system a little
>> harder.
>
> Whenever I get build failures with the load-adaptive MAKEOPTS and
> EMERGE_DEFAULT_OPTS, I check the build log to see if it's relatively
> obvious that something was depended upon before it was built. If so, I
> file a bug.
>
> Happens every month or so, for me.
>

There are log files? You're telling me I should read them? Gawd,
pretty soon you're gonna try to make a real admin of me instead of
just the oblivious happy home user that I am... ;-)

Good inputs. Thanks!

Cheers,
Mark

Re: [gentoo-user] gcc fails and then succeeds - definitely a problem?

[Alan McKinnon]

On Thu, 23 Feb 2012 11:46:03 -0800
Mark Knecht <markknecht@gmail.com> wrote:

> > Whenever I get build failures with the load-adaptive MAKEOPTS and
> > EMERGE_DEFAULT_OPTS, I check the build log to see if it's relatively
> > obvious that something was depended upon before it was built. If
> > so, I file a bug.
> >
> > Happens every month or so, for me.
> >  
> 
> There are log files? You're telling me I should read them? Gawd,
> pretty soon you're gonna try to make a real admin of me instead of
> just the oblivious happy home user that I am... ;-)

<sideways compliment>

Well, we wouldn't mention log files if we didn't feel you made the grade

:-)

</sideways compliment>


-- 
Alan McKinnnon
alan.mckinnon@gmail.com

Re: [gentoo-user] gcc fails and then succeeds - definitely a problem?

[Alan McKinnon]

On Thu, 23 Feb 2012 11:17:54 -0800
Grant <emailgrant@gmail.com> wrote:

> The gcc update just failed to compile on one of my systems with a
> segfault, but then succeeded after trying again even though I didn't
> change anything.  Does that indicate a hardware problem for sure?
> Should I run memtester?  Any other tests to run?  Nothing in dmesg.
> 
> - Grant
> 

Nah, most likely it's a -j<something bigger than 1> issue

Parallel builds are not deterministic so if the Makefile allows a race
condition to develop it's pot luck whether you'll be hit with it or not

-- 
Alan McKinnnon
alan.mckinnon@gmail.com

Re: [gentoo-user] gcc fails and then succeeds - definitely a problem?

[Grant]

>> The gcc update just failed to compile on one of my systems with a
>> segfault, but then succeeded after trying again even though I didn't
>> change anything.  Does that indicate a hardware problem for sure?
>> Should I run memtester?  Any other tests to run?  Nothing in dmesg.
>>
>> - Grant
>>
>
> Nah, most likely it's a -j<something bigger than 1> issue
>
> Parallel builds are not deterministic so if the Makefile allows a race
> condition to develop it's pot luck whether you'll be hit with it or not

I got sick of stuff like that so I run MAKEOPTS="-j1" on all of my systems.

- Grant

Re: [gentoo-user] gcc fails and then succeeds - definitely a problem?

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 692 bytes --]

On Thu, 23 Feb 2012 11:47:03 -0800, Grant wrote:

> > Parallel builds are not deterministic so if the Makefile allows a race
> > condition to develop it's pot luck whether you'll be hit with it or
> > not  
> 
> I got sick of stuff like that so I run MAKEOPTS="-j1" on all of my
> systems.

If it were a frequent occurrence, there may be some benefit in that. But
using only one of the CPUs 8 cores is such a waste when this sort of
thing happens only every few weeks. Usually trying again works, rarely
does using -j1 make a difference and when it does a bug report ensures
that it won't be an issue in future.


-- 
Neil Bothwick

PROSTITUTE: Receiver of swollen goods.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] gcc fails and then succeeds - definitely a problem?

[Grant]

>> > Parallel builds are not deterministic so if the Makefile allows a race
>> > condition to develop it's pot luck whether you'll be hit with it or
>> > not
>>
>> I got sick of stuff like that so I run MAKEOPTS="-j1" on all of my
>> systems.
>
> If it were a frequent occurrence, there may be some benefit in that. But
> using only one of the CPUs 8 cores is such a waste when this sort of
> thing happens only every few weeks. Usually trying again works, rarely
> does using -j1 make a difference and when it does a bug report ensures
> that it won't be an issue in future.

OK you've inspired me to give it another try.  So if I find a package
that doesn't build with -jn where n > 1 but does build with -j1 I
should file a bug?

- Grant

Re: [gentoo-user] gcc fails and then succeeds - definitely a problem?

[Michael Mol]

On Thu, Feb 23, 2012 at 4:00 PM, Grant <emailgrant@gmail.com> wrote:
>>> > Parallel builds are not deterministic so if the Makefile allows a race
>>> > condition to develop it's pot luck whether you'll be hit with it or
>>> > not
>>>
>>> I got sick of stuff like that so I run MAKEOPTS="-j1" on all of my
>>> systems.
>>
>> If it were a frequent occurrence, there may be some benefit in that. But
>> using only one of the CPUs 8 cores is such a waste when this sort of
>> thing happens only every few weeks. Usually trying again works, rarely
>> does using -j1 make a difference and when it does a bug report ensures
>> that it won't be an issue in future.
>
> OK you've inspired me to give it another try.  So if I find a package
> that doesn't build with -jn where n > 1 but does build with -j1 I
> should file a bug?

Pretty much. It can get more specific than that, but that much is
already a help.

Here's the relevant portions of my MAKEOPTS and EMERGE_DEFAULT_OPTS
which should speed things up for you about as much as possible.

MAKEOPTS="--jobs --load $n" # Where $n is num_CPUs * 1.25
EMERGE_DEFAULT_OPTS="--jobs --load-average=$m" # Where $m is num_CPUs * 1.5

With the "--jobs" parameters, I haven't needed to set $n or $m to
num_CPUS*2 to try to keep the load average up.

Here's my MAKEOPTS and EMERGE_DEFAULT_OPTS verbatim, for an eight-core machine:

MAKEOPTS="--jobs --load 10"
EMERGE_DEFAULT_OPTS="--jobs --load-average=12 --verbose --tree
--with-bdeps=y --keep-going"

If you want to keep things simple, just go with num_CPUs=n for both $m and $n.

-- 
:wq