[gentoo-amd64] MAKEOPTS="-j3" and dual opteron system

[gentoo-amd64] MAKEOPTS="-j3" and dual opteron system

[Hans-Gunther Borrmann]

Hello,

I tried to "emerge --oneshot  glibc" on a dual opteron system with 
"MAKEOPTS="-j3" in make.conf. The emerge began several times to loop at 
different places. The looping program always was "sh".  I could successfully 
emerge glibc after commenting out my MAKEOPTS setting in make.conf. What are 
the experiences with setting MAKEOPTS on smp servers?

Gunther Borrmann
________________________________________________________________
Hans-Gunther Borrmann <hans-gunther.borrmann@rz.uni-freiburg.de>
Rechenzentrum der Universitaet Freiburg
Hermann-Herder-Str. 10, D79104 FREIBURG
Tel.: +49 761/203-4652
Fax:  +49 761/203-4643
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] MAKEOPTS="-j3" and dual opteron system

[Jan Riedel]

Hans-Gunther Borrmann wrote:
> Hello,

hello gunther!

> I tried to "emerge --oneshot  glibc" on a dual opteron system with 
> "MAKEOPTS="-j3" in make.conf. The emerge began several times to loop at 
> different places. The looping program always was "sh".  I could successfully 
> emerge glibc after commenting out my MAKEOPTS setting in make.conf. What are 
> the experiences with setting MAKEOPTS on smp servers?
> 

i have MAKEOPTS="-j4" set in /etc/make.conf and never ran in trouble
when emerging glibc on my dual opteron system.

> Gunther Borrmann
> ________________________________________________________________
> Hans-Gunther Borrmann <hans-gunther.borrmann@rz.uni-freiburg.de>
> Rechenzentrum der Universitaet Freiburg
> Hermann-Herder-Str. 10, D79104 FREIBURG
> Tel.: +49 761/203-4652
> Fax:  +49 761/203-4643

regards
jan

ps: du kannst mir gerne auf deutsch mail zu meiner absenderadresse
senden, wenn du noch mehr wissen moechtest.
-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: MAKEOPTS="-j3" and dual opteron system

[Duncan]

Hans-Gunther Borrmann posted
<200603081035.15013.hans-gunther.borrmann@rz.uni-freiburg.de>, excerpted
below,  on Wed, 08 Mar 2006 10:35:14 +0100:

> I tried to "emerge --oneshot  glibc" on a dual opteron system with 
> "MAKEOPTS="-j3" in make.conf. The emerge began several times to loop at 
> different places. The looping program always was "sh".  I could successfully 
> emerge glibc after commenting out my MAKEOPTS setting in make.conf. What are 
> the experiences with setting MAKEOPTS on smp servers?

I'm running a dual Opteron (242) here, and routinely run MAKEOPTS="-j6
-l6" here, plus sometimes run multiple parallel emerges (using emerge -tp
first to see what the dependencies are, then emerge -a to verify that the
same dependency isn't in both).

As I'm running ~amd64, plus running way out front with still-masked gccs
(now gcc-4.0.2-rX and gcc-4.1.0), which sometime last year (with
gcc-4.0.0) required still experimental glibcs if I wanted to compile themm
with the new gcc, I tend to do more merging of everything, including
glibc, than most, and IDR ever having that sort of issue.

Occasionally, there's an ebuild that has parallel job issues, but they
tend to be pretty rare, because as they are found and bugged, the ebuild
will be adjusted to filter out the -jX and force serial compilation jobs. 
In fact, a quick: "grep MAKEOPTS /p/sys-libs/glibc/*|grep j1" returns a
huge number of hits, multiple per ebuild, where serial compilation is
already forced by the various glibc ebuilds.

Assuming you didn't manually filter all those forced -j1 calls, all I can
say is perhaps another one or more need added.  They apparently haven't
been triggered for most of us or it'd already be forced, but maybe your
machine just happens to trigger it in a different area. <shrug>  Either
that or there's something strange going on with your bash or make or the
like and it's filtering the wrong thing.

What glibc are you attempting to merge, what are your bash, portage, gcc,
and make versions, and where exactly is the error?  If you know bash
scripting well enough to follow what the ebuild is doing, it shouldn't be
hard to verify whether the problem make calls are forcing -j1 or not (some
in the glibc ebuilds do, some don't).

Also note that MAKEOPTS and MAKE_OPTS are two entirely different vars. 
There's a note in the xorg ebuilds mentioning that some users had set the
_ version instead of the run-together version, and it messed things up. 
The xorg ebuilds have a preemptive fix for the problem that the glibc
ebuilds are missing.  If you mistakenly set MAKE_OPTS instead of MAKEOPTS...

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman in
http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html


-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: MAKEOPTS="-j3" and dual opteron system

[Hans-Gunther Borrmann]

On Wednesday 08 March 2006 13:02, Duncan wrote:
> I'm running a dual Opteron (242) here, and routinely run MAKEOPTS="-j6
> -l6" here, plus sometimes run multiple parallel emerges (using emerge -tp
> first to see what the dependencies are, then emerge -a to verify that the
> same dependency isn't in both).
>
> As I'm running ~amd64, plus running way out front with still-masked gccs
> (now gcc-4.0.2-rX and gcc-4.1.0), which sometime last year (with
> gcc-4.0.0) required still experimental glibcs if I wanted to compile themm
> with the new gcc, I tend to do more merging of everything, including
> glibc, than most, and IDR ever having that sort of issue.
>
> Occasionally, there's an ebuild that has parallel job issues, but they
> tend to be pretty rare, because as they are found and bugged, the ebuild
> will be adjusted to filter out the -jX and force serial compilation jobs.
> In fact, a quick: "grep MAKEOPTS /p/sys-libs/glibc/*|grep j1" returns a
> huge number of hits, multiple per ebuild, where serial compilation is
> already forced by the various glibc ebuilds.
>
> Assuming you didn't manually filter all those forced -j1 calls, all I can
> say is perhaps another one or more need added.  They apparently haven't
> been triggered for most of us or it'd already be forced, but maybe your
> machine just happens to trigger it in a different area. <shrug>  Either
> that or there's something strange going on with your bash or make or the
> like and it's filtering the wrong thing.
>
> What glibc are you attempting to merge, what are your bash, portage, gcc,
> and make versions, and where exactly is the error?  If you know bash
> scripting well enough to follow what the ebuild is doing, it shouldn't be
> hard to verify whether the problem make calls are forcing -j1 or not (some
> in the glibc ebuilds do, some don't).

Thank you for this detailed reply. I always like to read your really very 
informative contributions to the gentoo mailing lists. As to your questions I 
cannot tell what the versions of bash portage etc. were. I emerged glibc 
during my migration from the 2004.3 profile to the 2005.0 profile which is 
now complete. During the migration new versions of bash, portage and so an 
were merged. Now all is new and works! I cannot reproduce the loop.  What 
happened was, that the emerge of glibc started to loop at a certain point. I 
then killed the emerge, issued the same command again and the merge started 
to loop at another point. I'll let MAKEOPTS set and observe what happens in 
future.

Thanks to all who answered.
-- 
________________________________________________________________
Hans-Gunther Borrmann <hans-gunther.borrmann@rz.uni-freiburg.de>
Rechenzentrum der Universitaet Freiburg
Hermann-Herder-Str. 10, D79104 FREIBURG
Tel.: +49 761/203-4652
Fax:  +49 761/203-4643
-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: Re: MAKEOPTS="-j3" and dual opteron system

[Duncan]

Hans-Gunther Borrmann posted
<200603101115.09815.hans-gunther.borrmann@rz.uni-freiburg.de>, excerpted
below,  on Fri, 10 Mar 2006 11:15:09 +0100:

> I cannot tell what the versions of bash portage etc. were. I
> emerged glibc during my migration from the 2004.3 profile to the 2005.0
> profile which is now complete. During the migration new versions of bash,
> portage and so an were merged. Now all is new and works! I cannot
> reproduce the loop.  What happened was, that the emerge of glibc started
> to loop at a certain point. I then killed the emerge, issued the same
> command again and the merge started to loop at another point. I'll let
> MAKEOPTS set and observe what happens in future.

Glad you got past it!  Too bad the issue couldn't be nailed down, bugged,
and fixed, but sometimes that happens.  It's frustrating in a way,
particularly for those of us who like to think of the computer as a
deterministic black box, put certain inputs in and you get certain outputs
out, as the bug hasn't been fixed, but the deterministic computer is
theory not always reality, and knowing that, it's livable frustration,
particularly when it's relieving in another, because one got past it.

OT:  Sometimes I wonder at the number of clauses my sentences develop and
whether anyone can make sense of the relationships but me.  =8^)  That
last sentence above...

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman in
http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html


-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: MAKEOPTS="-j3" and dual opteron system

[Hans-Gunther Borrmann]

On Wednesday 08 March 2006 13:02, Duncan wrote:
> I'm running a dual Opteron (242) here, and routinely run MAKEOPTS="-j6
> -l6" here, plus sometimes run multiple parallel emerges (using emerge -tp
> first to see what the dependencies are, then emerge -a to verify that the
> same dependency isn't in both).
>
> As I'm running ~amd64, plus running way out front with still-masked gccs
> (now gcc-4.0.2-rX and gcc-4.1.0), which sometime last year (with
> gcc-4.0.0) required still experimental glibcs if I wanted to compile themm
> with the new gcc, I tend to do more merging of everything, including
> glibc, than most, and IDR ever having that sort of issue.
>
> Occasionally, there's an ebuild that has parallel job issues, but they
> tend to be pretty rare, because as they are found and bugged, the ebuild
> will be adjusted to filter out the -jX and force serial compilation jobs.
> In fact, a quick: "grep MAKEOPTS /p/sys-libs/glibc/*|grep j1" returns a
> huge number of hits, multiple per ebuild, where serial compilation is
> already forced by the various glibc ebuilds.
>
> Assuming you didn't manually filter all those forced -j1 calls, all I can
> say is perhaps another one or more need added.  They apparently haven't
> been triggered for most of us or it'd already be forced, but maybe your
> machine just happens to trigger it in a different area. <shrug>  Either
> that or there's something strange going on with your bash or make or the
> like and it's filtering the wrong thing.
>
> What glibc are you attempting to merge, what are your bash, portage, gcc,
> and make versions, and where exactly is the error?  If you know bash
> scripting well enough to follow what the ebuild is doing, it shouldn't be
> hard to verify whether the problem make calls are forcing -j1 or not (some
> in the glibc ebuilds do, some don't).
>
> Also note that MAKEOPTS and MAKE_OPTS are two entirely different vars.
> There's a note in the xorg ebuilds mentioning that some users had set the
> _ version instead of the run-together version, and it messed things up.
> The xorg ebuilds have a preemptive fix for the problem that the glibc
> ebuilds are missing.  If you mistakenly set MAKE_OPTS instead of
> MAKEOPTS...

Thank you for your detailed answer. As to your questions I cannot tell what 
versions of bash, portage and etc. I used. The loop occured during the 
migration of the machine from profile 2004.3 to profile 2005.0. Now the 
migration is finished, all is up to date and the loop is not reproducible. 
I'll let MAKEOPTS set and watch what happens.

Thanks to all who answered.
-- 
________________________________________________________________
Hans-Gunther Borrmann <hans-gunther.borrmann@rz.uni-freiburg.de>
Rechenzentrum der Universitaet Freiburg
Hermann-Herder-Str. 10, D79104 FREIBURG
Tel.: +49 761/203-4652
Fax:  +49 761/203-4643
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] MAKEOPTS="-j3" and dual opteron system

[Brian Litzinger]

On Wed, Mar 08, 2006 at 10:35:14AM +0100, Hans-Gunther Borrmann wrote:
> I tried to "emerge --oneshot  glibc" on a dual opteron system with 
> "MAKEOPTS="-j3" in make.conf. The emerge began several times to loop at 
> different places. The looping program always was "sh".  I could successfully 
> emerge glibc after commenting out my MAKEOPTS setting in make.conf. What are 
> the experiences with setting MAKEOPTS on smp servers?

I with a dual opteron (~amd64) with -j9 and distcc with another machine.

I have not run into the looping problem you describe.

-- 
Brian Litzinger
-- 
gentoo-amd64@gentoo.org mailing list