[gentoo-user] emerge -j, make -j and make -l

[gentoo-user] emerge -j, make -j and make -l

[Michael Mol]

[-- Attachment #1: Type: text/plain, Size: 276 bytes --]

I just wanted to share an experience I had today with optimizing parallel
builds after discovering "-l" for Make...

I've got a little more tweaking I still want to do, but this is pretty
awesome...

http://funnybutnot.wordpress.com/2011/11/26/optimizing-parallel-builds/

ZZ

[-- Attachment #2: Type: text/html, Size: 401 bytes --]

Re: [gentoo-user] emerge -j, make -j and make -l

[Alex Schuster]

Michael Mol writes:

> http://funnybutnot.wordpress.com/2011/11/26/optimizing-parallel-builds/

404, but http://funnybutnot.wordpress.com/2011/11/26/ seems to work.

	Wonko

Re: [gentoo-user] emerge -j, make -j and make -l

[Michael Mol]

On Sat, Nov 26, 2011 at 10:34 AM, Alex Schuster <wonko@wonkology.org> wrote:
> Michael Mol writes:
>
>> http://funnybutnot.wordpress.com/2011/11/26/optimizing-parallel-builds/
>
> 404, but http://funnybutnot.wordpress.com/2011/11/26/ seems to work.

Should work now. Somehow, I triggered WP's "schedule a post" behavior.
Might still come up as 404 if you've got caching which caches
negatives.


-- 
:wq

Re: [gentoo-user] emerge -j, make -j and make -l

[Stefan G. Weichinger]

Am 26.11.2011 16:34, schrieb Alex Schuster:
> Michael Mol writes:
> 
>> http://funnybutnot.wordpress.com/2011/11/26/optimizing-parallel-builds/
> 
> 404, but http://funnybutnot.wordpress.com/2011/11/26/ seems to work.

Yes, got it now as well.

Thanks for "quoting" me, Michael ... but I also googled that command
somewhere ... not my idea ... ;-)

Stefan

Re: [gentoo-user] emerge -j, make -j and make -l

[Stefan G. Weichinger]

Am 2011-11-26 17:03, schrieb Stefan G. Weichinger:

> Thanks for "quoting" me, Michael ... but I also googled that command
> somewhere ... not my idea ... ;-)

Just went to that URL to cut and paste the command, the mentioned one
doesn't work!

My make.conf shows this comment/command:

gcc -march=native -E -v - </dev/null 2>&1 | sed -n 's/.* -v - //p'

which works for me.

Stefan

Re: [gentoo-user] emerge -j, make -j and make -l

[Michael Mol]

On Mon, Dec 19, 2011 at 2:58 PM, Stefan G. Weichinger <lists@xunil.at> wrote:
> Am 2011-11-26 17:03, schrieb Stefan G. Weichinger:
>
>> Thanks for "quoting" me, Michael ... but I also googled that command
>> somewhere ... not my idea ... ;-)
>
> Just went to that URL to cut and paste the command, the mentioned one
> doesn't work!
>
> My make.conf shows this comment/command:
>
> gcc -march=native -E -v - </dev/null 2>&1 | sed -n 's/.* -v - //p'
>
> which works for me.
>
> Stefan

Yeah, happened to me a couple days ago. Wordpress got a bit too smart
with the quotes, and it broke something else, too. Thanks for the
re-paste, I'll see if I can fix the post.

-- 
:wq

Re: [gentoo-user] emerge -j, make -j and make -l

[Stefan G. Weichinger]

Am 26.11.2011 16:22, schrieb Michael Mol:
> I just wanted to share an experience I had today with optimizing
> parallel builds after discovering "-l" for Make...
> 
> I've got a little more tweaking I still want to do, but this is pretty
> awesome...
> 
> http://funnybutnot.wordpress.com/2011/11/26/optimizing-parallel-builds/

URL ... not found

Re: [gentoo-user] emerge -j, make -j and make -l

[Mick]

[-- Attachment #1: Type: Text/Plain, Size: 448 bytes --]

On Saturday 26 Nov 2011 15:22:15 Michael Mol wrote:
> I just wanted to share an experience I had today with optimizing parallel
> builds after discovering "-l" for Make...
> 
> I've got a little more tweaking I still want to do, but this is pretty
> awesome...
> 
> http://funnybutnot.wordpress.com/2011/11/26/optimizing-parallel-builds/
> 
> ZZ

Thanks for sharing!  How do you determine the optimum value for -l?
-- 
Regards,
Mick

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] emerge -j, make -j and make -l

[Michael Mol]

On Sun, Nov 27, 2011 at 4:27 AM, Mick <michaelkintzios@gmail.com> wrote:
> On Saturday 26 Nov 2011 15:22:15 Michael Mol wrote:
>> I just wanted to share an experience I had today with optimizing parallel
>> builds after discovering "-l" for Make...
>>
>> I've got a little more tweaking I still want to do, but this is pretty
>> awesome...
>>
>> http://funnybutnot.wordpress.com/2011/11/26/optimizing-parallel-builds/
>>
>> ZZ
>
> Thanks for sharing!  How do you determine the optimum value for -l?

I'm making an educated guess. >.>

I figure that the optimal number of simultaneous CPU-consuming
processes is going to be the number of CPU cores, plus enough to keep
the CPU occupied while others are blocked on I/O. That's the same
reasoning that drives the selection of a -j number, really.

If I read make's man page correctly, -l acts as a threshold, choosing
not to spawn an additional child process if the system load average is
above a certain value Since system load is a count of actively running
and ready-to-run processes, you want it to be very close to your
number of logical cores[1].

Since it's going to be a spot decision for Make as to whether or not
to spawn another child (if it hits its limit, it's not going to check
again until after one of its children returns), there will be many
race cases where the load average is high when it looks, but some
other processes will return shortly afterward.[2] That means adding a
process or two for a fudge factor.

That's a lot of guess, though, and it still comes down to guess-and-check.

emerge -j8 @world # MAKEOPTS="-j16 -l10"

Was the first combination I tried. This completed in 89 minutes.

emerge -j8 @world # MAKEOPT="-j16 -l8"

Was the second. This took significantly longer.

I haven't tried higher than -l10; I needed this box to do be able to
do things, which meant installing more software. I've gone from 177
packages to 466.

[1] I don't have a hyperthreading system available, but I suspect that
this is also going to be true of logical cores; It's my understanding
that the overhead from overcommitting CPU comes primarily from context
switching between processors, and hyperthreading adds CPU hardware
specifically to reduce the need to context-switch in splitting
physical CPU resources between threads/processes. So while you'd lose
a little speed for an individual thread, you would gain it back in
aggregate over both threads.

[2] There would also be cases where the load average is low, such as
if a Make recipe calls for a significant bit of I/O before it consumes
a great deal of CPU, but a simple 7200rpm SATA disk appears to be
sufficiently fast that this case is less frequent.
-- 
:wq

Re: [gentoo-user] emerge -j, make -j and make -l

[Pandu Poluan]

[-- Attachment #1: Type: text/plain, Size: 2909 bytes --]

On Nov 27, 2011 5:12 PM, "Michael Mol" <mikemol@gmail.com> wrote:
>
> I figure that the optimal number of simultaneous CPU-consuming
> processes is going to be the number of CPU cores, plus enough to keep
> the CPU occupied while others are blocked on I/O. That's the same
> reasoning that drives the selection of a -j number, really.
>
> If I read make's man page correctly, -l acts as a threshold, choosing
> not to spawn an additional child process if the system load average is
> above a certain value Since system load is a count of actively running
> and ready-to-run processes, you want it to be very close to your
> number of logical cores[1].
>
> Since it's going to be a spot decision for Make as to whether or not
> to spawn another child (if it hits its limit, it's not going to check
> again until after one of its children returns), there will be many
> race cases where the load average is high when it looks, but some
> other processes will return shortly afterward.[2] That means adding a
> process or two for a fudge factor.
>
> That's a lot of guess, though, and it still comes down to guess-and-check.
>
> emerge -j8 @world # MAKEOPTS="-j16 -l10"
>
> Was the first combination I tried. This completed in 89 minutes.
>
> emerge -j8 @world # MAKEOPT="-j16 -l8"
>
> Was the second. This took significantly longer.
>
> I haven't tried higher than -l10; I needed this box to do be able to
> do things, which meant installing more software. I've gone from 177
> packages to 466.
>
> [1] I don't have a hyperthreading system available, but I suspect that
> this is also going to be true of logical cores; It's my understanding
> that the overhead from overcommitting CPU comes primarily from context
> switching between processors, and hyperthreading adds CPU hardware
> specifically to reduce the need to context-switch in splitting
> physical CPU resources between threads/processes. So while you'd lose
> a little speed for an individual thread, you would gain it back in
> aggregate over both threads.
>
> [2] There would also be cases where the load average is low, such as
> if a Make recipe calls for a significant bit of I/O before it consumes
> a great deal of CPU, but a simple 7200rpm SATA disk appears to be
> sufficiently fast that this case is less frequent.

Here's my experience:

I always experience emerge failures on my Gentoo VMs if I use
MAKEOPTS=-j>3. Not all packages, but many. Including, IIRC, glibc and gcc.

This happens even if I make sure that there's just one emerge job being
done. And this happens even if I allocate more vCPUs than -j, on VMware and
XenServer alike.

I don't know where the 'blame' lies, but I've found myself standardizing on
MAKEOPTS=-j3, and PORTAGE_DEFAULT_OPTS="--jobs
--load-average=<1.6*num_of_vCPU>"

(Yes, no explicit number of jobs. The newer portages are smart enough to
keep starting new jobs until the load number is reached)

Rgds,

[-- Attachment #2: Type: text/html, Size: 3442 bytes --]

Re: [gentoo-user] emerge -j, make -j and make -l

[Michael Mol]

On Sun, Nov 27, 2011 at 12:56 PM, Pandu Poluan <pandu@poluan.info> wrote:
> On Nov 27, 2011 5:12 PM, "Michael Mol" <mikemol@gmail.com> wrote:

[snip]

>
> Here's my experience:
>
> I always experience emerge failures on my Gentoo VMs if I use MAKEOPTS=-j>3.
> Not all packages, but many. Including, IIRC, glibc and gcc.

In my barebones 177-package state, I didn't get any build failures
from parallel building, either via emerge -j or make -j. I did get one
failure when I went to install X that worked fine on the second
attempt.

Parallel operations are finicky things; if you don't define the
relationships correctly, you can have things work fine most of the
time, and then a race condition between one make recipe and another
(or perhaps between one ebuild and another; a revdep-rebuild afterward
might not be a bad CYA) causes one thing to fail, just this one time.

My day job is C++ on Windows[1], and we do a *lot* with multithreaded
code. Race conditions are a PITA; you might not be able to reproduce a
race-induced failure on any of the workstations or test systems you
have, but then have it crop up consistently on a customer's system.
The same principles can and will apply with things like parallel make
and parallel emerge. I've even seen it happen in VS2005 and VS2008
parallel builds.

> This happens even if I make sure that there's just one emerge job being
> done. And this happens even if I allocate more vCPUs than -j, on VMware and
> XenServer alike.

FWIW, I've been running with MAKEOPTS=-j10 on my Phenom 9650 for over
a year. It's very rare that something breaks due to the parallel
build. I think it's happened perhaps three times, and each time was
resolvable with a retry. YMMV, of course; race conditions are finicky.

> I don't know where the 'blame' lies, but I've found myself standardizing on
> MAKEOPTS=-j3, and PORTAGE_DEFAULT_OPTS="--jobs
> --load-average=<1.6*num_of_vCPU>"
>
> (Yes, no explicit number of jobs. The newer portages are smart enough to
> keep starting new jobs until the load number is reached)

Sweet; I didn't know about Portage's --load-average; I'll definitely
switch to that instead of -j. Load-driven make plus load-driven
portage should work beautifully on my system.

I'll steal your 1.6 factor, and give:
    MAKEOPTS=-j <2*N> -l <1.6*N)
    PORTAGE_DEFAULT_OPTS="--jobs --load-average<1.6*N>"
a try.

How does that interact with distcc, by the way? I've got two of these
octo-core Xeon boxes, and I've still got my Phenom 9650--distcc on my
home network should become very, very nice. And this box is consuming
255W at the wall with monitor off, 326W with monitor on. That's not
bad. Though perhaps I should move to an apartment where heat isn't
free...

[1] Well, for most of this year, my task list has been more
PHP-oriented, but I'm still on tap for our C++ work.

-- 
:wq

Re: [gentoo-user] emerge -j, make -j and make -l

[Michael Mol]

On Sun, Nov 27, 2011 at 3:16 PM, Michael Mol <mikemol@gmail.com> wrote:
> I'll steal your 1.6 factor, and give:
>    MAKEOPTS=-j <2*N> -l <1.6*N)
>    PORTAGE_DEFAULT_OPTS="--jobs --load-average<1.6*N>"
> a try.

Ah. Which file does PORTAGE_DEFAULT_OPTS go in? It doesn't appear to
have an impact in /etc/make.conf

-- 
:wq

Re: [gentoo-user] emerge -j, make -j and make -l

[Alex Schuster]

Michael Mol writes:

> On Sun, Nov 27, 2011 at 3:16 PM, Michael Mol <mikemol@gmail.com> wrote:
> > I'll steal your 1.6 factor, and give:
> >    MAKEOPTS=-j <2*N> -l <1.6*N)
> >    PORTAGE_DEFAULT_OPTS="--jobs --load-average<1.6*N>"
> > a try.
> 
> Ah. Which file does PORTAGE_DEFAULT_OPTS go in? It doesn't appear to
> have an impact in /etc/make.conf

It's EMERGE_DEFAULT_OPTS in make.conf. See man make.conf.

	Wonko

Re: [gentoo-user] emerge -j, make -j and make -l

[Pandu Poluan]

[-- Attachment #1: Type: text/plain, Size: 621 bytes --]

On Nov 28, 2011 3:55 AM, "Alex Schuster" <wonko@wonkology.org> wrote:
>
> Michael Mol writes:
>
> > On Sun, Nov 27, 2011 at 3:16 PM, Michael Mol <mikemol@gmail.com> wrote:
> > > I'll steal your 1.6 factor, and give:
> > >    MAKEOPTS=-j <2*N> -l <1.6*N)
> > >    PORTAGE_DEFAULT_OPTS="--jobs --load-average<1.6*N>"
> > > a try.
> >
> > Ah. Which file does PORTAGE_DEFAULT_OPTS go in? It doesn't appear to
> > have an impact in /etc/make.conf
>
> It's EMERGE_DEFAULT_OPTS in make.conf. See man make.conf.
>

Yes, it's EMERGE_DEFAULT_OPTS >.<

I really should stop sending an email when it's 00:44 in the morning...

Rgds,

[-- Attachment #2: Type: text/html, Size: 967 bytes --]

Re: [gentoo-user] emerge -j, make -j and make -l

[Alan McKinnon]

On Sun, 27 Nov 2011 15:16:33 -0500
Michael Mol <mikemol@gmail.com> wrote:


[snip]

> My day job is C++ on Windows[1],

> [1] Well, for most of this year, my task list has been more
> PHP-oriented, but I'm still on tap for our C++ work.

You have my deepest, deepest, sympathies.

Gotta pay the rent something though, hey?

[I have a similar thing going myself with SLES]


-- 
Alan McKinnnon
alan.mckinnon@gmail.com

Re: [gentoo-user] emerge -j, make -j and make -l

[Pandu Poluan]

[-- Attachment #1: Type: text/plain, Size: 2756 bytes --]

On Nov 28, 2011 3:21 AM, "Michael Mol" <mikemol@gmail.com> wrote:
>
> On Sun, Nov 27, 2011 at 12:56 PM, Pandu Poluan <pandu@poluan.info> wrote:
> > On Nov 27, 2011 5:12 PM, "Michael Mol" <mikemol@gmail.com> wrote:
>
> [snip]
>
> >
> > Here's my experience:
> >
> > I always experience emerge failures on my Gentoo VMs if I use
MAKEOPTS=-j>3.
> > Not all packages, but many. Including, IIRC, glibc and gcc.
>
> In my barebones 177-package state, I didn't get any build failures
> from parallel building, either via emerge -j or make -j. I did get one
> failure when I went to install X that worked fine on the second
> attempt.
>

And in my barebones system (172 packages), there are several packages that
reliably fail, even when emerged on their own *sigh*

> > This happens even if I make sure that there's just one emerge job being
> > done. And this happens even if I allocate more vCPUs than -j, on VMware
and
> > XenServer alike.
>
> FWIW, I've been running with MAKEOPTS=-j10 on my Phenom 9650 for over
> a year. It's very rare that something breaks due to the parallel
> build. I think it's happened perhaps three times, and each time was
> resolvable with a retry. YMMV, of course; race conditions are finicky.
>

I have a hunch that the hypervisor had some effects. Most likely, another
VM in the same host has a high enough load that necessitates the Gentoo VM
to be shifted to another (physical) core, and this messes up the timing.

MAKEOPTS=-j3 always succeeds, though. So I'm disinclined to delve deeper
into the issue.

> > I don't know where the 'blame' lies, but I've found myself
standardizing on
> > MAKEOPTS=-j3, and PORTAGE_DEFAULT_OPTS="--jobs
> > --load-average=<1.6*num_of_vCPU>"
> >
> > (Yes, no explicit number of jobs. The newer portages are smart enough to
> > keep starting new jobs until the load number is reached)
>
> Sweet; I didn't know about Portage's --load-average; I'll definitely
> switch to that instead of -j. Load-driven make plus load-driven
> portage should work beautifully on my system.
>
> I'll steal your 1.6 factor, and give:
>    MAKEOPTS=-j <2*N> -l <1.6*N)
>    PORTAGE_DEFAULT_OPTS="--jobs --load-average<1.6*N>"
> a try.
>

Make sure that make supports non-integer values for -l, though.

> How does that interact with distcc, by the way? I've got two of these
> octo-core Xeon boxes, and I've still got my Phenom 9650--distcc on my
> home network should become very, very nice. And this box is consuming
> 255W at the wall with monitor off, 326W with monitor on. That's not
> bad. Though perhaps I should move to an apartment where heat isn't
> free...
>

Well, I don't use distcc, so I can't speculate. But with plain single
system compilation, my settings have been serving me real well :-)

Rgds,

[-- Attachment #2: Type: text/html, Size: 3551 bytes --]

Re: [gentoo-user] emerge -j, make -j and make -l

[Michael Mol]

On Sun, Nov 27, 2011 at 8:06 PM, Pandu Poluan <pandu@poluan.info> wrote:
> On Nov 28, 2011 3:21 AM, "Michael Mol" <mikemol@gmail.com> wrote:
>> On Sun, Nov 27, 2011 at 12:56 PM, Pandu Poluan <pandu@poluan.info> wrote:

[snip]

>> Sweet; I didn't know about Portage's --load-average; I'll definitely
>> switch to that instead of -j. Load-driven make plus load-driven
>> portage should work beautifully on my system.
>>
>> I'll steal your 1.6 factor, and give:
>>    MAKEOPTS=-j <2*N> -l <1.6*N)
>>    PORTAGE_DEFAULT_OPTS="--jobs --load-average<1.6*N>"
>> a try.

> Make sure that make supports non-integer values for -l, though.

I rounded up to 13.
-- 
:wq

Re: [gentoo-user] emerge -j, make -j and make -l

[Pandu Poluan]

[-- Attachment #1: Type: text/plain, Size: 1003 bytes --]

On Nov 28, 2011 12:35 PM, "Michael Mol" <mikemol@gmail.com> wrote:
>
> On Sun, Nov 27, 2011 at 8:06 PM, Pandu Poluan <pandu@poluan.info> wrote:
> > On Nov 28, 2011 3:21 AM, "Michael Mol" <mikemol@gmail.com> wrote:
> >> On Sun, Nov 27, 2011 at 12:56 PM, Pandu Poluan <pandu@poluan.info>
wrote:
>
> [snip]
>
> >> Sweet; I didn't know about Portage's --load-average; I'll definitely
> >> switch to that instead of -j. Load-driven make plus load-driven
> >> portage should work beautifully on my system.
> >>
> >> I'll steal your 1.6 factor, and give:
> >>    MAKEOPTS=-j <2*N> -l <1.6*N)
> >>    PORTAGE_DEFAULT_OPTS="--jobs --load-average<1.6*N>"
> >> a try.
>
> > Make sure that make supports non-integer values for -l, though.
>
> I rounded up to 13.

Shouldn't you be rounding down, instead? That said, as long as the number
falls between 1.6*N and 1.8*N (see my previous post), nothing should blow
up. Well, not spectacularly, at least :-)

BTW, FYI, --load-average accepts non-integer values.

Rgds,

[-- Attachment #2: Type: text/html, Size: 1500 bytes --]

Re: [gentoo-user] emerge -j, make -j and make -l

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 725 bytes --]

On Mon, 28 Nov 2011 00:56:17 +0700, Pandu Poluan wrote:

> I don't know where the 'blame' lies, but I've found myself
> standardizing on MAKEOPTS=-j3, and PORTAGE_DEFAULT_OPTS="--jobs
> --load-average=<1.6*num_of_vCPU>"
> 
> (Yes, no explicit number of jobs. The newer portages are smart enough to
> keep starting new jobs until the load number is reached)

The problem I found with that is the ebuilds load the system lightly to
start with, before they enter the compile phase, to portage starts dozens
of parallel ebuilds, then the system gets completely bogged down when
they start compiling.


-- 
Neil Bothwick

If Microsoft made cars:
"The airbag system would ask "are you sure?" before deploying."

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] emerge -j, make -j and make -l

[Pandu Poluan]

[-- Attachment #1: Type: text/plain, Size: 1478 bytes --]

On Nov 28, 2011 6:24 AM, "Neil Bothwick" <neil@digimed.co.uk> wrote:
>
> On Mon, 28 Nov 2011 00:56:17 +0700, Pandu Poluan wrote:
>
> > I don't know where the 'blame' lies, but I've found myself
> > standardizing on MAKEOPTS=-j3, and PORTAGE_DEFAULT_OPTS="--jobs
> > --load-average=<1.6*num_of_vCPU>"
> >
> > (Yes, no explicit number of jobs. The newer portages are smart enough to
> > keep starting new jobs until the load number is reached)
>
> The problem I found with that is the ebuilds load the system lightly to
> start with, before they enter the compile phase, to portage starts dozens
> of parallel ebuilds, then the system gets completely bogged down when
> they start compiling.
>

Yes, sometimes that would happen if at the beginning there are
network-bound ebuilds all downloading their respective distfiles. The load
stays low until they all start ./configure-ing roughly at the same time.
Then all hell breaks loose.

I successfully mitigate such "load-explosion" by doing a --fetchonly step
first, and keeping MAKEOPTS at low -j (which, in my case, is actually
required).

Just to add more info: I use USE=graphite (with some CFLAGS, uh,
'enhancements') with gcc-4.5.3. IIRC, I could push MAKEOPTS up to -j5 (and
even more, but I ran out of cores) when I was still using gcc-4.4.x and no
USE=graphite.

Won't file a bug report, though. I have a feeling that my bug report re:
emerge failure will be marked WONTFIX thanks to the 'ricer special' CFLAGS
:-P

Rgds,

[-- Attachment #2: Type: text/html, Size: 1769 bytes --]

[gentoo-user] Re: emerge -j, make -j and make -l

[Jack Byer]

Pandu Poluan wrote:

> On Nov 28, 2011 6:24 AM, "Neil Bothwick" <neil@digimed.co.uk> wrote:
>>
>> On Mon, 28 Nov 2011 00:56:17 +0700, Pandu Poluan wrote:
>>
>> > I don't know where the 'blame' lies, but I've found myself
>> > standardizing on MAKEOPTS=-j3, and PORTAGE_DEFAULT_OPTS="--jobs
>> > --load-average=<1.6*num_of_vCPU>"
>> >
>> > (Yes, no explicit number of jobs. The newer portages are smart 
enough
>> > to keep starting new jobs until the load number is reached)
>>
>> The problem I found with that is the ebuilds load the system lightly 
to
>> start with, before they enter the compile phase, to portage starts 
dozens
>> of parallel ebuilds, then the system gets completely bogged down 
when
>> they start compiling.
>>
> 
> Yes, sometimes that would happen if at the beginning there are
> network-bound ebuilds all downloading their respective distfiles. The 
load
> stays low until they all start ./configure-ing roughly at the same 
time.
> Then all hell breaks loose.
> 
> I successfully mitigate such "load-explosion" by doing a --fetchonly 
step
> first, and keeping MAKEOPTS at low -j (which, in my case, is actually
> required).
> 
> Just to add more info: I use USE=graphite (with some CFLAGS, uh,
> 'enhancements') with gcc-4.5.3. IIRC, I could push MAKEOPTS up to -j5 
(and
> even more, but I ran out of cores) when I was still using gcc-4.4.x 
and no
> USE=graphite.
> 
> Won't file a bug report, though. I have a feeling that my bug report 
re:
> emerge failure will be marked WONTFIX thanks to the 'ricer special' 
CFLAGS
> :-P
> 
> Rgds,

It would be nice if there were some way to mark particular packages 
that should never be compiled in parallel (like the trick for using a 
using a separate, non-tmpfs build directory for large packages). The 
"load-explosion" you describe is bad enough with regular packages but 
when firefox, xulrunner, chromium and libreoffice all decide to start 
compiling at the same time it turns into a complete nightmare.

Re: [gentoo-user] Re: emerge -j, make -j and make -l

[Dale]

Jack Byer wrote:
> It would be nice if there were some way to mark particular packages 
> that should never be compiled in parallel (like the trick for using a 
> using a separate, non-tmpfs build directory for large packages). The 
> "load-explosion" you describe is bad enough with regular packages but 
> when firefox, xulrunner, chromium and libreoffice all decide to start 
> compiling at the same time it turns into a complete nightmare. 

I think it has that already.  I have noticed several times that mine 
will only be working on one package and have a lot of packages left to 
update.  When that single package is done, it loads up several and does 
them at the same time.  Something tells emerge not to run them at the 
same time and I assume it is in the ebuild somewhere.

I have to have tmpfs in use when LOo comes up for a compile.  I have 
more memory than I have space on /var.  If I don't have portages work 
directory on tmpfs, the compile fills up /var and dies.  Ironic that 
what works for one fails for another.

Dale

:-)  :-)

-- 
I am only responsible for what I said ... Not for what you understood or how you interpreted my words!

Re: [gentoo-user] Re: emerge -j, make -j and make -l

[Pandu Poluan]

[-- Attachment #1: Type: text/plain, Size: 1160 bytes --]

On Nov 28, 2011 8:38 AM, "Dale" <rdalek1967@gmail.com> wrote:
>
> Jack Byer wrote:
>>
>> It would be nice if there were some way to mark particular packages that
should never be compiled in parallel (like the trick for using a using a
separate, non-tmpfs build directory for large packages). The
"load-explosion" you describe is bad enough with regular packages but when
firefox, xulrunner, chromium and libreoffice all decide to start compiling
at the same time it turns into a complete nightmare.
>
>
> I think it has that already.  I have noticed several times that mine will
only be working on one package and have a lot of packages left to update.
 When that single package is done, it loads up several and does them at the
same time.  Something tells emerge not to run them at the same time and I
assume it is in the ebuild somewhere.
>
> I have to have tmpfs in use when LOo comes up for a compile.  I have more
memory than I have space on /var.  If I don't have portages work directory
on tmpfs, the compile fills up /var and dies.  Ironic that what works for
one fails for another.
>

portage.env?

http://wiki.gentoo.org/wiki//etc/portage/env

Rgds,

[-- Attachment #2: Type: text/html, Size: 1431 bytes --]

Re: [gentoo-user] Re: emerge -j, make -j and make -l

[Dale]

[-- Attachment #1: Type: text/plain, Size: 1786 bytes --]

Pandu Poluan wrote:
>
>
> On Nov 28, 2011 8:38 AM, "Dale" <rdalek1967@gmail.com 
> <mailto:rdalek1967@gmail.com>> wrote:
> >
> > Jack Byer wrote:
> >>
> >> It would be nice if there were some way to mark particular packages 
> that should never be compiled in parallel (like the trick for using a 
> using a separate, non-tmpfs build directory for large packages). The 
> "load-explosion" you describe is bad enough with regular packages but 
> when firefox, xulrunner, chromium and libreoffice all decide to start 
> compiling at the same time it turns into a complete nightmare.
> >
> >
> > I think it has that already.  I have noticed several times that mine 
> will only be working on one package and have a lot of packages left to 
> update.  When that single package is done, it loads up several and 
> does them at the same time.  Something tells emerge not to run them at 
> the same time and I assume it is in the ebuild somewhere.
> >
> > I have to have tmpfs in use when LOo comes up for a compile.  I have 
> more memory than I have space on /var.  If I don't have portages work 
> directory on tmpfs, the compile fills up /var and dies.  Ironic that 
> what works for one fails for another.
> >
>
> portage.env?
>
> http://wiki.gentoo.org/wiki//etc/portage/env
>
> Rgds,
>

I just added it to fstab so that it mounts automatically.  If I am 
running some program that needs that much ram, I can always unmount it.  
If course, when knotify went wonky the other day and was using 14Gbs of 
ram, it didn't complain about tmpfs or anything.  It did get a bit slow 
until I killed it.  Don't have KDE running when you recompile some parts 
of it.  ;-)

Dale

:-)  :-)

-- 
I am only responsible for what I said ... Not for what you understood or how you interpreted my words!


[-- Attachment #2: Type: text/html, Size: 2699 bytes --]

Re: [gentoo-user] emerge -j, make -j and make -l

[Michael Mol]

On Sun, Nov 27, 2011 at 7:54 PM, Pandu Poluan <pandu@poluan.info> wrote:
>
> On Nov 28, 2011 6:24 AM, "Neil Bothwick" <neil@digimed.co.uk> wrote:
>>
>> On Mon, 28 Nov 2011 00:56:17 +0700, Pandu Poluan wrote:
>>
>> > I don't know where the 'blame' lies, but I've found myself
>> > standardizing on MAKEOPTS=-j3, and PORTAGE_DEFAULT_OPTS="--jobs
>> > --load-average=<1.6*num_of_vCPU>"
>> >
>> > (Yes, no explicit number of jobs. The newer portages are smart enough to
>> > keep starting new jobs until the load number is reached)
>>
>> The problem I found with that is the ebuilds load the system lightly to
>> start with, before they enter the compile phase, to portage starts dozens
>> of parallel ebuilds, then the system gets completely bogged down when
>> they start compiling.
>>
>
> Yes, sometimes that would happen if at the beginning there are network-bound
> ebuilds all downloading their respective distfiles. The load stays low until
> they all start ./configure-ing roughly at the same time. Then all hell
> breaks loose.
>
> I successfully mitigate such "load-explosion" by doing a --fetchonly step
> first, and keeping MAKEOPTS at low -j (which, in my case, is actually
> required).
>
> Just to add more info: I use USE=graphite (with some CFLAGS, uh,
> 'enhancements') with gcc-4.5.3. IIRC, I could push MAKEOPTS up to -j5 (and
> even more, but I ran out of cores) when I was still using gcc-4.4.x and no
> USE=graphite.
>
> Won't file a bug report, though. I have a feeling that my bug report re:
> emerge failure will be marked WONTFIX thanks to the 'ricer special' CFLAGS

As I noted, "-l" in MAKEOPTS takes care of the load explosion very nicely.
-- 
:wq

Re: [gentoo-user] emerge -j, make -j and make -l

[Pandu Poluan]

[-- Attachment #1: Type: text/plain, Size: 2109 bytes --]

On Nov 28, 2011 11:38 AM, "Michael Mol" <mikemol@gmail.com> wrote:
>
> On Sun, Nov 27, 2011 at 7:54 PM, Pandu Poluan <pandu@poluan.info> wrote:
> >
> > On Nov 28, 2011 6:24 AM, "Neil Bothwick" <neil@digimed.co.uk> wrote:
> >>
> >> On Mon, 28 Nov 2011 00:56:17 +0700, Pandu Poluan wrote:
> >>
> >> > I don't know where the 'blame' lies, but I've found myself
> >> > standardizing on MAKEOPTS=-j3, and PORTAGE_DEFAULT_OPTS="--jobs
> >> > --load-average=<1.6*num_of_vCPU>"
> >> >
> >> > (Yes, no explicit number of jobs. The newer portages are smart
enough to
> >> > keep starting new jobs until the load number is reached)
> >>
> >> The problem I found with that is the ebuilds load the system lightly to
> >> start with, before they enter the compile phase, to portage starts
dozens
> >> of parallel ebuilds, then the system gets completely bogged down when
> >> they start compiling.
> >>
> >
> > Yes, sometimes that would happen if at the beginning there are
network-bound
> > ebuilds all downloading their respective distfiles. The load stays low
until
> > they all start ./configure-ing roughly at the same time. Then all hell
> > breaks loose.
> >
> > I successfully mitigate such "load-explosion" by doing a --fetchonly
step
> > first, and keeping MAKEOPTS at low -j (which, in my case, is actually
> > required).
> >
> > Just to add more info: I use USE=graphite (with some CFLAGS, uh,
> > 'enhancements') with gcc-4.5.3. IIRC, I could push MAKEOPTS up to -j5
(and
> > even more, but I ran out of cores) when I was still using gcc-4.4.x and
no
> > USE=graphite.
> >
> > Won't file a bug report, though. I have a feeling that my bug report re:
> > emerge failure will be marked WONTFIX thanks to the 'ricer special'
CFLAGS
>
> As I noted, "-l" in MAKEOPTS takes care of the load explosion very nicely.

Most likely so. I am not aware of -l in MAKEOPTS before, so what I posted
was my workaround to prevent load explosion. Thanks to your very useful
tip, I now no longer have to worry about load explosion :-)

(I still like doing pre-fetchonly-ing, though. But now for a different main
reason :-)

Rgds,

[-- Attachment #2: Type: text/html, Size: 2805 bytes --]

Re: [gentoo-user] emerge -j, make -j and make -l

[Michael Mol]

On Mon, Nov 28, 2011 at 12:13 AM, Pandu Poluan <pandu@poluan.info> wrote:
> On Nov 28, 2011 11:38 AM, "Michael Mol" <mikemol@gmail.com> wrote:
>> On Sun, Nov 27, 2011 at 7:54 PM, Pandu Poluan <pandu@poluan.info> wrote:
>> > On Nov 28, 2011 6:24 AM, "Neil Bothwick" <neil@digimed.co.uk> wrote:
>> >> On Mon, 28 Nov 2011 00:56:17 +0700, Pandu Poluan wrote:
>> >>
>> >> > I don't know where the 'blame' lies, but I've found myself
>> >> > standardizing on MAKEOPTS=-j3, and PORTAGE_DEFAULT_OPTS="--jobs
>> >> > --load-average=<1.6*num_of_vCPU>"
>> >> >
>> >> > (Yes, no explicit number of jobs. The newer portages are smart enough
>> >> > to
>> >> > keep starting new jobs until the load number is reached)
>> >>
>> >> The problem I found with that is the ebuilds load the system lightly to
>> >> start with, before they enter the compile phase, to portage starts
>> >> dozens
>> >> of parallel ebuilds, then the system gets completely bogged down when
>> >> they start compiling.
>> >>
>> >
>> > Yes, sometimes that would happen if at the beginning there are
>> > network-bound
>> > ebuilds all downloading their respective distfiles. The load stays low
>> > until
>> > they all start ./configure-ing roughly at the same time. Then all hell
>> > breaks loose.
>> >
>> > I successfully mitigate such "load-explosion" by doing a --fetchonly
>> > step
>> > first, and keeping MAKEOPTS at low -j (which, in my case, is actually
>> > required).
>> >
>> > Just to add more info: I use USE=graphite (with some CFLAGS, uh,
>> > 'enhancements') with gcc-4.5.3. IIRC, I could push MAKEOPTS up to -j5
>> > (and
>> > even more, but I ran out of cores) when I was still using gcc-4.4.x and
>> > no
>> > USE=graphite.
>> >
>> > Won't file a bug report, though. I have a feeling that my bug report re:
>> > emerge failure will be marked WONTFIX thanks to the 'ricer special'
>> > CFLAGS
>>
>> As I noted, "-l" in MAKEOPTS takes care of the load explosion very nicely.
>
> Most likely so. I am not aware of -l in MAKEOPTS before, so what I posted
> was my workaround to prevent load explosion. Thanks to your very useful tip,
> I now no longer have to worry about load explosion :-)
>
> (I still like doing pre-fetchonly-ing, though. But now for a different main
> reason :-)

The explosion of information in this thread is going to make for a
*great* followup blog post. :)

Now I just wish there were a way to get Portage and Make to watch CPU
usage and raise or lower the load-average threshold depending on how
much CPU was going to 'sys', 'user' and 'wait'; Lower -l if a great
deal of time is spent in 'sys'; you're likely burning cycles in
context switches. Raise -l if a great deal of time is spent waiting on
I/O.

It'd also be helpful to be able to give keystone[1] packages and Make
recipes a more favorable NICE value than those less important, to
induce the scheduler to favor the important packages over
less-important packages when we've got more ready work than cores. I
don't think Make *can* have the smarts for that, but Portage could
conceivably do it for its own parallelization.

[1] Where many things depend on them, either directly or indirectly.
Getting these out of the way means more parallel-buildable packages
being available at the same time.

-- 
:wq

Re: [gentoo-user] emerge -j, make -j and make -l

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 1374 bytes --]

On Sun, 27 Nov 2011 23:34:14 -0500, Michael Mol wrote:

> >> The problem I found with that is the ebuilds load the system lightly
> >> to start with, before they enter the compile phase, to portage
> >> starts dozens of parallel ebuilds, then the system gets completely
> >> bogged down when they start compiling.

> > Yes, sometimes that would happen if at the beginning there are
> > network-bound ebuilds all downloading their respective distfiles. The
> > load stays low until they all start ./configure-ing roughly at the
> > same time. Then all hell breaks loose.
> >
> > I successfully mitigate such "load-explosion" by doing a --fetchonly
> > step first, and keeping MAKEOPTS at low -j (which, in my case, is
> > actually required).

It doesn't help here, as my cron script that runs emerge --sync already
follows it with emerge -f @world.

> As I noted, "-l" in MAKEOPTS takes care of the load explosion very
> nicely.

From the description, it should do just that, there may still be dozens
of ebuilds in progress, but only the first few will actually start
compiling. Adding it now. It should also helps when there are multiple
emerge processes running, my desktop acts as a build host for lower
powered systems too. Adding -l10 now.


-- 
Neil Bothwick

The trouble with the world is that everybody in it is three drinks behind.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] emerge -j, make -j and make -l

[Pandu Poluan]

On Mon, Nov 28, 2011 at 16:27, Neil Bothwick <neil@digimed.co.uk> wrote:
> On Sun, 27 Nov 2011 23:34:14 -0500, Michael Mol wrote:
>
>> >> The problem I found with that is the ebuilds load the system lightly
>> >> to start with, before they enter the compile phase, to portage
>> >> starts dozens of parallel ebuilds, then the system gets completely
>> >> bogged down when they start compiling.
>
>> > Yes, sometimes that would happen if at the beginning there are
>> > network-bound ebuilds all downloading their respective distfiles. The
>> > load stays low until they all start ./configure-ing roughly at the
>> > same time. Then all hell breaks loose.
>> >
>> > I successfully mitigate such "load-explosion" by doing a --fetchonly
>> > step first, and keeping MAKEOPTS at low -j (which, in my case, is
>> > actually required).
>
> It doesn't help here, as my cron script that runs emerge --sync already
> follows it with emerge -f @world.
>
>> As I noted, "-l" in MAKEOPTS takes care of the load explosion very
>> nicely.
>
> From the description, it should do just that, there may still be dozens
> of ebuilds in progress, but only the first few will actually start
> compiling. Adding it now. It should also helps when there are multiple
> emerge processes running, my desktop acts as a build host for lower
> powered systems too. Adding -l10 now.
>

Just tried adding "-l7.2" to MAKEOPTS ...

... and indeed, emerge -1avuND --changed-use @system @world seems
faster than without.

What's for sure, the emerge's output shows higher parallelism than without.

Hmmm... gotta try it with emerge -e @system @world sometime :-)

Rgds,
-- 
FdS Pandu E Poluan
~ IT Optimizer ~

 • LOPSA Member #15248
 • Blog : http://pepoluan.tumblr.com
 • Linked-In : http://id.linkedin.com/in/pepoluan

Re: [gentoo-user] emerge -j, make -j and make -l

[Stefan G. Weichinger]

Am 28.11.2011 10:27, schrieb Neil Bothwick:

> From the description, it should do just that, there may still be
> dozens of ebuilds in progress, but only the first few will actually
> start compiling. Adding it now. It should also helps when there are
> multiple emerge processes running, my desktop acts as a build host
> for lower powered systems too. Adding -l10 now.

Neil, you run a core-i7-2600 as well ... what is your current
best-practise with that CPU, concerning the values of N and -l ... ?

Thanks, greets, Stefan

Re: [gentoo-user] emerge -j, make -j and make -l

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 497 bytes --]

On Mon, 28 Nov 2011 17:36:08 +0100, Stefan G. Weichinger wrote:

> Neil, you run a core-i7-2600 as well ... what is your current
> best-practise with that CPU, concerning the values of N and -l ... ?

With the cooling system I currently have, I don't like to push it too
much (a new watercooler should arrive tomorrow), but MAKEOPTS="-j16 -l10"
appears to be a definite improvement over the old -j8 with no -l.


-- 
Neil Bothwick

Scrotum is a small planet near Uranus. True/False?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] emerge -j, make -j and make -l

[Stefan G. Weichinger]

Am 29.11.2011 12:08, schrieb Neil Bothwick:
> On Mon, 28 Nov 2011 17:36:08 +0100, Stefan G. Weichinger wrote:
> 
>> Neil, you run a core-i7-2600 as well ... what is your current 
>> best-practise with that CPU, concerning the values of N and -l
>> ... ?
> 
> With the cooling system I currently have, I don't like to push it
> too much (a new watercooler should arrive tomorrow), but
> MAKEOPTS="-j16 -l10" appears to be a definite improvement over the
> old -j8 with no -l.

I have it in two variables:

EMERGE_DEFAULT_OPTS="--jobs=8 --load-average=6"
MAKEOPTS="-j18"

and it doesn't push it much, even without any watercooling in there
(and without that "K" in the cpu-name ...)

Will try your -l10 as well, that might make a difference, sure.

thx, Stefan

Re: [gentoo-user] emerge -j, make -j and make -l

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 949 bytes --]

On Tue, 29 Nov 2011 14:47:49 +0100, Stefan G. Weichinger wrote:

> > With the cooling system I currently have, I don't like to push it
> > too much (a new watercooler should arrive tomorrow), but
> > MAKEOPTS="-j16 -l10" appears to be a definite improvement over the
> > old -j8 with no -l.  
> 
> I have it in two variables:
> 
> EMERGE_DEFAULT_OPTS="--jobs=8 --load-average=6"
> MAKEOPTS="-j18"
> 
> Will try your -l10 as well, that might make a difference, sure.
 
The trouble with --load-average in emerge is that it is only checked as
each ebuild is about to start, so you get the "load explosion" mentioned
previously when many ebuilds start and once and then get into their
compile phases. I'm using --jobs, with no value, in EMERGE_DEFAULT_OPTS
and the -l10 for make seems to take care of the load very nicely (sorry
about the pun there).


-- 
Neil Bothwick

If Satan ever loses his hair, there'll be hell toupee.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] emerge -j, make -j and make -l

[Stefan G. Weichinger]

Am 29.11.2011 16:39, schrieb Neil Bothwick:

> The trouble with --load-average in emerge is that it is only
> checked as each ebuild is about to start, so you get the "load
> explosion" mentioned previously when many ebuilds start and once
> and then get into their compile phases. I'm using --jobs, with no
> value, in EMERGE_DEFAULT_OPTS and the -l10 for make seems to take
> care of the load very nicely (sorry about the pun there).

ok with me ;-)

I now use:

EMERGE_DEFAULT_OPTS="--jobs"
MAKEOPTS="-j18 -l10"

and that is rather fast, yes .... thanks ...

I start to "feel" the potential of that new CPU, yes.

Stefan

Re: [gentoo-user] emerge -j, make -j and make -l

[Michael Mol]

On Sun, Nov 27, 2011 at 6:21 PM, Neil Bothwick <neil@digimed.co.uk> wrote:
> On Mon, 28 Nov 2011 00:56:17 +0700, Pandu Poluan wrote:
>
>> I don't know where the 'blame' lies, but I've found myself
>> standardizing on MAKEOPTS=-j3, and PORTAGE_DEFAULT_OPTS="--jobs
>> --load-average=<1.6*num_of_vCPU>"
>>
>> (Yes, no explicit number of jobs. The newer portages are smart enough to
>> keep starting new jobs until the load number is reached)
>
> The problem I found with that is the ebuilds load the system lightly to
> start with, before they enter the compile phase, to portage starts dozens
> of parallel ebuilds, then the system gets completely bogged down when
> they start compiling.

I did notice that, but it settles back down *very* quickly with the -l
parameter to make.


-- 
:wq

Re: [gentoo-user] emerge -j, make -j and make -l

[covici]

Mick <michaelkintzios@gmail.com> wrote:

> On Saturday 26 Nov 2011 15:22:15 Michael Mol wrote:
> > I just wanted to share an experience I had today with optimizing parallel
> > builds after discovering "-l" for Make...
> > 
> > I've got a little more tweaking I still want to do, but this is pretty
> > awesome...
> > 
> > http://funnybutnot.wordpress.com/2011/11/26/optimizing-parallel-builds/
> > 
> > ZZ
> 
> Thanks for sharing!  How do you determine the optimum value for -l?

How do you get emerge not to display number of jobs and load average --
I only want to compile one at a time -- much safer that way and it is
doing that, but now it displays all that load average and how many jobs,
etc. -- any way to get rid of that display?

-- 
Your life is like a penny.  You're going to lose it.  The question is:
How do
you spend it?

         John Covici
         covici@ccs.covici.com

Re: [gentoo-user] emerge -j, make -j and make -l

[Michael Mol]

On Sun, Nov 27, 2011 at 5:39 AM,  <covici@ccs.covici.com> wrote:
> Mick <michaelkintzios@gmail.com> wrote:
>
>> On Saturday 26 Nov 2011 15:22:15 Michael Mol wrote:
>> > I just wanted to share an experience I had today with optimizing parallel
>> > builds after discovering "-l" for Make...
>> >
>> > I've got a little more tweaking I still want to do, but this is pretty
>> > awesome...
>> >
>> > http://funnybutnot.wordpress.com/2011/11/26/optimizing-parallel-builds/
>> >
>> > ZZ
>>
>> Thanks for sharing!  How do you determine the optimum value for -l?
>
> How do you get emerge not to display number of jobs and load average --

The display is something emerge will show you if you've asked it to
build in parallel (which you did, by passing -j to emerge. That's
different from putting -j in MAKEOPTS.)

> I only want to compile one at a time -- much safer that way and it is
> doing that,

It's likely only doing that because there isn't anything it can
immediately build that doesn't have what it's *currently* working on
as a build dependency. I noted in my blog post that emerge's
parallelization has many of the same limitations as make's that's one
of the things I was talking about; there can be linchpin and keystone
packages which need to be built before many others. libc would be an
example. gcc is a frequent example.

> but now it displays all that load average and how many jobs,
> etc. -- any way to get rid of that display?

Forget the display; it sounds like you don't want emerge building in
parallel. In that event, don't pass "-j" to emerge. The display will
go away.

-- 
:wq

Re: [gentoo-user] emerge -j, make -j and make -l

[Dale]

covici@ccs.covici.com wrote:
> Mick<michaelkintzios@gmail.com>  wrote:
>
>> On Saturday 26 Nov 2011 15:22:15 Michael Mol wrote:
>>> I just wanted to share an experience I had today with optimizing parallel
>>> builds after discovering "-l" for Make...
>>>
>>> I've got a little more tweaking I still want to do, but this is pretty
>>> awesome...
>>>
>>> http://funnybutnot.wordpress.com/2011/11/26/optimizing-parallel-builds/
>>>
>>> ZZ
>> Thanks for sharing!  How do you determine the optimum value for -l?
> How do you get emerge not to display number of jobs and load average --
> I only want to compile one at a time -- much safer that way and it is
> doing that, but now it displays all that load average and how many jobs,
> etc. -- any way to get rid of that display?
>

Thank Zac for that.  He thinks he knows what you want.  ;-)  Apparently 
not huh?

http://forums.gentoo.org/viewtopic-t-901858.html

Just add --quiet-build=n to EMERGE_DEFAULT_OPTS in make.conf and it will 
do it the old way.

Hope that helps.

Dale

:-)  :-)

-- 
I am only responsible for what I said ... Not for what you understood or how you interpreted my words!

Re: [gentoo-user] emerge -j, make -j and make -l

[covici]

Dale <rdalek1967@gmail.com> wrote:

> covici@ccs.covici.com wrote:
> > Mick<michaelkintzios@gmail.com>  wrote:
> >
> >> On Saturday 26 Nov 2011 15:22:15 Michael Mol wrote:
> >>> I just wanted to share an experience I had today with optimizing parallel
> >>> builds after discovering "-l" for Make...
> >>>
> >>> I've got a little more tweaking I still want to do, but this is pretty
> >>> awesome...
> >>>
> >>> http://funnybutnot.wordpress.com/2011/11/26/optimizing-parallel-builds/
> >>>
> >>> ZZ
> >> Thanks for sharing!  How do you determine the optimum value for -l?
> > How do you get emerge not to display number of jobs and load average --
> > I only want to compile one at a time -- much safer that way and it is
> > doing that, but now it displays all that load average and how many jobs,
> > etc. -- any way to get rid of that display?
> >
> 
> Thank Zac for that.  He thinks he knows what you want.  ;-)
> Apparently not huh?
> 
> http://forums.gentoo.org/viewtopic-t-901858.html
> 
> Just add --quiet-build=n to EMERGE_DEFAULT_OPTS in make.conf and it
> will do it the old way.
> 
> Hope that helps.

Thanks much.  Don't these people have anything to do -- like fix ebuild
bugs?  Very strange indeed.

-- 
Your life is like a penny.  You're going to lose it.  The question is:
How do
you spend it?

         John Covici
         covici@ccs.covici.com

Re: [gentoo-user] emerge -j, make -j and make -l

[Dale]

covici@ccs.covici.com wrote:
> Dale<rdalek1967@gmail.com>  wrote:
>
>> covici@ccs.covici.com wrote:
>>> Mick<michaelkintzios@gmail.com>   wrote:
>>>
>>>> On Saturday 26 Nov 2011 15:22:15 Michael Mol wrote:
>>>>> I just wanted to share an experience I had today with optimizing parallel
>>>>> builds after discovering "-l" for Make...
>>>>>
>>>>> I've got a little more tweaking I still want to do, but this is pretty
>>>>> awesome...
>>>>>
>>>>> http://funnybutnot.wordpress.com/2011/11/26/optimizing-parallel-builds/
>>>>>
>>>>> ZZ
>>>> Thanks for sharing!  How do you determine the optimum value for -l?
>>> How do you get emerge not to display number of jobs and load average --
>>> I only want to compile one at a time -- much safer that way and it is
>>> doing that, but now it displays all that load average and how many jobs,
>>> etc. -- any way to get rid of that display?
>>>
>> Thank Zac for that.  He thinks he knows what you want.  ;-)
>> Apparently not huh?
>>
>> http://forums.gentoo.org/viewtopic-t-901858.html
>>
>> Just add --quiet-build=n to EMERGE_DEFAULT_OPTS in make.conf and it
>> will do it the old way.
>>
>> Hope that helps.
> Thanks much.  Don't these people have anything to do -- like fix ebuild
> bugs?  Very strange indeed.
>

Well, they had the poll but the dev thinks he knows better.  So, this is 
Gentoo and the devs rule the roost here.  It's pretty much been that way 
since I started using Gentoo back in 2003.  I don't expect it to change 
and you shouldn't either.  ;-)   Just do like I do, when they change 
something, override it with your own setting.  It works for me.

Dale

:-)  :-)

-- 
I am only responsible for what I said ... Not for what you understood or how you interpreted my words!

Re: [gentoo-user] emerge -j, make -j and make -l

[covici]

Dale <rdalek1967@gmail.com> wrote:

> covici@ccs.covici.com wrote:
> > Dale<rdalek1967@gmail.com>  wrote:
> >
> >> covici@ccs.covici.com wrote:
> >>> Mick<michaelkintzios@gmail.com>   wrote:
> >>>
> >>>> On Saturday 26 Nov 2011 15:22:15 Michael Mol wrote:
> >>>>> I just wanted to share an experience I had today with optimizing parallel
> >>>>> builds after discovering "-l" for Make...
> >>>>>
> >>>>> I've got a little more tweaking I still want to do, but this is pretty
> >>>>> awesome...
> >>>>>
> >>>>> http://funnybutnot.wordpress.com/2011/11/26/optimizing-parallel-builds/
> >>>>>
> >>>>> ZZ
> >>>> Thanks for sharing!  How do you determine the optimum value for -l?
> >>> How do you get emerge not to display number of jobs and load average --
> >>> I only want to compile one at a time -- much safer that way and it is
> >>> doing that, but now it displays all that load average and how many jobs,
> >>> etc. -- any way to get rid of that display?
> >>>
> >> Thank Zac for that.  He thinks he knows what you want.  ;-)
> >> Apparently not huh?
> >>
> >> http://forums.gentoo.org/viewtopic-t-901858.html
> >>
> >> Just add --quiet-build=n to EMERGE_DEFAULT_OPTS in make.conf and it
> >> will do it the old way.
> >>
> >> Hope that helps.
> > Thanks much.  Don't these people have anything to do -- like fix ebuild
> > bugs?  Very strange indeed.
> >
> 
> Well, they had the poll but the dev thinks he knows better.  So, this
> is Gentoo and the devs rule the roost here.  It's pretty much been
> that way since I started using Gentoo back in 2003.  I don't expect it
> to change and you shouldn't either.  ;-)   Just do like I do, when
> they change something, override it with your own setting.  It works
> for me.

Yep, that is the nice thing about gentoo.

Thanks.

-- 
Your life is like a penny.  You're going to lose it.  The question is:
How do
you spend it?

         John Covici
         covici@ccs.covici.com

Re: [gentoo-user] emerge -j, make -j and make -l

[Michael Hampicke]

Am 26.11.2011 16:22, schrieb Michael Mol:
> parallel builds

Sweet, I didn't even know about emerges -j option to do parallel builds.
Thx for sharing, I am sure I'll use this in the feature

Re: [gentoo-user] emerge -j, make -j and make -l

[Pandu Poluan]

[-- Attachment #1: Type: text/plain, Size: 484 bytes --]

On Nov 28, 2011 12:29 AM, "Michael Hampicke" <gentoo-user@hadt.biz> wrote:
>
>
> Am 26.11.2011 16:22, schrieb Michael Mol:
> > parallel builds
>
> Sweet, I didn't even know about emerges -j option to do parallel builds.
> Thx for sharing, I am sure I'll use this in the feature
>

emerge -j will be useful if the packages being emerged are independent from
one another. Packages that depend on another package will be emerged
sequentially after their dependencies are emerged.

Rgds,

[-- Attachment #2: Type: text/html, Size: 638 bytes --]

Re: [gentoo-user] emerge -j, make -j and make -l

[Mark Knecht]

On Sat, Nov 26, 2011 at 7:22 AM, Michael Mol <mikemol@gmail.com> wrote:
> I just wanted to share an experience I had today with optimizing parallel
> builds after discovering "-l" for Make...
>
> I've got a little more tweaking I still want to do, but this is pretty
> awesome...
>
> http://funnybutnot.wordpress.com/2011/11/26/optimizing-parallel-builds/
>
> ZZ

Good post Michael. Thanks.

I want to verify that in make.conf this is indeed MAKEOPTS we are
talking about and not EMERGE_DEFAULT_OPTS.

Currently for my i7-980x (6 physical cores + hyper threading = 12
logical cores) I have:

MAKEOPTS="-j3"
EMERGE_DEFAULT_OPTS="--with-bdeps y "
PORTAGE_IONICE_COMMAND="ionice -c 3 -p \${PID}"

I generally keep -j small day-to-day to allow emerge to work more or
less the background while I'm using the machine for other things. If I
was going to do an emerge -e @world then in the past I'd push it up
for 13. (N+1)

I've not used the -l option but it sounds interesting. If I understand
the then you're suggesting in /etc/make.conf

MAKEOPTS="-j13 -l7"

or something in that range for a full blown emerge -e @world?

- Mark

Re: [gentoo-user] emerge -j, make -j and make -l

[Dale]

Mark Knecht wrote:
> On Sat, Nov 26, 2011 at 7:22 AM, Michael Mol<mikemol@gmail.com>  wrote:
>> I just wanted to share an experience I had today with optimizing parallel
>> builds after discovering "-l" for Make...
>>
>> I've got a little more tweaking I still want to do, but this is pretty
>> awesome...
>>
>> http://funnybutnot.wordpress.com/2011/11/26/optimizing-parallel-builds/
>>
>> ZZ
> Good post Michael. Thanks.
>
> I want to verify that in make.conf this is indeed MAKEOPTS we are
> talking about and not EMERGE_DEFAULT_OPTS.
>
> Currently for my i7-980x (6 physical cores + hyper threading = 12
> logical cores) I have:
>
> MAKEOPTS="-j3"
> EMERGE_DEFAULT_OPTS="--with-bdeps y "
> PORTAGE_IONICE_COMMAND="ionice -c 3 -p \${PID}"
>
> I generally keep -j small day-to-day to allow emerge to work more or
> less the background while I'm using the machine for other things. If I
> was going to do an emerge -e @world then in the past I'd push it up
> for 13. (N+1)
>
> I've not used the -l option but it sounds interesting. If I understand
> the then you're suggesting in /etc/make.conf
>
> MAKEOPTS="-j13 -l7"
>
> or something in that range for a full blown emerge -e @world?
>
> - Mark
>
>


You should also look into this setting if you want to let portage run in 
the background:

PORTAGE_NICENESS=5

I have a 4 core system and run this:

MAKEOPTS="-j16 -l10"
EMERGE_DEFAULT_OPTS="-j8 "

I can't even tell portage is running.  Heck, I even watch videos and 
such.  No problems with slowdown at all.

Oh, the -l10 is sort of new.  The -j16 is changing a bit over time.  
Trying to test speed to see what is fastest.  So far, no compile race 
problems either.

Dale

:-)  :-)

-- 
I am only responsible for what I said ... Not for what you understood or how you interpreted my words!

Re: [gentoo-user] emerge -j, make -j and make -l

[Michael Mol]

On Sun, Nov 27, 2011 at 5:07 PM, Mark Knecht <markknecht@gmail.com> wrote:
> On Sat, Nov 26, 2011 at 7:22 AM, Michael Mol <mikemol@gmail.com> wrote:
>> I just wanted to share an experience I had today with optimizing parallel
>> builds after discovering "-l" for Make...
>>
>> I've got a little more tweaking I still want to do, but this is pretty
>> awesome...
>>
>> http://funnybutnot.wordpress.com/2011/11/26/optimizing-parallel-builds/
>>
>> ZZ
>
> Good post Michael. Thanks.
>
> I want to verify that in make.conf this is indeed MAKEOPTS we are
> talking about and not EMERGE_DEFAULT_OPTS.

It'd be a combination of them.

EMERGE_DEFAULT_OPTS, for my 8-way system, would have been -j8 (I'll be
changing this to reflect portage's load-aware behavior).

MAKEOPTS would be -j16, -l10. (Which actually goes up to about 12 or
13 based on that N*1.6 behavior)

>
> Currently for my i7-980x (6 physical cores + hyper threading = 12
> logical cores) I have:
>
> MAKEOPTS="-j3"
> EMERGE_DEFAULT_OPTS="--with-bdeps y "
> PORTAGE_IONICE_COMMAND="ionice -c 3 -p \${PID}"
>
> I generally keep -j small day-to-day to allow emerge to work more or
> less the background while I'm using the machine for other things. If I
> was going to do an emerge -e @world then in the past I'd push it up
> for 13. (N+1)
>
> I've not used the -l option but it sounds interesting. If I understand
> the then you're suggesting in /etc/make.conf
>
> MAKEOPTS="-j13 -l7"
>
> or something in that range for a full blown emerge -e @world?

Pretty much. Though I wouldn't do it just for @world. I'd leave it in
for all emerges.

-- 
:wq

Re: [gentoo-user] emerge -j, make -j and make -l

[Pandu Poluan]

[-- Attachment #1: Type: text/plain, Size: 1142 bytes --]

On Nov 28, 2011 11:32 AM, "Michael Mol" <mikemol@gmail.com> wrote:
>

----- >8 snip

>
> MAKEOPTS would be -j16, -l10. (Which actually goes up to about 12 or
> 13 based on that N*1.6 behavior)
>

----- >8 snip

Just in case anyone wonders where the multiplier "1.6" comes from:

There had been a discussion somewhere (I forgot where exactly, sorry) about
load numbers. The final conclusion was that the ideal load number for
today's processors is 2*N, because with the out-of-order capability of
modern processors, two instructions can overlap in the pipeline, even
without hyperthreading.

Unfortunately, striving for 2*N will inadvertently result in short bursts
of > 2*N, and this potentially induce a stall, which will be very costly.
1.8*N gives a 10% margin for burst activities, while 1.6*N gives a 20%
margin.

Since emerging packages has a sudden increase in load when autoconfigure
finishes and make fires up all the parallel building, I figure 20% margin
will be better.

Of course, that's before @mikemol made me aware of MAKEOPTS -l, so I am now
tempted to raise the load-average to 1.8*N, letting make handle the bursts.

Rgds,

[-- Attachment #2: Type: text/html, Size: 1346 bytes --]

Re: [gentoo-user] emerge -j, make -j and make -l

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 625 bytes --]

On Mon, 28 Nov 2011 12:26:48 +0700, Pandu Poluan wrote:

> Just in case anyone wonders where the multiplier "1.6" comes from:
> 
> There had been a discussion somewhere (I forgot where exactly, sorry)
> about load numbers. The final conclusion was that the ideal load number
> for today's processors is 2*N, because with the out-of-order capability
> of modern processors, two instructions can overlap in the pipeline, even
> without hyperthreading.

Is N the number of physical cores, or does it include hyperthreaded cores
too?


-- 
Neil Bothwick

Forget the Joneses...I can't keep up with The Simpsons.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] emerge -j, make -j and make -l

[Pandu Poluan]

On Mon, Nov 28, 2011 at 16:27, Neil Bothwick <neil@digimed.co.uk> wrote:
> On Mon, 28 Nov 2011 12:26:48 +0700, Pandu Poluan wrote:
>
>> Just in case anyone wonders where the multiplier "1.6" comes from:
>>
>> There had been a discussion somewhere (I forgot where exactly, sorry)
>> about load numbers. The final conclusion was that the ideal load number
>> for today's processors is 2*N, because with the out-of-order capability
>> of modern processors, two instructions can overlap in the pipeline, even
>> without hyperthreading.
>
> Is N the number of physical cores, or does it include hyperthreaded cores
> too?
>

And that... I forgot.

However, considering hyperthreading is just a 'trick' where a pair
threads of the same context can run together on a core, I'd wager that
N here should be considered as the physical core.

In other words, my guess is that 2*N is the "implicit hyperthreading"
performed by OOO processor cores. On hyperthreaded cores, the 2*N
ability of OOO processors is made "explicit".

TL;DR : 2*N for non-hyperthreaded cores, N for hyperthreaded cores.

(Disclaimer: Above is my pure speculation. Feel free to debunk it
based on your tests (: )

Rgds,
-- 
FdS Pandu E Poluan
~ IT Optimizer ~

 • LOPSA Member #15248
 • Blog : http://pepoluan.tumblr.com
 • Linked-In : http://id.linkedin.com/in/pepoluan

Re: [gentoo-user] emerge -j, make -j and make -l

[Jorge Martínez López]

Hi there

2011/11/28 Pandu Poluan <pandu@poluan.info>:
...
>
> TL;DR : 2*N for non-hyperthreaded cores, N for hyperthreaded cores.
>
> (Disclaimer: Above is my pure speculation. Feel free to debunk it
> based on your tests (: )
>
> Rgds,

So with a 6 core Phenom it would look like this:

MAKEOPTS="-j 12 -l 10"
EMERGE_DEFAULT_OPTS="--with-bdeps=y --jobs --load-average=10"

Does this make sense?

Thanks in advance!
-- 
Jorge Martínez López <jorgeml@gmail.com> http://www.jorgeml.net
      Google Talk / XMPP: jorgeml@gmail.com

Re: [gentoo-user] emerge -j, make -j and make -l

[Pandu Poluan]

2011/11/28 Jorge Martínez López <jorgeml@gmail.com>:
> Hi there
>
> 2011/11/28 Pandu Poluan <pandu@poluan.info>:
> ...
>>
>> TL;DR : 2*N for non-hyperthreaded cores, N for hyperthreaded cores.
>>
>> (Disclaimer: Above is my pure speculation. Feel free to debunk it
>> based on your tests (: )
>>
>> Rgds,
>
> So with a 6 core Phenom it would look like this:
>
> MAKEOPTS="-j 12 -l 10"
> EMERGE_DEFAULT_OPTS="--with-bdeps=y --jobs --load-average=10"
>
> Does this make sense?
>

You can 'push' MAKEOPTS to "-l 10.8", methinks. Although honestly I
don't think a difference of 0.8 will make any discernible effect :-)

(Yes, 'man make' says that "-l" accepts floating-point numbers. I just checked)

But your settings make perfect sense to me :-)

Rgds,
-- 
FdS Pandu E Poluan
~ IT Optimizer ~

 • LOPSA Member #15248
 • Blog : http://pepoluan.tumblr.com
 • Linked-In : http://id.linkedin.com/in/pepoluan

Re: [gentoo-user] emerge -j, make -j and make -l

[Michael Mol]

On Mon, Nov 28, 2011 at 12:26 AM, Pandu Poluan <pandu@poluan.info> wrote:
> On Nov 28, 2011 11:32 AM, "Michael Mol" <mikemol@gmail.com> wrote:

[snip]

> Unfortunately, striving for 2*N will inadvertently result in short bursts of
>> 2*N, and this potentially induce a stall, which will be very costly. 1.8*N
> gives a 10% margin for burst activities, while 1.6*N gives a 20% margin.

Actually, I suspect that 2*N is normally beneficial as a way to cover
for processes blocked on I/O.

I don't believe processes blocked on I/O are counted in the system's
instantaneous load, so they wouldn't be noticed when the kernel polls
to build a load average.

That tells me that, for a load-aware system, you want N for your
load-aware calculations, and 2*N (or thereabouts) for numbers which
aren't load-aware. Make's -j parameter would be an example of the
latter, as it serves as an upper limit when the load average is low,
and -l's algorithm keeps saying "go on, go on!"

I'm currently timing

MAKEOPTS="-j16 -l13"
EMERGE_DEFAULT_OPTS="--jobs --load-average=13"

with 493 packages (base plus X plus XFCE and chromium, and, of course,
USE flags), but I'll start another timed run with

MAKEOPTS="-j16 -l8"
EMERGE_DEFAULT_OPTS="--jobs --load-average=8"

once that's finished. Last night, I tried with -j16 -l10, and that
completed in 209 minutes, but that was still with the
PORTAGE_DEFAULT_OPTS typo, so that datapoint is mostly useless. This
one has already taken about 240 minutes. At least it's finished
building Chromium, now; I hope it doesn't still need to build gcc.
It's at 488/493.

(insert) Just finished:

real    208m23.880s
user    604m27.065s
sys     152m22.848s

Apparently, I misremembered when I started it.
(/insert)

FWIW, I strongly suspect that N should be your number of *logical*
cores, not your number of physical cores. I believe most of the
overhead

-- 
:wq

Re: [gentoo-user] emerge -j, make -j and make -l

[Michael Mol]

On Mon, Nov 28, 2011 at 1:51 PM, Michael Mol <mikemol@gmail.com> wrote:
> On Mon, Nov 28, 2011 at 12:26 AM, Pandu Poluan <pandu@poluan.info> wrote:

[snip]

> FWIW, I strongly suspect that N should be your number of *logical*
> cores, not your number of physical cores. I believe most of the
> overhead

...and finishing my sentence.

I believe most of the overhead comes from context switching between
two threads, and hyperthreading adds CPU hardware specifically for the
purpose of avoiding that costly context switch when controlling which
thread's instructions get sent up the pipeline.


-- 
:wq

Re: [gentoo-user] emerge -j, make -j and make -l

[Michael Mol]

On Mon, Nov 28, 2011 at 1:51 PM, Michael Mol <mikemol@gmail.com> wrote:
> I'm currently timing
>
> MAKEOPTS="-j16 -l13"
> EMERGE_DEFAULT_OPTS="--jobs --load-average=13"
>
> with 493 packages (base plus X plus XFCE and chromium, and, of course,
> USE flags), but I'll start another timed run with
>
> MAKEOPTS="-j16 -l8"
> EMERGE_DEFAULT_OPTS="--jobs --load-average=8"
>
> once that's finished. Last night, I tried with -j16 -l10, and that
> completed in 209 minutes, but that was still with the
> PORTAGE_DEFAULT_OPTS typo, so that datapoint is mostly useless. This
> one has already taken about 240 minutes. At least it's finished
> building Chromium, now; I hope it doesn't still need to build gcc.
> It's at 488/493.
>
> (insert) Just finished:
>
> real    208m23.880s
> user    604m27.065s
> sys     152m22.848s
>
> Apparently, I misremembered when I started it.
> (/insert)
>

MAKEOPTS="-j16 -l8"
EMERGE_DEFAULT_OPTS="--jobs --load-average=8"

finished at

real    217m46.366s
user    588m22.397s
sys     149m15.283s

I'll probably set it to -l10 and leave it there; I need to start using
this box for things.

-- 
:wq

Re: [gentoo-user] emerge -j, make -j and make -l

[Michael Mol]

[-- Attachment #1: Type: text/plain, Size: 1744 bytes --]

Quick update...I now have two of these things set up in a distcc cluster
with my Phenom 9650. ~530 packages in 228m 34s. There's an even larger
initial explosion of parallel emerge jobs, but it spreads out very
nicely...I may have to increase the -j parameter in MAKEOPTS. I'm also not
certain if distcc is properly deferring jobs to the local host, but I got
the impression the local machine was getting a doubled workload when I had
127.0.0.1 in/etc/distccd/hosts.

ZZ
On Nov 28, 2011 6:53 PM, "Michael Mol" <mikemol@gmail.com> wrote:

> On Mon, Nov 28, 2011 at 1:51 PM, Michael Mol <mikemol@gmail.com> wrote:
> > I'm currently timing
> >
> > MAKEOPTS="-j16 -l13"
> > EMERGE_DEFAULT_OPTS="--jobs --load-average=13"
> >
> > with 493 packages (base plus X plus XFCE and chromium, and, of course,
> > USE flags), but I'll start another timed run with
> >
> > MAKEOPTS="-j16 -l8"
> > EMERGE_DEFAULT_OPTS="--jobs --load-average=8"
> >
> > once that's finished. Last night, I tried with -j16 -l10, and that
> > completed in 209 minutes, but that was still with the
> > PORTAGE_DEFAULT_OPTS typo, so that datapoint is mostly useless. This
> > one has already taken about 240 minutes. At least it's finished
> > building Chromium, now; I hope it doesn't still need to build gcc.
> > It's at 488/493.
> >
> > (insert) Just finished:
> >
> > real    208m23.880s
> > user    604m27.065s
> > sys     152m22.848s
> >
> > Apparently, I misremembered when I started it.
> > (/insert)
> >
>
> MAKEOPTS="-j16 -l8"
> EMERGE_DEFAULT_OPTS="--jobs --load-average=8"
>
> finished at
>
> real    217m46.366s
> user    588m22.397s
> sys     149m15.283s
>
> I'll probably set it to -l10 and leave it there; I need to start using
> this box for things.
>
> --
> :wq
>

[-- Attachment #2: Type: text/html, Size: 2353 bytes --]

Re: [gentoo-user] emerge -j, make -j and make -l

[Michael Mol]

On Fri, Dec 9, 2011 at 11:35 PM, Michael Mol <mikemol@gmail.com> wrote:
> Quick update...I now have two of these things set up in a distcc cluster
> with my Phenom 9650. ~530 packages in 228m 34s. There's an even larger
> initial explosion of parallel emerge jobs, but it spreads out very
> nicely...I may have to increase the -j parameter in MAKEOPTS. I'm also not
> certain if distcc is properly deferring jobs to the local host, but I got
> the impression the local machine was getting a doubled workload when I had
> 127.0.0.1 in/etc/distccd/hosts.
>
> ZZ

Another update:

I've been playing with distcc a fair amount. One of the problems I
discovered is that distcc doesn't get leveraged all that much if you
have something like:
MAKEOPTS="-j16 -l8"
EMERGE_DEFAULT_OPTS="--jobs --load-average=8"

It seems that "./configure" scripts and other non-distributable load
pumps up the master system's load average enough that distributable
jobs don't get spawned. (And why should they? Make doesn't know that a
job will get shuttled off to another system.)

After getting Gentoo set up on my new dual-core i3 laptop, I
discovered that it seemed to emerge faster than my 8-core boxes with
MAKEOPS="-j16 -l4"
EMERGE_DEFAULT_OPTS="-jobs --load-average=3"

...It was doing a good job of shuffling jobs off to the two 8-way
machines. (The new laptop only has 100Mbit ethernet, so I've also
enabled LZO for distcc)

With that in mind, I'm going to be trying these settings on the 8-way machines:

MAKEOPTS="-j32 -l16"
EMERGE_DEFAULT_OPTS="--jobs --load-average=12"

-- 
:wq