From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from pigeon.gentoo.org ([208.92.234.80] helo=lists.gentoo.org) by finch.gentoo.org with esmtp (Exim 4.60) (envelope-from ) id 1RUtY1-0004RX-Hs for garchives@archives.gentoo.org; Mon, 28 Nov 2011 05:14:33 +0000 Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id EA00D21C0C6; Mon, 28 Nov 2011 05:14:24 +0000 (UTC) Received: from svr-us4.tirtonadi.com (svr-us4.tirtonadi.com [69.65.43.212]) by pigeon.gentoo.org (Postfix) with ESMTP id 2D56EE05E8 for ; Mon, 28 Nov 2011 05:13:09 +0000 (UTC) Received: from mail-bw0-f53.google.com ([209.85.214.53]) by svr-us4.tirtonadi.com with esmtpsa (TLSv1:RC4-SHA:128) (Exim 4.69) (envelope-from ) id 1RUtWg-001JDn-Qf for gentoo-user@lists.gentoo.org; Mon, 28 Nov 2011 12:13:10 +0700 Received: by bkaq10 with SMTP id q10so9457630bka.40 for ; Sun, 27 Nov 2011 21:13:05 -0800 (PST) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-user@lists.gentoo.org Reply-to: gentoo-user@lists.gentoo.org MIME-Version: 1.0 Received: by 10.205.141.70 with SMTP id jd6mr7733391bkc.13.1322457185501; Sun, 27 Nov 2011 21:13:05 -0800 (PST) Received: by 10.223.96.72 with HTTP; Sun, 27 Nov 2011 21:13:05 -0800 (PST) Received: by 10.223.96.72 with HTTP; Sun, 27 Nov 2011 21:13:05 -0800 (PST) In-Reply-To: References: <201111270927.57294.michaelkintzios@gmail.com> <20111127232108.08cbacfd@digimed.co.uk> Date: Mon, 28 Nov 2011 12:13:05 +0700 Message-ID: Subject: Re: [gentoo-user] emerge -j, make -j and make -l From: Pandu Poluan To: gentoo-user@lists.gentoo.org Content-Type: multipart/alternative; boundary=000e0ce0d68ec5c07104b2c48e4d X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - svr-us4.tirtonadi.com X-AntiAbuse: Original Domain - lists.gentoo.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - poluan.info X-Archives-Salt: a31d3db0-dbb3-4142-a987-34d8d8e66159 X-Archives-Hash: ae217ec01170765e3e95dc3d507a1161 --000e0ce0d68ec5c07104b2c48e4d Content-Type: text/plain; charset=UTF-8 On Nov 28, 2011 11:38 AM, "Michael Mol" wrote: > > On Sun, Nov 27, 2011 at 7:54 PM, Pandu Poluan wrote: > > > > On Nov 28, 2011 6:24 AM, "Neil Bothwick" wrote: > >> > >> On Mon, 28 Nov 2011 00:56:17 +0700, Pandu Poluan wrote: > >> > >> > I don't know where the 'blame' lies, but I've found myself > >> > standardizing on MAKEOPTS=-j3, and PORTAGE_DEFAULT_OPTS="--jobs > >> > --load-average=<1.6*num_of_vCPU>" > >> > > >> > (Yes, no explicit number of jobs. The newer portages are smart enough to > >> > keep starting new jobs until the load number is reached) > >> > >> The problem I found with that is the ebuilds load the system lightly to > >> start with, before they enter the compile phase, to portage starts dozens > >> of parallel ebuilds, then the system gets completely bogged down when > >> they start compiling. > >> > > > > Yes, sometimes that would happen if at the beginning there are network-bound > > ebuilds all downloading their respective distfiles. The load stays low until > > they all start ./configure-ing roughly at the same time. Then all hell > > breaks loose. > > > > I successfully mitigate such "load-explosion" by doing a --fetchonly step > > first, and keeping MAKEOPTS at low -j (which, in my case, is actually > > required). > > > > Just to add more info: I use USE=graphite (with some CFLAGS, uh, > > 'enhancements') with gcc-4.5.3. IIRC, I could push MAKEOPTS up to -j5 (and > > even more, but I ran out of cores) when I was still using gcc-4.4.x and no > > USE=graphite. > > > > Won't file a bug report, though. I have a feeling that my bug report re: > > emerge failure will be marked WONTFIX thanks to the 'ricer special' CFLAGS > > As I noted, "-l" in MAKEOPTS takes care of the load explosion very nicely. Most likely so. I am not aware of -l in MAKEOPTS before, so what I posted was my workaround to prevent load explosion. Thanks to your very useful tip, I now no longer have to worry about load explosion :-) (I still like doing pre-fetchonly-ing, though. But now for a different main reason :-) Rgds, --000e0ce0d68ec5c07104b2c48e4d Content-Type: text/html; charset=UTF-8


On Nov 28, 2011 11:38 AM, "Michael Mol" <mikemol@gmail.com> wrote:
>
> On Sun, Nov 27, 2011 at 7:54 PM, Pandu Poluan <pandu@poluan.info> wrote:
> >
> > On Nov 28, 2011 6:24 AM, "Neil Bothwick" <neil@digimed.co.uk> wrote:
> >>
> >> On Mon, 28 Nov 2011 00:56:17 +0700, Pandu Poluan wrote:
> >>
> >> > I don't know where the 'blame' lies, but I've found myself
> >> > standardizing on MAKEOPTS=-j3, and PORTAGE_DEFAULT_OPTS="--jobs
> >> > --load-average=<1.6*num_of_vCPU>"
> >> >
> >> > (Yes, no explicit number of jobs. The newer portages are smart enough to
> >> > keep starting new jobs until the load number is reached)
> >>
> >> The problem I found with that is the ebuilds load the system lightly to
> >> start with, before they enter the compile phase, to portage starts dozens
> >> of parallel ebuilds, then the system gets completely bogged down when
> >> they start compiling.
> >>
> >
> > Yes, sometimes that would happen if at the beginning there are network-bound
> > ebuilds all downloading their respective distfiles. The load stays low until
> > they all start ./configure-ing roughly at the same time. Then all hell
> > breaks loose.
> >
> > I successfully mitigate such "load-explosion" by doing a --fetchonly step
> > first, and keeping MAKEOPTS at low -j (which, in my case, is actually
> > required).
> >
> > Just to add more info: I use USE=graphite (with some CFLAGS, uh,
> > 'enhancements') with gcc-4.5.3. IIRC, I could push MAKEOPTS up to -j5 (and
> > even more, but I ran out of cores) when I was still using gcc-4.4.x and no
> > USE=graphite.
> >
> > Won't file a bug report, though. I have a feeling that my bug report re:
> > emerge failure will be marked WONTFIX thanks to the 'ricer special' CFLAGS
>
> As I noted, "-l" in MAKEOPTS takes care of the load explosion very nicely.

Most likely so. I am not aware of -l in MAKEOPTS before, so what I posted was my workaround to prevent load explosion. Thanks to your very useful tip, I now no longer have to worry about load explosion :-)

(I still like doing pre-fetchonly-ing, though. But now for a different main reason :-)

Rgds,

--000e0ce0d68ec5c07104b2c48e4d--