From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from pigeon.gentoo.org ([208.92.234.80] helo=lists.gentoo.org) by finch.gentoo.org with esmtp (Exim 4.60) (envelope-from ) id 1RUphl-0003A7-8x for garchives@archives.gentoo.org; Mon, 28 Nov 2011 01:08:21 +0000 Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 973C121C086; Mon, 28 Nov 2011 01:08:02 +0000 (UTC) Received: from svr-us4.tirtonadi.com (svr-us4.tirtonadi.com [69.65.43.212]) by pigeon.gentoo.org (Postfix) with ESMTP id 4549921C028 for ; Mon, 28 Nov 2011 01:06:47 +0000 (UTC) Received: from mail-bw0-f53.google.com ([209.85.214.53]) by svr-us4.tirtonadi.com with esmtpsa (TLSv1:RC4-SHA:128) (Exim 4.69) (envelope-from ) id 1RUpgF-004AYO-Ct for gentoo-user@lists.gentoo.org; Mon, 28 Nov 2011 08:06:47 +0700 Received: by bkaq10 with SMTP id q10so9210104bka.40 for ; Sun, 27 Nov 2011 17:06:43 -0800 (PST) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-user@lists.gentoo.org Reply-to: gentoo-user@lists.gentoo.org MIME-Version: 1.0 Received: by 10.204.16.67 with SMTP id n3mr44814312bka.6.1322442403197; Sun, 27 Nov 2011 17:06:43 -0800 (PST) Received: by 10.223.74.16 with HTTP; Sun, 27 Nov 2011 17:06:42 -0800 (PST) Received: by 10.223.74.16 with HTTP; Sun, 27 Nov 2011 17:06:42 -0800 (PST) In-Reply-To: References: <201111270927.57294.michaelkintzios@gmail.com> Date: Mon, 28 Nov 2011 08:06:42 +0700 Message-ID: Subject: Re: [gentoo-user] emerge -j, make -j and make -l From: Pandu Poluan To: gentoo-user@lists.gentoo.org Content-Type: multipart/alternative; boundary=00032555f6eaadb3bc04b2c11d04 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - svr-us4.tirtonadi.com X-AntiAbuse: Original Domain - lists.gentoo.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - poluan.info X-Archives-Salt: c388e555-3238-4fa8-9ac7-0b1ff037931c X-Archives-Hash: 1fbc2a16fd2f73f15d0d0994d8a3aedf --00032555f6eaadb3bc04b2c11d04 Content-Type: text/plain; charset=UTF-8 On Nov 28, 2011 3:21 AM, "Michael Mol" wrote: > > On Sun, Nov 27, 2011 at 12:56 PM, Pandu Poluan wrote: > > On Nov 27, 2011 5:12 PM, "Michael Mol" wrote: > > [snip] > > > > > Here's my experience: > > > > I always experience emerge failures on my Gentoo VMs if I use MAKEOPTS=-j>3. > > Not all packages, but many. Including, IIRC, glibc and gcc. > > In my barebones 177-package state, I didn't get any build failures > from parallel building, either via emerge -j or make -j. I did get one > failure when I went to install X that worked fine on the second > attempt. > And in my barebones system (172 packages), there are several packages that reliably fail, even when emerged on their own *sigh* > > This happens even if I make sure that there's just one emerge job being > > done. And this happens even if I allocate more vCPUs than -j, on VMware and > > XenServer alike. > > FWIW, I've been running with MAKEOPTS=-j10 on my Phenom 9650 for over > a year. It's very rare that something breaks due to the parallel > build. I think it's happened perhaps three times, and each time was > resolvable with a retry. YMMV, of course; race conditions are finicky. > I have a hunch that the hypervisor had some effects. Most likely, another VM in the same host has a high enough load that necessitates the Gentoo VM to be shifted to another (physical) core, and this messes up the timing. MAKEOPTS=-j3 always succeeds, though. So I'm disinclined to delve deeper into the issue. > > I don't know where the 'blame' lies, but I've found myself standardizing on > > MAKEOPTS=-j3, and PORTAGE_DEFAULT_OPTS="--jobs > > --load-average=<1.6*num_of_vCPU>" > > > > (Yes, no explicit number of jobs. The newer portages are smart enough to > > keep starting new jobs until the load number is reached) > > Sweet; I didn't know about Portage's --load-average; I'll definitely > switch to that instead of -j. Load-driven make plus load-driven > portage should work beautifully on my system. > > I'll steal your 1.6 factor, and give: > MAKEOPTS=-j <2*N> -l <1.6*N) > PORTAGE_DEFAULT_OPTS="--jobs --load-average<1.6*N>" > a try. > Make sure that make supports non-integer values for -l, though. > How does that interact with distcc, by the way? I've got two of these > octo-core Xeon boxes, and I've still got my Phenom 9650--distcc on my > home network should become very, very nice. And this box is consuming > 255W at the wall with monitor off, 326W with monitor on. That's not > bad. Though perhaps I should move to an apartment where heat isn't > free... > Well, I don't use distcc, so I can't speculate. But with plain single system compilation, my settings have been serving me real well :-) Rgds, --00032555f6eaadb3bc04b2c11d04 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable


On Nov 28, 2011 3:21 AM, "Michael Mol" <mikemol@gmail.com> wrote:
>
> On Sun, Nov 27, 2011 at 12:56 PM, Pandu Poluan <pandu@poluan.info> wrote:
> > On Nov 27, 2011 5:12 PM, "Michael Mol" <mikemol@gmail.com> wrote:
>
> [snip]
>
> >
> > Here's my experience:
> >
> > I always experience emerge failures on my Gentoo VMs if I use MAK= EOPTS=3D-j>3.
> > Not all packages, but many. Including, IIRC, glibc and gcc.
>
> In my barebones 177-package state, I didn't get any build failures=
> from parallel building, either via emerge -j or make -j. I did get one=
> failure when I went to install X that worked fine on the second
> attempt.
>

And in my barebones system (172 packages), there are several packages th= at reliably fail, even when emerged on their own *sigh*

> > This happens even if I make sure that there's just one eme= rge job being
> > done. And this happens even if I allocate more vCPUs than -j, on = VMware and
> > XenServer alike.
>
> FWIW, I've been running with MAKEOPTS=3D-j10 on my Phenom 9650 for= over
> a year. It's very rare that something breaks due to the parallel > build. I think it's happened perhaps three times, and each time wa= s
> resolvable with a retry. YMMV, of course; race conditions are finicky.=
>

I have a hunch that the hypervisor had some effects. Most likely, anothe= r VM in the same host has a high enough load that necessitates the Gentoo V= M to be shifted to another (physical) core, and this messes up the timing.<= /p>

MAKEOPTS=3D-j3 always succeeds, though. So I'm disinclined to delve = deeper into the issue.

> > I don't know where the 'blame' lies, but I've = found myself standardizing on
> > MAKEOPTS=3D-j3, and PORTAGE_DEFAULT_OPTS=3D"--jobs
> > --load-average=3D<1.6*num_of_vCPU>"
> >
> > (Yes, no explicit number of jobs. The newer portages are smart en= ough to
> > keep starting new jobs until the load number is reached)
>
> Sweet; I didn't know about Portage's --load-average; I'll = definitely
> switch to that instead of -j. Load-driven make plus load-driven
> portage should work beautifully on my system.
>
> I'll steal your 1.6 factor, and give:
> =C2=A0 =C2=A0MAKEOPTS=3D-j <2*N> -l <1.6*N)
> =C2=A0 =C2=A0PORTAGE_DEFAULT_OPTS=3D"--jobs --load-average<1.6= *N>"
> a try.
>

Make sure that make supports non-integer values for -l, though.

> How does that interact with distcc, by the way? I've got two of= these
> octo-core Xeon boxes, and I've still got my Phenom 9650--distcc on= my
> home network should become very, very nice. And this box is consuming<= br> > 255W at the wall with monitor off, 326W with monitor on. That's no= t
> bad. Though perhaps I should move to an apartment where heat isn't=
> free...
>

Well, I don't use distcc, so I can't speculate. But with plain s= ingle system compilation, my settings have been serving me real well :-)

Rgds,

--00032555f6eaadb3bc04b2c11d04--