From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from pigeon.gentoo.org ([208.92.234.80] helo=lists.gentoo.org) by finch.gentoo.org with esmtp (Exim 4.60) (envelope-from ) id 1RfYmI-0005cA-K9 for garchives@archives.gentoo.org; Tue, 27 Dec 2011 15:17:22 +0000 Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id EAF3C21C0DD; Tue, 27 Dec 2011 15:17:11 +0000 (UTC) Received: from mail-ey0-f181.google.com (mail-ey0-f181.google.com [209.85.215.181]) by pigeon.gentoo.org (Postfix) with ESMTP id B884B21C0A7 for ; Tue, 27 Dec 2011 15:16:09 +0000 (UTC) Received: by eaai1 with SMTP id i1so6130643eaa.40 for ; Tue, 27 Dec 2011 07:16:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=r2drq//Z3yHfFFuLDrXr4l03WQtP/JXasQLBc9UPl1U=; b=uyiACw58sNIfb8vGXSfIVqY/q2sfat+7chiKCFN8Wf3Sh5Bis1e7hzbMUaAUznmgow V6oIS5MFVysdPmwDz6/G4Yf+f/XrxBSjcODExP6N4sskmojd1Wp7E86/BEDxg54QxfH/ hf9cRM1aZLJ7aygyFTem7G+zDsIafCvMlZJx0= Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-user@lists.gentoo.org Reply-to: gentoo-user@lists.gentoo.org MIME-Version: 1.0 Received: by 10.204.145.69 with SMTP id c5mr7148917bkv.80.1324998968784; Tue, 27 Dec 2011 07:16:08 -0800 (PST) Received: by 10.204.226.72 with HTTP; Tue, 27 Dec 2011 07:16:08 -0800 (PST) In-Reply-To: References: Date: Tue, 27 Dec 2011 10:16:08 -0500 Message-ID: Subject: Re: [gentoo-user] emerge -j, make -j and make -l From: Michael Mol To: gentoo-user@lists.gentoo.org Content-Type: text/plain; charset=UTF-8 X-Archives-Salt: 38645d83-6315-4daa-9bb4-28b994894e7f X-Archives-Hash: 4e67b7dd90c5ae389a4b1064c6c40880 On Fri, Dec 9, 2011 at 11:35 PM, Michael Mol wrote: > Quick update...I now have two of these things set up in a distcc cluster > with my Phenom 9650. ~530 packages in 228m 34s. There's an even larger > initial explosion of parallel emerge jobs, but it spreads out very > nicely...I may have to increase the -j parameter in MAKEOPTS. I'm also not > certain if distcc is properly deferring jobs to the local host, but I got > the impression the local machine was getting a doubled workload when I had > 127.0.0.1 in/etc/distccd/hosts. > > ZZ Another update: I've been playing with distcc a fair amount. One of the problems I discovered is that distcc doesn't get leveraged all that much if you have something like: MAKEOPTS="-j16 -l8" EMERGE_DEFAULT_OPTS="--jobs --load-average=8" It seems that "./configure" scripts and other non-distributable load pumps up the master system's load average enough that distributable jobs don't get spawned. (And why should they? Make doesn't know that a job will get shuttled off to another system.) After getting Gentoo set up on my new dual-core i3 laptop, I discovered that it seemed to emerge faster than my 8-core boxes with MAKEOPS="-j16 -l4" EMERGE_DEFAULT_OPTS="-jobs --load-average=3" ...It was doing a good job of shuffling jobs off to the two 8-way machines. (The new laptop only has 100Mbit ethernet, so I've also enabled LZO for distcc) With that in mind, I'm going to be trying these settings on the 8-way machines: MAKEOPTS="-j32 -l16" EMERGE_DEFAULT_OPTS="--jobs --load-average=12" -- :wq