From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from pigeon.gentoo.org ([208.92.234.80] helo=lists.gentoo.org) by finch.gentoo.org with esmtp (Exim 4.60) (envelope-from ) id 1RUppr-0005bl-1g for garchives@archives.gentoo.org; Mon, 28 Nov 2011 01:16:43 +0000 Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 9004721C107; Mon, 28 Nov 2011 01:16:33 +0000 (UTC) Received: from smtp.gentoo.org (smtp.gentoo.org [140.211.166.183]) by pigeon.gentoo.org (Postfix) with ESMTP id AE2F821C058 for ; Mon, 28 Nov 2011 01:15:19 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp.gentoo.org (Postfix) with ESMTP id 1867E1B4006 for ; Mon, 28 Nov 2011 01:15:19 +0000 (UTC) X-Virus-Scanned: by amavisd-new using ClamAV at gentoo.org X-Spam-Flag: NO X-Spam-Score: -4.499 X-Spam-Level: X-Spam-Status: No, score=-4.499 tagged_above=-999 required=5.5 tests=[BAYES_00=-1.9, DKIM_ADSP_CUSTOM_MED=0.001, FREEMAIL_FROM=0.001, NML_ADSP_CUSTOM_MED=0.9, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-1.201] autolearn=ham Received: from smtp.gentoo.org ([127.0.0.1]) by localhost (smtp.gentoo.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vfsbdlxN9Qkt for ; Mon, 28 Nov 2011 01:15:12 +0000 (UTC) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by smtp.gentoo.org (Postfix) with ESMTP id 4272E65386 for ; Mon, 28 Nov 2011 01:15:10 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1RUpoI-0006gG-IR for gentoo-user@gentoo.org; Mon, 28 Nov 2011 02:15:06 +0100 Received: from cpe-76-183-217-112.tx.res.rr.com ([76.183.217.112]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 28 Nov 2011 02:15:06 +0100 Received: from ftn768 by cpe-76-183-217-112.tx.res.rr.com with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 28 Nov 2011 02:15:06 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: gentoo-user@lists.gentoo.org From: Jack Byer Subject: [gentoo-user] Re: emerge -j, make -j and make -l Date: Sun, 27 Nov 2011 19:10:25 -0600 Message-ID: References: <201111270927.57294.michaelkintzios@gmail.com> <20111127232108.08cbacfd@digimed.co.uk> Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-user@lists.gentoo.org Reply-to: gentoo-user@lists.gentoo.org Mime-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 7Bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: cpe-76-183-217-112.tx.res.rr.com User-Agent: KNode/4.7.3 X-Archives-Salt: 5faa18b8-940d-4fd8-af07-149cf86bbbd4 X-Archives-Hash: bc34ca19016a6b50e78936e0393d260b Pandu Poluan wrote: > On Nov 28, 2011 6:24 AM, "Neil Bothwick" wrote: >> >> On Mon, 28 Nov 2011 00:56:17 +0700, Pandu Poluan wrote: >> >> > I don't know where the 'blame' lies, but I've found myself >> > standardizing on MAKEOPTS=-j3, and PORTAGE_DEFAULT_OPTS="--jobs >> > --load-average=<1.6*num_of_vCPU>" >> > >> > (Yes, no explicit number of jobs. The newer portages are smart enough >> > to keep starting new jobs until the load number is reached) >> >> The problem I found with that is the ebuilds load the system lightly to >> start with, before they enter the compile phase, to portage starts dozens >> of parallel ebuilds, then the system gets completely bogged down when >> they start compiling. >> > > Yes, sometimes that would happen if at the beginning there are > network-bound ebuilds all downloading their respective distfiles. The load > stays low until they all start ./configure-ing roughly at the same time. > Then all hell breaks loose. > > I successfully mitigate such "load-explosion" by doing a --fetchonly step > first, and keeping MAKEOPTS at low -j (which, in my case, is actually > required). > > Just to add more info: I use USE=graphite (with some CFLAGS, uh, > 'enhancements') with gcc-4.5.3. IIRC, I could push MAKEOPTS up to -j5 (and > even more, but I ran out of cores) when I was still using gcc-4.4.x and no > USE=graphite. > > Won't file a bug report, though. I have a feeling that my bug report re: > emerge failure will be marked WONTFIX thanks to the 'ricer special' CFLAGS > :-P > > Rgds, It would be nice if there were some way to mark particular packages that should never be compiled in parallel (like the trick for using a using a separate, non-tmpfs build directory for large packages). The "load-explosion" you describe is bad enough with regular packages but when firefox, xulrunner, chromium and libreoffice all decide to start compiling at the same time it turns into a complete nightmare.