From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) by finch.gentoo.org (Postfix) with ESMTP id 4C13D138010 for ; Sun, 2 Sep 2012 01:27:57 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 2E48AE0495; Sun, 2 Sep 2012 01:27:41 +0000 (UTC) Received: from mail-bk0-f53.google.com (mail-bk0-f53.google.com [209.85.214.53]) by pigeon.gentoo.org (Postfix) with ESMTP id 673BDE039A for ; Sun, 2 Sep 2012 01:26:51 +0000 (UTC) Received: by bkwj4 with SMTP id j4so1781420bkw.40 for ; Sat, 01 Sep 2012 18:26:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=RHTfssHe83mg7gvz3KVJ+HtLnwwtdK7deAHPg+qxPy4=; b=OjOEHMFWa0WgqY9nxHUTL1VOtFpUOeffyc8IGXw8P1lKjM50LI2n/F4T8ODpBcLreg EMuozVV1kzqbqHhEVEFQHBqNv0+qiib4iR3WuTE+CfEGTjP+6lA42BfkY5vr6Xo5BwxB GC3QJK56G/guB9q1SSR35wDHV3LKeHNxFPBAXEhjUDz/ZceqKDT9VxRofmhRElf9ZiwC uHC8BGm1wH3eo3NNvOELN3e1+O2Zh73wuaMPHnEFtisbtUGwuksP4EWtXJSv6aYO0xWH jbVH4+7P8kVQqwxjgjl0sRsR4y8wXRtVa1H/8UN1Dd6Ywa+yKugmS/Y/o9K3DqEiwc8q 7Hqg== Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-dev@lists.gentoo.org Reply-to: gentoo-dev@lists.gentoo.org MIME-Version: 1.0 Received: by 10.205.133.11 with SMTP id hw11mr5698159bkc.46.1346549211248; Sat, 01 Sep 2012 18:26:51 -0700 (PDT) Received: by 10.204.112.211 with HTTP; Sat, 1 Sep 2012 18:26:51 -0700 (PDT) In-Reply-To: <20120902002002.GB25302@localhost> References: <20544.29691.208130.35494@a1i15.kph.uni-mainz.de> <20120831154521.5258c549@googlemail.com> <20120831111244.0c17b8aa@gentoo.org> <20120902002002.GB25302@localhost> Date: Sat, 1 Sep 2012 21:26:51 -0400 Message-ID: Subject: Re: [gentoo-dev] EJOBS variable for EAPI 5? (was: [RFC] Create a JOBS variable to replace -jX in MAKEOPTS) From: Michael Mol To: gentoo-dev@lists.gentoo.org Content-Type: text/plain; charset=UTF-8 X-Archives-Salt: a1e749e2-6c37-469d-8867-32881991978f X-Archives-Hash: a99cfcbb365a4d2d1b0e532de6a6ae70 On Sat, Sep 1, 2012 at 8:20 PM, Brian Harring wrote: > On Fri, Aug 31, 2012 at 11:12:44AM -0400, Alexis Ballier wrote: >> On Fri, 31 Aug 2012 15:45:21 +0100 >> Ciaran McCreesh wrote: >> >> > On Fri, 31 Aug 2012 10:21:15 +0200 >> > Ulrich Mueller wrote: >> > > Coming back to this old topic [1]. Is there still consensus that we >> > > should have such an EJOBS variable? (It shouldn't be called JOBS >> > > because this name is too generic, see the old discussion.) Then we >> > > could add it to EAPI 5. >> > > >> > > Ulrich >> > > >> > > [1] >> > > >> > >> > If we're doing this, do we tell users to stop setting MAKEOPTS for >> > EAPIs 5 and greater? >> >> How can this work ? I cant think of any simple solution. >> >> > Do we change the name of MAKEOPTS for EAPIs 5 and >> > greater instead? Do we put fancy code in the package mangler to deal >> > with it? >> >> IMHO EAPI-5 compliant PMs should do MAKEOPTS="$MAKEOPTS -j$EJOBS" for >> every EAPI; using EJOBS from ebuilds/eclasses is allowed only in EAPI 5 >> and greater. >> This is retroactive but could be classified 'PM internals' so its fine >> imho. > > This approach is fine imo, although I'd *potentially* look at adding a > magic $PROC_COUNT var that is the # of cpu threads on the system; > either that or defaulting jobs to it. > > I rather dislike requiring users to go jam a 2/4/8 in there when it's > easy to compute. That said, it's minor. On this point, I use dynamic load-based job counting. I'd be using distcc, too, but I don't have the spare hardware for a second build box right now. For a single box, I target a load average of $PROC_COUNT, with a max jobs in the vicinity of 2x$PROC_COUNT, to prevent I/O hangs from causing job count explosions. (Where I/O hiccups aren't a concern, I've found leaving --jobs open-ended worked fine, but builds always eventually expand to consume available I/O as upstream things get larger) I find this works very well; emerge and make both manage to lurk near the optimum point of keeping the CPU busy without spending too much time in context switches. In a distcc context, I would most likely target a load average of $LOCAL_PROC_COUNT, with a max jobs of (2*$LOCAL_PROC_COUNT+$REMOTE_PROC_COUNT). If ebuilds are try to be more aware of parallel-build contexts, I think it's important they're not limited to static job counts. -- :wq