From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from pigeon.gentoo.org ([208.92.234.80] helo=lists.gentoo.org) by finch.gentoo.org with esmtp (Exim 4.60) (envelope-from ) id 1RdgGl-0004Xc-6Q for garchives@archives.gentoo.org; Thu, 22 Dec 2011 10:53:03 +0000 Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 8F5F621C04B; Thu, 22 Dec 2011 10:52:49 +0000 (UTC) Received: from mail-ey0-f181.google.com (mail-ey0-f181.google.com [209.85.215.181]) by pigeon.gentoo.org (Postfix) with ESMTP id 944C721C074 for ; Thu, 22 Dec 2011 10:52:14 +0000 (UTC) Received: by eaai1 with SMTP id i1so2472071eaa.40 for ; Thu, 22 Dec 2011 02:52:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type; bh=DQIQoYLDlzvLlZaqs8GNBkWcsb3NKn7uMyPYMfDLScs=; b=xonC2uiZxCXVgwURxk49O9ZmOrinhFJAUQj0pHgiU5qMeCem2J1Ye7Pee0MjPyuiEw vqKYBQcLw/vOqPZE+oUsMNVDStuB8AjVUsVM89OMY++UtyDdzc1rfXEjCgUrTv1/Zn+F EQQzEB71dZFGj5Va66Iq3cNQk8Os4bxcxBxVo= Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-dev@lists.gentoo.org Reply-to: gentoo-dev@lists.gentoo.org MIME-Version: 1.0 Received: by 10.204.156.17 with SMTP id u17mr513396bkw.57.1324551133746; Thu, 22 Dec 2011 02:52:13 -0800 (PST) Sender: freemanrich@gmail.com Received: by 10.204.19.1 with HTTP; Thu, 22 Dec 2011 02:52:13 -0800 (PST) In-Reply-To: <20111222044323.GA18095@comet.roc.mn.charter.com> References: <20111222044323.GA18095@comet.roc.mn.charter.com> Date: Thu, 22 Dec 2011 05:52:13 -0500 X-Google-Sender-Auth: AATP6GgDWiaI1_JRNaw-gcggkeE Message-ID: Subject: Re: [gentoo-dev] Six month major project on Gentoo From: Rich Freeman To: gentoo-dev@lists.gentoo.org Content-Type: text/plain; charset=ISO-8859-1 X-Archives-Salt: feb55904-9bf5-434f-8ff8-4bd493030f0b X-Archives-Hash: 1e4850c9d54a5c1bdf43a88561ecc78f On Wed, Dec 21, 2011 at 11:43 PM, Donnie Berkholz wrote: > I looked into this 6 or 7 years ago. It wasn't feasible unless you were > on an extremely high-speed, low-latency network, beyond what was > typically accessible at the time outside of universities and LANs. Could > be worth exploring again now that 25-100 mbps connections are becoming > more common. I tried messing around with this with Amazon EC2. The problem was that due to latency I only really saw the benefit for VERY high levels of parallelization (think -j25+).. However, make isn't actually "distcc-aware" so it just runs 25 jobs of anything in parallel. So, anytime a makefile launched a ton of java or python jobs the host ground to a halt as it wasn't distributed and it was way more than the host could handle (especially java - which swapped like there was no tomorrow). If somebody were to do a distcc-ng for a large cluster one of the problems to solve would be having it not run jobs in parallel if it couldn't actually distribute them. Rich