Re: [gentoo-dev] Six month major project on Gentoo

[Francesco Riosa <vivo75@gmail.com>]

2011/12/22 Rich Freeman <rich0@gentoo.org>:
> On Wed, Dec 21, 2011 at 11:43 PM, Donnie Berkholz <dberkholz@gentoo.org> wrote:
>> I looked into this 6 or 7 years ago. It wasn't feasible unless you were
>> on an extremely high-speed, low-latency network, beyond what was
>> typically accessible at the time outside of universities and LANs. Could
>> be worth exploring again now that 25-100 mbps connections are becoming
>> more common.
>
> I tried messing around with this with Amazon EC2.  The problem was
> that due to latency I only really saw the benefit for VERY high levels
> of parallelization (think -j25+)..  However, make isn't actually
> "distcc-aware" so it just runs 25 jobs of anything in parallel.  So,
> anytime a makefile launched a ton of java or python jobs the host
> ground to a halt as it wasn't distributed and it was way more than the
> host could handle (especially java - which swapped like there was no
> tomorrow).
>
> If somebody were to do a distcc-ng for a large cluster one of the
> problems to solve would be having it not run jobs in parallel if it
> couldn't actually distribute them.
>
> Rich
>

Just wanted to point out that (if there is enough memory) recent
kernels manage much better parallelism, even excess of it, once
reached the maximum load augmenting threads only bring minimal loss of
"real" time.