* Re: [gentoo-dev] Six month major project on Gentoo
@ 2011-12-22 11:11 99% ` Francesco Riosa
0 siblings, 0 replies; 1+ results
From: Francesco Riosa @ 2011-12-22 11:11 UTC (permalink / raw
To: gentoo-dev
2011/12/22 Rich Freeman <rich0@gentoo.org>:
> On Wed, Dec 21, 2011 at 11:43 PM, Donnie Berkholz <dberkholz@gentoo.org> wrote:
>> I looked into this 6 or 7 years ago. It wasn't feasible unless you were
>> on an extremely high-speed, low-latency network, beyond what was
>> typically accessible at the time outside of universities and LANs. Could
>> be worth exploring again now that 25-100 mbps connections are becoming
>> more common.
>
> I tried messing around with this with Amazon EC2. The problem was
> that due to latency I only really saw the benefit for VERY high levels
> of parallelization (think -j25+).. However, make isn't actually
> "distcc-aware" so it just runs 25 jobs of anything in parallel. So,
> anytime a makefile launched a ton of java or python jobs the host
> ground to a halt as it wasn't distributed and it was way more than the
> host could handle (especially java - which swapped like there was no
> tomorrow).
>
> If somebody were to do a distcc-ng for a large cluster one of the
> problems to solve would be having it not run jobs in parallel if it
> couldn't actually distribute them.
>
> Rich
>
Just wanted to point out that (if there is enough memory) recent
kernels manage much better parallelism, even excess of it, once
reached the maximum load augmenting threads only bring minimal loss of
"real" time.
^ permalink raw reply [relevance 99%]
Results 1-1 of 1 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2011-12-14 11:06 [gentoo-dev] Six month major project on Gentoo Gaurav Saxena
2011-12-14 18:29 ` Alec Warner
2011-12-22 4:43 ` Donnie Berkholz
2011-12-22 10:52 ` Rich Freeman
2011-12-22 11:11 99% ` Francesco Riosa
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox