From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-14) on finch.gentoo.org X-Spam-Level: X-Spam-Status: No, score=-0.1 required=5.0 tests=DMARC_NONE, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI autolearn=unavailable autolearn_force=no version=4.0.0 Received: from twobit.net (host165.spe.iit.edu [198.37.27.165]) by chiba.3jane.net (Postfix) with SMTP id 18A9AABDA1 for ; Sun, 9 Jun 2002 17:03:28 -0500 (CDT) Received: (qmail 3966 invoked by uid 1000); 9 Jun 2002 21:49:13 -0000 Date: Sun, 9 Jun 2002 16:49:13 -0500 From: Nick Jones To: gentoo-dev@gentoo.org Subject: Re: [gentoo-dev] Distribuited build Message-ID: <20020609164913.A3863@twobit.net> References: <3D03BB1C.8010405@seul.org> <200206091638.57868.pauldv@cs.kun.nl> <3D03C5F3.2050600@seul.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.2i In-Reply-To: <3D03C5F3.2050600@seul.org>; from marko@seul.org on Sun, Jun 09, 2002 at 05:17:39PM -0400 Sender: gentoo-dev-admin@gentoo.org Errors-To: gentoo-dev-admin@gentoo.org X-BeenThere: gentoo-dev@gentoo.org X-Mailman-Version: 2.0.6 Precedence: bulk Reply-To: gentoo-dev@gentoo.org List-Help: List-Post: List-Subscribe: , List-Id: Gentoo Linux developer list List-Unsubscribe: , List-Archive: X-Archives-Salt: d8a8b9c7-1042-4cea-8750-5611ea65bd2d X-Archives-Hash: efb1c225e5f49600dcb6c13bcad36da0 OpenMosix is not a viable option for compiling with "make -j". The threads of a source file building do not exist long enough to be migrated. I tired it... Didn't do very well. > Paul de Vrieze wrote: > > On Sunday 09 June 2002 22:31, Marko Mikulicic wrote > > > > > > While I didn't test it yet, openmosix could be a solution for better > > utilization of your computing power. Parallel dependencies are sometimes > > difficult (cross dependencies), also it would require tweaks to emerge > > > > Well, I delibertely left out openmosix because it is automatically > supported with make -j n. However there is a big issue with it: > Openmosix process migration can distribute the load of a single system > through a cluster but if you install n packages they are build in > sequence so that only compiles from a single build get eventually > distributed. And it's even worst when some packages have parallel build > disabled. > Moreover openmosix is not applicable in a etherogeneous network, whith > freebsd, linux, win32, tru64, linux-alpha machines, while distcc can > execute cross-compilers on every architecture (windows too). > and openmosix requires all to run a special kernel, administrative > issues,.... I think that something lighter is needed, for it to be > really useful. I have access to a LAN with 16 PIV 1.5Gh but with any > kindof linux distros, versions, libraries, users. The only > administrative power I have is to run niced jobs and a good amount of > ram. > > However the problem is not openmosix vs. distcc, while both are > completely transparent with portage and both have the same problem: > sequential builds. > > As for cross dependencies, how does portage handle the now? There could > a queue for each pending build where dependent build are added. > RDEPEND does not conflict with paralell builds. > I agree this would require tweaks to emerge. I think it's not > urgent in portage1, but it could go in portage2, which with it's added > performance could resolve this kind of dependence in acceptable time. > What do you think ? > > Marko