From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) by finch.gentoo.org (Postfix) with ESMTP id CFBF913877A for ; Sat, 2 Aug 2014 16:53:55 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id B33E8E0AD2; Sat, 2 Aug 2014 16:53:49 +0000 (UTC) Received: from smtp.gentoo.org (smtp.gentoo.org [140.211.166.183]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id 9F83FE0AC9 for ; Sat, 2 Aug 2014 16:53:48 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp.gentoo.org (Postfix) with ESMTP id D48223400EE for ; Sat, 2 Aug 2014 16:53:47 +0000 (UTC) X-Virus-Scanned: by amavisd-new using ClamAV at gentoo.org X-Spam-Flag: NO X-Spam-Score: -1.684 X-Spam-Level: X-Spam-Status: No, score=-1.684 tagged_above=-999 required=5.5 tests=[AWL=-0.314, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-0.668, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=unavailable Received: from smtp.gentoo.org ([IPv6:::ffff:127.0.0.1]) by localhost (smtp.gentoo.org [IPv6:::ffff:127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zivQJ_mya7Ke for ; Sat, 2 Aug 2014 16:53:42 +0000 (UTC) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by smtp.gentoo.org (Postfix) with ESMTPS id E318034000E for ; Sat, 2 Aug 2014 16:53:41 +0000 (UTC) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1XDcYr-0000kE-DX for gentoo-user@gentoo.org; Sat, 02 Aug 2014 18:53:37 +0200 Received: from rrcs-71-40-157-251.se.biz.rr.com ([71.40.157.251]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sat, 02 Aug 2014 18:53:37 +0200 Received: from wireless by rrcs-71-40-157-251.se.biz.rr.com with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sat, 02 Aug 2014 18:53:37 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: gentoo-user@lists.gentoo.org From: James Subject: [gentoo-user] Re: Recommendations for scheduler Date: Sat, 2 Aug 2014 16:53:26 +0000 (UTC) Message-ID: References: <53DBCF34.6060601@gmail.com> <53DCB06A.5080600@gmail.com> <1424820.ClPMfftIKn@andromeda> <53DCEF97.4070901@gmail.com> Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-user@lists.gentoo.org Reply-to: gentoo-user@lists.gentoo.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: sea.gmane.org User-Agent: Loom/3.14 (http://gmane.org/) X-Loom-IP: 71.40.157.251 (Mozilla/5.0 (X11; Linux x86_64; rv:29.0) Gecko/20100101 Firefox/29.0 SeaMonkey/2.26.1) X-Archives-Salt: 7be49698-fa2f-4433-9d75-ab0b5297e5cc X-Archives-Hash: 49cc48fd7a358f35c5016912f70aaa4e Alan McKinnon gmail.com> writes: > Well, we've found 2 projects that at least in part seek to achieve our > general goals - chronos and Martin's new project. > Why don't we both fool around with them for a bit and get a sense of > what it will take to add features etc? Then we can meet back here and > discuss. Always better to build on an existing foundation Mesos looks promising for a variety of (Apache) reasons. Some key technologies folks may want google about that are related: Quincy (fair schedular) Chronos (scheduler) Hadoop (scheduler) HDFS (clusterd file system) http://gpo.zugaina.org/sys-cluster/apache-hadoop-common Zookeeper (Fault tolerance) SPARK ( optimized for interative jobs where a datase is resued in many parallel operations (advanced math/science and many other apps.) https://spark.apache.org/ Dryad Torque Mpiche2 MPI Globus tookit mesos_tech_report.pdf It looks as though Amazon, google, facebook and many others large in the Cluster/Cloud arena are using Mesos......? So let's all post what we find, particularly in overlays. hth, James