From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) by finch.gentoo.org (Postfix) with ESMTP id 5899513838B for ; Thu, 18 Sep 2014 15:32:52 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id C7414E0A83; Thu, 18 Sep 2014 15:32:45 +0000 (UTC) Received: from smtp.gentoo.org (smtp.gentoo.org [140.211.166.183]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id 7634BE09EA for ; Thu, 18 Sep 2014 15:32:44 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp.gentoo.org (Postfix) with ESMTP id 9920D34016B for ; Thu, 18 Sep 2014 15:32:43 +0000 (UTC) X-Virus-Scanned: by amavisd-new using ClamAV at gentoo.org X-Spam-Flag: NO X-Spam-Score: -1.661 X-Spam-Level: X-Spam-Status: No, score=-1.661 tagged_above=-999 required=5.5 tests=[AWL=-0.308, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-0.651, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=unavailable Received: from smtp.gentoo.org ([IPv6:::ffff:127.0.0.1]) by localhost (smtp.gentoo.org [IPv6:::ffff:127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hAAoa_KNpIOS for ; Thu, 18 Sep 2014 15:32:37 +0000 (UTC) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by smtp.gentoo.org (Postfix) with ESMTPS id 6DE7E3401E6 for ; Thu, 18 Sep 2014 15:32:35 +0000 (UTC) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1XUdh9-0007HZ-O7 for gentoo-user@gentoo.org; Thu, 18 Sep 2014 17:32:31 +0200 Received: from rrcs-71-40-157-251.se.biz.rr.com ([71.40.157.251]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 18 Sep 2014 17:32:31 +0200 Received: from wireless by rrcs-71-40-157-251.se.biz.rr.com with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 18 Sep 2014 17:32:31 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: gentoo-user@lists.gentoo.org From: James Subject: [gentoo-user] Re: File system testing Date: Thu, 18 Sep 2014 15:32:19 +0000 (UTC) Message-ID: References: <5419CEB1.6090900@guillemet.org> Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-user@lists.gentoo.org Reply-to: gentoo-user@lists.gentoo.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: sea.gmane.org User-Agent: Loom/3.14 (http://gmane.org/) X-Loom-IP: 71.40.157.251 (Mozilla/5.0 (X11; Linux x86_64; rv:29.0) Gecko/20100101 Firefox/29.0 SeaMonkey/2.26.1) X-Archives-Salt: df941a83-8856-42fd-bdef-5be76ce81e05 X-Archives-Hash: 0985278d3a8e456acd4fbb30c94444e2 Hervé Guillemet guillemet.org> writes: > > Le 16/09/2014 21:07, James a écrit : > > > > By now many are familiar with my keen interest in clustering gentoo > > systems. So, what most cluster technologies use is a distributed file > > system on top of the local (HD/SDD) file system. > Have you found this document : > http://hal.inria.fr/hal-00789086/PDF/a_survey_of_dfs.pdf Hello Herve, Yes, I read the document and it is a good introduction to some of my issues on which file system(s) to use for clustering. But, it's more of a survey than a comparison/benchmark study, which would be really beneficial. DFS are moving so fast now, and their setups and features are rarely a one to one match. For example, (currently) the best load balancing you find, is actually in the apps that run above the cluster software. [1] Some of the performance/resource-utilizations of the files systems/resources are determined by real-time analytics with graphical displays. I'm not sure that load balancing even belongs in a DFS, yet in the paper you reference, it was prominently discussed. Things are moving so fast there in the distributed-*/cluster/cluster-tools/cluster-apps space, one really need a system set up to apply almost daily patches for testing. I never realize just how much reading is necessary just to understand the current landscape in clustering. I'm trying to figure out an echo_system where gentoo folks can experiment wtih mesos clustering for scientific applications. After that, the more general case should be mature enough for general purpose applications. I'm avoiding the clustered web arena, as that is just too much for me to digest; so somebody else could champion that part of all of those Apache-cluster technologies. Thanks for the document link! James [1]