From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <gentoo-user+bounces-158078-garchives=archives.gentoo.org@lists.gentoo.org>
Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80])
by finch.gentoo.org (Postfix) with ESMTP id 8BD2413877A
for <garchives@archives.gentoo.org>; Mon, 18 Aug 2014 14:50:30 +0000 (UTC)
Received: from pigeon.gentoo.org (localhost [127.0.0.1])
by pigeon.gentoo.org (Postfix) with SMTP id C1363E0ACB;
Mon, 18 Aug 2014 14:50:25 +0000 (UTC)
Received: from mail-vc0-f193.google.com (mail-vc0-f193.google.com [209.85.220.193])
(using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits))
(No client certificate requested)
by pigeon.gentoo.org (Postfix) with ESMTPS id B6122E0984
for <gentoo-user@lists.gentoo.org>; Mon, 18 Aug 2014 14:50:24 +0000 (UTC)
Received: by mail-vc0-f193.google.com with SMTP id ij19so1829503vcb.8
for <gentoo-user@lists.gentoo.org>; Mon, 18 Aug 2014 07:50:23 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20120113;
h=mime-version:sender:in-reply-to:references:date:message-id:subject
:from:to:content-type;
bh=utx8CCwR39Zu5pQhc371PjHVnsPkTi+S4ciI3xrST9w=;
b=Z74R+WhELlek33DU/zokAgYT71YekMmG/qw5g5cP471f1uMwzocbeT6NCYkreN1exS
zbotTJe0e7FT81g+NMF1hYrPi6eWQ1h4M5OJGQnN136wEGcF6EjL6hQq7f0PQyGktFG6
JLLDPW/obduVnRrz79dML1SY/WBig01WbWEp6tF5WdnVo/Waa5bZdWbYKtcDGb9ibECv
paglUYTYHlKKVVRli4FZ/ZaICgnsUz6r0BOaoAExd4gGemF+oR0gw8lA34QtLxGb0dF9
KM4PHnOI0wyTsXuYAhVXs3erw7mpRQ7uH37sPGYN8Tyd7/3IWHcqr0k0fFiUn5H+5pqf
Ldbg==
Precedence: bulk
List-Post: <mailto:gentoo-user@lists.gentoo.org>
List-Help: <mailto:gentoo-user+help@lists.gentoo.org>
List-Unsubscribe: <mailto:gentoo-user+unsubscribe@lists.gentoo.org>
List-Subscribe: <mailto:gentoo-user+subscribe@lists.gentoo.org>
List-Id: Gentoo Linux mail <gentoo-user.gentoo.org>
X-BeenThere: gentoo-user@lists.gentoo.org
Reply-to: gentoo-user@lists.gentoo.org
MIME-Version: 1.0
X-Received: by 10.52.127.5 with SMTP id nc5mr573868vdb.59.1408373423880; Mon,
18 Aug 2014 07:50:23 -0700 (PDT)
Sender: freemanrich@gmail.com
Received: by 10.52.8.229 with HTTP; Mon, 18 Aug 2014 07:50:23 -0700 (PDT)
In-Reply-To: <1855316.WFR9YJczUb@andromeda>
References: <loom.20140806T175414-148@post.gmane.org>
<53F106B2.4090307@thegeezer.net>
<1855316.WFR9YJczUb@andromeda>
Date: Mon, 18 Aug 2014 10:50:23 -0400
X-Google-Sender-Auth: ol3FDQJ0V5H9eQh7XDSqqBln4S8
Message-ID: <CAGfcS_ndoBkzRbd30Qb9PJHaOLikxmnw3iXkjhRNk+GX+aEnLQ@mail.gmail.com>
Subject: Re: [gentoo-user] Clusters on Gentoo ?
From: Rich Freeman <rich0@gentoo.org>
To: gentoo-user@lists.gentoo.org
Content-Type: text/plain; charset=UTF-8
X-Archives-Salt: 220327b2-27ac-44c7-815e-32e94a7ff090
X-Archives-Hash: b6b002572f9e9a147d6ae4fe64f5cb50
On Mon, Aug 18, 2014 at 10:31 AM, J. Roeleveld <joost@antarean.org> wrote:
>
> I wouldn't use Hadoop for storage of files. It's only useful if you have a lot
> (and I do mean a LOT) of data where a query only returns a very small amount.
Not to mention a lot of data in a small number of files. I think the
minimum allocation size for Hadoop is measured in megabytes. I tried
using it to process gentoo-x86 and the number of files just clobbered
the thing. Since in my job the files were really just static data and
not the actual subject of the map/reduce I instead just replicated the
data to all the nodes and had them retrieve the data from the local
filesystem.
Hadoop is a very specialized tool. It does what it does very well,
but if you want to use it for something other than map/reduce then
consider carefully whether it is the right tool for the job.
--
Rich