public inbox for gentoo-user@lists.gentoo.org
 help / color / mirror / Atom feed
From: "J. Roeleveld" <joost@antarean.org>
To: gentoo-user@lists.gentoo.org
Subject: Re: [gentoo-user] File system testing
Date: Wed, 17 Sep 2014 09:45:14 +0200	[thread overview]
Message-ID: <1702192.KJn5UXzBYJ@andromeda> (raw)
In-Reply-To: <loom.20140916T203111-406@post.gmane.org>


On Tuesday, September 16, 2014 07:07:38 PM James wrote:
> Hello,
> 
> By now many are familiar with my keen interest in clustering gentoo
> systems. So, what most cluster technologies use is a distributed file
> system on top of the local (HD/SDD) file system. Naturally not
> all file systems, particularly the distributed file systems, have
> straightforward instructions. Also, an device file system, such as
> XFS and a distibuted (on top of the device file system) combination
> may not work very well when paired. So a variety of testing is
> something I'm researching. Eliminiation of either file system
> listed below, due to Gentoo User Experience is most welcome information,
> as well as tips and tricks to setting up any file system.
> 
> 
> Distributed File Systems (DFS):
> HDFS (poor performance)
> Lustre
> Ceph
> XtreemFS
> GlusterFS
> MooseFS
> FhGFS (BeeGFS) soon to be entirely open sourced?
> Any other distributed file systems I should consider using?
> 
> Local (Device) File Systems LFS:
> btrfs
> zfs
> ext4
> xfs
> 
> Obviously I do not what to test all combinations of DFS/LocalFS
> so your comments are extremely welcome as is any and all
> related information.
> 
> James

James,

Is my understanding correct that the top list all require one of the bottom 
list?
Eg. the "clustering" FSs only ensure the files on the LFSs are 
duplicated/spread over the various nodes?

I would normally expect the clustering FS to be either the full layer or a 
clustered block-device where an FS can be placed on top.

Otherwise it seems more like a network filesystem with caching options (See 
AFS).

I am also interested in these filesystems, but for a slightly different 
scenario:

- 2 servers in remote locations (different offices)
- 1 of these has all the files stored (server A) at the main office
- The other (server B - remote office) needs to "offer" all files from serverA

When server B needs to supply a file, it needs to check if the local copy is 
still the "valid" version. If yes, supply the local copy, otherwise download 
from server A. When a file is changed, server A needs to be updated.
While server B is sharing a file, the file needs to be locked on server A 
preventing simultaneous updates.

I prefer not to supply the same amount of storage at server B as server A has.
The remote location generally only needs access to 5% of the total amount of 
files stored on server A. But not always the same 5%.

Does anyone know of a filesystem that can handle this?

--
Joost


  reply	other threads:[~2014-09-17  7:45 UTC|newest]

Thread overview: 25+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-09-16 19:07 [gentoo-user] File system testing James
2014-09-17  7:45 ` J. Roeleveld [this message]
2014-09-17 15:55   ` [gentoo-user] " James
2014-09-17 19:34     ` J. Roeleveld
2014-09-17 20:20       ` Alec Ten Harmsel
2014-09-17 20:56         ` James
2014-09-18  8:24           ` J. Roeleveld
2014-09-18  9:48             ` Rich Freeman
2014-09-18 10:22               ` J. Roeleveld
2014-09-19 13:41             ` James
2014-09-19 14:56               ` Rich Freeman
2014-09-19 15:06                 ` J. Roeleveld
2014-09-19 15:02               ` J. Roeleveld
2014-09-18  8:04         ` J. Roeleveld
2014-09-18  9:17         ` Kerin Millar
2014-09-18 13:12           ` Alec Ten Harmsel
2014-09-19 15:21             ` Kerin Millar
2014-09-17 18:10 ` [gentoo-user] " Hervé Guillemet
2014-09-17 18:21   ` J. Roeleveld
2014-09-17 21:05     ` [gentoo-user] " James
2014-09-18  7:29       ` J. Roeleveld
2014-09-18  8:28     ` [gentoo-user] " Kerin Millar
2014-09-25 20:56     ` thegeezer
2014-09-18 15:32   ` [gentoo-user] " James
2014-09-25 20:47 ` [gentoo-user] " thegeezer

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1702192.KJn5UXzBYJ@andromeda \
    --to=joost@antarean.org \
    --cc=gentoo-user@lists.gentoo.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox