[gentoo-user] [OT] RAID 1 over network

[gentoo-user] [OT] RAID 1 over network

[Renat Golubchyk]

Hi all!

Short:
What is the best way to setup something similar to RAID 1 over a WAN?

Background:
Two LAMP servers are located in geographically different locations
connected through a load balancer to the net:

  LAMP A--------LAMP B
      \           / 
       \         / 
        \       / 
         \     / 
          \   / 
           \ / 
            V 
     load balancer LB
            | 
            |
           WAN

One purpose of the setup is to have data redundancy. Thus we have to
ensure that the data is replicated in a timely manner. Replicating MySQL
data is not difficult. The problem is the file system data like uploaded
documents and pictures. We can monitor changes in the file system and
initiate rsync to copy files over the network, but I think it's not a
good solution. What we are after is a network equivalent of RAID 1.
Are there any viable solutions that could work over a WAN?


Cheers,
Renat

--
Probleme kann man niemals mit derselben Denkweise loesen,
durch die sie entstanden sind.
                                              (Einstein)

[gentoo-user] Re: [OT] RAID 1 over network

[james]

Renat Golubchyk <ragermany <at> gmx.net> writes:


> What is the best way to setup something similar to RAID 1 over a WAN?


CFengine might be of use. I just popped over to the site, it
seems to be offering Open Source and Commercial support now...


just a thought, not a verified solution for you.


hth,


James

Re: [gentoo-user] [OT] RAID 1 over network

[Philipp Riegger]

On Tue, 2009-06-16 at 12:40 +0200, Renat Golubchyk wrote:
> Are there any viable solutions that could work over a WAN?

Look into DRBD, <http://www.drbd.org/>. It is open source but not yet in
the kernel, I think.

Philipp

Re: [gentoo-user] [OT] RAID 1 over network

[Mike Kazantsev]

[-- Attachment #1: Type: text/plain, Size: 1631 bytes --]

On Tue, 16 Jun 2009 12:40:46 +0200
Renat Golubchyk <ragermany@gmx.net> wrote:

> Hi all!
> 
> Short:
> What is the best way to setup something similar to RAID 1 over a WAN?

...

> One purpose of the setup is to have data redundancy. Thus we have to
> ensure that the data is replicated in a timely manner. Replicating MySQL
> data is not difficult. The problem is the file system data like uploaded
> documents and pictures. We can monitor changes in the file system and
> initiate rsync to copy files over the network, but I think it's not a
> good solution. What we are after is a network equivalent of RAID 1.
> Are there any viable solutions that could work over a WAN?

If replication after file is completely hit the disk is not acceptable
either because files are quite large or have to be really synchronous
then I'll second DRBD-with-HA-cluster-fs suggestion.

Otherwise, if it's acceptable to have each file replicated as soon as
it's fully stored on one node and link between nodes isn't good enough
clustered fs High-Availability features might play against you, but I
don't think rsync (via some sort of cron) is a good idea here, anyway.

You'd be better off using in-kernel dnotify / inotify reactors and
a simple script to track IN_MODIFY and IN_CREATE events, replicating
path in question (possibly via some sort of delta-transfer) as soon as
it hits the fs.
I even remember seeing inotify-tracking binary to use with common shell
scripts, although I believe any dedicated non-bash-hack implementation
should be better suited for such task.

-- 
Mike Kazantsev // fraggod.net

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] [OT] RAID 1 over network

[Daniel Iliev]

On Tue, 16 Jun 2009 18:58:56 +0600
Mike Kazantsev <mk.fraggod@gmail.com> wrote:

> On Tue, 16 Jun 2009 12:40:46 +0200
> Renat Golubchyk <ragermany@gmx.net> wrote:
> 
> > Hi all!
> > 
> > Short:
> > What is the best way to setup something similar to RAID 1 over a
> > WAN?
> 
> ...
> 
> > One purpose of the setup is to have data redundancy. Thus we have to
> > ensure that the data is replicated in a timely manner. Replicating
> > MySQL data is not difficult. The problem is the file system data
> > like uploaded documents and pictures. We can monitor changes in the
> > file system and initiate rsync to copy files over the network, but
> > I think it's not a good solution. What we are after is a network
> > equivalent of RAID 1. Are there any viable solutions that could
> > work over a WAN?
> 
> If replication after file is completely hit the disk is not acceptable
> either because files are quite large or have to be really synchronous
> then I'll second DRBD-with-HA-cluster-fs suggestion.
> 

DRBD is HA solution which is achieved by switching the role of the
nodes in case the "active node" goes offline. I think DRBD is not meant
for the schema OP has described, because only the "active node" is
accessible via FS. DRBD works between the FS and block device layers.
It catches the FS writes from the active node and sends them over the
network. DRBD on the backup node receives those and replicates them
directly to the disk driver. Thus you can't have mounted FS on the
backup node. If the active node goes offline, the backup node takes
over which means DRBD switches roles and the FS has to be mounted
afterwards.

-- 
Best regards,
Daniel

Re: [gentoo-user] [OT] RAID 1 over network

[Philipp Riegger]

On Tue, 2009-06-16 at 16:45 +0300, Daniel Iliev wrote:
> DRBD is HA solution which is achieved by switching the role of the
> nodes in case the "active node" goes offline. I think DRBD is not meant
> for the schema OP has described, because only the "active node" is
> accessible via FS. DRBD works between the FS and block device layers.
> It catches the FS writes from the active node and sends them over the
> network. DRBD on the backup node receives those and replicates them
> directly to the disk driver. Thus you can't have mounted FS on the
> backup node. If the active node goes offline, the backup node takes
> over which means DRBD switches roles and the FS has to be mounted
> afterwards.

But that might be a good solution with 2.6.30, NFS and FSCACHE.

Another solution would be to use ndb (network block devices), dm-raid
and a cluster filesystem. 

Philipp

Re: [gentoo-user] [OT] RAID 1 over network

[Dirk Heinrichs]

[-- Attachment #1: Type: text/plain, Size: 1038 bytes --]

Am Dienstag 16 Juni 2009 15:58:37 schrieb Philipp Riegger:
> On Tue, 2009-06-16 at 16:45 +0300, Daniel Iliev wrote:
> > DRBD is HA solution which is achieved by switching the role of the
> > nodes in case the "active node" goes offline. I think DRBD is not meant
> > for the schema OP has described, because only the "active node" is
> > accessible via FS. DRBD works between the FS and block device layers.
> > It catches the FS writes from the active node and sends them over the
> > network. DRBD on the backup node receives those and replicates them
> > directly to the disk driver. Thus you can't have mounted FS on the
> > backup node. If the active node goes offline, the backup node takes
> > over which means DRBD switches roles and the FS has to be mounted
> > afterwards.
>
> But that might be a good solution with 2.6.30, NFS and FSCACHE.
>
> Another solution would be to use ndb (network block devices), dm-raid
> and a cluster filesystem.

And finally, there's OpenAFS. Not really RAID, but maybe sufficient.

Bye...

	Dirk

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

Re: [gentoo-user] [OT] RAID 1 over network

[Daniel Iliev]

On Tue, 16 Jun 2009 17:54:34 +0200
Dirk Heinrichs <dirk.heinrichs@online.de> wrote:

> Am Dienstag 16 Juni 2009 15:58:37 schrieb Philipp Riegger:
[-snip-]
> > Another solution would be to use ndb (network block devices),
> > dm-raid and a cluster filesystem.
> 
> And finally, there's OpenAFS. Not really RAID, but maybe sufficient.
> 
> Bye...
> 
> 	Dirk


Perhaps. The unfortunate thing is that the answers can't be more
accurate than the question itself, but many times the OPs liberty to
reveal details is limited by some kind of NDA. In this particular case
a key detail is missing: which is more important - the traffic costs or
the hardware costs.

-- 
Best regards,
Daniel

Re: [gentoo-user] [OT] RAID 1 over network

[Daniel Iliev]

On Tue, 16 Jun 2009 15:58:37 +0200
Philipp Riegger <lists@anderedomain.de> wrote:

> 
> On Tue, 2009-06-16 at 16:45 +0300, Daniel Iliev wrote:
> > DRBD is HA solution which is achieved by switching the role of the
> > nodes in case the "active node" goes offline. I think DRBD is not
> > meant for the schema OP has described, because only the "active
> > node" is accessible via FS. DRBD works between the FS and block
> > device layers. It catches the FS writes from the active node and
> > sends them over the network. DRBD on the backup node receives those
> > and replicates them directly to the disk driver. Thus you can't
> > have mounted FS on the backup node. If the active node goes
> > offline, the backup node takes over which means DRBD switches roles
> > and the FS has to be mounted afterwards.
> 
> But that might be a good solution with 2.6.30, NFS and FSCACHE.
> 

Sorry, perhaps I'm missing something, but I couldn't understand
the solution you had in mind.
I'm just saying that the only scenario I have some experience with is
DRBD + ext3 and it won't work for load balancing.
If "LAMP A" and "LAMP B" as shown on the OP's schema were connected via
DRBD + a conventional FS (ext/xfs/reiser/etc.), then only one of those
systems would be able to serve client requests at a given moment.

-- 
Best regards,
Daniel