[gentoo-user] raid1 + lvm2

[gentoo-user] raid1 + lvm2

[Jarry]

Hi,

I'm going to re-install gentoo on a small hobby-server and because
I need both redundancy and flexibility, I thought in addition to
raid1 (2x 160GB ata-disk) this time I would also use lvm2:

/dev/md0 /boot   (~50MB)
/dev/md1 /       (2GB)
/dev/md2 <swap>  (2GB)
/dev/md3 lvm2    (rest for /var /tmp /usr /opt /chroot /home)

Is this generaly advisable solution (lvm2 over raid1), or is there
some risk in using raid1 together with lvm2 ?

One more question concerning partition type:
If I want to use raid1, I have to set all those 2x4 (hda+hdc) primary
partitions as type fd (raid autodetect). Is it not problem later for
lvm2 when preparing and creating volume-group? Because lvm-guide says
something about setting partition type to 8e (linux lvm), which I can
not do, if I want to use raid1...

Jarry
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] raid1 + lvm2

[Jim Burwell]

[-- Attachment #1: Type: text/plain, Size: 4946 bytes --]

Jarry wrote:

>Hi,
>
>I'm going to re-install gentoo on a small hobby-server and because
>I need both redundancy and flexibility, I thought in addition to
>raid1 (2x 160GB ata-disk) this time I would also use lvm2:
>
>/dev/md0 /boot   (~50MB)
>/dev/md1 /       (2GB)
>/dev/md2 <swap>  (2GB)
>/dev/md3 lvm2    (rest for /var /tmp /usr /opt /chroot /home)
>
>Is this generaly advisable solution (lvm2 over raid1), or is there
>some risk in using raid1 together with lvm2 ?
>
>One more question concerning partition type:
>If I want to use raid1, I have to set all those 2x4 (hda+hdc) primary
>partitions as type fd (raid autodetect). Is it not problem later for
>lvm2 when preparing and creating volume-group? Because lvm-guide says
>something about setting partition type to 8e (linux lvm), which I can
>not do, if I want to use raid1...
>
>Jarry
>  
>

I recently set up a server like this.  I have /boot, swap, and root 
mirrored using linux RAID (md), and swap and root partition is an LVM2 
partition.

There's no problem with setting the partition types of 0xfd.  LVM2 
doesn't have a problem with it.

One thing you need is a initrd or initramfs setup to get all this stuff 
up and running during boot.  I found the easiest way to do this was to 
use genkernel.  Here's are some quick notes on how I got this working 
using the gentoo-sources-2.6.14-r2 kernel:

    Create your partitions using fdisk (/boot, and LVM2 partition for
    swap and root) on both disks.
    Set up RAID1 mirroring for these partitons (/boot md0, LVM2
    swap/root md2)

    Add the md1 device as a physical volume, and create a volume group
    and logical volumes for swap and root inside it.

    Do high level formats on your new /boot partition and root volume,
    and mkswap on your swap volume..

    Use genkernel to configure your kernel making sure to include this
    option: --lvm2

    Make sure the linux RAID (md) stuff is compiled into the kernel. 
    The LVM2 (dm) stuff can be modules.

    Make sure you include these kernel flags in your grub.conf or
    lilo.conf file:

         root=/dev/ram0 init=/linuxrc real_root=/dev/vg00/root dolvm2
        lvmraid=/dev/md0 lvmraid=/dev/md1

(Those last two flags don't seem to be documented anywhere, but are 
required for the genkernel produced linuxrc scripts in the initramfs to 
start up the md devices before it scans for the LVM2 stuff)

All this is probably easier done, if possible, during the install from a 
livecd.  I did it backwards myself.  I got a basic gentoo system running 
on one drive with normal partitions, then transformed it into the 
RAID/LVM system.  I did this by creating the Linux RAID partitons on the 
second drive with 'missing components' using mdadm (I use mdadm instead 
of the traditional raidtools).  This allows you to create your RAID1 
volumes with only one drive, the volumes coming up in degraded mode.  
You can then do your LVM2 volume creation stuff and create your 
filesystems, etc, get them mounted, and copy your basic system from the 
non RAID drive to your new volumes.  Then you can update your fstab, set 
up your bootloader on the RAID drive, and test boot, etc, making sure 
the system comes up using the RAID/LVM2 setup, while having the original 
drive for backup in case things don't work.  Once you're confident 
things will come up on the RAID/LVM2 setup, you can repartition the 
original drive, and "hotadd" the partitions to your RAID volumes.  Linux 
RAID will then sync the formerly missing components, and voila, you now 
have a synced/clean RAID1 system.

Hopefully that all made sense.  I banged my head against it for a while 
getting this set up to boot up (the final key was the lvmraid= flags 
which cause the linxrc to start up the md devices.  I had to read 
through the linuxrc scripts to figure this one out).

Good luck.

-- 
+---------------------------------------------------------------------------+
|         Jim Burwell - Sr. Systems/Network/Security Engineer, JSBC         |
+---------------------------------------------------------------------------+
| "I never let my schooling get in the way of my education." - Mark Twain   |
| "UNIX was never designed to keep people from doing stupid things, because |
|  that policy would also keep them from doing clever things." - Doug Gwyn  |
| "Cool is only three letters away from Fool" - Mike Muir, Suicyco          |
| "..Government in its best state is but a necessary evil; in its worst     |
|  state an intolerable one.." - Thomas Paine, "Common Sense" (1776)        |
+---------------------------------------------------------------------------+
|   Email:  jimb at jay ess bee cee dot com           ICQ UIN:  1695089     |
+---------------------------------------------------------------------------+
|  Reply problems ?  Turn off the "sign" function in email prog.  Blame MS. |
+---------------------------------------------------------------------------+


[-- Attachment #2: Type: text/html, Size: 5577 bytes --]

Re: [gentoo-user] raid1 + lvm2

[Mike Williams]

On Saturday 10 December 2005 10:47, Jim Burwell wrote:
> One thing you need is a initrd or initramfs setup to get all this stuff
> up and running during boot.  I found the easiest way to do this was to
> use genkernel.  Here's are some quick notes on how I got this working
> using the gentoo-sources-2.6.14-r2 kernel:

Or just compile everything you need into the kernel. Something genkernel is 
perfectly capable of doing.

-- 
Mike Williams

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] raid1 + lvm2

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 1250 bytes --]

On Sat, 10 Dec 2005 10:37:58 +0100, Jarry wrote:

> I'm going to re-install gentoo on a small hobby-server and because
> I need both redundancy and flexibility, I thought in addition to
> raid1 (2x 160GB ata-disk) this time I would also use lvm2:
> 
> /dev/md0 /boot   (~50MB)
> /dev/md1 /       (2GB)
> /dev/md2 <swap>  (2GB)
> /dev/md3 lvm2    (rest for /var /tmp /usr /opt /chroot /home)

I've been running a similar setup for 18 months, it works well.

> Is this generaly advisable solution (lvm2 over raid1), or is there
> some risk in using raid1 together with lvm2 ?

Not that I've heard of or discovered.

> One more question concerning partition type:
> If I want to use raid1, I have to set all those 2x4 (hda+hdc) primary
> partitions as type fd (raid autodetect). Is it not problem later for
> lvm2 when preparing and creating volume-group? Because lvm-guide says
> something about setting partition type to 8e (linux lvm), which I can
> not do, if I want to use raid1...

The disk partitions should be set to type fd, because they are RAID
partitions. LVM is running on the RAID array, not directly on the
partitions, so it doesn't have a problem with this.


-- 
Neil Bothwick

What animal & what bit?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-user] raid1 + lvm2

[Thomas Harold]

Jarry wrote:

> Hi,
> 
> I'm going to re-install gentoo on a small hobby-server and because
> I need both redundancy and flexibility, I thought in addition to
> raid1 (2x 160GB ata-disk) this time I would also use lvm2:
> 
> /dev/md0 /boot   (~50MB)
> /dev/md1 /       (2GB)
> /dev/md2 <swap>  (2GB)
> /dev/md3 lvm2    (rest for /var /tmp /usr /opt /chroot /home)

That's how I typically setup my small office servers.  I create about 6 
LVM volumes during the initial installation.  The downside is that it 
makes recovery more difficult (additional volumes that have to be 
mounted).  I've also done a server where I made the root volume larger 
(16-24GB) and didn't bother with LVM2 during the initial install.

I also have a server where I did something slightly trickier with 
RAID1/LVM2:

hda1+hdc1 -> /dev/md0 /boot (~128MB)
hda2+hdc2 -> /dev/md1 (swap) (2GB)
hda3+hdc3 -> /dev/md2 / (root) (16GB)
hda4 -> /music (rest of disk), non-redundant
hdc4 -> /musicbackup (rest of disk, non-redundant)

Every few days (whenever I add new music), I update the contents of 
/musicbackup with what is stored on /music, then unmount /musicbackup 
again.  So, accidental deletions in my /music folder become easy to 
recover from.  And I don't care if my music is offline for a few 
hours/days before I can swap in a new drive.

> Is this generaly advisable solution (lvm2 over raid1), or is there
> some risk in using raid1 together with lvm2 ?

The only issue I've heard *rumor* of is that 2.6.14 kernels may not be 
playing nicely with LVM2.  (I'm using 2.6.13 on my servers at the 
moment.)  One of these days, I'll update to 2.6.14 and find out for sure.

> One more question concerning partition type:
> If I want to use raid1, I have to set all those 2x4 (hda+hdc) primary
> partitions as type fd (raid autodetect). Is it not problem later for
> lvm2 when preparing and creating volume-group? Because lvm-guide says
> something about setting partition type to 8e (linux lvm), which I can
> not do, if I want to use raid1...

All partitions end up as "fd", because the raid arrays need to be built 
prior to LVM2 getting a look at them.
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] raid1 + lvm2

[Richard Fish]

On 12/10/05, Mike Williams <mike@gaima.co.uk> wrote:
> On Saturday 10 December 2005 10:47, Jim Burwell wrote:
> > One thing you need is a initrd or initramfs setup to get all this stuff
> > up and running during boot. I found the easiest way to do this was to
> > use genkernel. Here's are some quick notes on how I got this working
> > using the gentoo-sources-2.6.14-r2 kernel:
>
> Or just compile everything you need into the kernel. Something genkernel is
> perfectly capable of doing.

While you can compile all of the necessary modules into the kernel, it
is not possible to have root on LVM2 without an initrd/initramfs.

-Richard

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] raid1 + lvm2

[Jarry]

Richard Fish wrote:

>>>One thing you need is a initrd or initramfs setup to get all this stuff
>>>up and running during boot. I found the easiest way to do this was to
>>>use genkernel. Here's are some quick notes on how I got this working
>>>using the gentoo-sources-2.6.14-r2 kernel:
>>
>>Or just compile everything you need into the kernel. Something genkernel is
>>perfectly capable of doing.
> 
> While you can compile all of the necessary modules into the kernel, it
> is not possible to have root on LVM2 without an initrd/initramfs.

And even if it is possible, LVM2 installation guide says it is not
recommended, so I will not try it...

One more thing I'm interested in: what impact does lvm2 have on disk i/o,
compared to "common" partitions? Probably lvm2 will make disk operations
a little slower, but how much? Or does it cause higher cpu-load too?

Jarry

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] raid1 + lvm2

[Richard Fish]

On 12/10/05, Jarry <jarry@gmx.net> wrote:
> One more thing I'm interested in: what impact does lvm2 have on disk i/o,
> compared to "common" partitions?

Insignificant, for both IO speeds and CPU load.

-Richard

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] raid1 + lvm2

[Thomas Harold]

Jarry wrote:

> One more thing I'm interested in: what impact does lvm2 have on disk i/o,
> compared to "common" partitions? Probably lvm2 will make disk operations
> a little slower, but how much? Or does it cause higher cpu-load too?

While I don't know of any benchmarks... the ability to resize partitions 
easily down the road outweighs any minor performance issues for my 
needs.  I would imagine that there is some slowdown and some CPU load, 
but nowhere near enough for me to notice (even on an old Celeron 566Mhz 
CPU).
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] raid1 + lvm2

[Jim Burwell]

[-- Attachment #1: Type: text/plain, Size: 4194 bytes --]

Thomas Harold wrote:

> Jarry wrote:
>
>> Hi,
>>
>> I'm going to re-install gentoo on a small hobby-server and because
>> I need both redundancy and flexibility, I thought in addition to
>> raid1 (2x 160GB ata-disk) this time I would also use lvm2:
>>
>> /dev/md0 /boot   (~50MB)
>> /dev/md1 /       (2GB)
>> /dev/md2 <swap>  (2GB)
>> /dev/md3 lvm2    (rest for /var /tmp /usr /opt /chroot /home)
>
<snip>

> The only issue I've heard *rumor* of is that 2.6.14 kernels may not be 
> playing nicely with LVM2.  (I'm using 2.6.13 on my servers at the 
> moment.)  One of these days, I'll update to 2.6.14 and find out for sure.


This doesn't appear to be true, at least as of 2.6.14-gentoo-r2.  I have 
a system up and running with root under LVM2 and RAID1:

    {root@gts/pts/1}~# cat /proc/version
    Linux version 2.6.14-gentoo-r2 (root@gts) (gcc version 3.3.6 (Gentoo
    3.3.6, ssp-3.3.6-1.0, pie-8.7.8)) #2 PREEMPT Thu Dec 1 15:00:58 PST 2005

    {root@gts/pts/1}~# mdadm -D /dev/md1
    /dev/md1:
            Version : 00.90.02
      Creation Time : Wed Nov 30 19:58:23 2005
         Raid Level : raid1
         Array Size : 292977280 (279.40 GiB 300.01 GB)
        Device Size : 292977280 (279.40 GiB 300.01 GB)
       Raid Devices : 2
      Total Devices : 2
    Preferred Minor : 1
        Persistence : Superblock is persistent

        Update Time : Sun Dec 11 11:08:30 2005
              State : clean
     Active Devices : 2
    Working Devices : 2
     Failed Devices : 0
      Spare Devices : 0

               UUID : 82c8fb92:89586f5c:8203e495:ab9c965a
             Events : 0.34669

        Number   Major   Minor   RaidDevice State
           0      33        2        0      active sync   /dev/hde2
           1      34        2        1      active sync   /dev/hdg2

    {root@gts/pts/1}~# pvs
      PV         VG   Fmt  Attr PSize   PFree
      /dev/md1   vg00 lvm2 a-   279.40G    0
    {root@gts/pts/1}~# vgs
      VG   #PV #LV #SN Attr  VSize   VFree
      vg00   1   2   0 wz--n 279.40G    0
    {root@gts/pts/1}~# lvs
      LV   VG   Attr   LSize   Origin Snap%  Move Copy%
      root vg00 -wi-ao 278.40G                         
      swap vg00 -wi-ao   1.00G                         


>> One more question concerning partition type:
>> If I want to use raid1, I have to set all those 2x4 (hda+hdc) primary
>> partitions as type fd (raid autodetect). Is it not problem later for
>> lvm2 when preparing and creating volume-group? Because lvm-guide says
>> something about setting partition type to 8e (linux lvm), which I can
>> not do, if I want to use raid1...
>
>
> All partitions end up as "fd", because the raid arrays need to be 
> built prior to LVM2 getting a look at them.


Yes.  I also have no problem with having 0xfd type partitions for the 
RAID slices, and LVM2 on top of those.  Even if it were a requirement to 
have type 0x8e partitions for LVM2, you could just partition your RAID 
devices (fdisk /dev/mdX) if your Kernel supports RAID dev partitioning.  
I'm glad it's not a requirement, as it would be seem a bit daft to have 
to do this.  :-)


-- 
+---------------------------------------------------------------------------+
|         Jim Burwell - Sr. Systems/Network/Security Engineer, JSBC         |
+---------------------------------------------------------------------------+
| "I never let my schooling get in the way of my education." - Mark Twain   |
| "UNIX was never designed to keep people from doing stupid things, because |
|  that policy would also keep them from doing clever things." - Doug Gwyn  |
| "Cool is only three letters away from Fool" - Mike Muir, Suicyco          |
| "..Government in its best state is but a necessary evil; in its worst     |
|  state an intolerable one.." - Thomas Paine, "Common Sense" (1776)        |
+---------------------------------------------------------------------------+
|   Email:  jimb@jsbc.cc                              ICQ UIN:  1695089     |
+---------------------------------------------------------------------------+
|  Reply problems ?  Turn off the "sign" function in email prog.  Blame MS. |
+---------------------------------------------------------------------------+


[-- Attachment #2: Type: text/html, Size: 6317 bytes --]