[gentoo-dev] Gentoo installation modifications for 3ware cards

[gentoo-dev] Gentoo installation modifications for 3ware cards

[Tucker I Sylvestro]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 993 bytes --]


Hello,

I spent the today (yesterday now, actually ;) installing gentoo linux on
my computer which contains a 3ware Escalade 6410 RAID card.  Since the
3ware drivers aren't compiled into the stock installation kernel that
comes on your 1.1a CD, I had to jump through some hoops to make it work,
and I wrote up a rough HOWTO as I went along.  It is attached, in case you
want to put it on the website somewhere.  I recommend doing so, as such a
document would have saved me several hours today.

Also, you might want to consider activating the 3ware drivers in the
installation kernel in your next release.  I don't know of any downsides
to doing so, and 3ware is a company that specifically targets the Linux
market by providing source code for the drivers.


Thanks for making an excellent distro, it flies so far, and I can't wait
to play with it more after I've gotten some sleep ;)

Tucker


PS.  This is the first HOWTO I've ever written, so any comments would be
very much appreciated.

[-- Attachment #2: 3wareGentooInstall-HOWTO --]
[-- Type: TEXT/PLAIN, Size: 8809 bytes --]

Gentoo Linux installation HOWTO modifications for installing onto a
computer with a 3ware Escalade 6000/7000 hardware IDE RAID card setup
(the processor is an Athlon Duron, but any x86 should be exactly the
same) booted from the RAID array.  There, that
should draw Google's attention...

 - by Tucker Sylvestro (tsyl1@mit.edu)


This document describes my experience with installing Gentoo Linux
onto an entirely generic computer except for the fact that it contains
two hard disks in a RAID 1 configuration using a 3Ware Escalade 6410
series IDE RAID card rather than a normal IDE controller.  The hard
drives are 80GB Maxtor Caviars, but that detail is most certainly
unimportant.  The unfortunate thing about the setup is this: the stock
Gentoo Linux installation kernel from their 1.1a distro CD isn't built
with the 3ware driver enabled.  Thus, you need to build the driver
separately, install it into the kernel and create the device file
representing the RAID array manually.  Below is a description of how I
did that.  Hopefully it will help save you a few hours, but of course
I haven't tested these instructions on any setup other than my own,
and I absolve myself of any liability for damages you incur by
following or attempting to follow these instructions.


A few notes before we begin:

1) These docs assume a lot of familiarity with Linux.  Specific
   details, namely on compiling the kernel, are left out.

2) I had a computer running Linux right next to me that I made use of
   in step 2, however you can probably get by with only one computer
   as long as you have a copy 3w-xxxx.o, the 3w-xxxx module on a
   floppy disk.  This module should be built for the same version of
   the kernel as on the Gentoo install CD, which was 2.4.18 for me.
   Note that the driver's name is actually "3w-xxxx," that is, "3w-"
   followed by the character x four times.  This note should make more
   sense after you've read step 2.

3) These docs only cover installing Gentoo, they don't talk about
   setting up 3ware's 3dm monitoring tools, which are targeted towards
   RedHat and SuSE only.  This is because I haven't set those up yet.
   I might add instructions for that later if and when I get around to
   installing them.

4) I don't offer technical support of either Gentoo Linux or 3ware
   RAID products.  Today was my first day working with either, and
   this document is the extent of the help I can give you.  However,
   the web has many fine resources, the best of them all being
   www.google.com.  Ask it and ye shall receive.


Ok:

Step 1: Run the standard Gentoo up to the point where it says to run
        fdisk.  By this point you should have networking configured
        and see command prompt.  However, because of the way the 3ware
        card works, you don't have a /dev/hda device to format, and
        fdisk will tell you as much if you try to run it.

Step 2: Build the 3ware drivers.  For this I turned to my already
        configured Linux machine sitting on the desk next to me.  The
        remainder of Step 2 occurs on this other machine rather than
        the one installing Gentoo.

        The first thing I did on this other machine was download the
        most recent Linux kernel sources I could find from kernel.org
        (which happened to be 2.4.18), then unpacked them into
        "/usr/src/linux-2.4.18" and made a symlink called
        "/usr/src/linux" that pointed to it.  I then ran `make
        xconfig` inside that directory.

	The only two options you need to change are: 
          - SCSI mode must be enabled
          - SCSI low-level drivers: enable "3ware Hardware ATA-RAID
                                    support" as a module.

        Now run `make dep`, `make bzImage` and `make modules`.  The
        3ware module is in the drivers/scsi subdirectory, and called
        "3w-xxxx.o".  Copy this to the machine running the Gentoo
        install.

Step 3: Installing the module.  Now go back to the computer installing
        Gentoo.  Install the module with `insmod -f 3w-xxxx.o`.  It
        printed a bunch of errors, but seemed to work fine anyways.
        Next run `modprobe 3w-xxxx`.  It should now print something
        like this:

<snip>
# modprobe 3w-xxxx
SCSI subsystem driver Revision: 1.00
3ware Storage Controller device driver for Linux v1.02.00.016
PCI: Found IRQ 11 for device 00:08.0
scsi0 : Found a 3ware Storage Controller at 0xdc00, IRQ: 11, P-chip: 5.7
scsi0 : 3ware Storage Controller
  Vendor: 3ware     Model: 3w-xxxx           Rev: 1.0
  Type:   Direct-Access                      ANSI SCSI revision: 00
</snip>

        Now wait for about two or three minutes.  Eventually you
        should see a notification pop up on your command line that
        looks like:

<snip>
3w-xxxx: scsi0: AEN: Initialization started: Unit #0.
</snip>

        I believe the reason for the delay is that the RAID card needs
        to boot up or initialize its own bios before it becomes
        active, although this is pure speculation on my part.

Step 4: Create the device file: type `mknod -m a /dev/sda b 8 0`.
        This creates the special file for a SCSI disk.  Yes, even
        though the 3ware Esplanade 6000/7000 is an IDE controller, it
        appears to be SCSI to Linux.  This is what that command does:
        mknod creates a special file, typically one which represents a
        hardware device.  "-m a" means set the mode to read/write, in
        other words the hard drive isn't read-only. "/dev/sda" is the
        name you'll use to refer to the device (remember: in UNIX,
        everything is a file!).  "b" means that the device is a block
        device, and "8 0" are the magical numbers telling Linux that
        this file represents the first SCSI device in the system.  8
        is known as the major number and (I think) is the same for all
        SCSI devices.  0 is called the minor number, and (I think) it
        represents the fact that the RAID controller is the first SCSI
        device in the system.
        
Step 5: Proceed with the rest of the installation, starting from the
        step where you partition the drive.  As a sanity check, when I
        ran `fdisk /dev/sda` in the next step, I got the following
        information printed to the screen:

<snip>
/dev/scsi/host0/bus0/target0/lun0: unknown partition table
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.

The number of cylinders for this disk is set to 9729.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problem with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
   (e.g., DOS FDISK, OS/2 FDISK)


Command (m for help):
</snip>


        Typing "p" at the prompt should bring up some numbers that
        looks accurate for your RAID setup.  Ex: if it says your hard
        drive has one block and/or one cylinder, something is wrong.



Step 6: In the kernel compilation stage, make sure you turn on SCSI
        support, and enable the 3w-xxxx driver by default in the
        "Low-level SCSI drivers" section.


GENERAL RULE: For the remainder of the installation, replace hdaX with
              sdaX everywhere the hard drive device is referenced in
              the installation instructions.



Step 7: Several hours later, after you're through with the rest of the
        Gentoo install, take out the CD or floppy and reboot, and make
        sure it recognizes the drives.  If you get a login prompt it
        obviously worked.  If not, and if it got past Grub or whatever
        bootloader you chose, look for output referring to a SCSI
        device or 3ware.  It should look exactly the same as the
        output you saw in Step 3 (at least it does on my machine) plus
        a few extra lines:

<snip>
SCSI subsystem driver Revision: 1.00
3ware Storage Controller device driver for Linux v1.02.00.016
PCI: Found IRQ 11 for device 00:08.0
scsi0 : Found a 3ware Storage Controller at 0xdc00, IRQ: 11, P-chip: 5.7
scsi0 : 3ware Storage Controller
  Vendor: 3ware     Model: 3w-xxxx           Rev: 1.0
  Type:   Direct-Access                      ANSI SCSI revision: 00
Attached scsi disk sda at scsi0, channel 0, id 0, lun 0
SCSI device sda: 156299440 512-byte hdwr sectors (80025 MB)
Partition check:
 /dev/scsi/host0/bus0/target0/lun0: p1 p2 p3
</snip>

Re: [gentoo-dev] Gentoo installation modifications for 3ware cards

[Jean-Michel Smith]

On Thursday 23 May 2002 03:11 am, Tucker I Sylvestro wrote:
> Hello,
>
> I spent the today (yesterday now, actually ;) installing gentoo linux on
> my computer which contains a 3ware Escalade 6410 RAID card.  Since the
> 3ware drivers aren't compiled into the stock installation kernel that
> comes on your 1.1a CD, I had to jump through some hoops to make it work,
> and I wrote up a rough HOWTO as I went along.  It is attached, in case you
> want to put it on the website somewhere.  I recommend doing so, as such a
> document would have saved me several hours today.
>
> Also, you might want to consider activating the 3ware drivers in the
> installation kernel in your next release.  I don't know of any downsides
> to doing so, and 3ware is a company that specifically targets the Linux
> market by providing source code for the drivers.

Hi Tucker,

Nice job on the howto.  I have an escalade 7850 on my gentoo/source-mage 
system at home (dual boot) and was able to install just fine from the gento 
1.1a disk.  The 3w-xxxx drivers were available on the cd, so all I had to do 
was a

modprobe 3w-xxxx

and my scsi device was recognized.  Of course, then you have modprobe the scsi 
disk module to see the drives themselves.  Are you sure you didn't overlook 
the drivers when you booted of the cdrom, or are you using an older cdrom to 
install perhaps?

I did not need to do any of the compilation steps you did to bootstrap the 
initial mount of /mnt/gentoo, or did I have to create /dev/sda or /dev/sdb 
IIRC.

Jean.

Re: [gentoo-dev] Gentoo installation modifications for 3ware cards

[Tucker I Sylvestro]

Nope, I didn't type modprobe 3w-xxxx until after I built and copied it in
the way I mentioned in the howto.  Nor did I know I needed to probe the
scsi driver in order to have sda created.  That sounds much easier than
then the process I went through ;)

I do think it would be nice to have that linked somewhere off the install
page, or at least somewhere in the docs.  I thought I'd been pretty
thorough about looking in all the right places before trying to roll my
own solution, and didn't find that described anywhere.

Purely out of curiousity, why don't I need to run modprobe now that my
computer's booting off the RAID array?  I compiled the module into my
kernel, so how is it different from the kernel on the CD (which did not
automatically recognize the drives) if the drivers were compiled into that
too?  I'm pretty sure I do have the 1.1a version since I bought it from
Edmunds Enterprises, straight off the link on the gentoo.org home page.

Thanks for the reply,
Tucker



On Thu, 23 May 2002, Jean-Michel Smith wrote:

> On Thursday 23 May 2002 03:11 am, Tucker I Sylvestro wrote:
> > Hello,
> >
> > I spent the today (yesterday now, actually ;) installing gentoo linux on
> > my computer which contains a 3ware Escalade 6410 RAID card.  Since the
> > 3ware drivers aren't compiled into the stock installation kernel that
> > comes on your 1.1a CD, I had to jump through some hoops to make it work,
> > and I wrote up a rough HOWTO as I went along.  It is attached, in case you
> > want to put it on the website somewhere.  I recommend doing so, as such a
> > document would have saved me several hours today.
> >
> > Also, you might want to consider activating the 3ware drivers in the
> > installation kernel in your next release.  I don't know of any downsides
> > to doing so, and 3ware is a company that specifically targets the Linux
> > market by providing source code for the drivers.
>
> Hi Tucker,
>
> Nice job on the howto.  I have an escalade 7850 on my gentoo/source-mage
> system at home (dual boot) and was able to install just fine from the gento
> 1.1a disk.  The 3w-xxxx drivers were available on the cd, so all I had to do
> was a
>
> modprobe 3w-xxxx
>
> and my scsi device was recognized.  Of course, then you have modprobe the scsi
> disk module to see the drives themselves.  Are you sure you didn't overlook
> the drivers when you booted of the cdrom, or are you using an older cdrom to
> install perhaps?
>
> I did not need to do any of the compilation steps you did to bootstrap the
> initial mount of /mnt/gentoo, or did I have to create /dev/sda or /dev/sdb
> IIRC.
>
> Jean.
>

Re: [gentoo-dev] Gentoo installation modifications for 3ware cards

[Colin Morey]

On Fri, 2002-05-24 at 06:13, Tucker I Sylvestro wrote:
> 
> Nope, I didn't type modprobe 3w-xxxx until after I built and copied it in
> the way I mentioned in the howto.  Nor did I know I needed to probe the
> scsi driver in order to have sda created.  That sounds much easier than
> then the process I went through ;)
> 
believe me it is :)

> I do think it would be nice to have that linked somewhere off the install
> page, or at least somewhere in the docs.  I thought I'd been pretty
> thorough about looking in all the right places before trying to roll my
> own solution, and didn't find that described anywhere.
> 
I'm going to be including this in a forth coming guide (server focused).

> Purely out of curiousity, why don't I need to run modprobe now that my
> computer's booting off the RAID array?  I compiled the module into my
> kernel, so how is it different from the kernel on the CD (which did not
> automatically recognize the drives) if the drivers were compiled into that
> too?  I'm pretty sure I do have the 1.1a version since I bought it from
> Edmunds Enterprises, straight off the link on the gentoo.org home page.
> 
the module on the cd is just that, a module,.. and as loading every
module on the cd would require a non-trivial memory load it's easier for
people to load them manually.
(i believe the current install documentation does actually mention the
use of 
"modprobe sd_mod"
for loading the relevant disk drivers)

there is a list of the modules available, (see Bug ID #2764 on
http://bugs.gentoo.org).

Colin Morey,
(Aka Peitolm/col)