[gentoo-server] boot from san with multipath support

[gentoo-server] boot from san with multipath support

[Rene Zbinden]

hello

has someone experience with installing gentoo on a san attached disk 
with mulitpath support? if someone can give me some hints, I would 
appreciate that.

thanks in advance
rene
-- 
gentoo-server@gentoo.org mailing list

Re: [gentoo-server] boot from san with multipath support

[gistolero]

Hi Rene,

Rene Zbinden wrote:
> hello
> 
> has someone experience with installing gentoo on a san attached disk 
> with mulitpath support? if someone can give me some hints, I would 
> appreciate that.
> 
> thanks in advance
> rene

I have tested multipath-tools from

http://christophe.varoqui.free.fr/wiki/wakka.php?wiki=Home

during the past three weeks and send some messages to the dm-devel mailing 
list. Here is a summary:




I.) ***** Installation *****

http://christophe.varoqui.free.fr/wiki/wakka.php?wiki=QuickInstallGuide

emerge multipath-tools




II.) ***** Configuration *****


1.) load dm-mapper and HBA kernel modules. lower HBA timeouts (see FAQ).


2.) create partitions on each lun (e.g.: lun0 has two paths sda and sdb - 
"fdisk sda" creates the partitions for sda _and_ sdb) - you have to do this 
step _before_ creating multipath devices (kpartx in 
/etc/dev.d/block/multipath.dev reads this partition information)


3.) create /etc/multipath.conf

- check multipath.conf.annotated and multipath.conf.synthetic in the 
multipath-tools tarball
- get lun wwid's with "scsi_id -g -u -s /block/sd?"
- create a "multipath { ... }" section for each lun (define "wwid" and "alias")
- create a "device { ... }" section for your storage (see "TestedEnvironments" 
located at the multipath-tools homepage) - set "path_grouping_policy" and 
"prio_callout" (setting "prio_callout" is important. if you choose a wrong 
value, you will see nothing after starting multipath)


4.) start multipathd (debug with "-d" or "-v4" (syslog))

Note: multipathd doesn't work with my hardware (I am using HP DL380, EVA 3000 
HSV100). But the daemon isn't needed necessarily:

My question at the dm-devel mailing list:
 >> ===> Multipathing seems to work without but not with multipathd.
 >> [...] Does I really need multipathd? I suppose so :-)
 >>

And the answer from christophe varoqui (multipath-tools developer):

 > multipathd is needed to reinstate paths.
 > In your case the rport disappears and reappears so the mecanism is all
 > hotplug-driven and thus may work without the daemon ... if memory
 > ressources permits hotplug and multipath(8) execution, that is.

Nicola (see below) has an EMC CX 300 storage system and uses multipathd 
without problems.



5.) run "multipath /dev/sd?" for each sd? device

multipath should create device-mapper table entries (see "dmsetup table"), 
block-device-files in /dev/mapper/ and links in /dev/ for the lun and for each 
partition

e.g.: you have created two partitions and your lun alias is "yellow" (defined 
in /etc/multipath.conf):

# dmsetup table
yellow1: ...
yellow: ...
yellow2: ...

# ls /dev/mapper/
yellow yellow1 yellow2 control

# ls -F /dev/yellow*
/dev/yellow@  /dev/yellow1@  /dev/yellow2@

and check "multipath [-v3] -ll"


6.) create filesystems and mount /dev/yellow1 and /dev/yellow2


you can delete the current configuration with "dmsetup remove yellow2; dmsetup 
remove yellow1; dmsetup remove yellow"




III.) ***** Test *****


I tested the multipath environment with a little perl script and a mysql database:

My trafficmaker-host executed this script 27 times (parallel):

...
for(my $count=1;$count<=1000000;$count++)
{
   ...
   my $sql="INSERT INTO $table VALUES($id,\"$value\")";
   my $return=$dbh->do($sql);
   ...
}
...
{
   my $sql="SELECT COUNT(*) FROM $table WHERE id=$id";
   my $sth=$dbh->prepare($sql);
   my $return=$sth->execute();
   ...
   $selectCount=$sth->fetchrow_array();
   ...;
}


The database host had to insert this 30 byte strings and I have started some 
copy-jobs (cp -a /usr/* /partition_mounted_with_multipath/ etc.) to increase 
the I/O load. During this test I have disabled and enabled the different 
HBA-Switch-Ports with the following result: It took 6 to 15 seconds before 
"multipath -l" showed that a path is down (15 seconds because the host had a 
30.0 CPU load and responded very slowly), but no INSERT got lost :-)))



IV.) ***** More Documentation *****

Check the homepage (I will add more documentation and helper scripts in the 
next days) and the dm-devel mailing list archive:

https://www.redhat.com/mailman/listinfo/dm-devel

E.g. Nicola Murino posted his experience with an EMC storage controller. He is 
using Gentoo too.


Hope that helps
Simon

-- 
gentoo-server@gentoo.org mailing list

AW: AW: [gentoo-server] boot from san with multipath support

[Witschel, Matthias, Infraserv-Hoechst/DE]

I know that the QLogic HBAs have built-in Multipathing support with enabled drivers (these should be availible from IBM or QLogic). This would prevent you from configuring Multipathing on OS level, thus making everything a bit easier. This I would recommend.

Im using QLogic HBAs with HP BL and DL Servers and they work fine under linux. 
With IBM HBAs I have no experience. They might work as well or even better.

Matthias Witschel

Infraserv GmbH & Co. Höchst KG
Geschäftsfeld IT-Services
Geb. C584, Raum U10e
Industriepark Höchst
65926 Frankfurt am Main
Tel. 069 305 84235
Fax 069 305 98 84235
mailto:matthias.witschel@infraserv.com

www.infraserv.com ++ www.industriepark-hoechst.com ++ www.ihr-nachbar.de
> Auf diesen Messen finden Sie uns: www.infraserv.com/messetermine <http://www.infraserv.com/messetermine>
> 


-----Ursprüngliche Nachricht-----
Von: Rene Zbinden [mailto:rene.zbinden@solnet.ch]
Gesendet: Montag, 19. September 2005 08:02
An: gentoo-server@lists.gentoo.org
Betreff: Re: AW: [gentoo-server] boot from san with multipath support


It will be an IBM x346, but we are not sure about the HBAs. We can 
choose between IBM Total Storage DS4000 or with Qlogig HBAs. Any 
Suggestions?

Witschel, Matthias, Infraserv-Hoechst/DE wrote:
> Up to now I have installed gentoo an SAN attached disks. I hvae for now experience with SLES9 and SAN. I'm running a bunch of HP Blades which use SAN storage. HP provides a devicedriver for these boxes, that enables multipathing within the driver.
> I plan to use this driver to enable SAN access for at least one gentoo box in the near future. Since HP ships the driver as sourcecode, there should be no problems.
> 
> What setup do you plan? What hardware do you plan to use?
> 
> Matthias Witschel
> 
> Infraserv GmbH & Co. Höchst KG
> Geschäftsfeld IT-Services
> Computing Services
> Geb. C584, Raum U10e
> Industriepark Höchst
> 65926 Frankfurt am Main
> mailto:matthias.witschel@infraserv.com
> 
> www.infraserv.com ++ www.industriepark-hoechst.com ++ www.ihr-nachbar.de
> 
>>Auf diesen Messen finden Sie uns: www.infraserv.com/messetermine <http://www.infraserv.com/messetermine>
>>
> 
> 
> -----Ursprüngliche Nachricht-----
> Von: Rene Zbinden [mailto:rene.zbinden@solnet.ch]
> Gesendet: Freitag, 16. September 2005 23:42
> An: gentoo-server@lists.gentoo.org
> Betreff: [gentoo-server] boot from san with multipath support
> 
> 
> hello
> 
> has someone experience with installing gentoo on a san attached disk 
> with mulitpath support? if someone can give me some hints, I would 
> appreciate that.
> 
> thanks in advance
> rene
-- 
gentoo-server@gentoo.org mailing list


-- 
gentoo-server@gentoo.org mailing list

Re: AW: AW: [gentoo-server] boot from san with multipath support

[Rene Zbinden]

We have a HDS 9980 System and I have heard of HDML. Somebody already 
tested that.
And what is the best way to install on a san disk? any pitfalls? Is it 
difficult to boot from a SAN disk?

Thanks in advance

Witschel, Matthias, Infraserv-Hoechst/DE wrote:
> I know that the QLogic HBAs have built-in Multipathing support with enabled drivers (these should be availible from IBM or QLogic). This would prevent you from configuring Multipathing on OS level, thus making everything a bit easier. This I would recommend.
> 
> Im using QLogic HBAs with HP BL and DL Servers and they work fine under linux. 
> With IBM HBAs I have no experience. They might work as well or even better.
> 
> Matthias Witschel
> 
> Infraserv GmbH & Co. Höchst KG
> Geschäftsfeld IT-Services
> Geb. C584, Raum U10e
> Industriepark Höchst
> 65926 Frankfurt am Main
> Tel. 069 305 84235
> Fax 069 305 98 84235
> mailto:matthias.witschel@infraserv.com
> 
> www.infraserv.com ++ www.industriepark-hoechst.com ++ www.ihr-nachbar.de
> 
>>Auf diesen Messen finden Sie uns: www.infraserv.com/messetermine <http://www.infraserv.com/messetermine>
>>
> 
> 
> -----Ursprüngliche Nachricht-----
> Von: Rene Zbinden [mailto:rene.zbinden@solnet.ch]
> Gesendet: Montag, 19. September 2005 08:02
> An: gentoo-server@lists.gentoo.org
> Betreff: Re: AW: [gentoo-server] boot from san with multipath support
> 
> 
> It will be an IBM x346, but we are not sure about the HBAs. We can 
> choose between IBM Total Storage DS4000 or with Qlogig HBAs. Any 
> Suggestions?
> 
> Witschel, Matthias, Infraserv-Hoechst/DE wrote:
> 
>>Up to now I have installed gentoo an SAN attached disks. I hvae for now experience with SLES9 and SAN. I'm running a bunch of HP Blades which use SAN storage. HP provides a devicedriver for these boxes, that enables multipathing within the driver.
>>I plan to use this driver to enable SAN access for at least one gentoo box in the near future. Since HP ships the driver as sourcecode, there should be no problems.
>>
>>What setup do you plan? What hardware do you plan to use?
>>
>>Matthias Witschel
>>
>>Infraserv GmbH & Co. Höchst KG
>>Geschäftsfeld IT-Services
>>Computing Services
>>Geb. C584, Raum U10e
>>Industriepark Höchst
>>65926 Frankfurt am Main
>>mailto:matthias.witschel@infraserv.com
>>
>>www.infraserv.com ++ www.industriepark-hoechst.com ++ www.ihr-nachbar.de
>>
>>
>>>Auf diesen Messen finden Sie uns: www.infraserv.com/messetermine <http://www.infraserv.com/messetermine>
>>>
>>
>>-----Ursprüngliche Nachricht-----
>>Von: Rene Zbinden [mailto:rene.zbinden@solnet.ch]
>>Gesendet: Freitag, 16. September 2005 23:42
>>An: gentoo-server@lists.gentoo.org
>>Betreff: [gentoo-server] boot from san with multipath support
>>
>>
>>hello
>>
>>has someone experience with installing gentoo on a san attached disk 
>>with mulitpath support? if someone can give me some hints, I would 
>>appreciate that.
>>
>>thanks in advance
>>rene
-- 
gentoo-server@gentoo.org mailing list

Re: [gentoo-server] boot from san with multipath support

[Lance Albertson]

[-- Attachment #1: Type: text/plain, Size: 1162 bytes --]

Rene Zbinden wrote:
> We have a HDS 9980 System and I have heard of HDML. Somebody already
> tested that.
> And what is the best way to install on a san disk? any pitfalls? Is it
> difficult to boot from a SAN disk?

We (where I work) have an HDS 9980 system as well and haven't been
having any luck finding reliable drivers for linux with multipath
support. Have any of you gotten that to work or point me to the right
direction?

> Witschel, Matthias, Infraserv-Hoechst/DE wrote:
> 
>> I know that the QLogic HBAs have built-in Multipathing support with
>> enabled drivers (these should be availible from IBM or QLogic). This
>> would prevent you from configuring Multipathing on OS level, thus
>> making everything a bit easier. This I would recommend.
>>
>> Im using QLogic HBAs with HP BL and DL Servers and they work fine
>> under linux. With IBM HBAs I have no experience. They might work as
>> well or even better.

-- 
Lance Albertson <ramereth@gentoo.org>
Gentoo Infrastructure | Operations Manager

---
GPG Public Key:  <http://www.ramereth.net/lance.asc>
Key fingerprint: 0423 92F3 544A 1282 5AB1  4D07 416F A15D 27F4 B742

ramereth/irc.freenode.net

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]