From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from pigeon.gentoo.org ([69.77.167.62] helo=lists.gentoo.org) by finch.gentoo.org with esmtp (Exim 4.60) (envelope-from ) id 1JyCP9-0007yb-SH for garchives@archives.gentoo.org; Mon, 19 May 2008 20:56:25 +0000 Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 428C8E04E3; Mon, 19 May 2008 20:56:23 +0000 (UTC) Received: from smtp.gentoo.org (smtp.gentoo.org [140.211.166.183]) by pigeon.gentoo.org (Postfix) with ESMTP id 7A25CE04E3 for ; Mon, 19 May 2008 20:56:22 +0000 (UTC) Received: from stork.gentoo.org (stork.gentoo.org [64.127.104.133]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by smtp.gentoo.org (Postfix) with ESMTP id 612E966BBB for ; Mon, 19 May 2008 20:56:21 +0000 (UTC) Received: from swift by stork.gentoo.org with local (Exim 4.68) (envelope-from ) id 1JyCP6-00064K-1X for gentoo-doc-cvs@lists.gentoo.org; Mon, 19 May 2008 20:56:20 +0000 To: gentoo-doc-cvs@lists.gentoo.org Subject: [gentoo-doc-cvs] cvs commit: hpc-howto.xml Message-Id: From: Sven Vermeulen Date: Mon, 19 May 2008 20:56:20 +0000 Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-doc-cvs@lists.gentoo.org Reply-to: docs-team@lists.gentoo.org X-Archives-Salt: 8b6ffe8e-394b-45c8-a4b0-79f7d85ffd74 X-Archives-Hash: ba8b98231f73513ac82a0358af057a38 swift 08/05/19 20:56:20 Modified: hpc-howto.xml Log: Coding style (sorry, length on uris not fixable) Revision Changes Path 1.14 xml/htdocs/doc/en/hpc-howto.xml file : http://sources.gentoo.org/viewcvs.py/gentoo/xml/htdocs/doc/en/hpc-howto.xml?rev=1.14&view=markup plain: http://sources.gentoo.org/viewcvs.py/gentoo/xml/htdocs/doc/en/hpc-howto.xml?rev=1.14&content-type=text/plain diff : http://sources.gentoo.org/viewcvs.py/gentoo/xml/htdocs/doc/en/hpc-howto.xml?r1=1.13&r2=1.14 Index: hpc-howto.xml =================================================================== RCS file: /var/cvsroot/gentoo/xml/htdocs/doc/en/hpc-howto.xml,v retrieving revision 1.13 retrieving revision 1.14 diff -u -r1.13 -r1.14 --- hpc-howto.xml 18 Dec 2006 21:47:19 -0000 1.13 +++ hpc-howto.xml 19 May 2008 20:56:20 -0000 1.14 @@ -1,5 +1,5 @@ - + @@ -28,7 +28,7 @@ permission to distribute this document as-is and update it when appropriate as long as the adelie linux R&D notice stays --> - + This document was written by people at the Adelie Linux R&D Center <http://www.adelielinux.com> as a step-by-step guide to turn a Gentoo @@ -44,22 +44,22 @@

-Gentoo Linux, a special flavor of Linux that can be automatically optimized -and customized for just about any application or need. Extreme performance, +Gentoo Linux, a special flavor of Linux that can be automatically optimized +and customized for just about any application or need. Extreme performance, configurability and a top-notch user and developer community are all hallmarks of the Gentoo experience.

-Thanks to a technology called Portage, Gentoo Linux can become an ideal secure +Thanks to a technology called Portage, Gentoo Linux can become an ideal secure server, development workstation, professional desktop, gaming system, embedded -solution or... a High Performance Computing system. Because of its +solution or... a High Performance Computing system. Because of its near-unlimited adaptability, we call Gentoo Linux a metadistribution.

-This document explains how to turn a Gentoo system into a High Performance -Computing system. Step by step, it explains what packages one may want to +This document explains how to turn a Gentoo system into a High Performance +Computing system. Step by step, it explains what packages one may want to install and helps configure them.

@@ -86,10 +86,10 @@

During the installation process, you will have to set your USE variables in -/etc/make.conf. We recommended that you deactivate all the +/etc/make.conf. We recommended that you deactivate all the defaults (see /etc/make.profile/make.defaults) by negating them in -make.conf. However, you may want to keep such use variables as x86, 3dnow, gpm, -mmx, nptl, nptlonly, sse, ncurses, pam and tcpd. Refer to the USE documentation +make.conf. However, you may want to keep such use variables as x86, 3dnow, gpm, +mmx, nptl, nptlonly, sse, ncurses, pam and tcpd. Refer to the USE documentation for more information.

@@ -114,8 +114,8 @@

-In step 15 ("Installing the kernel and a System Logger") for stability -reasons, we recommend the vanilla-sources, the official kernel sources +In step 15 ("Installing the kernel and a System Logger") for stability +reasons, we recommend the vanilla-sources, the official kernel sources released on http://www.kernel.org/, unless you require special support such as xfs.

@@ -125,7 +125,7 @@

-When you install miscellaneous packages, we recommend installing the +When you install miscellaneous packages, we recommend installing the following:

@@ -140,35 +140,35 @@

-A cluster requires a communication layer to interconnect the slave nodes to -the master node. Typically, a FastEthernet or GigaEthernet LAN can be used -since they have a good price/performance ratio. Other possibilities include -use of products like Myrinet, Myrinet, QsNet or others.

-A cluster is composed of two node types: master and slave. Typically, your +A cluster is composed of two node types: master and slave. Typically, your cluster will have one master node and several slave nodes.

-The master node is the cluster's server. It is responsible for telling the -slave nodes what to do. This server will typically run such daemons as dhcpd, -nfs, pbs-server, and pbs-sched. Your master node will allow interactive +The master node is the cluster's server. It is responsible for telling the +slave nodes what to do. This server will typically run such daemons as dhcpd, +nfs, pbs-server, and pbs-sched. Your master node will allow interactive sessions for users, and accept job executions.

-The slave nodes listen for instructions (via ssh/rsh perhaps) from the master -node. They should be dedicated to crunching results and therefore should not +The slave nodes listen for instructions (via ssh/rsh perhaps) from the master +node. They should be dedicated to crunching results and therefore should not run any unnecessary services.

-The rest of this documentation will assume a cluster configuration as per the -hosts file below. You should maintain on every node such a hosts file -(/etc/hosts) with entries for each node participating node in the +The rest of this documentation will assume a cluster configuration as per the +hosts file below. You should maintain on every node such a hosts file +(/etc/hosts) with entries for each node participating node in the cluster.

@@ -185,7 +185,7 @@

-To setup your cluster dedicated LAN, edit your /etc/conf.d/net +To setup your cluster dedicated LAN, edit your /etc/conf.d/net file on the master node.

@@ -202,7 +202,7 @@

-Finally, setup a DHCP daemon on the master node to avoid having to maintain a +Finally, setup a DHCP daemon on the master node to avoid having to maintain a network configuration on each slave node.

@@ -239,22 +239,22 @@

-The Network File System (NFS) was developed to allow machines to mount a disk +The Network File System (NFS) was developed to allow machines to mount a disk partition on a remote machine as if it were on a local hard drive. This allows for fast, seamless sharing of files across a network.

There are other systems that provide similar functionality to NFS which could -be used in a cluster environment. The Andrew File System -from IBM, recently open-sourced, provides a file sharing mechanism with -some additional security and performance features. The Coda File System is still in -development, but is designed to work well with disconnected clients. Many +be used in a cluster environment. The Andrew File System +from IBM, recently open-sourced, provides a file sharing mechanism with +some additional security and performance features. The Coda File System is still in +development, but is designed to work well with disconnected clients. Many of the features of the Andrew and Coda file systems are slated for inclusion in the next version of NFS (Version 4). -The advantage of NFS today is that it is mature, standard, well understood, +The advantage of NFS today is that it is mature, standard, well understood, and supported robustly across a variety of platforms.

@@ -277,8 +277,8 @@

-On the master node, edit your /etc/hosts.allow file to allow -connections from slave nodes. If your cluster LAN is on 192.168.1.0/24, +On the master node, edit your /etc/hosts.allow file to allow +connections from slave nodes. If your cluster LAN is on 192.168.1.0/24, your hosts.allow will look like:

@@ -287,7 +287,7 @@

-Edit the /etc/exports file of the master node to export a work +Edit the /etc/exports file of the master node to export a work directory structure (/home is good for this).

@@ -304,8 +304,8 @@

-To mount the nfs exported filesystem from the master, you also have to -configure your salve nodes' /etc/fstab. Add a line like this +To mount the nfs exported filesystem from the master, you also have to +configure your salve nodes' /etc/fstab. Add a line like this one:

@@ -314,7 +314,7 @@

-You'll also need to set up your nodes so that they mount the nfs filesystem by +You'll also need to set up your nodes so that they mount the nfs filesystem by issuing this command:

@@ -329,15 +329,15 @@

-SSH is a protocol for secure remote login and other secure network services -over an insecure network. OpenSSH uses public key cryptography to provide -secure authorization. Generating the public key, which is shared with remote -systems, and the private key which is kept on the local system, is done first +SSH is a protocol for secure remote login and other secure network services +over an insecure network. OpenSSH uses public key cryptography to provide +secure authorization. Generating the public key, which is shared with remote +systems, and the private key which is kept on the local system, is done first to configure OpenSSH on the cluster.

-For transparent cluster usage, private/public keys may be used. This process +For transparent cluster usage, private/public keys may be used. This process has two steps:

@@ -374,12 +374,12 @@ -Host keys must have an empty passphrase. RSA is required for host-based +Host keys must have an empty passphrase. RSA is required for host-based authentication.

-For host based authentication, you will also need to edit your +For host based authentication, you will also need to edit your /etc/ssh/shosts.equiv.

@@ -397,7 +397,7 @@ # $OpenBSD: sshd_config,v 1.42 2001/09/20 20:57:51 mouring Exp $ # This sshd was compiled with PATH=/usr/bin:/bin:/usr/sbin:/sbin -# This is the sshd server system-wide configuration file. See sshd(8) +# This is the sshd server system-wide configuration file. See sshd(8) # for more information. # HostKeys for protocol version 2 @@ -405,7 +405,7 @@

-If your application require RSH communications, you will need to emerge +If your application require RSH communications, you will need to emerge net-misc/netkit-rsh and sys-apps/xinetd.

@@ -417,7 +417,7 @@

-Then configure the rsh deamon. Edit your /etc/xinet.d/rsh file. +Then configure the rsh deamon. Edit your /etc/xinet.d/rsh file.

@@ -456,7 +456,7 @@
 
  # Adelie Linux Research & Development Center
-# /etc/hosts.allow     
+# /etc/hosts.allow
 
 ALL:192.168.1.0/255.255.255.0
 
@@ -489,20 +489,20 @@
 
 
 
-The Network Time Protocol (NTP) is used to synchronize the time of a computer 
-client or server to another server or reference time source, such as a radio 
-or satellite receiver or modem. It provides accuracies typically within a 
-millisecond on LANs and up to a few tens of milliseconds on WANs relative to 
-Coordinated Universal Time (UTC) via a Global Positioning Service (GPS) 
+The Network Time Protocol (NTP) is used to synchronize the time of a computer
+client or server to another server or reference time source, such as a radio
+or satellite receiver or modem. It provides accuracies typically within a
+millisecond on LANs and up to a few tens of milliseconds on WANs relative to
+Coordinated Universal Time (UTC) via a Global Positioning Service (GPS)
 receiver, for example. Typical NTP configurations utilize multiple redundant
-servers and diverse network paths in order to achieve high accuracy and 
+servers and diverse network paths in order to achieve high accuracy and
 reliability.
 
 
 
-Select a NTP server geographically close to you from Public NTP Time 
-Servers, and configure your /etc/conf.d/ntp and 
+Select a NTP server geographically close to you from Public NTP Time
+Servers, and configure your /etc/conf.d/ntp and
 /etc/ntp.conf files on the master node.
 
 
@@ -549,7 +549,7 @@

-Edit your /etc/ntp.conf file on the master to setup an external +Edit your /etc/ntp.conf file on the master to setup an external synchronization source:

@@ -565,7 +565,7 @@ restrict ntp2.cmc.ec.gc.ca stratum 10 driftfile /etc/ntp.drift.server -logfile /var/log/ntp +logfile /var/log/ntp broadcast 192.168.1.255 restrict default kod restrict 127.0.0.1 @@ -573,7 +573,7 @@

-And on all your slave nodes, setup your synchronization source as your master +And on all your slave nodes, setup your synchronization source as your master node.

@@ -594,7 +594,7 @@ restrict master stratum 11 driftfile /etc/ntp.drift.server -logfile /var/log/ntp +logfile /var/log/ntp restrict default kod restrict 127.0.0.1 @@ -608,7 +608,7 @@ -NTP will not update the local clock if the time difference between your +NTP will not update the local clock if the time difference between your synchronization source and the local clock is too great. @@ -691,10 +691,10 @@

-The Portable Batch System (PBS) is a flexible batch queueing and workload +The Portable Batch System (PBS) is a flexible batch queueing and workload management system originally developed for NASA. It operates on networked, -multi-platform UNIX environments, including heterogeneous clusters of -workstations, supercomputers, and massively parallel systems. Development of +multi-platform UNIX environments, including heterogeneous clusters of +workstations, supercomputers, and massively parallel systems. Development of PBS is provided by Altair Grid Technologies.

@@ -703,12 +703,12 @@ -OpenPBS ebuild does not currently set proper permissions on var-directories +OpenPBS ebuild does not currently set proper permissions on var-directories used by OpenPBS.

-Before starting using OpenPBS, some configurations are required. The files +Before starting using OpenPBS, some configurations are required. The files you will need to personalize for your system are:

@@ -762,10 +762,10 @@

-To submit a task to OpenPBS, the command qsub is used with some -optional parameters. In the example below, "-l" allows you to specify +To submit a task to OpenPBS, the command qsub is used with some +optional parameters. In the example below, "-l" allows you to specify the resources required, "-j" provides for redirection of standard out and -standard error, and the "-m" will e-mail the user at beginning (b), end (e) +standard error, and the "-m" will e-mail the user at beginning (b), end (e) and on abort (a) of the job.

@@ -775,8 +775,8 @@

-Normally jobs submitted to OpenPBS are in the form of scripts. Sometimes, you -may want to try a task manually. To request an interactive shell from OpenPBS, +Normally jobs submitted to OpenPBS are in the form of scripts. Sometimes, you +may want to try a task manually. To request an interactive shell from OpenPBS, use the "-I" parameter.

@@ -802,16 +802,16 @@

-Message passing is a paradigm used widely on certain classes of parallel -machines, especially those with distributed memory. MPICH is a freely -available, portable implementation of MPI, the Standard for message-passing +Message passing is a paradigm used widely on certain classes of parallel +machines, especially those with distributed memory. MPICH is a freely +available, portable implementation of MPI, the Standard for message-passing libraries.

-The mpich ebuild provided by Adelie Linux allows for two USE flags: -doc and crypt. doc will cause documentation to be -installed, while crypt will configure MPICH to use ssh instead +The mpich ebuild provided by Adelie Linux allows for two USE flags: +doc and crypt. doc will cause documentation to be +installed, while crypt will configure MPICH to use ssh instead of rsh.

@@ -821,7 +821,7 @@

-You may need to export a mpich work directory to all your slave nodes in +You may need to export a mpich work directory to all your slave nodes in /etc/exports:

@@ -830,15 +830,15 @@

-Most massively parallel processors (MPPs) provide a way to start a program on -a requested number of processors; mpirun makes use of the appropriate +Most massively parallel processors (MPPs) provide a way to start a program on +a requested number of processors; mpirun makes use of the appropriate command whenever possible. In contrast, workstation clusters require that each -process in a parallel job be started individually, though programs to help -start these processes exist. Because workstation clusters are not already -organized as an MPP, additional information is required to make use of them. -Mpich should be installed with a list of participating workstations in the -file machines.LINUX in the directory -/usr/share/mpich/. This file is used by mpirun to choose +process in a parallel job be started individually, though programs to help +start these processes exist. Because workstation clusters are not already +organized as an MPP, additional information is required to make use of them. +Mpich should be installed with a list of participating workstations in the +file machines.LINUX in the directory +/usr/share/mpich/. This file is used by mpirun to choose processors to run on.

@@ -848,11 +848,11 @@

 # Change this file to contain the machines that you want to use
-# to run MPI jobs on.  The format is one host name per line, with either
+# to run MPI jobs on. The format is one host name per line, with either
 #    hostname
 # or
 #    hostname:n
-# where n is the number of processors in an SMP.  The hostname should
+# where n is the number of processors in an SMP. The hostname should
 # be the same as the result from the command "hostname"
 master
 node01
@@ -863,18 +863,18 @@

-Use the script tstmachines in /usr/sbin/ to ensure that -you can use all of the machines that you have listed. This script performs -an rsh and a short directory listing; this tests that you both have -access to the node and that a program in the current directory is visible on -the remote node. If there are any problems, they will be listed. These +Use the script tstmachines in /usr/sbin/ to ensure that +you can use all of the machines that you have listed. This script performs +an rsh and a short directory listing; this tests that you both have +access to the node and that a program in the current directory is visible on +the remote node. If there are any problems, they will be listed. These problems must be fixed before proceeding.

-The only argument to tstmachines is the name of the architecture; this -is the same name as the extension on the machines file. For example, the -following tests that a program in the current directory can be executed by +The only argument to tstmachines is the name of the architecture; this +is the same name as the extension on the machines file. For example, the +following tests that a program in the current directory can be executed by all of the machines in the LINUX machines list.

@@ -883,7 +883,7 @@ -This program is silent if all is well; if you want to see what it is doing, +This program is silent if all is well; if you want to see what it is doing, use the -v (for verbose) argument: @@ -905,24 +905,24 @@

-If tstmachines finds a problem, it will suggest possible reasons and +If tstmachines finds a problem, it will suggest possible reasons and solutions. In brief, there are three tests:

- Can processes be started on remote machines? tstmachines attempts - to run the shell command true on each machine in the machines files by + Can processes be started on remote machines? tstmachines attempts + to run the shell command true on each machine in the machines files by using the remote shell command.
- Is current working directory available to all machines? This - attempts to ls a file that tstmachines creates by running ls using the + Is current working directory available to all machines? This + attempts to ls a file that tstmachines creates by running ls using the remote shell command.
Can user programs be run on remote systems? This checks that shared - libraries and other components have been properly installed on all + libraries and other components have been properly installed on all machines.

@@ -939,7 +939,7 @@

-For further information on MPICH, consult the documentation at http://www-unix.mcs.anl.gov/mpi/mpich/docs/mpichman-chp4/mpichman-chp4.htm.

@@ -973,44 +973,44 @@

-The original document is published at the Adelie Linux R&D Centre web site, -and is reproduced here with the permission of the authors and Cyberlogic's Adelie Linux R&D +The original document is published at the Adelie Linux R&D Centre web site, +and is reproduced here with the permission of the authors and Cyberlogic's Adelie Linux R&D Centre.

http://www.gentoo.org, Gentoo Foundation, Inc.
- http://www.adelielinux.com, + http://www.adelielinux.com, Adelie Linux Research and Development Centre
- http://nfs.sourceforge.net, + http://nfs.sourceforge.net, Linux NFS Project
- http://www-unix.mcs.anl.gov/mpi/mpich/, + http://www-unix.mcs.anl.gov/mpi/mpich/, Mathematics and Computer Science Division, Argonne National Laboratory
http://ntp.org
- http://www.eecis.udel.edu/~mills/, + http://www.eecis.udel.edu/~mills/, David L. Mills, University of Delaware
- http://www.ietf.org/html.charters/secsh-charter.html, + http://www.ietf.org/html.charters/secsh-charter.html, Secure Shell Working Group, IETF, Internet Society
- http://www.linuxsecurity.com/, + http://www.linuxsecurity.com/, Guardian Digital
- http://www.openpbs.org/, + http://www.openpbs.org/, Altair Grid Technologies, LLC.

-- gentoo-doc-cvs@lists.gentoo.org mailing list