[gentoo-soc] Weekly progress report [june 9-15/16] Gentoo Clustering CD

[gentoo-soc] Weekly progress report [june 9-15/16] Gentoo Clustering CD

[Eric Thibodeau]

[-- Attachment #1.1: Type: text/plain, Size: 3319 bytes --]

_*LDAP:*_
I spent many hours (way over the 30 hours I had promised myself to 
pass/week on SoC) creating an LDAP-as-auth-backend auto-install script. 
It's not simple because Gentoo's philosophy is that ebuilds do as little 
as possible and the admin does the work. I have no problems with this 
approach but it's, by definition, countering my efforts of providing a 
"turn-key" solution Clustering LiveCD. Although most of the work that is 
being done by the script should be done by an ebuild, I had to chose a 
stand alone script beacuse:

1- I _absolutely_ have to modify/create some files in /etc
2- Once _some_ of the files created, I have to initiate the ldap database
3- Then _successfully_ start the slapd daemon
4- and _only_ then shall I finish the /etc file modifications (ie: 
changing /etc/nsswitch.conf to also use ldap as a backend)

Obviously, since this script is supposed to be called from within the 
catalyst process, Joe user should not have to use it but my intention is 
that the script could also be used later on for people wishing to 
implement LDAP without having to learn _all_ that is required to get 
that going on their system (obviously with a BFW: "This is a one shot 
deal, don't expect it to work, you should read the docs, it's poison, it 
will reformat your car's carburator, etc..." I'm also leaving in the 
possiblity that the same script + config file approach could  be used to 
_add_ LDAP databases in the future (such as a shared Addressbook)

Well, even though all of this seems far from clustering and HPC, the 
whole central auth and management is an issue when it comes to a 
cluster. One has to remember that a cluster is like a department 
isolated on it's own network and everyone is supposed to be able to log 
onto _any_ machine and expect them to all behave the exact same way.

Stuff that would be nice to also have in LDAP which isn't presently part 
of my script/template:

* Automount definitions
* TLS
*

_*Catalyst:
*_I updated the spec files to use a new snapshot since I will want to be 
using net-nds/openldap-2.4.10 and it's quite recent in the tree. In the 
process I noticed I could get to Stage3 with no problems but that 
liveCD-stage1.spec now completely barfs with a huge list of loop 
dependency errors. I backtracked to the original snapshot and the errors 
are also there. I'll have to investigate by removing my profile overlay, 
it's probably due to some change I did in there and didn't rebuild the 
liveCD since. It's not critical for the moment so I'll set that aside 
for the time being (adding a bug ton soc.gexp.o)

_*Clustering:*_
Jsbronder's on fire, I'll definately have to look into his *empi* and 
*eselect mpi* work, being more than just relevant to clustering ;)

_*Special thanks:*_
robbat2: for all his help and patience with my obvious n00bism 
concerning LDAP ACLs and some config directives ;)
Damm (#ldap): Has helped me with a few questions and made me waste much 
time on nssov...which I thwarted him into trying to create an ebuild now :P

Eric
PS: also available under the project's News : 
http://soc.gentooexperimental.org/projects/gentoo-cluster-seed/news
Current files not currently available on the web site...so here they 
are. I DIDN'T run ispell on them...so please, no harsh comments on my 
keyboard dyslexia ;)

[-- Attachment #1.2: Type: text/html, Size: 3708 bytes --]

[-- Attachment #2: cluster_ldap_skel.conf --]
[-- Type: text/plain, Size: 3638 bytes --]

#!/bin/bash
# By Eric Thibodeau
# 13 June 2008
#
# NOTES: 
#	* it is usually suggested to keep all definitions in 
#	_lower case_... but do as youi wish!
#	* all *_DESC variables are optionnal and are simply used
#	in the description field of the LDAP db.
#
# What is the name you want to give the LDAP domain?
# say we wanted the domain to be gentoo.local you would
# put the following:
DOMAIN="gentoo.local"
DOMAIN_DESC="This is the top of the LDAP hierarchy"
# Which part of your organisation is this machine filling in
# for? In the present example, this is the cluster so we'll call
# this Organizational Unit (ou) cluster:
OU="cluster"
OU_DESC="Clustering department branch. All units defined under this branch are for use by the cluster"

# We will create some specific branches under that cluster,
# logically, we'll have users and groups to manage these, so
# we will minimally impose the definition of these two:
USERS_OU="users"
USERS_OU_DESC="Cluster specific Users"
GROUPS_OU="groups"
GROUPS_OU_DESC="Cluster specific Groups"

# OUTHER_OU is parsed to automatically create other sub-OU under
# the one defined above (as OU). This could be, for example:
#OTHER_OU="aliases networks hosts"

# if you want descriptions to be added to each of these groups
# automatically in the LDAP database, dedine a separate _DESC
# vairable for each. For example:
#aliases_DESC="This is the container for user aliases"
#hosts_DESC="This is the container for static host descriptions"

# Although this is often a philosophical debate, we'll stick 
# with having an admin for the ldap user database and one
# for managing it's contents. Here is the dirrerence in their
# role:
#
# The ADMIN_DN will be the user used to create the ldap db
# and have total control over it. This user is typically useful
# only at creation and dumping/migration of the database. This
# user _always_ has TOTAL access to the LDAP db where it's 
# defined. One typically _doesn't_ use this user to mange the
# LDAP database, the user defined in ADMIN_DN is the one to use.
#
LDAP_ADMIN_DN="admin"
# This is the paswsord to use for LDAP management tasks and is the
# one that is stored in /etc/openldap/slapd.conf (but we at least
# hash it ;) 
LDAP_ADMIN_DN_PWD="default"

# The following user will essentially be identical to root, you're better
# off not renaming him.
ADMIN_DN="root"
ADMIN_DN_DESC="root account (under LDAP)"

# The following is the name of the group used to identify people
# with full access to the LDAP db. One advantage is that the members
# of this group can be dynamically changed within the LDAP db. NOTE: 
# Since it's given the same gid as wheel, it's functionnaly equivalent!!!
ADMIN_GROUP_DN="wheel"
ADMIN_GROUP_DN_DESC="Users in this group can freely modify the LDAP directory at will"

# The following is the place to put the resulting generated files
# usually we'd want this to be ROOT="/" ...if you trust the script 
# entirely ;). We use the environment's $ROOT if one is available...
[[ -z $ROOT ]] && ROOT="/"

# The LDAP_AS_AUTH determines wether we change /etc/nsswitch.conf
# AND /etc/pam.d/system-auth so that LDAP can _also_ be used as
# and authetication backend. We anticipate the script could also be
# used as a means to create other LDAP configs (ie: shared addressbook)
# But for the meantimne, leave it to yes:
LDAP_AS_AUTH="#yes"

# The LDIF_OUT defines the name of the ldif file that will be automatically
# created by the script. It's only really useful if you want to keep that file 
# afterwards for xyz reason (obviously, LDIF_OUT_KEEP has to be set to yes):
LDIF_OUT="./create_db.ldif"
LDIF_OUT_KEEP="yes"

[-- Attachment #3: ldap-setup.sh --]
[-- Type: application/x-sh, Size: 10735 bytes --]

Re: [gentoo-soc] Weekly progress report [june 9-15/16] Gentoo Clustering CD

[Donnie Berkholz]

On 22:07 Mon 16 Jun     , Eric Thibodeau wrote:
> _*LDAP:*_
> I spent many hours (way over the 30 hours I had promised myself to 
> pass/week on SoC) creating an LDAP-as-auth-backend auto-install script. 
> It's not simple because Gentoo's philosophy is that ebuilds do as little as 
> possible and the admin does the work.

Hmmm ... you might want to read 
http://www.gentoo.org/main/en/philosophy.xml again and consider how it 
applies to this project.

Thanks,
Donnie
-- 
gentoo-soc@lists.gentoo.org mailing list

Re: [gentoo-soc] Weekly progress report [june 9-15/16] Gentoo Clustering CD

[Eric Thibodeau]

[-- Attachment #1: Type: text/plain, Size: 4045 bytes --]

Well, as you point it out:
     "If the tool forces the user to do things a particular way, then 
the tool is working against, rather than for, the user."

    My experience with infrastructures and especially configuration 
management tools is that you can't have both automatic-easy-no-hands-on 
configuration and manual configuration of the same application 
simultaneously. If one attempts to make a "perfect" configuration tool, 
he will quickly realize the human being is unpredictable and sometimes, 
god forbid, stupid. Compensating for the human factor is a never ending 
quest which eventually force the choice between total control over the 
configuration by the tools or total configuration by the user/admin. If 
you don't believe me, look at a popular OS called Windows, made by 
Microsoft. Their attempt at providing a unified "interface" to all 
configuration and management aspects of the machines inadvertently fails 
as one almost always has to go into the "registry" to fix something a 
user (also called viruse) did or didn't do.

    Furthermore, totally interfaced control over a complex backend is 
also failure-proned. One of my IT colleagues experienced weird name 
resolution problems that were eventually resolved by deleting entries in 
an AD using OpenLDAP tools accessing the said AD through a translation 
backend (the interface to the AD failed to report the keys...even the 
search engine for the advanced management)

    Obviously, this view is a can-O-worms and I really don't want to 
start this philosophical debate. My point is actually that I want the 
project to adhere to the Gentoo philosophy as closely as possible and I 
can hardly see how this can be the case when I _impose _ some 
configuration decisions (ie: I'm not asking the user what ACL to put 
into slapd and how to construct system-auth).

    This said, my approach at modifying the configuration files is using 
a "general" .conf file (ie: we could call it either a domain-def.conf or 
local-machine-def.conf file) which (would/should) sources as much 
information form the gentoo-specific /etc/conf.d file and patch in where 
specifics aren't defined. If this can be accepted by the powers that my 
be in Gentoo as an approach to "guiding" the initial setup of a machine 
(as performed by src_install (or should it be pkg_setup) ) then fine!

    But, like I said when choosing to make the script, some of the steps 
need to be accomplished in an authoritative manner and in a specific 
sequence, which cannot be guaranteed by portage.

Now, following what Donnie pointed me to on the Seed Linux mailing list 
(http://groups.google.com/group/seed-linux-dev/browse_frm/thread/d5ab069f47de4b76? 
) and more specifically this post about global configuration management: 
http://groups.google.com/group/seed-linux-dev/browse_thread/thread/73cb8a4fef940903 
, I believe we have something interesting that may result from merging 
my script into proper Seed Linux Ebuilds and getting some consensus on 
the configuration definition.

Eric

PS: To answer the original e-mails question: yeah, I think the 
philosophy applies to the project. At the end, the user is supposed to 
end up with a working system that remains 100% a Gentoo system (with 
minimal overlay interaction) most of the hard work being 
auto-configuration to _get going_... The user is then free to go and 
break it all by modifying the config files, the I will painstakingly 
keep as close to the proposed ones from the original ebuilds ;)

Donnie Berkholz wrote:
> On 22:07 Mon 16 Jun     , Eric Thibodeau wrote:
>   
>> _*LDAP:*_
>> I spent many hours (way over the 30 hours I had promised myself to 
>> pass/week on SoC) creating an LDAP-as-auth-backend auto-install script. 
>> It's not simple because Gentoo's philosophy is that ebuilds do as little as 
>> possible and the admin does the work.
>>     
>
> Hmmm ... you might want to read 
> http://www.gentoo.org/main/en/philosophy.xml again and consider how it 
> applies to this project.
>
> Thanks,
> Donnie
>   


[-- Attachment #2: Type: text/html, Size: 4893 bytes --]