[gentoo-dev] Gentoo Enterprise deployment tools

[gentoo-dev] Gentoo Enterprise deployment tools

[Thierry Carrez]

[-- Attachment #1: Type: text/plain, Size: 1164 bytes --]

Hi folks,

I would like to get your opinion on Enterprise-oriented desktop
deployment tools for Gentoo Linux (or the lack of).

As a small company CIO, I deployed Gentoo on a small scale here but
quickly ran into scaling problems and the lack of tools to help.

There is no obvious way to freeze a Portage tree (or to design a
specific profile) for testing on a golden workstation, to build a set of
update packages (ServicePack) and push it to the workstations, or to
have centralized accountability of what's installed where. There is no
easy way to avoid having to keep a synchronized copy of the portage tree
on all systems, even when using yourown-binaries.

With automatic deployments, would we run into difficult-to-solve
etc-update problems ? Should/could the ServicePack system take care of
that ?

Even in a simpler setup (preprod > production) we don't have the tools
to push a software configuration change from a test machine to a
production one.

What tools are missing ? Is it our job to provide them ? Can it
reasonably be done ? Am I just wrong to want to use Gentoo in that
direction ?

Next week: Gentoo-as-a-metadistribution tools :)

-- 
Koon

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Gentoo Enterprise deployment tools

[Alec Warner]

Thierry Carrez wrote:
> Hi folks,
> 
> I would like to get your opinion on Enterprise-oriented desktop
> deployment tools for Gentoo Linux (or the lack of).
> 
> As a small company CIO, I deployed Gentoo on a small scale here but
> quickly ran into scaling problems and the lack of tools to help.
> 
> There is no obvious way to freeze a Portage tree (or to design a
> specific profile) for testing on a golden workstation, to build a set of
> update packages (ServicePack) and push it to the workstations, or to
> have centralized accountability of what's installed where. There is no
> easy way to avoid having to keep a synchronized copy of the portage tree
> on all systems, even when using yourown-binaries.

Network mountable trees seem to work well enough although an emerge 
--metadata is still required on clients.
I would disagree also on the profile argument.  Profiles are very 
powerful, very details, and have decent manpages as well as literally 
tons of examples.  What specifically is stopping you from rolling your 
own profile?

The rest of that stuff is a generally well known about issue ( at least 
in the portage community ).  Many features of portage-2.1 will be 
helpful in this type of situation.

> With automatic deployments, would we run into difficult-to-solve
> etc-update problems ? Should/could the ServicePack system take care of
> that ?
  I wouldn't use etc-update for this on a enterprise rollout personally. 
    If you need config cfengine does a nice job, as well as using 
cvs/rcs/something-else

> 
> Even in a simpler setup (preprod > production) we don't have the tools
> to push a software configuration change from a test machine to a
> production one.
   What exactly are you looking for here?
> 
> What tools are missing ? Is it our job to provide them ? Can it
> reasonably be done ? Am I just wrong to want to use Gentoo in that
> direction ?
   Portage needs work; I know the devs are working on it, I know there 
are other people who are doing there own things.  I see a lot of 
portage-2.1 features that greatly simplify what you are trying to do ( 
repositories, config rewrite..etc.. ).  I think portage and what it 
covers is a big part of this.  Recollecting a conversation with jstubbs 
about portage he mentioned that he wouldn't want the portage-team to 
maintain a Enterprise-like distribution program, but that the new API 
would be great to write one against ;)

I also know Chotchki was looking at doing his senior thesis on a network 
aware portage that did some cool things.  A lot of this is just waiting 
( and helping :) :) ) the portage devs get the work done that needs to 
get done.

I know Cardoe and genstef? are working on a seperate package manager 
that just handles binaries but uses all the current portage stuff, so 
you might want to talk to them as well.

> 
> Next week: Gentoo-as-a-metadistribution tools :)
> 

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Gentoo Enterprise deployment tools

[Thierry Carrez]

Alec Warner wrote:

>> There is no obvious way to freeze a Portage tree (or to design a
>> specific profile) for testing on a golden workstation, to build a set of
>> update packages (ServicePack) and push it to the workstations, or to
>> have centralized accountability of what's installed where. There is no
>> easy way to avoid having to keep a synchronized copy of the portage tree
>> on all systems, even when using yourown-binaries.
> 
> Network mountable trees seem to work well enough although an emerge
> --metadata is still required on clients.
> I would disagree also on the profile argument.  Profiles are very
> powerful, very details, and have decent manpages as well as literally
> tons of examples.  What specifically is stopping you from rolling your
> own profile?
> 
> The rest of that stuff is a generally well known about issue ( at least
> in the portage community ).  Many features of portage-2.1 will be
> helpful in this type of situation.

I probably wasn't clear :)

I don't say that it cannot be done, and I don't ask what's the best way
to do it. I just ask *if* we should try to provide higher-level tools
(and/or doc) to help in doing so. It's not obvious (especially for
non-developers) how to proceed in that situation, even if a lot of
people have designed their own solution in their corner.

>> With automatic deployments, would we run into difficult-to-solve
>> etc-update problems ? Should/could the ServicePack system take care of
>> that ?
> 
>  I wouldn't use etc-update for this on a enterprise rollout personally.
>    If you need config cfengine does a nice job, as well as using
> cvs/rcs/something-else

Again, the technology is out there, it's just not tightly integrated.
Should we leave it as-is and let everyone design his own tools to
connect the dots or should we ?

>> Even in a simpler setup (preprod > production) we don't have the tools
>> to push a software configuration change from a test machine to a
>> production one.
> 
> What exactly are you looking for here?

Should we provide high-level software that defines an update pack (new
binaries + configuration changes), allows to test it on a preproduction
system and (once tested) to push it to registered production systems ?
Or let everyone write his own treefreezing + network mounts + shell
scripts + cfengine / CVS magic combo to do it ?

>   Portage needs work; I know the devs are working on it, I know there
> are other people who are doing there own things.  I see a lot of
> portage-2.1 features that greatly simplify what you are trying to do (
> repositories, config rewrite..etc.. ).  I think portage and what it
> covers is a big part of this.  Recollecting a conversation with jstubbs
> about portage he mentioned that he wouldn't want the portage-team to
> maintain a Enterprise-like distribution program, but that the new API
> would be great to write one against ;)

I don't think it should be the role of the portage-team either.

> I also know Chotchki was looking at doing his senior thesis on a network
> aware portage that did some cool things.  A lot of this is just waiting
> ( and helping :) :) ) the portage devs get the work done that needs to
> get done.
> 
> I know Cardoe and genstef? are working on a seperate package manager
> that just handles binaries but uses all the current portage stuff, so
> you might want to talk to them as well.

I sure hope they will comment on that thread :)

-- 
Koon
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Gentoo Enterprise deployment tools

[Omkhar Arasaratnam]

Thierry Carrez wrote:

>Hi folks,
>
>I would like to get your opinion on Enterprise-oriented desktop
>deployment tools for Gentoo Linux (or the lack of).
>
>As a small company CIO, I deployed Gentoo on a small scale here but
>quickly ran into scaling problems and the lack of tools to help.
>
>There is no obvious way to freeze a Portage tree (or to design a
>specific profile) for testing on a golden workstation,
>
Why not? Just don't update the portage tree

> to build a set of
>update packages (ServicePack) and push it to the workstations, or to
>have centralized accountability of what's installed where. 
>
set make.conf to point to a static tree stored somewhere which you
update as time permits. You could even create an /etc/init.d script that
would check for updates (emerge --sync) against your QA'ed frozen tree
at each boot.

>There is no
>easy way to avoid having to keep a synchronized copy of the portage tree
>on all systems, even when using yourown-binaries.
>  
>
see above

>With automatic deployments, would we run into difficult-to-solve
>etc-update problems ? Should/could the ServicePack system take care of
>that ?
>  
>
etc-update would continue to be a problem

>Even in a simpler setup (preprod > production) we don't have the tools
>to push a software configuration change from a test machine to a
>production one.
>
>What tools are missing ? Is it our job to provide them ? Can it
>reasonably be done ? Am I just wrong to want to use Gentoo in that
>direction ?
>
>Next week: Gentoo-as-a-metadistribution tools :)
>
>  
>
I think most of the assumptions that you're making involve giving your
user population root access.
Don't
Lock down your make.conf and only roll up update when you deem is
neccassary using your internal tree. etc-update may still be an issue....

-- 

Omkhar Arasaratnam - Gentoo PPC64 Developer
omkhar@gentoo.org - http://dev.gentoo.org/~omkhar
Gentoo Linux / PPC64 Linux: http://ppc64.gentoo.org

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Gentoo Enterprise deployment tools

[Brian Harring]

On Tue, Jun 21, 2005 at 08:35:52PM +0200, Thierry Carrez wrote:
> I don't say that it cannot be done, and I don't ask what's the best way
> to do it. I just ask *if* we should try to provide higher-level tools
> (and/or doc) to help in doing so. It's not obvious (especially for
> non-developers) how to proceed in that situation, even if a lot of
> people have designed their own solution in their corner.
Best way to do it?  Scary notion, not the way we're doing it 
currently.

Push mode is preferable imo, 'cept no code exists to support that.  
Someone could write the necessary client/server code, but that would 
have issues when bound into existing portage apis...

> >> With automatic deployments, would we run into difficult-to-solve
> >> etc-update problems ? Should/could the ServicePack system take care of
> >> that ?
> > 
> >  I wouldn't use etc-update for this on a enterprise rollout personally.
> >    If you need config cfengine does a nice job, as well as using
> > cvs/rcs/something-else
> 
> Again, the technology is out there, it's just not tightly integrated.
> Should we leave it as-is and let everyone design his own tools to
> connect the dots or should we ?
Not sure if the technology persay is out there honestly.  If it were a 
cluster, cloned boxes, I'd say minimalize CONFIG_PROTECT, and 
(assuming you write the client/server cruft above) slip in config pkgs 
that get installed alongside... or, just jam the config changes into 
the pkg (not clean but it's possible).  Or just trigger 
staggered reboot's on the boxes if you've got a fast network and pxe 
boot + imaging setup (I like the other method a bit more however :)

If you're managing a half dozen servers, each server running it's own 
customized httpd.conf, I don't see an easy way to handle that 
(would love to hear any ideas people have on that one).

Basically, kind of curious of how one could easily handle config 
management of multiple boxes, with config's potentially being wildly 
different from system to system (talking about a bit more then just 
/etc/conf.d/net.* and /etc/hostname differences here).  I suspect just 
wrapping the config changes into a bingpkg, and sliding them out 
alongside on a push would suffice, but that's just one possible 
method.

> >> Even in a simpler setup (preprod > production) we don't have the tools
> >> to push a software configuration change from a test machine to a
> >> production one.
> > 
> > What exactly are you looking for here?
> 
> Should we provide high-level software that defines an update pack (new
> binaries + configuration changes), allows to test it on a preproduction
> system and (once tested) to push it to registered production systems ?
> Or let everyone write his own treefreezing + network mounts + shell
> scripts + cfengine / CVS magic combo to do it ?
How do you push it?  I don't mean, what protocol/underlying, I'm 
asking how do you actually push _portage_ to do what you want?  Either 
you try and abuse the craptastic api in stable to pull it off, or you 
probably resort to a catalyst akin trick of calling emerge via system.

Neither is a proper solution.  Api is required, further, preferably 
portage innards being designed such that you can swap in your own 
remote subsystem (whether cache tree or config) so it's a matter of 
pushing commands down the client/server pipes, with the portage 
config/installation on that box pulling what it needs (remote tree == 
having to pull all relevant files if building, binpkg is easier 
however).

> >   Portage needs work; I know the devs are working on it, I know there
> > are other people who are doing there own things.  I see a lot of
> > portage-2.1 features that greatly simplify what you are trying to do (
> > repositories, config rewrite..etc.. ).  I think portage and what it
> > covers is a big part of this.  Recollecting a conversation with jstubbs
> > about portage he mentioned that he wouldn't want the portage-team to
> > maintain a Enterprise-like distribution program, but that the new API
> > would be great to write one against ;)
> 
> I don't think it should be the role of the portage-team either.

I draw a slightly finer line... portaged, some hypothetical 
client/server ap, not our business to implement, just provide an api 
for them to use.  Thing is, if they're going remote, they'll either 
need to be able to trigger sync's on the boxes local tree 
(innefficient as all hell), or the tree is remote.  If the tree is 
remote, that falls on portage devs head to provide a framework so the 
tree can be remote, in other words abstraction/framework design.

Further... if you're pushing updates out, you need some method to 
query the vdb from the target- even if you're dealing with pushing 
updates down to a set of identical installations, you need to identify 
(easily/cleanly) what needs to be built, and what needs to be pushed 
down the line.  Dancing around it, but you need access to the vdb for 
that system definition, which probably would be stored locally... in 
which case, the system targets probably would need to have a remote 
vdb.

Implementing all of the crazy and fun stuff isn't portage (the 
project) business (interest in it personally, but other things have 
much higher priority).  To do the crazy/fun stuff requires a sane 
design so stuff can be swapped in/out as required, which falls on our 
heads though, and is what's being kicked around/worked on now.

> > I know Cardoe and genstef? are working on a seperate package manager
> > that just handles binaries but uses all the current portage stuff, so
> > you might want to talk to them as well.
> 
> I sure hope they will comment on that thread :)
Kind of curious what they're attempting myself, since I've not heard 
much thus far...
~harring
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Gentoo Enterprise deployment tools

[Thierry Carrez]

Omkhar Arasaratnam wrote:

> I think most of the assumptions that you're making involve giving your
> user population root access.
> Don't

??
The assumptions I am making are clearly not involving giving a user
population root access. I just point to the lack of tools to maintain
semi-frozen trees and to automate software updates on a large enterprise
desktop deployment, and try to see if this gathers interest. I fail to
see where I need to give wheel to user.

My position would rather be that workstations don't need a portage tree
or an emerge command. A software deployment server could push package
installation on workstations and keep track of what's installed where
and with which configuration files.

-- 
Thierry Carrez (Koon)
Gentoo Linux Security
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Gentoo Enterprise deployment tools

[Nathan L. Adams]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Thierry Carrez wrote:
> Omkhar Arasaratnam wrote:
> 
> 
>>I think most of the assumptions that you're making involve giving your
>>user population root access.
>>Don't
> 
> 
> ??
> The assumptions I am making are clearly not involving giving a user
> population root access. I just point to the lack of tools to maintain
> semi-frozen trees and to automate software updates on a large enterprise
> desktop deployment, and try to see if this gathers interest. I fail to
> see where I need to give wheel to user.
> 
> My position would rather be that workstations don't need a portage tree
> or an emerge command. A software deployment server could push package
> installation on workstations and keep track of what's installed where
> and with which configuration files.
> 

Would the semi-frozen tree you're looking for be embodied by GLEP 14? (I
think it would for my needs.) Imagine if you could do something like this:

# emerge --securityupdates world

And you would get only the updates required by GLSA's that affect /your/
machine.

http://www.gentoo.org/proj/en/glep/glep-0014.html

This is similar to what Enterprises do with Windows installations, and I
honestly think GLEP 14 could become portage's killer feature benefiting
'normal' users and enterprise users alike.

The ability to push updates from a central location is, of course, a
separate matter. :)

Nathan

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCuVgR2QTTR4CNEQARAqr3AJ9YqU09o9ZzeAVOapVCu5DnfyXCJwCgnDT6
RzGQ8mNP9qxH6RePsnIpK6M=
=a99r
-----END PGP SIGNATURE-----
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Gentoo Enterprise deployment tools

[M. Edward (Ed) Borasky]

Thierry Carrez wrote:

>Hi folks,
>
>I would like to get your opinion on Enterprise-oriented desktop
>deployment tools for Gentoo Linux (or the lack of).
>
>As a small company CIO, I deployed Gentoo on a small scale here but
>quickly ran into scaling problems and the lack of tools to help.
>
>There is no obvious way to freeze a Portage tree (or to design a
>specific profile) for testing on a golden workstation, to build a set of
>update packages (ServicePack) and push it to the workstations, or to
>have centralized accountability of what's installed where. There is no
>easy way to avoid having to keep a synchronized copy of the portage tree
>on all systems, even when using yourown-binaries.
>
>With automatic deployments, would we run into difficult-to-solve
>etc-update problems ? Should/could the ServicePack system take care of
>that ?
>
>Even in a simpler setup (preprod > production) we don't have the tools
>to push a software configuration change from a test machine to a
>production one.
>
>What tools are missing ? Is it our job to provide them ? Can it
>reasonably be done ? Am I just wrong to want to use Gentoo in that
>direction ?
>
>Next week: Gentoo-as-a-metadistribution tools :)
>  
>
I'm not sure some of the assumptions you and other posters have made are
valid for the specific case of a "enterprise network of workstations and
servers" in a Linux environment, or, for that matter, in a Windows-based
enterprise. First of all, why would a "workstation" need to have *any*
software installed on its hard drive at all? You can boot the OS off the
network, load executables off the network, read shared data off the
network, and even create a cluster for large computations over the
network. The only thing that needs to reside on the workstation's hard
drive is the unique data representing the workstation user's
*legitimate* business requirements and contributions, and any desktop
customizations/unique configuration files. In the ancient days, when
workstation hard drives were tiny, that's how it was done.

PCs in enterprises, whether Linux, Windows, Macs or other flavors,
resemble home computers only because some enterprises think it's
necessary to "retain talented employees", not because they actually need
to have a browser, OS, office suite, email, virus scanner, etc., loaded
on their desktop machines. I work in a Windows environment, and I've got
all that stuff on my PC, taking up disk space that could be used by the
large amounts of data I work with as a performance engineer. The only
software on my machine that needs to be there that wouldn't normally be
on everyone's machine in the enterprise is a few high-end analysis
packages like R and Maxima.

So ... the problem actually reduces to deployment/management tools for
*servers*, I think, plus a crew of IT folks who can rescue users who
manage to hose up the home directory on their workstation, the only
place in the building they can actually write into. :)
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Gentoo Enterprise deployment tools

[Mike Doty]

Thierry Carrez wrote:
> Omkhar Arasaratnam wrote:
> 
> 
>>I think most of the assumptions that you're making involve giving your
>>user population root access.
>>Don't
> 
> 
> ??
> The assumptions I am making are clearly not involving giving a user
> population root access. I just point to the lack of tools to maintain
> semi-frozen trees and to automate software updates on a large enterprise
> desktop deployment, and try to see if this gathers interest. I fail to
> see where I need to give wheel to user.
> 
> My position would rather be that workstations don't need a portage tree
> or an emerge command. A software deployment server could push package
> installation on workstations and keep track of what's installed where
> and with which configuration files.
> 
Why not have your build server make .rpms(or .tgz for that matter) and 
automagicly untar to /.  Alternatively, I'm exploring cvs to manage /etc 
for my clustered nodes, that may be something you want to consider as well.

Mike
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Gentoo Enterprise deployment tools

[Jim Northrup]

M. Edward (Ed) Borasky wrote:

>Thierry Carrez wrote:
>
>  
>
>>Hi folks,
>>
>>I would like to get your opinion on Enterprise-oriented desktop
>>deployment tools for Gentoo Linux (or the lack of).
>>
>>As a small company CIO, I deployed Gentoo on a small scale here but
>>quickly ran into scaling problems and the lack of tools to help.
>>    
>>
I have deployed gentoo on openmosix for the reason of scale, where only
one machine was the subject of configuration and point of access, while
all neighbor nodes were less specific in configuration but all shared
kernel.

openmosix is more secure than plain old IP, given its custom marked
packets (or something keen like that) and is well suited to LAMP which
can occasionally expect a cgi process to take a dump or hang or freeze,
upon the loss of a node holding its virtual process.  open source
databases have lousy replicatoin, or at least did when i was undertaking
this.  its unlikely the IO-bound processes will migrate, but the
linked-list traversals deservedly take a hike(ie migrate) when the app
design falls short.

so the question of how to begin locking down packages, etc for HA,
that's where staging and testing plans are of key importance.

this is where my own syadmin handywork started with a cloning system:
the mothership on one side, and a rescue disc, gig-e hub, and a
nc/gnutar combo to clone and stage mutations/development.


-- 
gentoo-dev@gentoo.org mailing list