[gentoo-dev] Stow it in the warehouse

[gentoo-dev] Stow it in the warehouse

[Garry Roseman]

On Fri, 2002-07-05 at 11:33, Wout Mertens wrote:
> Hey there,
> 
> I agree, stow is cool and at work we use it to maintain our 12GB
> /usr/local tree that is exported over nfs to our workstations.
> Some drawbacks, however:
> 
> - You have a slight speed loss because of the symlinking adding
>   extra lookups. Luckily, Linux has very good caching :)
> - Some packages hate it when you symlink stuff; e.g. sudo needs its
>   sudoers file to be a regular file. Granted, this is a configuration
file
>   and as such may not need to be symlinked in a general gentoo
context, so
>   this could be solved by just creating a regular file in the
>   pkg_postinst.
> - stow -R can grind your server to a screeching halt if you have many
>   files. I'm sure this is solveable by rewriting the code a little,
and I
>   don't know if recent versions have trouble with it since we don't
try it
>   anymore ;)
> - You have a lot of symlinks in your /usr, which makes ls -l a bit
less
>   attractive to look at. Of course, there's still ls -lL ;-)
> 
> I've been considering nagging to drobbins about changing Portage so
that
> merging actually means 
> [build the image somewhere where it will remain and install by
symlinking]

That would be a nice option.  Portage developers already do all of the
hard work that we stowers often face in assuring that a package will
install into a tree outside of its target location.  For example, it
would be nice if I could emerge a package to a "warehouse" such as
/usr/local/stow and just build the package in <warehouse>/<package>
(instead of /var/tmp/portage/<section>/<package>/image).  Symlinking
should be left for a separate manual step.

One advantage of emerging into a warehouse is that it becomes possible
to define software build environments that are different for different
users or for different projects.  This is done with Stow by stowing to a
special target directory, often a home directory.  For example, a user
who wants to do development with GCC-3.1 could 
  cd <warehouse>;  stow -t ~/ gcc-3.1 etc.
and assuming that his PATH includes ~/bin and the loader is configured
to search his $HOME/lib for libraries, etc., then that user can proceed
to use gcc 3.1 without interfering with the gcc that is used for normal
software installation on the system.  I have seen some discussion of
using stow for creating and archiving entire project-specific build
environments, but I haven't tried it yet.  It seems like a good match
for portage.  
 
> This would have the advantage that you can always revert to a certain
> version of a package with certainty, since no files are removed from
the
> previous package. And you see where a file comes from really quickly
by
> looking at the symlink, which is very useful.

It becomes possible, maybe even easy (so far I find it easy on my 700 MB
/usr/local), to revert to the previous release or to jump back and forth
between package versions.  As I suggested above, it also may become
possible to define, track, and archive the entire build environment for
software development projects.  That is essential for me --- it must be
possible to recreate the environment that produced software that I have
running in field installations, potentially a different version for each
installation.

> 
> And actually, I think it would be possible to let portage optionally
have
> that feature, because the only thing it changes is where the files are
> installed and what merge and unmerge do. All the rest would stay the
> same. That way, people who want it can turn it on and the rest aren't
> bothered.
> 
> Thoughts?

I think there's potential for using portage to build into a warehouse
and use stow to withdraw custom environments from the warehouse.  But we
need experience doing it --- does anyone know how to force emerge to
install into a warehouse as I have described it?

It appears to me that all we can do right now with portage is to define
BUILD_PREFIX to point to the warehouse and portage would put the image
into a <package> subfolder but it would create an "image" sub-subfolder
-- that's not what we want.  The "image" subfolder is superfluous. We
would also want emerge to track this package version separately from
those installed into /.  If an update is released for a warehoused
package then let emerge -u world just install the updated version
alongside it in the warehouse and let me know that it is available. 
Updating a warehoused package would not automatically put the new
version on the path for execution. 

Right now using portage to install packages into the local warehouse
requires use of "ebuild <package>.ebuild install" followed by "mv
<image>/* <warehouse>/<package>" and there is no automatic updating of
packages warehoused that way.

Given a black-and-white choice between an elaborated system of
dependency tracking and file tracking via a database, and a simple
warehouse of available versions that can be quickly symlinked into an
environment (and allowing user and project-specific environments) ---
but with no "query" tool other than "ls" --- I'd take the warehouse and
dump the database!

But maybe it's not a black and white choice?

-- 
Fuper, Memphis, LPIC-1