[gentoo-amd64] Re: backups and world updates

[Duncan <1i5t5.duncan@cox.net>]

Mark posted <1f81f7e005072806534ed43866@mail.gmail.com>, excerpted below, 
on Thu, 28 Jul 2005 09:53:20 -0400:

> Thanks to everyone who helped me get my system back after a couple of
> world update snafus. Now of course I'm a little gun-shy about using
> options like emerge --update world. So what's my best bet to keep my
> system up to date, while protecting it from my own lack of understanding
> of updating config files?
> 
> Here's what I'm intending to do so far:
> 
> 1. Prior to running any large system update, back up /etc to another
> location 2. use dispatch-conf instead of etc-update
> 
> Can anyone make any other suggestions? Which emerge options are best for
> full system updates? Thanks

One of the most useful things I've found here, is to add "buildpkg" to
your make.conf FEATURES line.  Portage will then emerge as normal thru the
compile and fake-install stages, then instead of qmerging directly to your
live file system, it will tarball up the resulting fake-install, attach
the ebuild and some useful metadata to to make merging it easier, and file
away the resulting binary package in your portage ${PKGDIR}
(${PORTDIR}/packages, by default).  It will then untar the binpkg it just
created, and merge that, thereby ensuring the binpkg works as intended
(thus avoiding the nasty surprise of a dud binpkg if you end up needing it
later).

This way, eventually, you'll have several back-versions of each package
available in binary form, so if a new version of a package fails, you can
simply emerge --packageonly your previous, known working, version, without
having to go thru all the trouble recompiling what was just on your system
before you merged the broken version!

Among other things, this protects you from a gcc failure, as it'll be a
simple matter to just remerge your last working gcc from the binary
package, which won't require a working gcc to compile since it's a
precompiled binary package!

If portage itself (or python, which portage needs to function) breaks,
again, no problem, because altho you can't use the broken portage to
remerge itself, the binary package is actually just a tbz2 (tar.bz2)
tarball including the files themselves, with that bit of portage metadata
glued onto the end.  Thus, (after backing up any config files you want to
keep), simply extract the tarball over root, and it'll deploy a working
portage once again.  At that point, the portage database will have
incorrect information about what version of portage (or python) itself is
merged, but since portage is now working again, you'll be able to emerge
--packageonly the package over itself, to get the portage database back in
line with reality again, and clean up the loose ends.

If tar or bzip2 break, you'll be unable to remerge them directly, using
this technique, because portage needs them to extract the binary package. 
However, again, that's not an insurmountable problem.  There are in fact
two solutions.  First, both those packages are comparatively easy to
remerge the conventional way, from source, if necessary.  Thus, you can
do FEATURES=-buildpkg emerge bzip2 (or tar), to remerge without having
portage try to use the broken binary packaging ability.  Alternatively,
you can copy working copies of the appropriate executable from your rescue
partition or the LiveCD, over top of the borked copies, then use emerge -K
to remerge your last previously working version.

FEATURES=buildpkg comes in handy for other things as well, particularly
for troubleshooting.  Since I have a backup copy saved, if something quits
working, it's easy for me to temporarily remerge and old version to check
if it worked with it or if something else (possibly a library or other
dependency) caused the breakage.

***  VERY HANDY within the context of broken config files, since the
binary packages save the default configs, if I want to see how my existing
config compares to the default config, all I have to do is grab the
default config out of the binary package and compare it with my existing
version.

Likewise, I can quickly compare scripts and default configs between
versions, and compare files existing in the various versions with what's
on my live system.

Space required to store all those binary packages?  1-5 gigs, depending on
how often you want to clean out all the old binary packages, how many
packages you have installed, and whether you run amd64 or ~amd64 (the
latter updating more frequently, resulting in more package versions to
store).  I have my $PKGDIR on its own dedicated partition, 1 gig, but
running ~amd64, have to clean out old versions more frequently than I'd
like.  A 5 gig partition would have allowed me to do it only once a year
or so.

So, what if you like this idea, but haven't been doing it, and want to
jumpstart your binary package collection with the packages you have
currently merged?  That's where quickpkg comes in.  quickpkg allows you to
make binary packages based on what's on your system at the moment, thus,
avoiding having to recompile to get the binary package.  Use "equery
list" to get a list of what's currently merged.  The list it returns isn't
really formatted as we want, however, but with a bit of text processing
wizardry...

equery list|cut -d" " -f2|grep -v ^\*$ > package.list

(The cut command cuts out the second field, -f2, with fields delineated by
a space -d" ".  The grep command is there to remove what otherwise ends up
as the first line, after the cut, a single "*".  The result is redirected
to the file package.list.)

That gives you a list of packages.  You may want to open it in an editor
and check it out, or split it into several smaller lists, to process one
at a time.  When you are satisfied, try this (substituting the filenames
for the smaller lists if appropriate):

for pkg in `cat package.list` ; do quickpkg $pkg; done

(This simply creates a bash for loop, iterating over all the lines in the
cat-ed file, executing quickpkg for each line.)

quickpkg spits out info about what is doing, and will deal with slotted
packages as needed, quickpkg-ing all merged versions, so that should do
it.  Keep in mind, however, that it's still a big job, even if it's not
having to actually compile all those packages, so it'll take some time,
which is why I suggested splitting up the list, above.  Of course, you
could simply do it while you were sleeping or away at work or something,
if desired.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman in
http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html


-- 
gentoo-amd64@gentoo.org mailing list