[gentoo-amd64] Re: Outside of portage (was Re: Kernel-3.3.0 and Nvidia-drivers)

[Duncan <1i5t5.duncan@cox.net>]

Barry Schwartz posted on Thu, 22 Mar 2012 16:32:55 -0500 as excerpted:

> Frank Peters <frank.peters@comcast.net> skribis:
>> Out of habit, I still use the kernel sources from kernel.org.
> 
> This makes me curious what other people are using from outside of
> Portage. I install Grub outside of Portage, on the grounds that (a) it
> is a bad thing to upgrade unless you have to; and (b) the Grub 2 ebuilds
> were broken anyway, and I didn’t want to switch back to Grub 1 when
> moving from Exherbo back to Gentoo. So I used the default GNU install
> procedure in /usr/local. If the machine boots, the machine boots, and
> I’m satisfied.
> 
> (I also use only package.use and not make.conf for use flags, a habit
> picked up from using Paludis, and recently got rid of package.mask and
> use package.accepted_keywords for all my masking. Lots of redundancy in
> Portage these days, maybe a bit too much.)

FWIW, I use almost all ebuilds (but for the kernel), but beyond ~amd64/no-
multilib (and ~x86 on my netbook, with a 32-bit chroot build image on my 
main machine, with an rsync script to sync up), I use three overlays 
(kde, mozilla, x11) in addition to my own, unmasking stuff I care about 
that takes too long to show up in ~arch, and a number of -9999 live-git 
packages.

For the kernel, I've been running direct Linus-kernel live-git for quite 
some time now, using my own git-pull, git-whatchanged, make-oldconfig, 
build and install scripts, tho I don't normally update during the 2-weeks 
or so merge window between release and the following -rc1.  Sometimes I 
don't update until rc2 or rc3 if I'm really busy.

This works quite well, giving me a chance to see what's coming.  
Additionally, I get a chance to find, report, and generally get fixed, 
bugs, before they end up in a release kernel.  That's actually the most 
important bit and the reason I run a number of live-git packages.  Git-
bisect is pretty easy, and in a number of cases, I've found that the 
releases simply don't have detailed enough changelogs for me to properly 
trace bugs -- the git commit logs and occasionally the commit sources 
themselves are MUCH more informative, even tho I don't claim to really 
know C/C++ or the like (or even python/perl, but I do OK with bash =:^).

That's precisely why I run openrc-9999 as well.  I had a couple 
experiences where openrc had a regression, and there really isn't a 
proper changelog for it, at least that's easy to view before one actually 
does the install, to be prepared for what's going to be changing.

So eventually I just started running the live-9999 version of openrc, 
updating it perhaps twice a week or so.   I had already created a couple 
scripts to let me view git-whatchanged on the three (all git-based) 
overlays I run, and I expanded them a bit so I could check git-whatchanged 
with openrc-9999 and whatever other live-git based packages I run, as 
well.

Now, with updates a couple times a week, I not only check the whatchanged 
log before I actually do the install, but if anything looks interesting, 
I do a git-show on that specific commit, as well, to see the actual patch.

Then I'm prepared for the etc-updates afterward, and have an idea when/
what might go wrong on the first post-update reboot.  If something /does/ 
go wrong, I can either use init=bash on grub's commandline, or assemble 
md9 (my backup rootfs and /usr/local partitions) instead of md3 (my main 
rootfs and use root=/dev/md9p1 instead of the kernel built-in
root=/dev/md3p1, thus letting me boot, and either fix the problem, or 
emerge the previous binpkg to get back to the old version.

Then I can file the bug, and with openrc as with the kernel, I've 
reported and gotten fixed a number of bugs before they ever made it to a 
full release.  But what's really nice about openrc, unlike the kernel, is 
that enough of it is in shell scripts that I can actually propose patches 
some of the time, spottting and correcting the incorrectly used 
commandline option, or at least understanding and explaining why a change 
to the scripts won't work, even if I'm not sure how to actually make a 
working change that does what they were obviously trying to do.

The only other "regular" live/9999 version package I run is pan, the news 
client, which I use via gmane for following this list, BTW.  I've been 
interested in nntp clients for years, every since I ran the IE/OE4 betas 
back in the MSWindows 95 era, before MSWindows 98.  So when MS decided to 
do the eXPrivacy thing and I switched to Linux in late 2001, after I'd 
gotten settled on Linux and started looking around for a community 
project to contribute to, the nntp client I had chosen was a logical 
first-choice.  So I've been involved with pan upstream since 2002 or so, 
and have been running a live-vcs version since before gnome and with it 
pan, switched from svn to git.

Pan even lost its former primary developer, Charles Kerr, who moved on to 
other things, and the project looked dead for a few years.  But I and a 
couple others kept the lights on in the user list, which continued to 
function, and eventually we attracted one developer, than another and 
another, and now, pan has an active and healthy development community 
once again. =:^)

So of course I want to run pan's live-git/9999 version.  =:^)  There's 
one in the tree, but I run a slightly modified one here, kept in sync 
with upstream, altho I don't always know the proper way to setup the 
newer USE flags, so some of them only have the side I use setup and 
tested.  Plus, the newer version has features that aren't in a full 
release yet, altho there's another release due this spring/summer that 
should have most of them.

Other than that, it depends on what I'm interested in ATM.  Right now, 
I'm following btrfs, so have the -9999/git-live version of btrfs-progs 
installed, tho I'm not actually running a btrfs filesystem yet, as the 
feature I'm most interested in (multi-way mirroring, the feature they 
/call/ raid1 actually isn't, it's only two-way mirroring) isn't there 
yet... they say kernel 3.5-ish.

For a couple months recently I was running the live-git/9999 version of 
phonon-vlc, because it had some patches needed for either the kde 4.8 pre-
releases (which I ran, from the kde overlay... I've not tried full kde 
live tho, at least not since I gave up on it pre-4.0) or for gcc-4.6.x, 
which I unmasked and did a full system rebuild with, a couple months ago.

Awhile back, I was running most of the xorg and mesa stack from live 
packages, because that's where the support for my radeon hd4650 graphics 
card was.  I've not run the xorg/mesa stack live since that support was 
released, but I do sometimes run a couple live-git/9999 xorg packages, 
which are sometimes required to run the latest xorg-server release, or 
because I want to test a new feature in the kernel or mesa, and need a 
live xf86-video-ati to do it, which pulls in a live something else, which 
pulls in two or three other live xorg-releated packages.


You mentioned grub2.  I unmasked and ran the 1.99 version in the tree, 
and have been running the 2.00_betaX versions as well.  Aside from a bit 
of trouble with the initial 1.99 install, and a 2.00_beta1 that would 
boot to the grub menu, but would reboot every time I tried to actually 
boot a kernel, it's been fine.  However, I can mention that while my 
system's legacy-BIOS, I upgraded to GPT partitioning a couple years ago, 
and had the foresight to setup a BIOS-reserved partition on each of my 
four drives.  I've actually been quite happy indeed with how grub2 works 
with that. =:^)

I also run multiple md/raid devices, mostly 4-way RAID-1, and of course, 
grub2 works WAY better with md/raid than did grub-legacy.  But, while 
with most of the system I have a working and backup raid of the same 
size, both 4-way, that wouldn't work for /boot and be able to choose one 
or the other.  So what I did for /boot is split the 4-way in half, into 
two two-way mds, for /boot and the backup /boot.  That sort-of worked OK 
for grub1, getting the job done but managing it was quite complicated.  
By contrast, grub2 makes the boot and backup-boot md/raid management a 
BREEZE! =:^)

But the four drives, two as my main /boot, and the other two as a backup 
/boot, makes upgrading grub a very nearly risk-free exercise, especially 
with GPT and a dedicated BIOS partition for grub2 to use as well. =:^)  I 
have the following set in /etc/portage/env/sys-boot/grub:

# To keep grub from trying to mount /boot,
# on a normally deactivated md.
DONT_MOUNT_BOOT=1

# Just to be safe, since I handle this myself
INSTALL_MASK="$INSTALL_MASK /etc/default /etc/grub.d grub2-mkconfig"
PKG_INSTALL_MASK="$INSTALL_MASK"


I don't know what the package would do if I didn't set the dont-mount 
var, as it can't mount it anyway, since that md/raid device is normally 
not active/assembled.  But with that, it won't even try it.

That means the grub ebuild simply merges it to /usr/* as it normally 
would, and leaves /boot alone.

Then I activate the main boot md and mount it, and run grub2-install 
/dev/sda (the first of the four drives, normally set in BIOS to boot) 
myself.

Then I reboot.  If it doesn't work, I can set the BIOS to boot from sdc 
or sdd instead, where the backup boot is located, with a grub that's 
still the older version.  I can then either fix the issue or revert to 
the older grub, from the backup boot.

When I'm satisfied that the first install is working correctly, then I 
assemble the raid and mount the main /boot again, and run grub2-install 
/dev/sdb (the second spindle on the main /boot).  Then I unmount/stop it 
and assemble and mount the boot backup md/raid, and run grub2-install 
for /dev/sdc and /dev/sdd.

As you can see from the above INSTALL_MASKs I handle the configuration 
manually.  grub2-mkconfig is all but worthless on my system, taking 
longer to install a simple config file than the kernel does to BUILD, or 
at least it did when I tried it back on grub-1.99.  When I profiled/
traced the problem thru the script, it was due to about 50 calls at ~10 
seconds each to grub-probe!

So before I test the backup boot and new grub install to it, I copy over 
any grubenv or *.cfg changes from the main boot to the backup boot.

Then I can reboot again, setting the BIOS to boot from the third or 
fourth drive, to test that.  Once that's tested, I reboot once more and 
set the BIOS back to booting from the first drive and main /boot.


So as you can see, upgrading grub isn't any more dangerous for me than 
upgrading any other package on the system, especially something like 
openrc, which as I said, I run the live version of and generally upgrade 
a couple times a week.  Grub does happen to be a bit more work to 
upgrade, since unlike the rest of the system, at least at this point in 
grub2's life, I upgrade on both the main and backup together, tho only 
after testing main.  For other packages, I only upgrade the backup once 
every few months, after I've rebooted since the last upgrade thereby 
testing that, and everything, all the daemons and bootscripts, xorg/mesa, 
kde, etc, seems to be running normally, preferably when I'm running a 
full kde release version, not a prerelease, which I'm now doing about two 
months out of every six.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman