[gentoo-dev] kernelmod-rebuild-0.1

[gentoo-dev] kernelmod-rebuild-0.1

[Paul Varner]

[Also posted to gentoo-user and forum]

In order to scratch an itch that I've been having with either a new
kernel install or a recompilation of a kernel with a new config and
forgetting packages that installed kernel modules, I've created
kernelmod-rebuild to assist in the process of rebuilding packages.

This script is based on revdep-rebuild (i.e. I started with it and
changed it to do what I want) but instead of looking for broken library
dependencies, it looks for packages that have installed kernel modules.

What this script does is locate the packages that have installed modules
into /lib/modules, determines which ebuilds to rebuild based on that
list and rebuilds them.

What this script doesn't do is locate the packages such as vmware that
installed modules outside of the emerge process (since portage is
unaware of those modules). Unfortunately, you will still have to
remember to rerun the configuration process for those packages.

Here is the usage statement from the script

garath root # kernelmod-rebuild --help
Usage: /usr/bin/kernelmod-rebuild [OPTIONS] [--] [EMERGE_OPTIONS]

Rebuild packages that have installed kernel modules

  -X, --package-names  recompile based on package names, not exact versions
  -q, --quiet          be less verbose

Calls emerge, all other options are used for it (e. g. -p, --pretend).
Here is a sample run of the script on my system:

garath root # kernelmod-rebuild --package-names --quiet --pretend --verbose

Finding packages which install kernel modules...
Packages will be recompiled.

Collecting list of packages to rebuild... done.

Cleaning list of packages to rebuild... done.

Evaluating package order... done.

All prepared. Starting rebuild...

These are the packages that I would merge, in order:

Calculating dependencies ...done!
[ebuild   R   ] media-sound/alsa-driver-0.9.8  -oss
[ebuild   R   ] media-video/nvidia-kernel-1.0.4496-r3

Now you can remove -p (or --pretend) from arguments and re-run kernelmod-rebuild.
This script is useful for when you decide to try out the latest
test-kernel, but can't remember all of the packages that you've
installed that build modules.  It is also useful when you make changes
to your kernel config and recompiled that kernel (as that can
occasionally break modules depending upon what you changed).

The ebuild for this script is located at
http://varnerfamily.org/pvarner/gentoo/kernelmod-rebuild-0.1.ebuild.tar.gz

To install the ebuild, download the tar file and extract it to your
portage overlay directory.  For example, I have set the variable
PORTDIR_OVERLAY in my make.conf file to /usr/local/portage.  Here is how
I installed the package on my machine: 

garath root # cd /usr/local/portage
garath portage # wget http://varnerfamily.org/pvarner/gentoo/kernelmod-rebuild-0.1.ebuild.tar.gz
garath portage # tar xvzf kernelmod-rebuild-0.1.ebuild.tar.gz
garath portage # rm kernelmod-rebuild-0.1.ebuild.tar.gz
garath portage # emerge -pv kernelmod-rebuild
garath portage # emerge -v kernelmod-rebuild

If you are on a different architecture than x86, you either need to edit
the ebuild or set ACCEPT_KEYWORDS="~arch" as I have marked those
architectures unstable/testing since I don't have machines to test on. 
It should work properly, but I couldn't test it :)

If you want to report a bug, either reply to this thread or send me an
email to the address listed in the man page.

Finally, I have submitted the ebuild to bugzilla (bug# 34921)
<http://bugs.gentoo.org/show_bug.cgi?id=34921>  with the request that
the script be included in the gentoolkit package.  Hopefully, it will be
accepted as part of the toolkit, but even if it isn't, I hope that some
of you can find it useful.

Regards,
Paul

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] kernelmod-rebuild-0.1

[Philippe Lafoucrière]

Great idea, and great work !

Please gentoo gurus, add this tool to gentoolkit !

[+1]


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] kernelmod-rebuild-0.1

[William Kenworthy]

Something I couldnt see from from your explanation: does it leave existing
modules in place if you change your kernel?  If it does it will be the
answer to my prayers as I am currently building kernels ...

BillK

> [Also posted to gentoo-user and forum]
>
> In order to scratch an itch that I've been having with either a new
> kernel install or a recompilation of a kernel with a new config and
> forgetting packages that installed kernel modules, I've created
> kernelmod-rebuild to assist in the process of rebuilding packages.
>

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] kernelmod-rebuild-0.1

[Paul Varner]

On Wed, 2003-12-03 at 01:29, William Kenworthy wrote:
> Something I couldnt see from from your explanation: does it leave existing
> modules in place if you change your kernel?  If it does it will be the
> answer to my prayers as I am currently building kernels ...
> 

Based upon bug #1477, if you are using portage 2.0.49-r18 your current
modules will stay in place.  However, I did notice that version is not
marked stable yet, so you might not be using it.

Regards,
Paul

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] kernelmod-rebuild-0.1

[Daniel Robbins]

[-- Attachment #1: Type: text/plain, Size: 361 bytes --]

On Wed, 2003-12-03 at 00:29, William Kenworthy wrote:
> Something I couldnt see from from your explanation: does it leave existing
> modules in place if you change your kernel?  If it does it will be the
> answer to my prayers as I am currently building kernels ...

The latest Portage (-r18) should leave existing modules in place.

Regards,

Daniel

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] kernelmod-rebuild-0.1

[Paul Varner]

Daniel Robbins wrote:
>> Something I couldnt see from from your explanation: does it leave
existing
>> modules in place if you change your kernel?  If it does it will be the
>> answer to my prayers as I am currently building kernels ...
>
> The latest Portage (-r18) should leave existing modules in place.

As I discovered earlier today, the latest 2.0.49-r18 in the portage tree
does not contain the fixed portage.py file to prevent existing modules from
being removed.

Until the fixes make it into a portage a workaround is to set the
CONFIG_PROTECT variable on the command line where you run the emerge
command.  For example, while I was testing this I had gentoo-test-r1 as my
main kernel.  For the testing, I installed gentoo-sources-r9.  After I
changed /usr/src/linux to point to /usr/src/linux-2.4.20-gentoo-r9, I ran
kernelmod-rebuild with the following command:

env CONFIG_PROTECT="/lib/modules/2.4.22-gentoo-test-r1"
kernelmod-rebuild --quiet -- --verbose

This resulted in my third-party kernel modules being rebuilt for the
gentoo-sources kernel and left the modules for the gentoo-test kernel in
place.  I would not recommend setting this in the make.conf file as when you
reinstall one of the packages for a kernel where you have already installled
the package, you typically want the old modules to be overwritten.

Regards,
Paul



--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] kernelmod-rebuild-0.1

[Markus Nigbur]

[-- Attachment #1: Type: text/plain, Size: 2300 bytes --]

On Wed, 03 Dec 2003 00:42:54 -0600
Paul Varner <gentoo-dev@varnerfamily.org> wrote:

> [Also posted to gentoo-user and forum]
> 
> In order to scratch an itch that I've been having with either a new
> kernel install or a recompilation of a kernel with a new config and
> forgetting packages that installed kernel modules, I've created
> kernelmod-rebuild to assist in the process of rebuilding packages.
> 
> This script is based on revdep-rebuild (i.e. I started with it and
> changed it to do what I want) but instead of looking for broken
> library dependencies, it looks for packages that have installed kernel
> modules.
> 
> What this script does is locate the packages that have installed
> modules into /lib/modules, determines which ebuilds to rebuild based
> on that list and rebuilds them.
> 
> What this script doesn't do is locate the packages such as vmware that
> installed modules outside of the emerge process (since portage is
> unaware of those modules). Unfortunately, you will still have to
> remember to rerun the configuration process for those packages.
> 
> Here is the usage statement from the script
> 
> garath root # kernelmod-rebuild --help
> Usage: /usr/bin/kernelmod-rebuild [OPTIONS] [--] [EMERGE_OPTIONS]
> 
> Rebuild packages that have installed kernel modules
> 
>   -X, --package-names  recompile based on package names, not exact
>   versions-q, --quiet          be less verbose
> 
> Calls emerge, all other options are used for it (e. g. -p, --pretend).
> Here is a sample run of the script on my system:
> 
> garath root # kernelmod-rebuild --package-names --quiet --pretend
> --verbose
> 
> Finding packages which install kernel modules...
> Packages will be recompiled.
> 
> Collecting list of packages to rebuild... done.
> 
> Cleaning list of packages to rebuild... done.
> 
> Evaluating package order... done.
> 
> All prepared. Starting rebuild...
> 
> These are the packages that I would merge, in order:
> 
> Calculating dependencies ...done!
> [ebuild   R   ] media-sound/alsa-driver-0.9.8  -oss
> [ebuild   R   ] media-video/nvidia-kernel-1.0.4496-r3

mhmm..

pyrania@powerix pyrania $ qpkg -f /lib/modules
media-libs/svgalib *
media-video/nvidia-kernel *
sys-apps/i2c *
x11-base/xfree-drm *

-- 
Markus Nigbur
Gentoo Developer
http://www.gentoo.org

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] kernelmod-rebuild-0.1

[Paul Varner]

Markus Nigbur wrote:
>> Calculating dependencies ...done!
>> [ebuild   R   ] media-sound/alsa-driver-0.9.8  -oss
>> [ebuild   R   ] media-video/nvidia-kernel-1.0.4496-r3
>
> mhmm..
>
> pyrania@powerix pyrania $ qpkg -f /lib/modules
> media-libs/svgalib *
> media-video/nvidia-kernel *
> sys-apps/i2c *
> x11-base/xfree-drm *

I am aware of that and the first comment in the script says that it evolved
from the simple script 'emerge -v `qpkg -I -nc -f /lib/modules`'

The main audience for this script isn't developers and experienced gentoo
administrators.  It is intended for use by your average gentoo
administrator.  What prompted me to write it was seeing people on the
gentoo-user mailing list having problems, because they forgot to rebuild
packages after recompiling a kernel.  Finally, since I have been testing it
and looking at the behavior of the various packages during a kernel
recompile, I find it to be very useful as it does the work for you of
locating and building those packages for you and thus saves you time and
effort.  For me, one of the purposes of using computers is the ability to
automate repetitious or error-prone tasks.  This is one of those that I have
found and seen to be error-prone and which to me benefits from being
automated.

Regards,
Paul



--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] kernelmod-rebuild-0.1

[Peter Ruskin]

On Thursday 04 Dec 2003 17:15, Paul Varner wrote:
> I am aware of that and the first comment in the script says that it
> evolved from the simple script 'emerge -v `qpkg -I -nc -f
> /lib/modules`'

Here's my kernel build script that includes this:

##################################################
#!/bin/sh
# /usr/local/bin/build-kernel

# start this script from the kernel source directory,
# for example /usr/src/linux-2.4.20-xfs-r2
# ...test for that...
grep -q "Linux kernel" ./README
if [ "$(echo $?)" != "0" ] ; then
	echo "ERROR: You don't appear to be in a kernel source directory"
	echo
	exit 1
fi

# remove symlink
if [ "$( file ../linux | grep 'symbolic link' )" != "" ]; then
	rm -f ../linux
fi

# This kernel variable produces something like "-2.4.20-xfs-r2"
kernel=$(pwd | cut -dx -f2-9)

# make symlink
cd ..
ln -sf linux${kernel} linux
ln -sf . ../linux

cd linux${kernel}

# comment these 3 lines if you don't need mrproper
#cp .config ..
#make mrproper
#cp ../.config .

# check timestamp of .config
CONFTIME=$(ls -l .config | awk '{print $6,$7,$8}')

# use old if you told it to, or preferably xconfig if it can, else
# menuconfig.
if [ "$1" == "old" ]; then
	make oldconfig
else
	make xconfig
	[ $(echo $?) -eq 0 ] || make menuconfig
fi

if [ "$(ls -l .config | awk '{print $6,$7}')" == "$CONFTIME" ] ; then
	echo
	echo "    .config file unchanged...aborting"
	exit 0
fi

# NOTE**  make dep not required for 2.6 kernels
PATCHLEVEL=$(grep 'PATCHLEVEL =' /usr/src/linux/Makefile | \
	cut -d= -f2 | cut -d' ' -f2)
if [ ${PATCHLEVEL} -gt 5 ]; then
	make clean
	if [ $(echo $?) -ne 0 ] ; then
		echo
		echo "    make clean failed"
		exit 1
	fi
else
	make dep clean
	if [ $(echo $?) -ne 0 ] ; then
		echo
		echo "    make dep clean failed"
		exit 1
	fi
fi

make bzImage
if [ $(echo $?) -ne 0 ] ; then
	echo
	echo "    make bzImage failed"
	exit 1
fi

make modules
if [ $(echo $?) -ne 0 ] ; then
	echo
	echo "    make modules failed"
	exit 1
fi

make modules_install
if [ $(echo $?) -ne 0 ] ; then
	echo
	echo "    make modules_install failed"
	exit 1
fi

[ "$(grep boot /etc/mtab)" == "" ] && mount /boot
cp arch/i386/boot/bzImage /boot/vmlinuz$kernel
cp System.map /boot/System.map$kernel
cp .config /boot/.config$kernel
sync

# if just rebuilding booted kernel...
if [[ "-$(uname -r)" == "$kernel" ]] ; then
	if [ ${PATCHLEVEL} -gt 5 ]; then
		env AUTOCLEAN="no" ACCEPT_KEYWORDS="~x86" \
emerge -v $(qpkg -I -nc -f /lib/modules | \
grep -v i2c | grep -v lm-sensors)
	else
		env AUTOCLEAN="no" ACCEPT_KEYWORDS="~x86" \
emerge -v $(qpkg -I -nc -f /lib/modules)
	fi
else
	echo
	# build-kernel-2 contains the first if clause above
	echo "Run build-kernel-2 after next reboot to"
	echo "update lm_sensors, nvidia-kernel, etc."
fi

echo
echo "    All done.  Now edit /boot/grub/grub.conf and reboot"
##################################################

Peter
-- 
======================================================================
Gentoo Linux:	Portage 2.0.49-r18 (default-x86-1.4, gcc-3.2.3, 
glibc-2.3.2-r3, 2.6.0-test11-gentoo-r1-win4lin)	i686 AMD Athlon(tm) XP 
3200+
======================================================================


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] kernelmod-rebuild-0.1

[Paul Varner]

All:

Since this thread was mentioned today in the Gentoo Weekly News, There
is definitely a demand for this type of script.  Anyhow I'm now working
on 0.2 based upon the feedback that I have received.  

Besides adding a couple of new options, I'm also looking at speeding up
the script where possible.  One of the things that I brought over from
revdep-rebuild was the calling of emerge to order the packages.  This
has the result of the script using the emerge three times.  Two times to
determine the order in which the packages should be recompiled and the
third time to actually recompile the packages.

The more that I look at this, the more that I think that it is
unnecessary work for what I am trying to do.  Can anyone think of why I
should care or not care about the order that I pass packages to emerge
when recompiling existing packages that contain kernel modules?  I can't
think of any, but I'm also not knowledgeable of the portage internals
yet.

Regards,
Paul

--
gentoo-dev@gentoo.org mailing list