public inbox for gentoo-dev@lists.gentoo.org
 help / color / mirror / Atom feed
* [gentoo-dev] Lib_users, a post-upgrade helper
@ 2010-07-28  9:38 Tobias Klausmann
  2010-07-28 12:53 ` Jeroen Roovers
                   ` (2 more replies)
  0 siblings, 3 replies; 9+ messages in thread
From: Tobias Klausmann @ 2010-07-28  9:38 UTC (permalink / raw
  To: gentoo-dev

Hi! 

Short version: Upgrades happen, libs get deleted/replaced; my
script (links below) tells you what processes to restart.

Long version:

I've recently realized that many admins aren't aware of a problem
when upgrading their systems. Usually, people are told to run
revdep-rebuild and do sundry other stuff. But sometimes, a
certain point is missed: deleted-but-mapped files. Yes, we all
know that upgrading say OpenSSL or gnutls means that we should
restart all the services that use those libraries. But sometimes,
we might miss some obscure daemon or a job running in a screen
(so looking at /etc/init.d doesn't help us).

Fortunately, all the relevant information can be found in /proc:
the per-process maps file lists files (and if they're deleted)
and the cmdline file gives us the name of the processes.

So it's a simple shell onliner, right? Yes, that can be done (and
please refrain from posting your solution to that particular
exercise). But Shell onliners tend to be forgotten, and robust
they ain't, especially if you add in stuff like not listing a
deleted shmem segment or similar stuff.

To make a long posting boring: I've hacked up a Python script
that does all that and prints out a nice summary. It's available
from here:

http://schwarzvogel.de/software-misc.shtml

Or, for those who want to go to the tgz directly:

http://schwarzvogel.de/pkgs/lib_users-0.1.tar.gz

Comments/suggestions appreciated.

Regards,
Tobias



^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [gentoo-dev] Lib_users, a post-upgrade helper
  2010-07-28  9:38 [gentoo-dev] Lib_users, a post-upgrade helper Tobias Klausmann
@ 2010-07-28 12:53 ` Jeroen Roovers
  2010-07-28 13:08   ` Jeroen Roovers
  2010-07-28 13:23   ` Tobias Klausmann
  2010-07-28 15:01 ` "Paweł Hajdan, Jr."
  2010-07-28 20:14 ` Tobias Klausmann
  2 siblings, 2 replies; 9+ messages in thread
From: Jeroen Roovers @ 2010-07-28 12:53 UTC (permalink / raw
  To: gentoo-dev

On Wed, 28 Jul 2010 11:38:12 +0200
Tobias Klausmann <klausman@gentoo.org> wrote:

> To make a long posting boring: I've hacked up a Python script
> that does all that and prints out a nice summary. It's available
> from here:
> 
> http://schwarzvogel.de/software-misc.shtml
> 
> Or, for those who want to go to the tgz directly:
> 
> http://schwarzvogel.de/pkgs/lib_users-0.1.tar.gz

Is it like checkrestart[1], originally from debian-goodies but hacked
to work on Gentoo Linux?


Cheers,
     jer


[1] app-admin/checkrestart



^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [gentoo-dev] Lib_users, a post-upgrade helper
  2010-07-28 12:53 ` Jeroen Roovers
@ 2010-07-28 13:08   ` Jeroen Roovers
  2010-07-28 13:23   ` Tobias Klausmann
  1 sibling, 0 replies; 9+ messages in thread
From: Jeroen Roovers @ 2010-07-28 13:08 UTC (permalink / raw
  To: gentoo-dev

[-- Attachment #1: Type: text/plain, Size: 1614 bytes --]

On Wed, 28 Jul 2010 14:53:01 +0200
Jeroen Roovers <jer@gentoo.org> wrote:

> On Wed, 28 Jul 2010 11:38:12 +0200
> Tobias Klausmann <klausman@gentoo.org> wrote:
> 
> > To make a long posting boring: I've hacked up a Python script
> > that does all that and prints out a nice summary. It's available
> > from here:
> > 
> > http://schwarzvogel.de/software-misc.shtml
> > 
> > Or, for those who want to go to the tgz directly:
> > 
> > http://schwarzvogel.de/pkgs/lib_users-0.1.tar.gz
> 
> Is it like checkrestart[1], originally from debian-goodies but hacked
> to work on Gentoo Linux?

I've tried now and at first I thought python was complaining, then I
discovered that this was the actual intended output. ;-)

The problem I've always had with checkrestart is that it's too neat
(I'd like to see a list of paths to files for each process, for
instance), too much Debian (no verbose mode, discarding instead of
presenting information it can't deal with) and not enough Gentoo. I've
looked into finding time to discover in which ways I could add the
necessary Gentoo-like bells and whistles, but that little project
hasn't moved forward in a good while.

In short, checkrestart's user interface could do with some extras, and
in your lib_users "hack" all of a sudden we need not rely on lsof
anymore (and yes, app-admin/checkrestart could easily drop the
lsb_release DEPEND since we already know which OS we're on - before
anyone reminds me).

Maybe we should put checkrestart as it is right now into a Gentoo
repository and start extending and improving it based on your
experience hacking up lib_users?


     jer

[-- Attachment #2: checkrestart-vs-lib_users --]
[-- Type: application/octet-stream, Size: 4335 bytes --]

Found 31 processes using old versions of upgraded files
(14 distinct programs)
(11 distinct packages)

Of these, 3 seem to contain init scripts which can be used to restart them:
The following packages seem to have init scripts that could be used
to restart them:
net-misc/openssh:
	545	/usr/sbin/sshd
	549	/usr/sbin/sshd
	4439	/usr/sbin/sshd
	2695	/usr/sbin/sshd
	2691	/usr/sbin/sshd
net-fs/nfs-utils:
	4257	/usr/sbin/rpc.mountd
sys-process/at:
	25152	/usr/sbin/atd

These are the init scripts:
/etc/init.d/sshd restart
/etc/init.d/nfs restart
/etc/init.d/nfsmount restart
/etc/init.d/rpc.statd restart
/etc/init.d/rpc.idmapd restart
/etc/init.d/rpc.pipefs restart
/etc/init.d/atd restart
/etc/init.d/atop restart

These processes do not seem to have an associated init script to restart them:
app-cdr/graveman:
	29794	/usr/bin/graveman
www-client/opera:
	26169	/usr/lib/opera/opera
	32032	/usr/lib/opera/operapluginwrapper
	26188	/usr/lib/opera/operapluginwrapper
app-shells/bash:
	2700	/bin/bash
	8878	/bin/bash
	577	/bin/bash
	8823	/bin/bash
	8824	/bin/bash
	556	/bin/bash
x11-misc/slim:
	15225	/usr/bin/slim
sys-process/procps:
	8822	/usr/bin/top
x11-base/xorg-server:
	15232	/usr/bin/Xorg
sys-apps/shadow:
	574	/bin/su
app-misc/screen:
	2719	/usr/bin/screen
	7806	/usr/bin/screen
	593	/usr/bin/screen
	8820	/usr/bin/screen
(545) "sshd: jeroen [priv]" uses: /var/tmp/portage/sys-libs/cracklib-2.8.16/image/lib/libcrack.so.2.8.1 (+14 more)
(549) "sshd: jeroen@pts/4" uses: /var/tmp/portage/sys-libs/cracklib-2.8.16/image/lib/libcrack.so.2.8.1 (+14 more)
(556) "-bash " uses: /usr/lib/gconv/gconv-modules.cache
(574) "su - " uses: /var/tmp/portage/sys-libs/cracklib-2.8.16/image/lib/libcrack.so.2.8.1 (+12 more)
(577) "-su " uses: /usr/lib/gconv/gconv-modules.cache
(593) "screen -rd " uses: /lib/libpam.so.0.82.1
(2691) "sshd: jeroen [priv]" uses: /var/tmp/portage/sys-libs/cracklib-2.8.16/image/lib/libcrack.so.2.8.1 (+14 more)
(2695) "sshd: jeroen@pts/0" uses: /var/tmp/portage/sys-libs/cracklib-2.8.16/image/lib/libcrack.so.2.8.1 (+14 more)
(2700) "-bash " uses: /usr/lib/gconv/gconv-modules.cache
(2719) "screen -rd " uses: /lib/libpam.so.0.82.1
(4257) "/usr/sbin/rpc.mountd " uses: /var/tmp/portage/sys-apps/util-linux-2.17.2/image/lib/libuuid.so.1.3.0 (+1 more)
(4439) "/usr/sbin/sshd " uses: /lib/libpam.so.0.82.1
(4707) "/sbin/agetty 38400 tty2 linux " uses: /var/tmp/portage/sys-apps/util-linux-2.17.2/image/sbin/agetty
(4708) "/sbin/agetty 38400 tty3 linux " uses: /var/tmp/portage/sys-apps/util-linux-2.17.2/image/sbin/agetty
(4709) "/sbin/agetty 38400 tty4 linux " uses: /var/tmp/portage/sys-apps/util-linux-2.17.2/image/sbin/agetty
(4710) "/sbin/agetty 38400 tty5 linux " uses: /var/tmp/portage/sys-apps/util-linux-2.17.2/image/sbin/agetty
(4711) "/sbin/agetty 38400 tty6 linux " uses: /var/tmp/portage/sys-apps/util-linux-2.17.2/image/sbin/agetty
(7806) "SCREEN " uses: /lib/libpam.so.0.82.1 (+2 more)
(8820) "SCREEN " uses: /lib/libpam.so.0.82.1 (+2 more)
(8821) "su - jeroen " uses: /var/tmp/portage/sys-apps/shadow-4.1.4.2-r3/image/bin/su (+13 more)
(8822) "top " uses: /var/tmp/portage/sys-libs/ncurses-5.7-r4/image/lib/libncurses.so.5.7
(8823) "-/bin/bash " uses: /var/tmp/portage/sys-libs/ncurses-5.7-r4/image/lib/libncurses.so.5.7 (+1 more)
(8824) "-su " uses: /var/tmp/portage/sys-libs/ncurses-5.7-r4/image/lib/libncurses.so.5.7 (+1 more)
(8878) "-/bin/bash " uses: /var/tmp/portage/sys-libs/ncurses-5.7-r4/image/lib/libncurses.so.5.7 (+1 more)
(15225) "/usr/bin/slim " uses: /var/tmp/portage/sys-libs/pam-1.1.1-r2/image/lib/security/pam_deny.so (+7 more)
(15232) "/usr/bin/X -nolisten tcp -br -deferglyphs 16 -auth /var/run/slim.auth vt07 " uses: /var/tmp/portage/x11-drivers/nvidia-drivers-96.43.18/image/usr/lib/xorg/modules/drivers/nvidia_drv.so (+3 more)
(25152) "/usr/sbin/atd " uses: /lib/libpam.so.0.82.1
(26169) "/usr/lib/opera/opera " uses: /usr/lib/opengl/nvidia/lib/libGLcore.so.96.43.18 (+4 more)
(26188) "/usr/lib/opera//operapluginwrapper 96 99 /opt/Adobe/flash-player/libflashplayer.so " uses: /usr/lib/opengl/nvidia/lib/libnvidia-tls.so.96.43.18 (+4 more)
(29794) "graveman " uses: /usr/share/mime/mime.cache (+5 more)
(32032) "/usr/lib/opera//operapluginwrapper 64 80 /usr/lib/nsbrowser/plugins/gecko-mediaplayer-wmp.so " uses: /usr/lib/opengl/nvidia/lib/libnvidia-tls.so.96.43.18 (+4 more)

[-- Attachment #3: lib_users-0.1.ebuild --]
[-- Type: application/octet-stream, Size: 542 bytes --]

# Copyright 1999-2010 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: $

EAPI=3

DESCRIPTION="goes through /proc/*/maps and finds all cases of libraries being
mapped but marked as deleted"
HOMEPAGE="http://schwarzvogel.de/software-misc.shtml"
SRC_URI="http://schwarzvogel.de/pkgs/lib_users-0.1.tar.gz"

LICENSE="GPL-2"
SLOT="0"
KEYWORDS="~x86"
IUSE=""

DEPEND=""
RDEPEND="${DEPEND}"

src_test() {
	python test_lib_users.py
}

src_install() {
	newbin lib_users.py lib_users
	dodoc README TODO
}

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [gentoo-dev] Lib_users, a post-upgrade helper
  2010-07-28 12:53 ` Jeroen Roovers
  2010-07-28 13:08   ` Jeroen Roovers
@ 2010-07-28 13:23   ` Tobias Klausmann
  2010-07-28 15:23     ` Jeroen Roovers
  1 sibling, 1 reply; 9+ messages in thread
From: Tobias Klausmann @ 2010-07-28 13:23 UTC (permalink / raw
  To: gentoo-dev

Hi! 

On Wed, 28 Jul 2010, Jeroen Roovers wrote:
> > To make a long posting boring: I've hacked up a Python script
> > that does all that and prints out a nice summary. It's available
> > from here:
> > 
> > http://schwarzvogel.de/software-misc.shtml
> 
> Is it like checkrestart[1], originally from debian-goodies but hacked
> to work on Gentoo Linux?

(Note: this is stream-of-conciousness, so it might be a bit
rambling).

It is very similar in scope, but the implementations differ a
lot. While lib_users has no external dependencies aside from a
Python interpreter, checkrestart uses lsof. While that approach
has the advantage of not only catching mapped files but every
open FD, I am thinking about implementing something similar with
lib_users (but using /proc/<pid>/fd/). Can someone chime in on
the question of dlopen()ed libs and whether they turn up as mapped
files?

One annoying thing about checkrestart is that it refuses to work
for non-root users:
$ checkrestart
This program must be run as root
in order to collect information about all open file descriptors
$

This is a /bug/ - some users might just want to know what /they/
have to restart. 

Checkrestart doesn't have a --help function (it also has no
options at all, but at least displaying version info would've
been nice). 

For a Debian package, manpage etc. are also suspiciously absent. 

Checkrestart tries to tell you which init scripts are relevant. I
think that is neither robust nor desirable but a case of
over-engineering.

Checkrestart does not handle the interpreter problem at all,
while lib_users at least tries to give you enough information to
find out what to do.

I've also taken a look at the source code of checkrestart and
it's obvious it has been hacked to work on Gentoo. Also, it has
several pieces of dead code (checking for lsof output that isn't
ever generated, for example). Is the maintainer of the hacks
still around and doing updates? Its last release was in 2008. How
about upstream for the original checkrestart?

Or, to put it in cold, dead numbers:

$ grep -cv ^# $(which checkrestart)|wc -l
399
$ grep -cv ^# lib_users
106

I'll maintain lib_users anyway since I use it on my own systems,
so it won't go away if nobody is interested in it but me.

Regards,
Tobias




^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [gentoo-dev] Lib_users, a post-upgrade helper
  2010-07-28  9:38 [gentoo-dev] Lib_users, a post-upgrade helper Tobias Klausmann
  2010-07-28 12:53 ` Jeroen Roovers
@ 2010-07-28 15:01 ` "Paweł Hajdan, Jr."
  2010-07-28 15:57   ` Tobias Klausmann
  2010-07-28 20:14 ` Tobias Klausmann
  2 siblings, 1 reply; 9+ messages in thread
From: "Paweł Hajdan, Jr." @ 2010-07-28 15:01 UTC (permalink / raw
  To: gentoo-dev

[-- Attachment #1: Type: text/plain, Size: 400 bytes --]

On 7/28/10 2:38 AM, Tobias Klausmann wrote:
> To make a long posting boring: I've hacked up a Python script
> that does all that and prints out a nice summary.
> Comments/suggestions appreciated.

It would be interesting to integrated it with portage, similarly to how
it now displays "you have xx configuration files to update". It could
print "you have xx services to restart".

Paweł


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [gentoo-dev] Lib_users, a post-upgrade helper
  2010-07-28 13:23   ` Tobias Klausmann
@ 2010-07-28 15:23     ` Jeroen Roovers
  2010-07-28 15:55       ` Tobias Klausmann
  0 siblings, 1 reply; 9+ messages in thread
From: Jeroen Roovers @ 2010-07-28 15:23 UTC (permalink / raw
  To: gentoo-dev

On Wed, 28 Jul 2010 15:23:55 +0200
Tobias Klausmann <klausman@gentoo.org> wrote:

> While that approach has the advantage of not only catching mapped
> files but every open FD, I am thinking about implementing something
> similar with lib_users (but using /proc/<pid>/fd/).

The great advantage would be that it also discovers changed runtime
configuration files. Not that it would also disregard "trivial"
updates, of course, but by pointing out the filenames, an admin having
just run etc-update could easily figure that out.


     jer



^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [gentoo-dev] Lib_users, a post-upgrade helper
  2010-07-28 15:23     ` Jeroen Roovers
@ 2010-07-28 15:55       ` Tobias Klausmann
  0 siblings, 0 replies; 9+ messages in thread
From: Tobias Klausmann @ 2010-07-28 15:55 UTC (permalink / raw
  To: gentoo-dev

Hi! 

On Wed, 28 Jul 2010, Jeroen Roovers wrote:
> > While that approach has the advantage of not only catching mapped
> > files but every open FD, I am thinking about implementing something
> > similar with lib_users (but using /proc/<pid>/fd/).
> 
> The great advantage would be that it also discovers changed runtime
> configuration files. Not that it would also disregard "trivial"
> updates, of course, but by pointing out the filenames, an admin having
> just run etc-update could easily figure that out.

I suspect that wouldn't work in most cases since programs
typically close FDs after reading the config file(s)*. The one
program I know of that doesn't, uses the FD to find out the file
changed and does file-was-deleted handling internally.

Still, if config files aren't the only useful case for checking
the FDs, it might be worthwhile to implement.

Regards,
Tobias

* This is a /good/ thing.



^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [gentoo-dev] Lib_users, a post-upgrade helper
  2010-07-28 15:01 ` "Paweł Hajdan, Jr."
@ 2010-07-28 15:57   ` Tobias Klausmann
  0 siblings, 0 replies; 9+ messages in thread
From: Tobias Klausmann @ 2010-07-28 15:57 UTC (permalink / raw
  To: gentoo-dev

Hi! 

On Wed, 28 Jul 2010, "Paweł Hajdan, Jr." wrote:
> It would be interesting to integrated it with portage, similarly to how
> it now displays "you have xx configuration files to update". It could
> print "you have xx services to restart".

Definitely. One of the reasons why I expressly mention dual- or
multiple-licensing in the README is so that it's clear that
inclusion of the code (or adaption) is encouraged. I know very
little about portage internals (or any other package manager, for
that matter), so it's not high priority for me.

Regards,
Tobias



^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [gentoo-dev] Lib_users, a post-upgrade helper
  2010-07-28  9:38 [gentoo-dev] Lib_users, a post-upgrade helper Tobias Klausmann
  2010-07-28 12:53 ` Jeroen Roovers
  2010-07-28 15:01 ` "Paweł Hajdan, Jr."
@ 2010-07-28 20:14 ` Tobias Klausmann
  2 siblings, 0 replies; 9+ messages in thread
From: Tobias Klausmann @ 2010-07-28 20:14 UTC (permalink / raw
  To: gentoo-dev

Hi! 

And hot off the tar, here's v0.2:
http://www.schwarzvogel.de/software-misc.shtml
http://www.schwarzvogel.de/pkgs/lib_users-0.2.tar.gz

Several suggestions were implemented, including more Pythonic
code in a few spots and new patterns for the blacklist.

Please test and comment.

Regards,
Tobias



^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2010-07-28 20:14 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2010-07-28  9:38 [gentoo-dev] Lib_users, a post-upgrade helper Tobias Klausmann
2010-07-28 12:53 ` Jeroen Roovers
2010-07-28 13:08   ` Jeroen Roovers
2010-07-28 13:23   ` Tobias Klausmann
2010-07-28 15:23     ` Jeroen Roovers
2010-07-28 15:55       ` Tobias Klausmann
2010-07-28 15:01 ` "Paweł Hajdan, Jr."
2010-07-28 15:57   ` Tobias Klausmann
2010-07-28 20:14 ` Tobias Klausmann

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox