[gentoo-user] Finding orphaned libs

[gentoo-user] Finding orphaned libs

[Alan McKinnon]

Hi,

Is there an easy way to detect the orphaned libs on and old machine who's 
install dates back to 2004? The only idea I can come up with is 

for I in /usr/lib/*.so.* ; do equery belongs $I ; done

which strikes me as being a tad overkill...


-- 
alan dot mckinnon at gmail dot com

Re: [gentoo-user] Finding orphaned libs

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 587 bytes --]

On Mon, 8 Jun 2009 22:44:18 +0200, Alan McKinnon wrote:

> Is there an easy way to detect the orphaned libs on and old machine
> who's install dates back to 2004? The only idea I can come up with is 
> 
> for I in /usr/lib/*.so.* ; do equery belongs $I ; done

qfile --orphans /usr/lib/*.so.*

or, maybe cleaner

qfile --orphans $(find -H /usr/lib /lib -type f)

which avoids checking all the symlinks.

Then run symlinks remove any dangling links left over.


-- 
Neil Bothwick

It may be that your sole purpose in life is simply to serve as a
warning to others.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Finding orphaned libs

[Alan McKinnon]

On Monday 08 June 2009 22:54:08 Neil Bothwick wrote:
> On Mon, 8 Jun 2009 22:44:18 +0200, Alan McKinnon wrote:
> > Is there an easy way to detect the orphaned libs on and old machine
> > who's install dates back to 2004? The only idea I can come up with is
> >
> > for I in /usr/lib/*.so.* ; do equery belongs $I ; done
>
> qfile --orphans /usr/lib/*.so.*
>
> or, maybe cleaner
>
> qfile --orphans $(find -H /usr/lib /lib -type f)
>
> which avoids checking all the symlinks.
>
> Then run symlinks remove any dangling links left over.

Thanks, that's just the biscuit :-)

I really need to start using those q* commands. After 5 years on gentoo, I 
reckon it's about time...

-- 
alan dot mckinnon at gmail dot com

Re: [gentoo-user] Finding orphaned libs

[Dale]

Alan McKinnon wrote:
> On Monday 08 June 2009 22:54:08 Neil Bothwick wrote:
>   
>> On Mon, 8 Jun 2009 22:44:18 +0200, Alan McKinnon wrote:
>>     
>>> Is there an easy way to detect the orphaned libs on and old machine
>>> who's install dates back to 2004? The only idea I can come up with is
>>>
>>> for I in /usr/lib/*.so.* ; do equery belongs $I ; done
>>>       
>> qfile --orphans /usr/lib/*.so.*
>>
>> or, maybe cleaner
>>
>> qfile --orphans $(find -H /usr/lib /lib -type f)
>>
>> which avoids checking all the symlinks.
>>
>> Then run symlinks remove any dangling links left over.
>>     
>
> Thanks, that's just the biscuit :-)
>
> I really need to start using those q* commands. After 5 years on gentoo, I 
> reckon it's about time...
>
>   

Careful, it will end up like etcat and the rest.  You learn them and
start to use them, then they go away and you have to start over again.

Dale

:-)  :-) 

Re: [gentoo-user] Finding orphaned libs

[Willie Wong]

On Mon, Jun 08, 2009 at 09:54:08PM +0100, Penguin Lover Neil Bothwick squawked:
> On Mon, 8 Jun 2009 22:44:18 +0200, Alan McKinnon wrote:
> 
> > Is there an easy way to detect the orphaned libs on and old machine
> > who's install dates back to 2004? The only idea I can come up with is 
> > 
> > for I in /usr/lib/*.so.* ; do equery belongs $I ; done
> 
> qfile --orphans /usr/lib/*.so.*
> 
> or, maybe cleaner
> 
> qfile --orphans $(find -H /usr/lib /lib -type f)
> 
> which avoids checking all the symlinks.
> 
> Then run symlinks remove any dangling links left over.

Whoa, that's useful. Thanks for Neil for giving the solution, and to
Alan for even asking the question. Having a box whose install dates to 

sep~ # head /var/log/emerge.log
Started emerge on: Nov 03, 2002
 *** emerge  >=sys-apps/baselayout-1.7.9-r1 >=sys-apps/texinfo-4.2-r1 sys-devel/gettext >=sys-devel/binutils-2.13.90.0.4 >=sys-devel/gcc-3.2
>>> emerge (1 of 7) sys-apps/baselayout-1.8.4.1 to /
::: completed emerge (1 of 7) sys-apps/baselayout-1.8.4.1 to /
>>> emerge (2 of 7) sys-apps/texinfo-4.2-r5 to /

This seems to be a worthy enterprise. 

BTW, though, $(find -H ...) will have too big a list. Perhaps better
to do 

find -H /usr/lib /lib -type f | xargs qfile -o

W
-- 
When my cats aren't happy, I'm not happy. Not because I care about their mood 
but because I know they're just sitting there thinking up ways to get even.
    ~Penny Ward Moser
Sortir en Pantoufles: up 913 days, 22:12

Re: [gentoo-user] Finding orphaned libs

[Joerg Schilling]

Willie Wong <wwong@math.princeton.edu> wrote:

> > qfile --orphans $(find -H /usr/lib /lib -type f)
> > 
> > which avoids checking all the symlinks.

In case that you find implementation works correctly, this will
not change anything unless either /usr/lib or /lib are symlinks.

If you however use find -L, -type f will find even all symlinks in case they
are pointing to a file.

In negative logic: "find -L /usr/lib -type l" will find all symlinks
in /usr/lib that do not point anywhere.

 
> BTW, though, $(find -H ...) will have too big a list. Perhaps better
> to do 
>
> find -H /usr/lib /lib -type f | xargs qfile -o

This is why find has ".... -exec ... {} +" since 19 years.....

xargs will have problems in case that a file name contains spaces, tabs or 
newlines.
Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: [gentoo-user] Finding orphaned libs

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 952 bytes --]

On Tue, 09 Jun 2009 12:05:05 +0200, Joerg Schilling wrote:

> > > qfile --orphans $(find -H /usr/lib /lib -type f)
> > > 
> > > which avoids checking all the symlinks.  
> 
> In case that you find implementation works correctly, this will
> not change anything unless either /usr/lib or /lib are symlinks.
> 
> If you however use find -L, -type f will find even all symlinks in case
> they are pointing to a file.

The idea is not have find NOT follow symlinks, except when the given path
is a symlink (which it often is on a 64 bit machine).

Checking the symlinks is a waste of time, it's faster to remove only real
files that are orphaned, then remove any dangling symlinks after.
Otherwise qfile has to check about three times as many objects.


-- 
Neil Bothwick

Handy Guide to Modern Science:
   1. If it's green or it wiggles, it's biology.
   2. If it stinks, it's chemistry.
   3. If it doesn't work, it's physics.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Finding orphaned libs

[Dale]

Neil Bothwick wrote:
> On Mon, 8 Jun 2009 22:44:18 +0200, Alan McKinnon wrote:
>
>   
>> Is there an easy way to detect the orphaned libs on and old machine
>> who's install dates back to 2004? The only idea I can come up with is 
>>
>> for I in /usr/lib/*.so.* ; do equery belongs $I ; done
>>     
>
> qfile --orphans /usr/lib/*.so.*
>
> or, maybe cleaner
>
> qfile --orphans $(find -H /usr/lib /lib -type f)
>
> which avoids checking all the symlinks.
>
> Then run symlinks remove any dangling links left over.
>
>
>   

Questions:  You knew I subscribed to this list.  lol  I ran the command:

find -H /usr/lib /lib -type f | xargs qfile -o

and I got a lot of hits.  I'm just going to post snippets of the
directories here.  It is a LONG list otherwise.

/usr/lib/gimp/2.0/plug-ins/
/usr/lib/perl5/site_perl/5.8.8/i486-linux/sys/
/usr/lib/python2.5/site-packages/dbus/
/usr/lib/python2.5/site-packages/Numeric/
/usr/lib/python2.5/site-packages/PyQt4/uic/
/usr/lib/python2.5/site-packages/Pyrex/Plex/
/usr/lib/portage/pym/

Some of those directories have additional directories in them but I
tried to hit the highlights.  Now to my questions.  Can these be
deleted?  Should one make certain these doesn't belong to some package
somewhere?  Should I update my backups BEFORE deleting these?  o_O

If you want the whole list, let me know.  Be forewarned tho, it is a
huge list even if I leave out the kernel sources for my old trusty 2.6.23.

Thanks much.

Dale

:-)  :-) 

Re: [gentoo-user] Finding orphaned libs

[Willie Wong]

On Mon, Jun 08, 2009 at 06:52:23PM -0500, Penguin Lover Dale squawked:
> Questions:  You knew I subscribed to this list.  lol  I ran the command:
> 
> find -H /usr/lib /lib -type f | xargs qfile -o
> 
> and I got a lot of hits.  I'm just going to post snippets of the
> directories here.  It is a LONG list otherwise.
> 
> /usr/lib/gimp/2.0/plug-ins/
> /usr/lib/perl5/site_perl/5.8.8/i486-linux/sys/
> /usr/lib/python2.5/site-packages/dbus/
> /usr/lib/python2.5/site-packages/Numeric/
> /usr/lib/python2.5/site-packages/PyQt4/uic/
> /usr/lib/python2.5/site-packages/Pyrex/Plex/
> /usr/lib/portage/pym/
> 
> Some of those directories have additional directories in them but I
> tried to hit the highlights.  Now to my questions.  Can these be
> deleted?  Should one make certain these doesn't belong to some package
> somewhere?  Should I update my backups BEFORE deleting these?  o_O

Depends if you do have those packages installed. Numeric is from
dev-python/numeric, for example.

Also, be careful when you parse the output of the command. Most of the
.pyc and .pyo files in the python2.5 directories are byte-compiled
version that python generated dynamically the first time they are
used. For example: ..../Numeric/numeric_version.py was installed by
the ebuild and thuse qfile tells me it belongs to dev-python/numeric,
but .../Numeric/numeric_version.pyc is listed as an "orphan". While it
is safe to delete, it will just be regenerated again later, wasting
computing cycles. 

Ditto for some of the files in the perl directory. 

W
-- 
I am not a physics major. I am not a physics major. I will never be a physics 
major. I never will want to be a physics major. I can never be a physics 
major.                           ~Daniel Jonathan Peng
Sortir en Pantoufles: up 913 days, 22:59

Re: [gentoo-user] Finding orphaned libs

[Dale]

Willie Wong wrote:
> On Mon, Jun 08, 2009 at 06:52:23PM -0500, Penguin Lover Dale squawked:
>   
>> Questions:  You knew I subscribed to this list.  lol  I ran the command:
>>
>> find -H /usr/lib /lib -type f | xargs qfile -o
>>
>> and I got a lot of hits.  I'm just going to post snippets of the
>> directories here.  It is a LONG list otherwise.
>>
>> /usr/lib/gimp/2.0/plug-ins/
>> /usr/lib/perl5/site_perl/5.8.8/i486-linux/sys/
>> /usr/lib/python2.5/site-packages/dbus/
>> /usr/lib/python2.5/site-packages/Numeric/
>> /usr/lib/python2.5/site-packages/PyQt4/uic/
>> /usr/lib/python2.5/site-packages/Pyrex/Plex/
>> /usr/lib/portage/pym/
>>
>> Some of those directories have additional directories in them but I
>> tried to hit the highlights.  Now to my questions.  Can these be
>> deleted?  Should one make certain these doesn't belong to some package
>> somewhere?  Should I update my backups BEFORE deleting these?  o_O
>>     
>
> Depends if you do have those packages installed. Numeric is from
> dev-python/numeric, for example.
>
> Also, be careful when you parse the output of the command. Most of the
> .pyc and .pyo files in the python2.5 directories are byte-compiled
> version that python generated dynamically the first time they are
> used. For example: ..../Numeric/numeric_version.py was installed by
> the ebuild and thuse qfile tells me it belongs to dev-python/numeric,
> but .../Numeric/numeric_version.pyc is listed as an "orphan". While it
> is safe to delete, it will just be regenerated again later, wasting
> computing cycles. 
>
> Ditto for some of the files in the perl directory. 
>
> W
>   

That's sort of what I was thinking.  It was generated when it was
started up the first time.  I also noticed some things that I installed
in the Seamonkey directory too.  Adblock was one of those.

Tho there was a good many of them, I suspect most all of them belongs to
something here.  I may just leave well enough alone, may poke around a
little but nothing risky.

Thanks.

Dale

:-)  :-) 

Re: [gentoo-user] Finding orphaned libs

[Alan McKinnon]

On Tuesday 09 June 2009 02:27:39 Dale wrote:
> That's sort of what I was thinking.  It was generated when it was
> started up the first time.  I also noticed some things that I installed
> in the Seamonkey directory too.  Adblock was one of those.
>
> Tho there was a good many of them, I suspect most all of them belongs to
> something here.  I may just leave well enough alone, may poke around a
> little but nothing risky.

Yes, you do want to be very careful, especially any required files that 
ebuilds generate in post_install() - portage usually doesn't know about those.

I'd stick to .so libs initially (revdep-rebuild can find your mistakes!). In 
my case, I had:
liblcms.so.1.0.13
liblcms.so.1.0.18
and eic showed that there is no.13 version. So apparently, this comes from a 
time when I had a portage that didn't cleanly remove stuff.

-- 
alan dot mckinnon at gmail dot com

Re: [gentoo-user] Finding orphaned libs

[Mike Kazantsev]

[-- Attachment #1: Type: text/plain, Size: 1122 bytes --]

On Mon, 08 Jun 2009 19:27:39 -0500
Dale <rdalek1967@gmail.com> wrote:

> > Also, be careful when you parse the output of the command. Most of the
> > .pyc and .pyo files in the python2.5 directories are byte-compiled
> > version that python generated dynamically the first time they are
> > used. For example: ..../Numeric/numeric_version.py was installed by
> > the ebuild and thuse qfile tells me it belongs to dev-python/numeric,
> > but .../Numeric/numeric_version.pyc is listed as an "orphan". While it
> > is safe to delete, it will just be regenerated again later, wasting
> > computing cycles. 
> 
> That's sort of what I was thinking.  It was generated when it was
> started up the first time.  I also noticed some things that I installed
> in the Seamonkey directory too.  Adblock was one of those.

That's not quite correct: .py[co] are generated by emerge right after
package installaton and these won't come back unless you use these libs
as root, since python won't have write access to these paths and will
be byte-compiling each script on-the-fly.

-- 
Mike Kazantsev // fraggod.net

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Finding orphaned libs

[Dale]

Mike Kazantsev wrote:
> On Mon, 08 Jun 2009 19:27:39 -0500
> Dale <rdalek1967@gmail.com> wrote:
>
>   
>>> Also, be careful when you parse the output of the command. Most of the
>>> .pyc and .pyo files in the python2.5 directories are byte-compiled
>>> version that python generated dynamically the first time they are
>>> used. For example: ..../Numeric/numeric_version.py was installed by
>>> the ebuild and thuse qfile tells me it belongs to dev-python/numeric,
>>> but .../Numeric/numeric_version.pyc is listed as an "orphan". While it
>>> is safe to delete, it will just be regenerated again later, wasting
>>> computing cycles. 
>>>       
>> That's sort of what I was thinking.  It was generated when it was
>> started up the first time.  I also noticed some things that I installed
>> in the Seamonkey directory too.  Adblock was one of those.
>>     
>
> That's not quite correct: .py[co] are generated by emerge right after
> package installaton and these won't come back unless you use these libs
> as root, since python won't have write access to these paths and will
> be byte-compiling each script on-the-fly.
>
>   

All the more reason for me to leave them alone then huh?  ;-)  We all
know what happens to portage when python pukes up last weeks meal.  :/

Dale

:-)  :-) 

Re: [gentoo-user] Finding orphaned libs

[Mike Kazantsev]

[-- Attachment #1: Type: text/plain, Size: 818 bytes --]

On Wed, 10 Jun 2009 02:53:52 -0500
Dale <rdalek1967@gmail.com> wrote:

> > That's not quite correct: .py[co] are generated by emerge right after
> > package installaton and these won't come back unless you use these libs
> > as root, since python won't have write access to these paths and will
> > be byte-compiling each script on-the-fly.
> 
> All the more reason for me to leave them alone then huh?  ;-)  We all
> know what happens to portage when python pukes up last weeks meal.  :/

Nah, you can't crash python by stealing it's bytecode, everything will
work, just won't have a nice warm-start.

Of course, that's not a good thing, so emerge compiles them on purpose,
and unless you really short on space or inodes there's not much point
in removing them.

-- 
Mike Kazantsev // fraggod.net

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Finding orphaned libs

[Volker Armin Hemmann]

On Montag 08 Juni 2009, Neil Bothwick wrote:
> qfile --orphans $(find -H /usr/lib /lib -type f)

qfile --orphans $(find -H /usr/lib /lib -type f)
zsh: argument list too long: qfile

bash
energy ~ # qfile --orphans $(find -H /usr/lib /lib -type f)
bash: /usr/bin/qfile: Argument list too long

bash --version
GNU bash, version 4.0.24(2)-release (x86_64-pc-linux-gnu)
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

qfile -V
portage-utils-0.2: compiled on May 10 2009
$Id: qfile.c,v 1.45 2007/05/24 14:47:18 solar Exp $
file written for Gentoo by <solar and vapier @ gentoo.org>

Re: [gentoo-user] Finding orphaned libs

[Joerg Schilling]

Volker Armin Hemmann <volkerarmin@googlemail.com> wrote:

> On Montag 08 Juni 2009, Neil Bothwick wrote:
> > qfile --orphans $(find -H /usr/lib /lib -type f)
>
> qfile --orphans $(find -H /usr/lib /lib -type f)
> zsh: argument list too long: qfile

This is not a zsh limitation but a Linux limitation.
You may have longer arg lists if you use Solaris ;-)

You may avoid the problem with find . -exec prog args {} +

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: [gentoo-user] Finding orphaned libs

[Volker Armin Hemmann]

On Dienstag 09 Juni 2009, Joerg Schilling wrote:
> Volker Armin Hemmann <volkerarmin@googlemail.com> wrote:
> > On Montag 08 Juni 2009, Neil Bothwick wrote:
> > > qfile --orphans $(find -H /usr/lib /lib -type f)
> >
> > qfile --orphans $(find -H /usr/lib /lib -type f)
> > zsh: argument list too long: qfile
>
> This is not a zsh limitation but a Linux limitation.
> You may have longer arg lists if you use Solaris ;-)
>
> You may avoid the problem with find . -exec prog args {} +

ah, thank you - I just copied from Neil (and had a look into the manpage which 
uses the same line more or less).

Re: [gentoo-user] Finding orphaned libs

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 704 bytes --]

On Tue, 9 Jun 2009 16:04:41 +0200, Volker Armin Hemmann wrote:

> > This is not a zsh limitation but a Linux limitation.
> > You may have longer arg lists if you use Solaris ;-)
> >
> > You may avoid the problem with find . -exec prog args {} +  
> 
> ah, thank you - I just copied from Neil (and had a look into the
> manpage which uses the same line more or less).

I've only used that command with smaller directories, where the
limitation doesn't occur. find -exec does the job but is rather slow.
Piping through xargs is faster and the limitations Joerg mentions, like
spaces in filenames, don't apply in the case of */lib.


-- 
Neil Bothwick

All general statements are false.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Finding orphaned libs

[Joerg Schilling]

Neil Bothwick <neil@digimed.co.uk> wrote:

> On Tue, 9 Jun 2009 16:04:41 +0200, Volker Armin Hemmann wrote:
>
> > > This is not a zsh limitation but a Linux limitation.
> > > You may have longer arg lists if you use Solaris ;-)
> > >
> > > You may avoid the problem with find . -exec prog args {} +  
> > 
> > ah, thank you - I just copied from Neil (and had a look into the
> > manpage which uses the same line more or less).
>
> I've only used that command with smaller directories, where the
> limitation doesn't occur. find -exec does the job but is rather slow.
> Piping through xargs is faster and the limitations Joerg mentions, like
> spaces in filenames, don't apply in the case of */lib.

Please try to inform yourself about execplus in order to learn that this is the 
only unlimited was to do the job.

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: [gentoo-user] Finding orphaned libs

[Mickaël Bucas]

2009/6/9 Joerg Schilling <Joerg.Schilling@fokus.fraunhofer.de>
>
> Volker Armin Hemmann <volkerarmin@googlemail.com> wrote:
>
> > On Montag 08 Juni 2009, Neil Bothwick wrote:
> > > qfile --orphans $(find -H /usr/lib /lib -type f)
> >
> > qfile --orphans $(find -H /usr/lib /lib -type f)
> > zsh: argument list too long: qfile
>
> This is not a zsh limitation but a Linux limitation.
> You may have longer arg lists if you use Solaris ;-)
>
> You may avoid the problem with find . -exec prog args {} +
>
> Jörg
>
> --
>  EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
>       js@cs.tu-berlin.de                (uni)
>       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
>  URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
>

The right way to handle any size for the list returned by "find", is
by using "xargs" :

find -H /usr/lib /lib -type f | xargs -d'\n' qfile --orphans

"xargs" fills the command line up to the maximum size, and then create
another command line.

Mickaël Bucas

Re: [gentoo-user] Finding orphaned libs

[Joerg Schilling]

Mickaël Bucas <mbucas@gmail.com> wrote:


> > You may avoid the problem with find . -exec prog args {} +

> The right way to handle any size for the list returned by "find", is
> by using "xargs" :
>
> find -H /usr/lib /lib -type f | xargs -d'\n' qfile --orphans

No, this is definitely wrong: the right way to handle this is execplus 
(since 19 years).


Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: [gentoo-user] Finding orphaned libs

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 395 bytes --]

On Tue, 09 Jun 2009 16:15:21 +0200, Joerg Schilling wrote:

> > find -H /usr/lib /lib -type f | xargs -d'\n' qfile --orphans  
> 
> No, this is definitely wrong: the right way to handle this is execplus 
> (since 19 years).

If it's been around 19 years, why doesn't Google know anything about it?
What is it?


-- 
Neil Bothwick

The facts, although interesting, are irrelevant.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Finding orphaned libs

[Paul Hartman]

On Tue, Jun 9, 2009 at 9:36 AM, Neil Bothwick<neil@digimed.co.uk> wrote:
> If it's been around 19 years, why doesn't Google know anything about it?
> What is it?

from the manpage of "find", it even compares it to the xargs method:

-exec command {} +
This  variant of the -exec action runs the specified command on the
selected files, but the command line is built by appending each
selected file name at the end; the total number of invocations of the
command will be much less than the number of matched files.  The
command line is built in much the same way that xargs builds its
command lines.  Only one instance of `{}' is allowed within the
command.   The  command is executed in the starting directory.

Re: [gentoo-user] Finding orphaned libs

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 795 bytes --]

On Tue, 9 Jun 2009 09:42:34 -0500, Paul Hartman wrote:

> -exec command {} +
> This  variant of the -exec action runs the specified command on the
> selected files, but the command line is built by appending each
> selected file name at the end; the total number of invocations of the
> command will be much less than the number of matched files.  The
> command line is built in much the same way that xargs builds its
> command lines.  Only one instance of `{}' is allowed within the
> command.   The  command is executed in the starting directory.

How does this handle quoted filenames?

Does -exec command "{}" +

do

command "file 1 file 2 file 3"

or

command "file 1" "file 2" "file 3"



-- 
Neil Bothwick

All generalizations are false, including this one.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Finding orphaned libs

[Joerg Schilling]

Neil Bothwick <neil@digimed.co.uk> wrote:

> On Tue, 9 Jun 2009 09:42:34 -0500, Paul Hartman wrote:
>
> > -exec command {} +
> > This  variant of the -exec action runs the specified command on the
> > selected files, but the command line is built by appending each
> > selected file name at the end; the total number of invocations of the
> > command will be much less than the number of matched files.  The
> > command line is built in much the same way that xargs builds its
> > command lines.  Only one instance of `{}' is allowed within the
> > command.   The  command is executed in the starting directory.
>
> How does this handle quoted filenames?

correctly.

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: [gentoo-user] Finding orphaned libs

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 272 bytes --]

On Tue, 09 Jun 2009 16:59:39 +0200, Joerg Schilling wrote:

> > How does this handle quoted filenames?  
> 
> correctly.

Are you part Vorlon ;-)


-- 
Neil Bothwick

The world is a tragedy to those who feel, but a comedy to those who
think.(Horace Walpole)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Finding orphaned libs

[Etaoin Shrdlu]

On Tuesday 9 June 2009, 16:52, Neil Bothwick wrote:
> On Tue, 9 Jun 2009 09:42:34 -0500, Paul Hartman wrote:
> > -exec command {} +
> > This  variant of the -exec action runs the specified command on the
> > selected files, but the command line is built by appending each
> > selected file name at the end; the total number of invocations of
> > the command will be much less than the number of matched files.  The
> > command line is built in much the same way that xargs builds its
> > command lines.  Only one instance of `{}' is allowed within the
> > command.   The  command is executed in the starting directory.
>
> How does this handle quoted filenames?
>
> Does -exec command "{}" +
>
> do
>
> command "file 1 file 2 file 3"
>
> or
>
> command "file 1" "file 2" "file 3"

The latter.

Re: [gentoo-user] Finding orphaned libs

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 313 bytes --]

On Tue, 9 Jun 2009 17:21:32 +0200, Etaoin Shrdlu wrote:

> > command "file 1 file 2 file 3"
> >
> > or
> >
> > command "file 1" "file 2" "file 3"  
> 
> The latter.

Thanks, and bye-bye xargs (not that I used it very often).


-- 
Neil Bothwick

Cross-country skiing is great in small countries.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Finding orphaned libs

[Etaoin Shrdlu]

On Tuesday 9 June 2009, 16:36, Neil Bothwick wrote:
> On Tue, 09 Jun 2009 16:15:21 +0200, Joerg Schilling wrote:
> > > find -H /usr/lib /lib -type f | xargs -d'\n' qfile --orphans
> >
> > No, this is definitely wrong: the right way to handle this is
> > execplus (since 19 years).
>
> If it's been around 19 years, why doesn't Google know anything about
> it? What is it?

Well, google does not know everything :)

Basically, using + instead of ; after -exec allows to run the specified 
command less times, each time with the highest possible number of 
arguments (instead of running it once per file, which is what happens 
with ;). And yes, that's been in POSIX for a long time now. Example:


$ ls
file1  file2  file3  file4  file5

$ find . -type f -exec sh -c 'echo "number of arguments: $#"' sh {} \;
number of arguments: 1
number of arguments: 1
number of arguments: 1
number of arguments: 1
number of arguments: 1

$ find . -type f -exec sh -c 'echo "number of arguments: $#"' sh {} +
number of arguments: 5

So when you have to run a command on a very big number of files, say 1000 
or more, with ; you spawn 1000 processes, with + you span just one or 
two (well, depending on the maximum command line length on the system 
anyway). This is of course much less resource intensive.

Basically, using -exec with + does what xargs does, but without the need 
to care for strange characters in file names (well, a bit simplified, 
but you get the idea).

Re: [gentoo-user] Finding orphaned libs

[Alan McKinnon]

On Tuesday 09 June 2009 17:20:49 Etaoin Shrdlu wrote:
> On Tuesday 9 June 2009, 16:36, Neil Bothwick wrote:
> > On Tue, 09 Jun 2009 16:15:21 +0200, Joerg Schilling wrote:
> > > > find -H /usr/lib /lib -type f | xargs -d'\n' qfile --orphans
> > >
> > > No, this is definitely wrong: the right way to handle this is
> > > execplus (since 19 years).
> >
> > If it's been around 19 years, why doesn't Google know anything about
> > it? What is it?
>
> Well, google does not know everything :)
>
> Basically, using + instead of ; after -exec allows to run the specified
> command less times, each time with the highest possible number of
> arguments (instead of running it once per file, which is what happens
> with ;). And yes, that's been in POSIX for a long time now. Example:
>
>
> $ ls
> file1  file2  file3  file4  file5
>
> $ find . -type f -exec sh -c 'echo "number of arguments: $#"' sh {} \;
> number of arguments: 1
> number of arguments: 1
> number of arguments: 1
> number of arguments: 1
> number of arguments: 1
>
> $ find . -type f -exec sh -c 'echo "number of arguments: $#"' sh {} +
> number of arguments: 5
>
> So when you have to run a command on a very big number of files, say 1000
> or more, with ; you spawn 1000 processes, with + you span just one or
> two (well, depending on the maximum command line length on the system
> anyway). This is of course much less resource intensive.

Some numbers.

These are from memory, I ran these commands today while preparing a machine 
for an installation of a package. The directory tree at the starting point had 
about 5000 files, more than 80% with a UID not attached to an account:

chown -R <user>:<group> *
about 2 seconds

find . -nouser -o -nogroup -exec chown <user>:<group> {} +
about 30 seconds (wild guess)

find . -nouser -o -nogroup -exec chown <user>:<group> {} \;
I killed this one after 5 minutes and it was nowhere near complete

Admittedly, this was on a vmware guest with a rather poor disk configuration, 
but it does illustrate that the naive "find \;" performs extremely poorly. 
chown on it's own is foolish as the whole point of the exercise was to find 
files meeting certain criteria, and there was definitely some that didn't.

execplus is a fine middle ground giving the best possible bang for buck.

-- 
alan dot mckinnon at gmail dot com

Re: [gentoo-user] Finding orphaned libs

[Mickaël Bucas]

2009/6/9 Joerg Schilling <Joerg.Schilling@fokus.fraunhofer.de>:
> Mickaël Bucas <mbucas@gmail.com> wrote:
>
>
>> > You may avoid the problem with find . -exec prog args {} +
>
>> The right way to handle any size for the list returned by "find", is
>> by using "xargs" :
>>
>> find -H /usr/lib /lib -type f | xargs -d'\n' qfile --orphans
>
> No, this is definitely wrong: the right way to handle this is execplus
> (since 19 years).
>
> Jörg

I think I was too assertive.
I prefer "xargs", and it's still handy for more elaborate command lines

Mickaël Bucas