[gentoo-user] [OT] -print0 but with 'locate'

[gentoo-user] [OT] -print0 but with 'locate'

[Meino.Cramer]

Hi,

I want to 'locate' a bunch of files and feed the output into 
'| xargs md5sum'.
Unfortunately some of them are infected with the "file name"-virus
(space in the filename).
With find there is the -print0 option which corresponds to '-0' of
the xargs options. As of my knowledge, locate does not have such 
a thing.
I dont like the idea to 'find' (read: 'search again')all the files,
which locate already knows...so
Is there any trick/option/whatever to get the files with spaced
filenames processed by md5sum in combination with 'locate'?

Thank you very much in advance for any help!
Best regards
Meino

Re: [gentoo-user] [OT] -print0 but with 'locate'

[Joerg Schilling]

<Meino.Cramer@gmx.de> wrote:

> Hi,
>
> I want to 'locate' a bunch of files and feed the output into 
> '| xargs md5sum'.
> Unfortunately some of them are infected with the "file name"-virus
> (space in the filename).
> With find there is the -print0 option which corresponds to '-0' of
> the xargs options. As of my knowledge, locate does not have such 
> a thing.

This is one of the reasons, why POSIX did not include -print0, as it would need 
to add new related options to an unknown amount of other programs.

Another reason of curse is that "find . -exec cmd {} +" exists longer than GNU 
find and it's -print0. Execplus was added on 1989 by David Korn.

Jörg

-- 
 EMail:joerg@schily.net                    (home) Jörg Schilling D-13353 Berlin
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.org/private/ http://sourceforge.net/projects/schilytools/files/'

Re: [gentoo-user] [OT] -print0 but with 'locate'

[Meino.Cramer]

Joerg Schilling <Joerg.Schilling@fokus.fraunhofer.de> [16-09-03 12:28]:
> <Meino.Cramer@gmx.de> wrote:
> 
> > Hi,
> >
> > I want to 'locate' a bunch of files and feed the output into 
> > '| xargs md5sum'.
> > Unfortunately some of them are infected with the "file name"-virus
> > (space in the filename).
> > With find there is the -print0 option which corresponds to '-0' of
> > the xargs options. As of my knowledge, locate does not have such 
> > a thing.
> 
> This is one of the reasons, why POSIX did not include -print0, as it would need 
> to add new related options to an unknown amount of other programs.
> 
> Another reason of curse is that "find . -exec cmd {} +" exists longer than GNU 
> find and it's -print0. Execplus was added on 1989 by David Korn.
> 
> Jörg
> 
> -- 
>  EMail:joerg@schily.net                    (home) Jörg Schilling D-13353 Berlin
>        joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
>  URL:  http://cdrecord.org/private/ http://sourceforge.net/projects/schilytools/files/'
> 


...and how does it solve my problem?

Re: [gentoo-user] [OT] -print0 but with 'locate'

[Alan McKinnon]

On 03/09/2016 12:43, Meino.Cramer@gmx.de wrote:
> Joerg Schilling <Joerg.Schilling@fokus.fraunhofer.de> [16-09-03 12:28]:
>> <Meino.Cramer@gmx.de> wrote:
>>
>>> Hi,
>>>
>>> I want to 'locate' a bunch of files and feed the output into 
>>> '| xargs md5sum'.
>>> Unfortunately some of them are infected with the "file name"-virus
>>> (space in the filename).
>>> With find there is the -print0 option which corresponds to '-0' of
>>> the xargs options. As of my knowledge, locate does not have such 
>>> a thing.
>>
>> This is one of the reasons, why POSIX did not include -print0, as it would need 
>> to add new related options to an unknown amount of other programs.
>>
>> Another reason of curse is that "find . -exec cmd {} +" exists longer than GNU 
>> find and it's -print0. Execplus was added on 1989 by David Korn.
>>
>> Jörg
>>
>> -- 
>>  EMail:joerg@schily.net                    (home) Jörg Schilling D-13353 Berlin
>>        joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
>>  URL:  http://cdrecord.org/private/ http://sourceforge.net/projects/schilytools/files/'
>>
> 
> 
> ...and how does it solve my problem?

In several ways.

locate may or may not be up to date as it runs on a cron.
find OTOH tells you what files you really have right now.

Your objection to find IMHO is not really valid. It doesn't read all the
files, it simply reads all the *directories* and grabs the dentries for
the files from there. As you intend using locate, all you need is the
names and so find has no need to touch any file inodes it finds at all.
[ -mtime can really slow things down, but you won't be doing that ]

Do it once and even if you repeat the find many times in succession, all
the data find needs is now in RAM so it's not much more expensive than
locate.

So with that in mind there's no reason to not use find, then you get all
the -print0 goodness that you *really* need.

The alternative is to struggle endlessly with bash quoting rules to get
quotes around your filenames and quote the arguments to md5sum. Or some
magic with awk (all of which is many many times more cpu work than your
objection to find)

So use find :-)


-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] [OT] -print0 but with 'locate'

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 628 bytes --]

On Sat, 3 Sep 2016 11:52:46 +0200, Meino.Cramer@gmx.de wrote:

> I want to 'locate' a bunch of files and feed the output into 
> '| xargs md5sum'.  
> Unfortunately some of them are infected with the "file name"-virus
> (space in the filename).
> With find there is the -print0 option which corresponds to '-0' of
> the xargs options. As of my knowledge, locate does not have such 
> a thing.

it does

locate -0 blah | xargs md5sum -0


-- 
Neil Bothwick

I heard someone tried the monkeys-on-typewriters bit trying for the plays
of W. Shakespeare but all they got was the collected works of Francis
Bacon

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 163 bytes --]

Re: [gentoo-user] [OT] -print0 but with 'locate'

[Alan Mackenzie]

Hello, Meino.

On Sat, Sep 03, 2016 at 11:52:46AM +0200, Meino.Cramer@gmx.de wrote:
> Hi,

> I want to 'locate' a bunch of files and feed the output into 
> '| xargs md5sum'.
> Unfortunatexly some of them are infected with the "file name"-virus
> (space in the filename).
> With find there is the -print0 option which corresponds to '-0' of
> the xargs options. As of my knowledge, locate does not have such 
> a thing.
> I dont like the idea to 'find' (read: 'search again')all the files,
> which locate already knows...so
> Is there any trick/option/whatever to get the files with spaced
> filenames processed by md5sum in combination with 'locate'?

I'm afraid I don't know `locate' at all (it isn't on my system), but
does it have something like `ls''s -1 option which prints out each
filename on a separate line?  In this case you could put quote marks
around each file name, for example like this:

locate .... -1 ..... | sed "s/^.*$/\'&\'/" | xargs md5sum

> Thank you very much in advance for any help!
> Best regards
> Meino

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: [gentoo-user] [OT] -print0 but with 'locate'

[Meino.Cramer]

Neil Bothwick <neil@digimed.co.uk> [16-09-03 13:20]:
> On Sat, 3 Sep 2016 11:52:46 +0200, Meino.Cramer@gmx.de wrote:
> 
> > I want to 'locate' a bunch of files and feed the output into 
> > '| xargs md5sum'.  
> > Unfortunately some of them are infected with the "file name"-virus
> > (space in the filename).
> > With find there is the -print0 option which corresponds to '-0' of
> > the xargs options. As of my knowledge, locate does not have such 
> > a thing.
> 
> it does
> 
> locate -0 blah | xargs md5sum -0
> 
> 
> -- 
> Neil Bothwick
> 
> I heard someone tried the monkeys-on-typewriters bit trying for the plays
> of W. Shakespeare but all they got was the collected works of Francis
> Bacon


...ok thanks. THATS is what I was looking for...

[gentoo-user] Re: [OT] -print0 but with 'locate'

[Kai Krakow]

Am Sat, 3 Sep 2016 11:52:46 +0200
schrieb Meino.Cramer@gmx.de:

> Hi,
> 
> I want to 'locate' a bunch of files and feed the output into 
> '| xargs md5sum'.
> Unfortunately some of them are infected with the "file name"-virus
> (space in the filename).
> With find there is the -print0 option which corresponds to '-0' of
> the xargs options. As of my knowledge, locate does not have such 
> a thing.
> I dont like the idea to 'find' (read: 'search again')all the files,
> which locate already knows...so
> Is there any trick/option/whatever to get the files with spaced
> filenames processed by md5sum in combination with 'locate'?

In that case try

$ locate ... | while read name; do md5sum "$name"; done

This works for results having one filename per line.

-- 
Regards,
Kai

Replies to list-only preferred.

[gentoo-user] Re: -print0 but with 'locate'

[Ian Zimmerman]

On 2016-09-03 13:00, Alan McKinnon wrote:

> Your objection to find IMHO is not really valid. It doesn't read all the
> files, it simply reads all the *directories* and grabs the dentries for
> the files from there. As you intend using locate, all you need is the
> names and so find has no need to touch any file inodes it finds at all.
> [ -mtime can really slow things down, but you won't be doing that ]
> 
> Do it once and even if you repeat the find many times in succession, all
> the data find needs is now in RAM so it's not much more expensive than
> locate.

This depends on where the starting point for find is, IMO.

If I do this over a limited part of the system then yes.  But not if I
do find / -print ; in this latter case I observe that _some_ of the
dentries are cached but not all.

Do I have too many files? :-)

-- 
Please *no* private Cc: on mailing lists and newsgroups
Why does the arrow on Hillary signs point to the right?

[gentoo-user] Re: -print0 but with 'locate'

[Ian Zimmerman]

On 2016-09-03 11:52, Meino.Cramer@gmx.de wrote:

> Is there any trick/option/whatever to get the files with spaced
> filenames processed by md5sum in combination with 'locate'?

In case neither "find -exec +" nor "locate -0" existed (both of which
others in this thread have already pointed out), I would do it like this:

locate foo | awk '{ gsub(/ /,"\\ ",$0); print "md5sum " $0 }' | sh

Note that this is only one extra process (the awk) over and above the
md5sum invocations.  I expect the overhead is minimal, and perhaps this
is even competitive with the straight "find" solution.

-- 
Please *no* private Cc: on mailing lists and newsgroups
Why does the arrow on Hillary signs point to the right?

Re: [gentoo-user] Re: -print0 but with 'locate'

[Joerg Schilling]

Ian Zimmerman <itz@primate.net> wrote:

> In case neither "find -exec +" nor "locate -0" existed (both of which

Then you must have an OS that was not updated for 27 years ;-)

Jörg

-- 
 EMail:joerg@schily.net                    (home) Jörg Schilling D-13353 Berlin
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.org/private/ http://sourceforge.net/projects/schilytools/files/'