[gentoo-dev] Re: [gentoo-commits] gentoo-x86 commit in app-misc/note: ChangeLog note-1.3.3.ebuild

[gentoo-dev] Re: [gentoo-commits] gentoo-x86 commit in app-misc/note: ChangeLog note-1.3.3.ebuild

[Donnie Berkholz]

On 07:40 Mon 15 Oct     , Christian Faulhammer (opfer) wrote:
> 1.1                  app-misc/note/note-1.3.3.ebuild
> 
> file : http://sources.gentoo.org/viewcvs.py/gentoo-x86/app-misc/note/note-1.3.3.ebuild?rev=1.1&view=markup
> plain: http://sources.gentoo.org/viewcvs.py/gentoo-x86/app-misc/note/note-1.3.3.ebuild?rev=1.1&content-type=text/plain

> 	for v in mysql text dbm general; do
> 		if ! use ${v}; then
> 			for u in `find "${D}" -type f -name *${v}.*pm`; do
> 				rm ${u}

Looks like 'u' could have spaces in it, since it's based on D.

Thanks,
Donnie
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: [gentoo-commits] gentoo-x86 commit in app-misc/note: ChangeLog note-1.3.3.ebuild

[Bo Ørsted Andresen]

[-- Attachment #1: Type: text/plain, Size: 1104 bytes --]

On Monday 15 October 2007 09:42:50 Donnie Berkholz wrote:
> On 07:40 Mon 15 Oct     , Christian Faulhammer (opfer) wrote:
> > 1.1                  app-misc/note/note-1.3.3.ebuild
> >
> > file : http://sources.gentoo.org/viewcvs.py/gentoo-x86/app-misc/note/note-1.3.3.ebuild?rev=1.1&view=markup
> > plain: http://sources.gentoo.org/viewcvs.py/gentoo-x86/app-misc/note/note-1.3.3.ebuild?rev=1.1&content-type=text/plain
> >
> > 	for v in mysql text dbm general; do
> > 		if ! use ${v}; then
> > 			for u in `find "${D}" -type f -name *${v}.*pm`; do
> > 				rm ${u}
>
> Looks like 'u' could have spaces in it, since it's based on D.

Written like this u certainly can't since the output of the find expression is
subjected to word splitting before u gets set. Hence this will fail if ${D}
contains spaces even if ${u} gets quoted. The best way to solve this is to use
find ... -print0 | xargs -0 rm. Unfortunately that solution means using GNU
extensions that I'm not sure are guaranteed to be available in the ebuild
environment. If they aren't maybe they should be.

-- 
Bo Andresen

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Re: [gentoo-commits] gentoo-x86 commit in app-misc/note: ChangeLog note-1.3.3.ebuild

[Donnie Berkholz]

On 09:56 Mon 15 Oct     , Bo Ørsted Andresen wrote:
> Written like this u certainly can't since the output of the find expression is
> subjected to word splitting before u gets set. Hence this will fail if ${D}
> contains spaces even if ${u} gets quoted. The best way to solve this is to use
> find ... -print0 | xargs -0 rm. Unfortunately that solution means using GNU
> extensions that I'm not sure are guaranteed to be available in the ebuild
> environment. If they aren't maybe they should be.

Cool, I hadn't dealt with this before. Thanks for pointing that out! We 
were just talking about whether to require GNU find the other day, but I 
don't recall the conclusion. Does anyone else?

Thanks,
Donnie
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: [gentoo-commits] gentoo-x86 commit in app-misc/note: ChangeLog note-1.3.3.ebuild

[Ulrich Mueller]

>>>>> On Mon, 15 Oct 2007, Bo Ørsted Andresen wrote:

> The best way to solve this is to use find ... -print0 | xargs -0 rm.

cd "${D}"; find . ... | xargs rm -f
and you won't need any GNU extensions.

Ulrich
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: [gentoo-commits] gentoo-x86 commit in app-misc/note: ChangeLog note-1.3.3.ebuild

[Roy Marples]

On Mon, 2007-10-15 at 09:56 +0200, Bo Ørsted Andresen wrote:
> On Monday 15 October 2007 09:42:50 Donnie Berkholz wrote:
> > On 07:40 Mon 15 Oct     , Christian Faulhammer (opfer) wrote:
> > > 1.1                  app-misc/note/note-1.3.3.ebuild
> > >
> > > file : http://sources.gentoo.org/viewcvs.py/gentoo-x86/app-misc/note/note-1.3.3.ebuild?rev=1.1&view=markup
> > > plain: http://sources.gentoo.org/viewcvs.py/gentoo-x86/app-misc/note/note-1.3.3.ebuild?rev=1.1&content-type=text/plain
> > >
> > > 	for v in mysql text dbm general; do
> > > 		if ! use ${v}; then
> > > 			for u in `find "${D}" -type f -name *${v}.*pm`; do
> > > 				rm ${u}
> >
> > Looks like 'u' could have spaces in it, since it's based on D.
> 
> Written like this u certainly can't since the output of the find expression is
> subjected to word splitting before u gets set. Hence this will fail if ${D}
> contains spaces even if ${u} gets quoted. The best way to solve this is to use
> find ... -print0 | xargs -0 rm. Unfortunately that solution means using GNU
> extensions that I'm not sure are guaranteed to be available in the ebuild
> environment. If they aren't maybe they should be.
> 

find "${D}" -type f -name *${v}.*pm -delete

Thanks

Roy

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: [gentoo-commits] gentoo-x86 commit in app-misc/note: ChangeLog note-1.3.3.ebuild

[Roy Marples]

On Mon, 2007-10-15 at 01:01 -0700, Donnie Berkholz wrote:
> On 09:56 Mon 15 Oct     , Bo Ørsted Andresen wrote:
> > Written like this u certainly can't since the output of the find expression is
> > subjected to word splitting before u gets set. Hence this will fail if ${D}
> > contains spaces even if ${u} gets quoted. The best way to solve this is to use
> > find ... -print0 | xargs -0 rm. Unfortunately that solution means using GNU
> > extensions that I'm not sure are guaranteed to be available in the ebuild
> > environment. If they aren't maybe they should be.
> 
> Cool, I hadn't dealt with this before. Thanks for pointing that out! We 
> were just talking about whether to require GNU find the other day, but I 
> don't recall the conclusion. Does anyone else?

I think most people wanted to force all GNU tools.
So once it's all keyworded BSD and installed outside of our normal $PATH
or g prefixed (gfind, gsed, etc) then you can start
DEPEND="your-find-of-choice" and use it in the ebuild.

But we're not there yet so lets use something that all platforms
support. Like say find's -delete action which should fit the bill quite
nicely.

Thanks

Roy

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: [gentoo-commits] gentoo-x86 commit in app-misc/note: ChangeLog note-1.3.3.ebuild

[Fabian Groffen]

On 15-10-2007 09:35:35 +0100, Roy Marples wrote:
> find "${D}" -type f -name *${v}.*pm -delete

Looks like you rely on your shell here to assume that you meant
"*${v}.*pm" because there is nothing that matches that pattern by
coincidence.  If it does, your find probably doesn't do what you expect
it to.


-- 
Fabian Groffen
Gentoo on a different level
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: [gentoo-commits] gentoo-x86 commit in app-misc/note: ChangeLog note-1.3.3.ebuild

[Roy Marples]

On Mon, 2007-10-15 at 10:43 +0200, Fabian Groffen wrote:
> On 15-10-2007 09:35:35 +0100, Roy Marples wrote:
> > find "${D}" -type f -name *${v}.*pm -delete
> 
> Looks like you rely on your shell here to assume that you meant
> "*${v}.*pm" because there is nothing that matches that pattern by
> coincidence.  If it does, your find probably doesn't do what you expect
> it to.

It was like that in the original code snippet, I assumed that is what
the author intended? Anyway, you're probably right.

Thanks

Roy

-- 
gentoo-dev@gentoo.org mailing list

[gentoo-dev] Re: Re: [gentoo-commits] gentoo-x86 commit in app-misc/note: ChangeLog note-1.3.3.ebuild

[Steve Long]

Roy Marples wrote:

> On Mon, 2007-10-15 at 10:43 +0200, Fabian Groffen wrote:
>> On 15-10-2007 09:35:35 +0100, Roy Marples wrote:
>> > find "${D}" -type f -name *${v}.*pm -delete
>> 
>> Looks like you rely on your shell here to assume that you meant
>> "*${v}.*pm" because there is nothing that matches that pattern by
>> coincidence.  If it does, your find probably doesn't do what you expect
>> it to.
> 
> It was like that in the original code snippet, I assumed that is what
> the author intended? Anyway, you're probably right.
> 
Well if portability is a concern, -delete is not specified in POSIX[1]
afaict. -exec is, it turns out, including -exec blah {} + which really made
me wonder why people still have such a thing for xargs. It turns out GNU
didn't include this til recently, and it isn't in OpenBSD either for one[2]
(..disgraceful, if you ask me ;)

The unintended globbing is indeed unsafe, in the general case. I'd do this: 

find "$D" -type f -name '*'"$v"'.*pm' -exec rm {} +

The shell will still treat that all as one argument (this method is
typically used to insert variables into awk commands, or sed ones which
use ".) The + will make the command execution more efficient for commands
that take multiple filenames. The one caveat with + is that the {} must
appear at the end of the command.[3]

[1] http://www.opengroup.org/onlinepubs/009695399/utilities/find.html
[2]
http://www.openbsd.org/cgi-bin/man.cgi?query=find&apropos=0&sektion=1&manpath=OpenBSD+Current&arch=i386&format=html
[3] http://wooledge.org/mywiki/UsingFind (highly recommended)


-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: Re: [gentoo-commits] gentoo-x86 commit in app-misc/note: ChangeLog note-1.3.3.ebuild

[Gilles Dartiguelongue]

[-- Attachment #1: Type: text/plain, Size: 918 bytes --]


Le lundi 15 octobre 2007 à 23:43 +0100, Steve Long a écrit : 
> Roy Marples wrote:
> The unintended globbing is indeed unsafe, in the general case. I'd do this: 
> 
> find "$D" -type f -name '*'"$v"'.*pm' -exec rm {} +
> 
> The shell will still treat that all as one argument (this method is
> typically used to insert variables into awk commands, or sed ones which
> use ".) The + will make the command execution more efficient for commands
> that take multiple filenames. The one caveat with + is that the {} must
> appear at the end of the command.[3]

I haven't reread the fine mans for a while but iirc, + could lead you to
hit the argument buffer limit of your shell. xargs has --max-args to get
around that and is also way faster than any -exec variant (at least
that's what the bench I did on a PIV2.6Ghz showed me about 4 months ago)

-- 
Gilles Dartiguelongue <eva@gentoo.org>
Gentoo

[-- Attachment #2: Ceci est une partie de message numériquement signée --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

[gentoo-dev] Re: Re: Re: [gentoo-commits] gentoo-x86 commit in app-misc/note: ChangeLog note-1.3.3.ebuild

[Steve Long]

Gilles Dartiguelongue wrote:
> I haven't reread the fine mans for a while but iirc, + could lead you to
> hit the argument buffer limit of your shell. xargs has --max-args to get
> around that and is also way faster than any -exec variant (at least
> that's what the bench I did on a PIV2.6Ghz showed me about 4 months ago)
> 
It doesn't: find will just execute the command as many times as needed (each
time with an argument length maximally less than ARG_MAX. Well, it did when
we tested this a couple of months ago.) So it's optimal in that sense. I
have no idea on the timing measurements, but I don't like pipes and
subshells as a rule (externals are bad enough!;>) It'd be interesting to
see measurements over 100000 loops with long argument lists requiring
multiple invocations per pass, I suppose.

An array expansion can lead to an argument length greater than ARG_MAX
however, eg funcToFix "${list[@]}"

(I'd be kind of perturbed if the last case were infeasible since it would
imply poor memory handling in BASH, and limit the scope of its usability.)


-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: Re: Re: [gentoo-commits] gentoo-x86 commit in app-misc/note: ChangeLog note-1.3.3.ebuild

[Elias Pipping]

[-- Attachment #1: Type: text/plain, Size: 1779 bytes --]

On Tue, Oct 16, 2007 at 03:14:12AM +0100, Steve Long wrote:
> Gilles Dartiguelongue wrote:
> > I haven't reread the fine mans for a while but iirc, + could lead you to
> > hit the argument buffer limit of your shell. xargs has --max-args to get
> > around that and is also way faster than any -exec variant (at least
> > that's what the bench I did on a PIV2.6Ghz showed me about 4 months ago)
> > 
> It doesn't: find will just execute the command as many times as needed (each
> time with an argument length maximally less than ARG_MAX. Well, it did when
> we tested this a couple of months ago.) So it's optimal in that sense. I
> have no idea on the timing measurements, but I don't like pipes and
> subshells as a rule (externals are bad enough!;>) It'd be interesting to
> see measurements over 100000 loops with long argument lists requiring
> multiple invocations per pass, I suppose.

Here's a quick benchmark:

% \
  repeat 3 \time find -name '*.ebuild' | xargs cat >/dev/null; echo; \
  repeat 3 \time find -name '*.ebuild' -exec cat {} ';' >/dev/null; echo; \
  repeat 3 \time find -name '*.ebuild' -exec cat {} '+' >/dev/null
       66.28 real         0.77 user         4.71 sys
       30.95 real         0.76 user         4.73 sys
       30.08 real         0.76 user         4.67 sys

      174.72 real        18.10 user       112.92 sys
      208.69 real        18.07 user       117.05 sys
      173.64 real        18.10 user       110.79 sys

       62.61 real         1.26 user        10.88 sys
       54.98 real         1.25 user        10.46 sys
       41.84 real         1.25 user        10.00 sys

the third variant doesn't get any faster than on the last line.

( tested in zsh, with bsd time and gnu find )


-- Elias

[-- Attachment #2: Type: application/pgp-signature, Size: 186 bytes --]