[gentoo-user] OT: cut replacement with bash builtins

[gentoo-user] OT: cut replacement with bash builtins

[Florian Philipp]

[-- Attachment #1: Type: text/plain, Size: 832 bytes --]

Hi list!

I'm currently streamlining some of my shell scripts to avoid unnecessary
process calls where bash itself is powerful enough.

At the moment, I want to replace stuff like this:
string='foo:bar:foo'
second_field=$(echo $string | cut -d : -f 2) # should read "bar"

My current solution is using two string operations:
string='foo:bar:foo'
# remove everything up to and including first ':'
second_and_following=${string#*:}
# remove everything from the first ':' following
second_field=${second_and_following%%:*}

Of course, I normally do this in a single line with a subshell but it
still looks cumbersome. Is there a way to do it in a single operation
without a temporary variable? The following does not work:
string='foo:bar:foo'
second_field=${string#:%%:*}

Thanks in advance!
Florian Philipp


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

[gentoo-user] Re: OT: cut replacement with bash builtins

[hamilton]

 On Sun, 27 Feb 2011 21:01:46 +0100, Florian Philipp wrote:
> Hi list!
>
> I'm currently streamlining some of my shell scripts to avoid 
> unnecessary
> process calls where bash itself is powerful enough.

 My experience (take it for whatever you think it's worth) is that
 doing so often just makes things harder to follow and maintain.
 It's very unlikely that the overhead of a fork+exec is appreciably
 slowing your process down.  Having said that (and in that vein) there
 is something more straightforward which may be useful:

 [...]

> My current solution is using two string operations:
> string='foo:bar:foo'
> # remove everything up to and including first ':'
> second_and_following=${string#*:}
> # remove everything from the first ':' following
> second_field=${second_and_following%%:*}

 second_field = $(echo $string | awk -F: '{print $2}')


-- 
 Jon Hamilton
 hamilton@pobox.com

Re: [gentoo-user] OT: cut replacement with bash builtins

[Alex Schuster]

Florian Philipp writes:

> I'm currently streamlining some of my shell scripts to avoid unnecessary
> process calls where bash itself is powerful enough.
> 
> At the moment, I want to replace stuff like this:
> string='foo:bar:foo'
> second_field=$(echo $string | cut -d : -f 2) # should read "bar"
> 
> My current solution is using two string operations:
> string='foo:bar:foo'
> # remove everything up to and including first ':'
> second_and_following=${string#*:}
> # remove everything from the first ':' following
> second_field=${second_and_following%%:*}

That's how I do these things, too.

> Of course, I normally do this in a single line with a subshell but it

Hmm, I don't get this. Subshell?

> still looks cumbersome. Is there a way to do it in a single operation
> without a temporary variable? The following does not work:
> string='foo:bar:foo'
> second_field=${string#:%%:*}

I don't think so. But you can write a shell function for this:

getfield()
{
	local str=${1#*:}
	echo "${str%%:*}
}

string='foo:bar:foo'
second_field=$( getfield "$string" )

But if you need to do this very often in a loop, sometimes going back to
cut can speed things up, when you place it outside:

See

for string in $( < inputfile )
do
	second_field=$( getfield "$string" )
	do_something_with $second_field
done

vs.

secondfields=( $( cut -d : -f 2 inputfile ) )
for secondfield in ${secondfields[@]}
do
	do_something_with $second_field
done	

So, in th e2nd example, cut is called only once, but processes all input
lines. The result is put into an array.

Of course, all stuff should be put into double quotes in case there is
whitescape involved. Setting IFS to $'\n' may be necessary for this when
creating the array.

	Wonko

Re: [gentoo-user] Re: OT: cut replacement with bash builtins

[Florian Philipp]

[-- Attachment #1: Type: text/plain, Size: 1443 bytes --]

Am 27.02.2011 21:09, schrieb hamilton:
> On Sun, 27 Feb 2011 21:01:46 +0100, Florian Philipp wrote:
>> Hi list!
>>
>> I'm currently streamlining some of my shell scripts to avoid unnecessary
>> process calls where bash itself is powerful enough.
> 
> My experience (take it for whatever you think it's worth) is that
> doing so often just makes things harder to follow and maintain.
> It's very unlikely that the overhead of a fork+exec is appreciably
> slowing your process down.  Having said that (and in that vein) there
> is something more straightforward which may be useful:
> 

Oh, I completely agree with you. In 90% of all cases, simple and clean
code is preferable. And if I really card about performance, I'd use C or
Perl. However, there are still such rare cases where everything works
fine and you see no reason to code it all again but those three forks in
the innermost loop just kill your performance.

In such cases, I optimize it and put the clean version in the comments.
That way, clarity doesn't suffer too much.

> [...]
> 
>> My current solution is using two string operations:
>> string='foo:bar:foo'
>> # remove everything up to and including first ':'
>> second_and_following=${string#*:}
>> # remove everything from the first ':' following
>> second_field=${second_and_following%%:*}
> 
> second_field = $(echo $string | awk -F: '{print $2}')
> 
> 

Regards,
Florian Philipp


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: [gentoo-user] OT: cut replacement with bash builtins

[Florian Philipp]

[-- Attachment #1: Type: text/plain, Size: 2493 bytes --]

Am 27.02.2011 22:06, schrieb Alex Schuster:
> Florian Philipp writes:
> 
>> I'm currently streamlining some of my shell scripts to avoid unnecessary
>> process calls where bash itself is powerful enough.
>>
>> At the moment, I want to replace stuff like this:
>> string='foo:bar:foo'
>> second_field=$(echo $string | cut -d : -f 2) # should read "bar"
>>
>> My current solution is using two string operations:
>> string='foo:bar:foo'
>> # remove everything up to and including first ':'
>> second_and_following=${string#*:}
>> # remove everything from the first ':' following
>> second_field=${second_and_following%%:*}
> 
> That's how I do these things, too.
> 
>> Of course, I normally do this in a single line with a subshell but it
> 
> Hmm, I don't get this. Subshell?

second_field=$(second_and_following=${string#*:}; echo
${second_and_following%%:*})

Putting it in parentheses creates a subshell. New variables don't leave
the scope. I guess I should have said command substitution but both
concepts apply here.

> 
>> still looks cumbersome. Is there a way to do it in a single operation
>> without a temporary variable? The following does not work:
>> string='foo:bar:foo'
>> second_field=${string#:%%:*}
> 
> I don't think so. But you can write a shell function for this:
> 
> getfield()
> {
> 	local str=${1#*:}
> 	echo "${str%%:*}
> }
> 
> string='foo:bar:foo'
> second_field=$( getfield "$string" )
> 
> But if you need to do this very often in a loop, sometimes going back to
> cut can speed things up, when you place it outside:
> 
> See
> 
> for string in $( < inputfile )
> do
> 	second_field=$( getfield "$string" )
> 	do_something_with $second_field
> done
> 
> vs.
> 
> secondfields=( $( cut -d : -f 2 inputfile ) )
> for secondfield in ${secondfields[@]}
> do
> 	do_something_with $second_field
> done	
> 
> So, in th e2nd example, cut is called only once, but processes all input
> lines. The result is put into an array.
> 

Agreed. Using one long pipe (when applicable) is the real strength of
shell programming.

> Of course, all stuff should be put into double quotes in case there is
> whitescape involved. Setting IFS to $'\n' may be necessary for this when
> creating the array.
> 
> 	Wonko
> 

Agreed, again. I removed the quotes from my examples to improve
readability. It was already bad enough with all those regular expressions.

Thanks for the input.

Florian Philipp


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]