[gentoo-osx] SheBang script

[gentoo-osx] SheBang script

[Grobian]

[-- Attachment #1: Type: text/plain, Size: 505 bytes --]

Ok, I didn't really know where to put it, but I wanted to share my 
efforts with some people in an early phase.  This is the result of half 
an hour script-fu hacking.  I think it might be useful with regard to 
the upcoming portage.

See introductionary comment in the attach to get an idea what it does.

It currently only generates diff -u output, but it's quite easy to "| 
patch" it, and much safer for now to see what is happening, and if that 
is desired ;)

-- 
Fabian Groffen
Gentoo for Mac OS X

[-- Attachment #2: shebang --]
[-- Type: text/plain, Size: 2300 bytes --]

#!/usr/bin/env sh

# SheBang(tm) - a Gentoo script to rape files in a given directory for
# the purpose of supporting a variable 'prefix'.
#
# Fabian Groffen <grobian at gentoo (dot] org>
# 0.1 - initial mockup (20050913)
#
# This script searches for executable script files in the given
# directory and all its sub-directories and tries to process them in
# such a way, that the actual PATH environment controls how they are
# executed.  An example would be a script file having at the first line:
#    #!/usr/bin/perl
# this will be changed into:
#    #!/usr/bin/env perl
# where the actual location of 'perl' is resolved at runtime by 'env'
# using the current PATH environment.  The location of 'env' can be set
# using the variable ENV_PATH below.

# path to the env executable
ENV_PATH="/usr/bin/env"

if [ -z $1 ];
then
	echo "$0 needs an argument: the directory to process" > /dev/stderr
	exit 1;
fi

# make a list with candidate files... well actually find 'em all and let
# file(1) tell us if it is text somehow.  Let awk do some magic to read
# the file and see if the first line starts with "#!".
CANDS=`find $1 -type f -exec file '{}' \; | grep text | cut -d: -f1 | \
awk '
{
	file = $1
	getline < file
	if ($0 ~ /^#!/) print file
	close(file)
}
'`
# we basically have only candidate files in our hands now, so we should
# continue with the script-fu to see if we can do something with the
# "#!/some/where/file" thing...
for cand in $CANDS;
do
	# first get the first line of the file without the #!
	line=`head -n1 $cand | cut -c3-`
	# then see if we know this executable thing
	command=`echo $line | cut -d' ' -f 1`
	name=`basename $command`
	case "$name" in
		sh)
			fine=1;
		;;
		bash)
			fine=1;
		;;
		perl)
			fine=1;
		;;
		awk)
			fine=1;
		;;
		*)
			# we don't know these, and maybe we shouldn't touch them
			fine=0;
		;;
	esac

	if [ $fine -eq 1 ];
	then
		# we know these, and they are fine
		awk -v name="$name" \
				-v file="$cand" \
				-v envpath="$ENV_PATH" \
				-v command="$command" '
			BEGIN {
				for (i = 1; (getline line < file) > 0; i++) {
					if (i == 1) {
						out = "#!" envpath " " name
						out = out substr($0, length(command) + 2 + 1)
						print out
					} else {
						print line
					}
				}
				close(file)
			}
		' | diff -u $cand -
	fi
done

Re: [gentoo-osx] SheBang script

[Kito]

On Sep 13, 2005, at 3:58 PM, Grobian wrote:
[...]

> # This script searches for executable script files in the given
> # directory and all its sub-directories and tries to process them in
> # such a way, that the actual PATH environment controls how they are
> # executed.  An example would be a script file having at the first  
> line:

Ok, I was thinking something along these lines to handle  
install_names for executables and dylibs as well, basically just loop  
over the ${D} and either warn or fix if install_names contain paths  
outside the global prefix.

Brian: Is this sort of thing best handled in bash ala bashrc, or is  
there a good entry point in the portage code to plug this stuff in  
pythonically?

--Kito

[...]
-- 
gentoo-osx@gentoo.org mailing list

Re: [gentoo-osx] SheBang script

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 1228 bytes --]

On Wed, Sep 14, 2005 at 11:06:03AM -0500, Kito wrote:
> 
> On Sep 13, 2005, at 3:58 PM, Grobian wrote:
> [...]
> 
> ># This script searches for executable script files in the given
> ># directory and all its sub-directories and tries to process them in
> ># such a way, that the actual PATH environment controls how they are
> ># executed.  An example would be a script file having at the first  
> >line:
> 
> Ok, I was thinking something along these lines to handle  
> install_names for executables and dylibs as well, basically just loop  
> over the ${D} and either warn or fix if install_names contain paths  
> outside the global prefix.
> 
> Brian: Is this sort of thing best handled in bash ala bashrc, or is  
> there a good entry point in the portage code to plug this stuff in  
> pythonically?

Kicking around something that would allow setups the ability to plug in 
chunks of code for doing things prior/post merge...

So, hooking within the python side would be possible, and prefered.  
Basically is how collision-protect/setuid scans will be implemented, 
and probably triggering ldconfig runs even (assuming it works sanely).

Less cruft jammed into bash, the better. :)
~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-osx] SheBang script

[Grobian]

Brian Harring wrote:
>> Brian: Is this sort of thing best handled in bash ala bashrc, or is  
>> there a good entry point in the portage code to plug this stuff in  
>> pythonically?
> 
> Kicking around something that would allow setups the ability to plug in 
> chunks of code for doing things prior/post merge...
> 
> So, hooking within the python side would be possible, and prefered.  
> Basically is how collision-protect/setuid scans will be implemented, 
> and probably triggering ldconfig runs even (assuming it works sanely).
> 
> Less cruft jammed into bash, the better. :)
> ~harring

I personally think this script should be seen as 'proof-of-concept'. 
It's not too fast, and it involves a lot of inefficient work.  I'm no 
python coder, but I think, given this 'proof-of-concept' it may be 
fairly simple to write a function (?) or two in python that performs 
this job a few orders of a magnitude faster and more controllable.

The problem with these kinds of scripts is that their portability is not 
really assured.  I dubbed whether I would write a quick C version, but 
again I stumbled upon the portability question and decided not to do so, 
because portage, as inner core of Gentoo, should be as much as possible 
unaffected by environmental differences.  Python is (like Java) a 
solution to that.  So, ultimately, this script should be rewritten in 
Python.


-- 
Fabian Groffen
Gentoo for Mac OS X
-- 
gentoo-osx@gentoo.org mailing list

Re: [gentoo-osx] SheBang script

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 1679 bytes --]

On Wed, Sep 14, 2005 at 06:42:55PM +0200, Grobian wrote:
> 
> 
> Brian Harring wrote:
> >>Brian: Is this sort of thing best handled in bash ala bashrc, or is  
> >>there a good entry point in the portage code to plug this stuff in  
> >>pythonically?
> >
> >Kicking around something that would allow setups the ability to plug in 
> >chunks of code for doing things prior/post merge...
> >
> >So, hooking within the python side would be possible, and prefered.  
> >Basically is how collision-protect/setuid scans will be implemented, 
> >and probably triggering ldconfig runs even (assuming it works sanely).
> >
> >Less cruft jammed into bash, the better. :)
> >~harring
> 
> I personally think this script should be seen as 'proof-of-concept'. 
> It's not too fast, and it involves a lot of inefficient work.  I'm no 
> python coder, but I think, given this 'proof-of-concept' it may be 
> fairly simple to write a function (?) or two in python that performs 
> this job a few orders of a magnitude faster and more controllable.
> 
> The problem with these kinds of scripts is that their portability is not 
> really assured.  I dubbed whether I would write a quick C version, but 
> again I stumbled upon the portability question and decided not to do so, 
> because portage, as inner core of Gentoo, should be as much as possible 
> unaffected by environmental differences.  Python is (like Java) a 
> solution to that.  So, ultimately, this script should be rewritten in 
> Python.
module, not script is prefered.

Assuming I actually finish triggers and they don't suck, it'll be a 
passing the func in, not exec'ing a strict.
~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-osx] SheBang script

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 600 bytes --]

On Wed, Sep 14, 2005 at 11:45:55AM -0500, Brian Harring wrote:
> module, not script is prefered.
> 
> Assuming I actually finish triggers and they don't suck, it'll be a 
> passing the func in, not exec'ing a strict.
Damn, I really should read the emails instead of just blindly type and 
send :)

Clarifying/rephrasing; pre/post merge triggers will basically be funcs 
hooked to run at certain phases.  Calling a script could be done, but 
if it's python code it's calling it should be a loaded module imo, 
rather then the inneficient spawning of a new python interpretter...
~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-osx] SheBang script

[Grobian]

Brian Harring wrote:
>> The problem with these kinds of scripts is that their portability is not 
>> really assured.  I dubbed whether I would write a quick C version, but 
>> again I stumbled upon the portability question and decided not to do so, 
>> because portage, as inner core of Gentoo, should be as much as possible 
>> unaffected by environmental differences.  Python is (like Java) a 
>> solution to that.  So, ultimately, this script should be rewritten in 
>> Python.
> module, not script is prefered.

As I said, I'm a python illiterate :)

> Assuming I actually finish triggers and they don't suck, it'll be a 
> passing the func in, not exec'ing a strict.

hmmm... sounds good.  For my understanding, does it mean that the func 
could be executed based on (trigger) conditions, like ppc-macos or 
PREFIX != "/", pref.userWantsSheBang == True or something like that?


-- 
Fabian Groffen
Gentoo for Mac OS X
-- 
gentoo-osx@gentoo.org mailing list

Re: [gentoo-osx] SheBang script

[Grobian]

Brian Harring wrote:
> Clarifying/rephrasing; pre/post merge triggers will basically be funcs 
> hooked to run at certain phases.  Calling a script could be done, but 
> if it's python code it's calling it should be a loaded module imo, 
> rather then the inneficient spawning of a new python interpretter...

Ok, I think we want the same ;)  The functionality could be part of some 
"prefixUtility" module perhaps?


-- 
Fabian Groffen
Gentoo for Mac OS X
-- 
gentoo-osx@gentoo.org mailing list

Re: [gentoo-osx] SheBang script

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 776 bytes --]

On Wed, Sep 14, 2005 at 06:53:28PM +0200, Grobian wrote:
> 
> 
> Brian Harring wrote:
> >Clarifying/rephrasing; pre/post merge triggers will basically be funcs 
> >hooked to run at certain phases.  Calling a script could be done, but 
> >if it's python code it's calling it should be a loaded module imo, 
> >rather then the inneficient spawning of a new python interpretter...
> 
> Ok, I think we want the same ;)  The functionality could be part of some 
> "prefixUtility" module perhaps?
Don't care much name wise, just would have to be defined in the config 
as a trigger.

The reasoning pretty much comes down to being able to shove all arch 
specific crap out of the core, and into pluggable chunks that are 
loaded by configuration settings.
~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-osx] SheBang script

[Grobian]

A small update on the workings of this script:
I found out that for example:
#!/usr/bin/env awk -f
doesn't work as the shell (bash and tcsh) seem to split on the first 
space, and passes "awk -f" as argument to env, which in turn doesn't 
find a command called "awk -f".

I haven't found a solution for this problem yet, but I think that 
/usr/bin/env is not the solution to our problem here.  We either need to 
write our own version of 'env' that does what we want, or maybe find our 
way through another program, perhaps a shell like /bin/sh.  (I think we 
need to keep it portable...)


Grobian wrote:
> 
> 
> Brian Harring wrote:
>>> The problem with these kinds of scripts is that their portability is 
>>> not really assured.  I dubbed whether I would write a quick C 
>>> version, but again I stumbled upon the portability question and 
>>> decided not to do so, because portage, as inner core of Gentoo, 
>>> should be as much as possible unaffected by environmental 
>>> differences.  Python is (like Java) a solution to that.  So, 
>>> ultimately, this script should be rewritten in Python.
>> module, not script is prefered.
> 
> As I said, I'm a python illiterate :)
> 
>> Assuming I actually finish triggers and they don't suck, it'll be a 
>> passing the func in, not exec'ing a strict.
> 
> hmmm... sounds good.  For my understanding, does it mean that the func 
> could be executed based on (trigger) conditions, like ppc-macos or 
> PREFIX != "/", pref.userWantsSheBang == True or something like that?
> 
> 

-- 
Fabian Groffen
Gentoo for Mac OS X
-- 
gentoo-osx@gentoo.org mailing list