[gentoo-dev] initscripts in python

[gentoo-dev] initscripts in python

[Justin Whitney]

Hi,

I don't really like talking about these kinds of vague ideas (like I'm
doing here), because that doesn't get them written - but this one I want
feedback on first before I dive in... so...

For a lot of reasons I'd like to implement the initscrits in something
other than shell script.  Something like python, say.

Reasons for doing this would include:

*writing (advanced) shellscripts requires learning awk/sed, and various
other minor tools (mostly because their features aren't supported by the
language).  Use of a language with these features builtin lowers the
learning requirement, or at least puts it all under one roof.
*improved performance and bytecode-compilability
*Speedups due to fewer exec calls (for awk/sed/etc)

Reasons NOT to do this would include:

*breaking from standard would mean packages with provided initscripts
would require a rewrite.
*slight increase in boot requirements (interpreter and libs must exist
at least minimally on root partition)
*probably needs a bit more memory

other bits:

*compatibility could obviously be maintained, as existing shell scripts
could still be run without changes.


Note: I am by no means proposing this as a standard feature of gentoo.

That said, since gentoo already uses python for portage, selecting
python as the language to use makes sense.  Aside from the re-writes,
and some other details, I don't see much disadvantage to the above
design.

Comments appreciated.

--Justin Whitney


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] initscripts in python

[Daniel Armyr]

I definately like the idea, especially since I am one of those who knows 
most programming-languages, but hasn't a clue about how to use all those 
wierd little mini-programs.
However, I am afraid  the cons you proposed are quite substantial and 
cannot be overlooked.
//Daniel Armyr

Justin Whitney wrote:

>Hi,
>
>I don't really like talking about these kinds of vague ideas (like I'm
>doing here), because that doesn't get them written - but this one I want
>feedback on first before I dive in... so...
>
>For a lot of reasons I'd like to implement the initscrits in something
>other than shell script.  Something like python, say.
>
>Reasons for doing this would include:
>
>*writing (advanced) shellscripts requires learning awk/sed, and various
>other minor tools (mostly because their features aren't supported by the
>language).  Use of a language with these features builtin lowers the
>learning requirement, or at least puts it all under one roof.
>*improved performance and bytecode-compilability
>*Speedups due to fewer exec calls (for awk/sed/etc)
>
>Reasons NOT to do this would include:
>
>*breaking from standard would mean packages with provided initscripts
>would require a rewrite.
>*slight increase in boot requirements (interpreter and libs must exist
>at least minimally on root partition)
>*probably needs a bit more memory
>
>other bits:
>
>*compatibility could obviously be maintained, as existing shell scripts
>could still be run without changes.
>
>
>Note: I am by no means proposing this as a standard feature of gentoo.
>
>That said, since gentoo already uses python for portage, selecting
>python as the language to use makes sense.  Aside from the re-writes,
>and some other details, I don't see much disadvantage to the above
>design.
>
>Comments appreciated.
>
>--Justin Whitney
>
>
>--
>gentoo-dev@gentoo.org mailing list
>
>
>
>  
>


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] initscripts in python

[merv]

I for one would enjoy the clean-feel of Python in initscrit composition.
That said, I would miss the very "Unix atmospherics" of awk and sed 
and the run-time interactivity of the shell scripting (I mean, it's potential 
for shorthand notation).

It may be a difficult one to call though.  I imagine that many would feel 
similar sentiments about the scripting advantages of both Python and 
shell scripting.  Maybe practicality would have the last say, however. 
On machines with very limited resources (I have a box running Gentoo 
on a Pentium 133Mhz and < 1GB harddrive with 64Mb RAM), Python 
would be a death sentence for the box.  Shell scripting is the only 
option in such environments for job control etc etc.

Perhaps we might experiment with a build-time option for incorporating 
*both* Python and shell scripting tools such as awk and sed in to the 
shell (by having the Python *shell extras* turned on/off via a 
compilation flag?).  In such environments both would be available for 
use as a shell scripting resource.  Or perhaps again, we might set 
about incorporating shell scripting notation in to Python and attempt to 
ferment a hybrid for shell scripting purposes, initscrits and the like.

Interesting idea that has made me think more closely about what a 
shell scripting language is and what it is for.

Will be good to see if this goes anywhere.

--Merv Hammer

On 15 Apr 2003 at 21:29, Justin Whitney wrote:

> Hi,
> 
> I don't really like talking about these kinds of vague ideas (like I'm
> doing here), because that doesn't get them written - but this one I want
> feedback on first before I dive in... so...
> 
> For a lot of reasons I'd like to implement the initscrits in something
> other than shell script.  Something like python, say.
> 
> Reasons for doing this would include:
> 
> *writing (advanced) shellscripts requires learning awk/sed, and various
> other minor tools (mostly because their features aren't supported by the
> language).  Use of a language with these features builtin lowers the
> learning requirement, or at least puts it all under one roof.
> *improved performance and bytecode-compilability
> *Speedups due to fewer exec calls (for awk/sed/etc)
> 
> Reasons NOT to do this would include:
> 
> *breaking from standard would mean packages with provided initscripts
> would require a rewrite.
> *slight increase in boot requirements (interpreter and libs must exist
> at least minimally on root partition)
> *probably needs a bit more memory
> 
> other bits:
> 
> *compatibility could obviously be maintained, as existing shell scripts
> could still be run without changes.
> 
> 
> Note: I am by no means proposing this as a standard feature of gentoo.
> 
> That said, since gentoo already uses python for portage, selecting
> python as the language to use makes sense.  Aside from the re-writes,
> and some other details, I don't see much disadvantage to the above
> design.
> 
> Comments appreciated.
> 
> --Justin Whitney
> 
> 
> --
> gentoo-dev@gentoo.org mailing list
> 


--
Merv Hammer
mailto: merv@spidernet.com.cy
------------------------------------------------------------------------------
-Working with Unix is like wrestling a worthy opponent... 
-Working with windows is like attacking a small whining child 
 who is carrying a .38

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] initscripts in python

[Jeff Rose]

Could you use a hybrid of both bash scripts and python by using the
'-c' argument to run single python commands?

-Jeff

On Wed, 16 Apr 2003, merv wrote:

> I for one would enjoy the clean-feel of Python in initscrit composition.
> That said, I would miss the very "Unix atmospherics" of awk and sed
> and the run-time interactivity of the shell scripting (I mean, it's potential
> for shorthand notation).
>
> It may be a difficult one to call though.  I imagine that many would feel
> similar sentiments about the scripting advantages of both Python and
> shell scripting.  Maybe practicality would have the last say, however.
> On machines with very limited resources (I have a box running Gentoo
> on a Pentium 133Mhz and < 1GB harddrive with 64Mb RAM), Python
> would be a death sentence for the box.  Shell scripting is the only
> option in such environments for job control etc etc.
>
> Perhaps we might experiment with a build-time option for incorporating
> *both* Python and shell scripting tools such as awk and sed in to the
> shell (by having the Python *shell extras* turned on/off via a
> compilation flag?).  In such environments both would be available for
> use as a shell scripting resource.  Or perhaps again, we might set
> about incorporating shell scripting notation in to Python and attempt to
> ferment a hybrid for shell scripting purposes, initscrits and the like.
>
> Interesting idea that has made me think more closely about what a
> shell scripting language is and what it is for.
>
> Will be good to see if this goes anywhere.
>
> --Merv Hammer
>
> On 15 Apr 2003 at 21:29, Justin Whitney wrote:
>
> > Hi,
> >
> > I don't really like talking about these kinds of vague ideas (like I'm
> > doing here), because that doesn't get them written - but this one I want
> > feedback on first before I dive in... so...
> >
> > For a lot of reasons I'd like to implement the initscrits in something
> > other than shell script.  Something like python, say.
> >
> > Reasons for doing this would include:
> >
> > *writing (advanced) shellscripts requires learning awk/sed, and various
> > other minor tools (mostly because their features aren't supported by the
> > language).  Use of a language with these features builtin lowers the
> > learning requirement, or at least puts it all under one roof.
> > *improved performance and bytecode-compilability
> > *Speedups due to fewer exec calls (for awk/sed/etc)
> >
> > Reasons NOT to do this would include:
> >
> > *breaking from standard would mean packages with provided initscripts
> > would require a rewrite.
> > *slight increase in boot requirements (interpreter and libs must exist
> > at least minimally on root partition)
> > *probably needs a bit more memory
> >
> > other bits:
> >
> > *compatibility could obviously be maintained, as existing shell scripts
> > could still be run without changes.
> >
> >
> > Note: I am by no means proposing this as a standard feature of gentoo.
> >
> > That said, since gentoo already uses python for portage, selecting
> > python as the language to use makes sense.  Aside from the re-writes,
> > and some other details, I don't see much disadvantage to the above
> > design.
> >
> > Comments appreciated.
> >
> > --Justin Whitney
> >
> >
> > --
> > gentoo-dev@gentoo.org mailing list
> >
>
>
> --
> Merv Hammer
> mailto: merv@spidernet.com.cy
> ------------------------------------------------------------------------------
> -Working with Unix is like wrestling a worthy opponent...
> -Working with windows is like attacking a small whining child
>  who is carrying a .38
>
> --
> gentoo-dev@gentoo.org mailing list
>


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] initscripts in python

[Paul de Vrieze]

[-- Attachment #1: signed data --]
[-- Type: text/plain, Size: 728 bytes --]

On Wednesday 16 April 2003 07:38, merv wrote:
> I for one would enjoy the clean-feel of Python in initscrit composition.
> That said, I would miss the very "Unix atmospherics" of awk and sed
> and the run-time interactivity of the shell scripting (I mean, it's
> potential for shorthand notation).
>

As most init scripts are essentially nobrainer oneliners, I believe using 
anything but sh is overkill. In the cases where something special is required 
it is very well possible to use python/any other language either embedded or 
instead of the standard interpreter. (You could even make init scripts in c)

Paul

-- 
Paul de Vrieze
Researcher
Mail: pauldv@cs.kun.nl
Homepage: http://www.cs.kun.nl/~pauldv

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] initscripts in python

[merv]

Sure could. Can see the logic of the argument.  And then there could 
be the implementation of a "*.psh" for a python-lang shell script.  Just 
ideas, but this could be an option too. Then we are starting to address 
script files and runtime shell interaction.

Anyone have any thoughts on how to get the shell to accept Python 
commands/arguments/statements and what-not in runtime from the 
prompt?  Would it be a big undertaking to use C-level translation of a 
new set of Python macros/symbols passed to the the kernel?  I 
suppose that a decision at that point would involve some consideration 
of whether the objective was to incorporate the Python languages itself 
(to greater or lesser extent) into the shell  or whether it was the syntax 
and structure of Python that was to be *lifted*.  Another way to see it 
would be the potential for bloating scripts with *import*s as a result of 
Python languages transplantation on the one hand, while having such 
useful things as f.tell() and f.seek() available on the other hand.  If the 
syntax were incorporated into the shell, would there be a neat way of 
by-passing the need for a full-blown python interpreter?  I think this 
could be worth some serious attention, but I just wonder if anyone has 
any cool ideas or some time to share on this admittedly vague (so far) 
possibility.



On 16 Apr 2003 at 1:25, Jeff Rose wrote:

> Could you use a hybrid of both bash scripts and python by using the
> '-c' argument to run single python commands?
> 
> -Jeff


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] initscripts in python

[Sven Vermeulen]

[-- Attachment #1: Type: text/plain, Size: 1403 bytes --]

On Wed, Apr 16, 2003 at 10:19:31AM +0200, Paul de Vrieze wrote:
> As most init scripts are essentially nobrainer oneliners, I believe using 
> anything but sh is overkill. In the cases where something special is required 
> it is very well possible to use python/any other language either embedded or 
> instead of the standard interpreter. (You could even make init scripts in c)

I agree. The cons for using Python-based initscripts are too important. 

First of all it's important that the system still boots, even when Python
doesn't work due to a bad ebuild or any other circumstance. Saying that this
is the same for when the shell itself doesn't work isn't true, since you'll
need the shell in order to fix the problem. You don't need Python.

Second is that there are lots of users that can shellscript (it's not that
difficult) but who don't know how to code in Python. They'll seriously object
when Gentoo would start using Python initscripts.

This would also mean that Python should be moved from /usr/lib to /lib,
increasing the size of the / partition and removing the possibility for
shared Python in a network (admin POV).

I also think that the initialisation time would increase, leaving us with no
real advantage whatsoever. 

If you want speed, just code them in C and compile :)

	Sven Vermeulen

-- 
	Fighting for peace is like fucking for virginity.

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] initscripts in python

[merv]

Yep, one of the things that would be hard to do would persuade many 
of either the *need* for this or of its advantage.

To move away from pros and cons for a moment, I see a need and an 
advantage :

[*need* is a word treated as meaning a potential *use for* something, 
for now, a bit elastic perhaps]:

1) [need or use:] by incorporating Python functionality (or other 
programming lang) into a more extensible style shell, sh becomes 
more widely accessibly to a number of community member who have 
minimal->no sh scripting ability, but who are familiar with programming 
of one flavour or another. I would have to agree, and say that my first 
impulse is to just encourage them to learn to *bash* or whatever, but 
everyone has their reasons.  If a user was able to incorporate the 
functionality (but not the whole language) of a lang with which they 
were familiar (or their favourite lang) at build time, shell scripting would 
become a common language for even more people of the Gentoo 
family.

2)[advantage or effect:] to undertake such a re-write of the way that the 
shell is built would, conceivably, result in a change in the way that the 
community thinks of the shell.  Perhaps shell extensibility (for which I 
mean a kind of individual user-level customisation during build) would 
result in the shell itself being perceived as a kind of IDE or runtime 
environment, as well as a way to get around and make things happen.  
This kind of *shell development* might act as a corollary to kernel 
development. If kernel development is seen as cultivating and 
extending the functionality and capability of the kernel, then perhaps 
shell development could be seen as refining and extending the 
adaptability and usability of the shell at the level of the individual user, 
providing greater choice and customisability.  Would this not be a 
natural implementation of the kind of hyper-tweak choice-driven 
philosophy of Gentoo?

and of course, any extensibility options to be built in at build-time could 
of course be ommitted (packaged as *default off*), only turned *on* by 
those who deliberately wanted to make use of it.

It has always felt to me that the power of sh scripting was its flexibility. 
To extend the functionality of the sh-scripting env in this way would 
consolidate, in my view, the usefulness of sh-scripting by enhancing it, 
or at least extending it for those who wanted to make use of it.


>SH-scripting is in itself already very powerfull. It's not because lots of
>initscripts are bad written (because of historic reasons) that it is to 
>blame
>on sh. You can create functions with sh. You can do so many 
>powerfull things
>with it (no, object oriented programming it cannot, but that's not 
>powerfull
>either :)
>
>http://www.tldp.org/LDP/abs/html/complexfunct.html
>		
>Not that I dislike Python, au contraire, I like the immens easiness for
>parsing XML with it, controlling databases with the DB-API... but 
>initscripts
>don't require this possibility.
>
>	Sven Vermeulen
--
	Fighting for peace is like fucking for virginity.
-
Merv Hammer
mailto: merv@spidernet.com.cy

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] initscripts in python

[Justin Whitney]

Thanks for the arguments thus far.

So added to the cons list is:

*addition of newer less-tested interpreter could add instability
*very small/short scripts incur overhead

Here are two mitigating arguments to counter the first statement.

First - a separate, stable build of python could be used as the
system-boot interpreter.  it might make sense to statically link it and
so forth to lower dependencies... perhaps even to optimize it in some
respects, for its task.

Second, a shell script like the runscript.sh that is already used could
be used to launch the interpreter and check for a runtime failure.  On
failure it could launch the .sh version of the bootscript.  This means
keeping two copies of the shellscripts obviously (which wouldn't be a
bad idea anyway...)  this would at least have the benefit of salvaging
the boot...

As for the overhead problem with very small scripts, I don't see that
going away... in fact converting such scripts would be a waste of time
in the first place, because it doesn't make use of any of the advantages
proposed by the change.  It would make more sense to set up the boot
process to be able to run both kinds so you don't waste your time.

--Justin


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] initscripts in python

[Brad Laue]

On Tue, 2003-04-15 at 21:29, Justin Whitney wrote:
> Reasons for doing this would include:
> 
> *writing (advanced) shellscripts requires learning awk/sed, and various
> other minor tools (mostly because their features aren't supported by the
> language).  Use of a language with these features builtin lowers the
> learning requirement, or at least puts it all under one roof.
> *improved performance and bytecode-compilability
> *Speedups due to fewer exec calls (for awk/sed/etc)
> 

My main problem with this is an apparent flaw in logic; one has to learn
python in addition to sh now, although sh scripts will be a normal part
of UNIX administration for a long time to come. As part of learning sh,
one masters the associated tools (awk/sed as mentioned examples).

Adding python syntax to the mix increases the learning curve for not
much benefit to an existing system (I'm not so sure there would be a
speedup either).

Might be interesting to see as a practical demonstration, though.

Brad
--
// -- http://www.BRAD-X.com/ -- //


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] initscripts in python

[Abhishek Amit]

On 08:38 Wed 16 Apr     , merv wrote:
> It may be a difficult one to call though.  I imagine that many would feel 
> similar sentiments about the scripting advantages of both Python and 
> shell scripting.  Maybe practicality would have the last say, however. 
> On machines with very limited resources (I have a box running Gentoo 
> on a Pentium 133Mhz and < 1GB harddrive with 64Mb RAM), Python 
> would be a death sentence for the box.  Shell scripting is the only 
> option in such environments for job control etc etc.
> 
If you run gentoo python is a requirment that you can not avoid in any
way. Portage is completly python. So I don't really see how python could
be too heavy unless you mean specifically for running these scripts.


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] initscripts in python

[Joseph Carter]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wed, Apr 16, 2003 at 10:19:31AM +0200, Paul de Vrieze wrote:
> > I for one would enjoy the clean-feel of Python in initscrit composition.
> > That said, I would miss the very "Unix atmospherics" of awk and sed
> > and the run-time interactivity of the shell scripting (I mean, it's
> > potential for shorthand notation).
> 
> As most init scripts are essentially nobrainer oneliners, I believe using 
> anything but sh is overkill. In the cases where something special is required 
> it is very well possible to use python/any other language either embedded or 
> instead of the standard interpreter. (You could even make init scripts in c)

I don't know..  Python has this great feature that sh does not: it byte
compiles its scripts and will execute the pre-compiled one if it doesn't
have to do anything else.  If everything required to boot a Gentoo system
were done in Python thusly, it'd boot pretty fast.  (This I consider a
very worthwhile goal!)

- -- 
Joseph Carter <knghtbrd@efn.org>                    Only l33t on Thursdays
 
<stu> apt: !bugs
<apt> !bugs are stupid
<dpkg> apt: are stupid?  what's that?
<apt> dpkg: i don't know
<dpkg> apt: Beauty is in the eye of the beer holder...
<apt> i already had it that way, dpkg.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)
Comment: 1024D/20F62261F1857A3E79FC44F98FF7D7A3DCF9DAB3

iEYEARECAAYFAj6eQGAACgkQj/fXo9z52rNxIQCgi2k1eB+6GldIRHEL8JIO2I+o
aEoAoLCn/9qV8erbx2QreXptUfSQZWOX
=kAEI
-----END PGP SIGNATURE-----

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] initscripts in python

[George Shapovalov]

Though I still fail to see grand benefits, I must admit this is a neat little 
idea :) (perheaps more of an excersise kind though).
A glimpse into possible implementation immediately makes me think about using 
"appropriate" shell for the task, the one "designed to blend a traditional 
shell with the power of Python". A short search on google turns up two such 
projects:
http://pysh.sourceforge.net/  and
http://sourceforge.net/projects/pyshell/

Though looks like both weren't active for quite some time lately...
So, if anybody is motivated anough, I guess that person can try to pick-up 
wherever those teams left it ;). Then its a matter of selecting appropriate 
shell as your "basic" one and picking the corresponding baselayout (and 
creating both + the appropriate profile specification of curse :)).

While still not answering why we would want init scripts in python, this can 
at least create on its way something to stay :).

Anyway, just my 0.02$ ;).

George


On Wednesday 16 April 2003 22:49, Joseph Carter wrote:
> On Wed, Apr 16, 2003 at 10:19:31AM +0200, Paul de Vrieze wrote:
> > > I for one would enjoy the clean-feel of Python in initscrit
> > > composition. That said, I would miss the very "Unix atmospherics" of
> > > awk and sed and the run-time interactivity of the shell scripting (I
> > > mean, it's potential for shorthand notation).
> >
> > As most init scripts are essentially nobrainer oneliners, I believe using
> > anything but sh is overkill. In the cases where something special is
> > required it is very well possible to use python/any other language either
> > embedded or instead of the standard interpreter. (You could even make
> > init scripts in c)
>
> I don't know..  Python has this great feature that sh does not: it byte
> compiles its scripts and will execute the pre-compiled one if it doesn't
> have to do anything else.  If everything required to boot a Gentoo system
> were done in Python thusly, it'd boot pretty fast.  (This I consider a
> very worthwhile goal!)


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] initscripts in python

[Sven Vermeulen]

[-- Attachment #1: Type: text/plain, Size: 798 bytes --]

On Wed, Apr 16, 2003 at 03:44:28PM -0700, Abhishek Amit wrote:
> If you run gentoo python is a requirment that you can not avoid in any
> way. Portage is completly python. So I don't really see how python could
> be too heavy unless you mean specifically for running these scripts.

Portage is completely python, yes, but you cannot know what the future will
hold. I've personally already been working on a Gentoo-without-Portage (well,
if you can still call it Gentoo that is :) for fun.

Creating the initscripts in Python will force you to stick with Python as a
primary dependency, even if the Gentoo developers someday would say "hey,
perhaps we can make a Gentoo-project with a C-coded Portage".

Wkr,
	Sven Vermeulen

-- 
	Fighting for peace is like fucking for virginity.

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] initscripts in python

[Sven Vermeulen]

[-- Attachment #1: Type: text/plain, Size: 308 bytes --]

On Wed, Apr 16, 2003 at 05:08:39PM -0400, Brad Laue wrote:
> Might be interesting to see as a practical demonstration, though.

I'd second this. I'm not convinced wrt the speedgain, but a live
demonstration could prove it :)

	Sven Vermeulen

-- 
	Fighting for peace is like fucking for virginity.

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] initscripts in python

[Paul de Vrieze]

[-- Attachment #1: signed data --]
[-- Type: text/plain, Size: 780 bytes --]

On Thursday 17 April 2003 07:49, Joseph Carter wrote:
>
> I don't know..  Python has this great feature that sh does not: it byte
> compiles its scripts and will execute the pre-compiled one if it doesn't
> have to do anything else.  If everything required to boot a Gentoo system
> were done in Python thusly, it'd boot pretty fast.  (This I consider a
> very worthwhile goal!)

The time in the init scripts is not spend actually waiting for bash to 
evaluate the script. It is in actually starting the applications. Some have 
builtin delays, and some are just big. Python would not help that at all. The 
only thing that would help are paralel init scripts.

Paul

-- 
Paul de Vrieze
Researcher
Mail: pauldv@cs.kun.nl
Homepage: http://www.cs.kun.nl/~pauldv

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] initscripts in python

[merv]

hi

for the box I mentioned,  I have not installed the portage tree.  After 
Stage1 I kinda lept sideways and continued manually (not emerging) 
building packages (of which there were very few) by compiling on 
another machine and then network-installing to the little box (i did 
install gcc in the little box to ensure maximal optimisation compile 
time).  I don't have the spare capacity for the portage tree since 
whatever spare space there is goes to the logging files of the firewalls 
it is now running.

On 16 Apr 2003 at 15:44, Abhishek Amit wrote:

> On 08:38 Wed 16 Apr     , merv wrote:
> > It may be a difficult one to call though.  I imagine that many would feel 
> > similar sentiments about the scripting advantages of both Python and 
> > shell scripting.  Maybe practicality would have the last say, however. 
> > On machines with very limited resources (I have a box running Gentoo 
> > on a Pentium 133Mhz and < 1GB harddrive with 64Mb RAM), Python 
> > would be a death sentence for the box.  Shell scripting is the only 
> > option in such environments for job control etc etc.
> > 
> If you run gentoo python is a requirment that you can not avoid in any
> way. Portage is completly python. So I don't really see how python could
> be too heavy unless you mean specifically for running these scripts.
> 
> 
> --
> gentoo-dev@gentoo.org mailing list
> 


--
Merv Hammer
mailto: merv@spidernet.com.cy

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] initscripts in python

[merv]

I agree,  it would be more of an exercise, first off.

IMHO, i feel that the concept of being able to select shell runtime 
variables such as script language or syntax would absolutely open up 
new possibilities.  I agree with those who argue that any so-called 
"gain" would be debateable, but surely three important gains would be 
these:

1) gives more choice to fellow gentooers
2) is an exercise in rethinking things that are almost Unix "lore"
3) depending on the degree of success, we'll know it can be done!

Thanks for the links too.

On 16 Apr 2003 at 23:27, George Shapovalov wrote:

> Though I still fail to see grand benefits, I must admit this is a neat little 
> idea :) (perheaps more of an excersise kind though).
> A glimpse into possible implementation immediately makes me think about using 
> "appropriate" shell for the task, the one "designed to blend a traditional 
> shell with the power of Python". A short search on google turns up two such 
> projects:
> http://pysh.sourceforge.net/  and
> http://sourceforge.net/projects/pyshell/
> 
> Though looks like both weren't active for quite some time lately...
> So, if anybody is motivated anough, I guess that person can try to pick-up 
> wherever those teams left it ;). Then its a matter of selecting appropriate 
> shell as your "basic" one and picking the corresponding baselayout (and 
> creating both + the appropriate profile specification of curse :)).
> 
> While still not answering why we would want init scripts in python, this can 
> at least create on its way something to stay :).
> 
> Anyway, just my 0.02$ ;).
> 
> George
> 
> 
> On Wednesday 16 April 2003 22:49, Joseph Carter wrote:
> > On Wed, Apr 16, 2003 at 10:19:31AM +0200, Paul de Vrieze wrote:
> > > > I for one would enjoy the clean-feel of Python in initscrit
> > > > composition. That said, I would miss the very "Unix atmospherics" of
> > > > awk and sed and the run-time interactivity of the shell scripting (I
> > > > mean, it's potential for shorthand notation).
> > >
> > > As most init scripts are essentially nobrainer oneliners, I believe using
> > > anything but sh is overkill. In the cases where something special is
> > > required it is very well possible to use python/any other language either
> > > embedded or instead of the standard interpreter. (You could even make
> > > init scripts in c)
> >
> > I don't know..  Python has this great feature that sh does not: it byte
> > compiles its scripts and will execute the pre-compiled one if it doesn't
> > have to do anything else.  If everything required to boot a Gentoo system
> > were done in Python thusly, it'd boot pretty fast.  (This I consider a
> > very worthwhile goal!)
> 
> 
> --
> gentoo-dev@gentoo.org mailing list
> 


--
Merv Hammer
mailto: merv@spidernet.com.cy

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] initscripts in python

[merv]

Yep, I agree on your two cons:

->a new interpreter would have to reach stable pretty quick since we 
are mucking around with pretty system-vital stuff here (and since we 
are using python syntax [in this hypothetical instance] to message to 
the kernel, a new interpreter and a fairly fundamental re-write of the 
shell may be the end result, though it might be politic not to revise what 
was not necessary to revise)

->the structure of the interpreter and ways in which it parse shell syntax 
of a python dialect would have to be addressed with the aim of 
minimising overhead on one-liner files, in particular (small scripts incur 
overhead, unavoidably, at least conventionally, but lots of small 
python-interpreted scripts might have real potential for runtime bloat)

About your two mitigating factors:  indeed, python seems to be 
evolving as a native gentoo language (perhaps this is a bit overstating 
the point, given that python is built on C APIs, but consider that portage 
is a central characteristic and feature of gentoo and that the portage 
tree is a testimony to the practical usefulness and potential of python) 
so to weave python or python-dialect syntax into a "system-boot 
interpreter" would be a natural extension of its role in gentoo and, if 
optimised for that purpose at that point, perhaps even a fast, efficient 
and inexpensive alternative.

[Maybe it is worth saying at this point that I am trying very deliberately 
to drive a line between python the language as it is and python-
language characteristics, such as its clean syntax, it "everything is an 
object" approach to life and so on which make it very attractive for 
"collective" development where code-readability and recycleability 
really begin to matter.  I am not suggesting, at this point, a wholesale 
transplant of the python language to sh but ways of building-in some of 
its features or characteristics to ADD TO sh, to be an addition, rather 
than replace sh].

On the issue you raise in your second mitigating factor, a kind of 
runtime boot-failover could be one example of the things made 
possible by introducing a more "extensible" shell.

Just to note: i envisage several levels of control or options.  There 
would, perhaps, be an initial build-time option to include python as a 
part of an "actively extensible" shell. sh would be default and required.  
python could then be toggled on and off at any time during by the use 
of command line switches or arguments passed with shell commands. 
additionally, optimisation switches could be passed as arguments to 
shell commands, or in script files, that enable the python shell syntax 
to behave exactly as required in given conditions or environments. For 
example, (and these lame suggestions are only for illustrating my 
point):

pye -a			to activate a python-shell environment
pye -k			to kill it and return to sh
pye -a -psh		to activate a mixed sh/python shell environment
pye -k  psh		to kill the mixed environment and return to sh
pye -k  psh -p		to kill the mixed environment but allow python 
    shell to persist 
pye -a -swoff		to activate the python shell environment 
    (when swap off) except when swap is activated 
    and performance might be an issue (if the 
    python shell environment proved to have 
    performance issues)

etc etc etc...you see my point

of course, the python shell environment could be invoked and killed in 
scripts in similar ways.

maybe we would in such cases be faced with the infinitive question of 
whether to trade performance for functionality, vice versa, or whether 
the trade-off in itself was necessary

the end result would be a new way to think of and use the shell

but now i pontificate :)

On 16 Apr 2003 at 16:54, Justin Whitney wrote:

> Thanks for the arguments thus far.
> 
> So added to the cons list is:
> 
> *addition of newer less-tested interpreter could add instability
> *very small/short scripts incur overhead
> 
> Here are two mitigating arguments to counter the first statement.
> 
> First - a separate, stable build of python could be used as the
> system-boot interpreter.  it might make sense to statically link it and
> so forth to lower dependencies... perhaps even to optimize it in some
> respects, for its task.
> 
> Second, a shell script like the runscript.sh that is already used could
> be used to launch the interpreter and check for a runtime failure.  On
> failure it could launch the .sh version of the bootscript.  This means
> keeping two copies of the shellscripts obviously (which wouldn't be a
> bad idea anyway...)  this would at least have the benefit of salvaging
> the boot...
> 
> As for the overhead problem with very small scripts, I don't see that
> going away... in fact converting such scripts would be a waste of time
> in the first place, because it doesn't make use of any of the advantages
> proposed by the change.  It would make more sense to set up the boot
> process to be able to run both kinds so you don't waste your time.
> 
> --Justin
> 
> 
> --
> gentoo-dev@gentoo.org mailing list
> 


--
Merv Hammer
mailto: merv@spidernet.com.cy

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] initscripts in python

[merv]

I can see your point.  But there would, I believe, be very deliberate 
measures taken to ensure that the use of python in initscrits, since this 
is how the discussion started, did not result in a kind of python lock-
down of Gentoo.

>From my own point of view, the underlying theme of the idea is the 
concept of a more "extensible" shell.  Sh would remain the default 
required language of the shell environment.  It is a poewrful and widely 
shared tool and remove it would be self-defeating.  But the optional 
inclusion of the Python-style shell syntax or dialect at build time, and 
later thru interactive activation/deactivation switches at the prompt and 
in scripts, would mean that python shell environment could be switched 
on and off at will, could be used in a mixed environment with sh, and of 
course could be unmerged and compiled out at any time.  That the 
portage tree should have been written in python is perhaps a sign of 
Gentoo's progressive streak, but yes, non-python portage is a real 
possibility as could multi-language, selectable portage in the future.

The issue is not really about using Python per se.  Python is already a 
large part of Gentoo and is, in many senses, relatively accessible to 
the wider community (I know that argument is all much of a 
muchness).  Maybe the extensible shell could evolve to provide 
support for other languages.  As an anchor in our debate, and as a 
reference, isn't tcsh an example of the spirit of an extensible shell?  
Here we have C-like syntax operating at the shell level messaging the 
kernel.  Performance benefits of the C language itself on one side, isn't 
this (in a more developed [meaning, dynamic rather than static] form) 
the kind ofthing we are talking about?

Would not the future look, in these circumstances, something like a 
world in which the gentooer, at build time, could select to have a shell 
(with default sh) augmented with one or more language-dialects 
(syntax libraries, say Perl, Python and Ruby, or even JS and 
PHP...what the heck?!) of his/her choice for interactive use in the shell 
environment?

On 17 Apr 2003 at 8:45, Sven Vermeulen wrote:

> On Wed, Apr 16, 2003 at 03:44:28PM -0700, Abhishek Amit wrote:
> > If you run gentoo python is a requirment that you can not avoid in any
> > way. Portage is completly python. So I don't really see how python could
> > be too heavy unless you mean specifically for running these scripts.
> 
> Portage is completely python, yes, but you cannot know what the future will
> hold. I've personally already been working on a Gentoo-without-Portage (well,
> if you can still call it Gentoo that is :) for fun.
> 
> Creating the initscripts in Python will force you to stick with Python as a
> primary dependency, even if the Gentoo developers someday would say "hey,
> perhaps we can make a Gentoo-project with a C-coded Portage".
> 
> Wkr,
> 	Sven Vermeulen
> 
> -- 
> 	Fighting for peace is like fucking for virginity.
> 


--
Merv Hammer
mailto: merv@spidernet.com.cy

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] initscripts in python

[Caleb Shay]

On Thu, 2003-04-17 at 16:56, merv wrote:
<snip>

> 
> >From my own point of view, the underlying theme of the idea is the 
> concept of a more "extensible" shell. 

<snip>

I saw this line and thought "A plugin extensible shell, how cool would
that be?  You want perl regexps?  Load the preg extension.  Don't use
tab-completion?  Don't load the tab-completion extension and save some
ram.  A quick google turns up:

http://sourceforge.net/projects/dxsh/

Doesn't look like it's seen much activity lately, but what a great idea!

Caleb




--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] initscripts in python

[merv]

thanks for the link and the encouragement

i have been in this discussion now for a couple of days (since the first 
post entitled "initscripts in python") and am preparing to sit down and 
plan this "extensible shell plugin" somewhat more deliberately and 
carefully.

several people have posted links to previous attempts/projects at this 
kind of things and all of them seem to have fizzled out over time.  of 
course this is part of the nature of community development but 
perhaps it is also an indication of the size of this task.

I am a C programmer by profession so I guess that's where I'll start but 
I want to build in Python/C APIs as early as possible since Python has 
been the topic of the discussion and seems a particularly good place to 
start in view of its syntactical/semantic "cleanness", besides being 
Gentoo fodder.  I intend very much that an extensible shell with Python 
plugin should be a testing bed and that others will follow, including 
Perl, Ruby etc etc ... whatever there is a demand/need/curiosity for.  
And the implementation will need breaking down into build-
time/emerge packaging, initscripts/boot-time related code, extensible 
(dynamically loadable) shell environment code, script-
loading/reading/interpreting code, etc etc etc

watch this space...patiently (I have four babies and seven kittens 
besides a full time job :) )

"IDEAS MAKE SOFTWARE BEAUTIFUL" - so keep 'em comin' !

On 17 Apr 2003 at 18:26, Caleb Shay wrote:

> On Thu, 2003-04-17 at 16:56, merv wrote:
> <snip>
> 
> > 
> > >From my own point of view, the underlying theme of the idea is the 
> > concept of a more "extensible" shell. 
> 
> <snip>
> 
> I saw this line and thought "A plugin extensible shell, how cool would
> that be?  You want perl regexps?  Load the preg extension.  Don't use
> tab-completion?  Don't load the tab-completion extension and save some
> ram.  A quick google turns up:
> 
> http://sourceforge.net/projects/dxsh/
> 
> Doesn't look like it's seen much activity lately, but what a great idea!
> 
> Caleb
> 
> 
> 
> 
> --
> gentoo-dev@gentoo.org mailing list
> 


--
Merv Hammer
mailto: merv@spidernet.com.cy

--
gentoo-dev@gentoo.org mailing list