[gentoo-user] Per package /bin/sh selection

[gentoo-user] Per package /bin/sh selection

[Alessandro Barbieri]

[-- Attachment #1: Type: text/plain, Size: 136 bytes --]

How can I change /bin/sh in a per package way using /etc/portage/env?
I need to set bash some packages while I run dash for the system.

[-- Attachment #2: Type: text/html, Size: 179 bytes --]

Re: [gentoo-user] Per package /bin/sh selection

[Michael Orlitzky]

On 4/7/20 11:13 AM, Alessandro Barbieri wrote:
> How can I change /bin/sh in a per package way using /etc/portage/env?
> I need to set bash some packages while I run dash for the system.

If it's the ./configure script failing, you can set

  CONFIG_SHELL="/bin/bash"

to override it only in that configure script (but please report a bug
first; configure scripts are supposed to be POSIX sh).

Re: [gentoo-user] Per package /bin/sh selection

[Alessandro Barbieri]

[-- Attachment #1: Type: text/plain, Size: 660 bytes --]

I already filed bugs here:

https://bugs.gentoo.org/716504
https://bugs.gentoo.org/716496

I need the workaround for the two above plus this
https://bugs.gentoo.org/714094

Il Mar 7 Apr 2020, 18:10 Michael Orlitzky <mjo@gentoo.org> ha scritto:

> On 4/7/20 11:13 AM, Alessandro Barbieri wrote:
> > How can I change /bin/sh in a per package way using /etc/portage/env?
> > I need to set bash some packages while I run dash for the system.
>
> If it's the ./configure script failing, you can set
>
>   CONFIG_SHELL="/bin/bash"
>
> to override it only in that configure script (but please report a bug
> first; configure scripts are supposed to be POSIX sh).
>
>

[-- Attachment #2: Type: text/html, Size: 1269 bytes --]

Re: [gentoo-user] Per package /bin/sh selection

[Michael Orlitzky]

On 4/7/20 2:48 PM, Alessandro Barbieri wrote:
> I already filed bugs here:
> 
> https://bugs.gentoo.org/716504
> https://bugs.gentoo.org/716496
> 
> I need the workaround for the two above plus
> this https://bugs.gentoo.org/714094
> 

Thanks, this will be a PITA for a while (again). Another developer had
patched /bin/dash so that it was effectively broken, to the point where
./configure scripts would decide on their own use bash instead (even if
you set /bin/sh to point to dash). This "fixed" the errors, but meant
that everyone who thought he switched to dash was actually still using
slow old bash for every build. I undid that patch (so that now dash acts
like it does upstream), but as a result there's going to be a bunch of
previously-hidden bashisms in ./configure scripts that need to be fixed.

Re: [gentoo-user] Per package /bin/sh selection

[Joerg Schilling]

Michael Orlitzky <mjo@gentoo.org> wrote:

> Thanks, this will be a PITA for a while (again). Another developer had
> patched /bin/dash so that it was effectively broken, to the point where
> ./configure scripts would decide on their own use bash instead (even if
> you set /bin/sh to point to dash). This "fixed" the errors, but meant

Are there no unit tests for dash?

When ever I change something in bosh, I run the unit tests to verify that I did 
not introduce a bug. One of the unit tests is to run a configure and compare 
the results with the results frm a reference shell.

BTW: Did you ever think about replacing dash by bosh?

Bosh has the advantage to support multi byte characters.

Jörg

-- 
 EMail:joerg@schily.net                    (home) Jörg Schilling D-13353 Berlin
    joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL: http://cdrecord.org/private/ http://sf.net/projects/schilytools/files/'

Re: [gentoo-user] Per package /bin/sh selection

[Mike Gilbert]

On Wed, Apr 8, 2020 at 8:44 AM Joerg Schilling
<Joerg.Schilling@fokus.fraunhofer.de> wrote:
>
> Michael Orlitzky <mjo@gentoo.org> wrote:
>
> > Thanks, this will be a PITA for a while (again). Another developer had
> > patched /bin/dash so that it was effectively broken, to the point where
> > ./configure scripts would decide on their own use bash instead (even if
> > you set /bin/sh to point to dash). This "fixed" the errors, but meant
>
> Are there no unit tests for dash?

There do not appear to be any tests in the dash source repo.

https://git.kernel.org/pub/scm/utils/dash/dash.git/

Wikipedia says that dash is a fork of NetBSD's ash, and I do see tests
in their CVS repo. That might be worth looking into.

http://cvsweb.netbsd.org/bsdweb.cgi/src/tests/bin/sh/

> When ever I change something in bosh, I run the unit tests to verify that I did
> not introduce a bug. One of the unit tests is to run a configure and compare
> the results with the results frm a reference shell.
>
> BTW: Did you ever think about replacing dash by bosh?

We use bash as the default /bin/sh, but users are free to replace it
with whatever shell they like, so long as it is reasonably
POSIX-compliant. Other shells are obviously less tested in Gentoo.

Re: [gentoo-user] Per package /bin/sh selection

[Joerg Schilling]

Mike Gilbert <floppym@gentoo.org> wrote:

> Wikipedia says that dash is a fork of NetBSD's ash, and I do see tests
> in their CVS repo. That might be worth looking into.
>
> http://cvsweb.netbsd.org/bsdweb.cgi/src/tests/bin/sh/

I see this is the variant from Rihard Elz, so it may make sense.
The original ash is too buggy as a reference.

> > When ever I change something in bosh, I run the unit tests to verify that I did
> > not introduce a bug. One of the unit tests is to run a configure and compare
> > the results with the results frm a reference shell.
> >
> > BTW: Did you ever think about replacing dash by bosh?
>
> We use bash as the default /bin/sh, but users are free to replace it
> with whatever shell they like, so long as it is reasonably
> POSIX-compliant. Other shells are obviously less tested in Gentoo.

Well, bosh has been tested to work as /bin/sh on Gentoo.
BTW: On Solaris, bosh is faster than dash (because Solaris has a fully working 
vfork()). On Linux bosh is "only" of the same speed as dash since vfork() on 
Linux does not borrow the parents address space description but copies it.

Jörg

-- 
 EMail:joerg@schily.net                    (home) Jörg Schilling D-13353 Berlin
    joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL: http://cdrecord.org/private/ http://sf.net/projects/schilytools/files/'

Re: [gentoo-user] Per package /bin/sh selection

[Mike Gilbert]

On Wed, Apr 8, 2020 at 1:17 PM Joerg Schilling
<Joerg.Schilling@fokus.fraunhofer.de> wrote:
>
> Mike Gilbert <floppym@gentoo.org> wrote:
>
> > Wikipedia says that dash is a fork of NetBSD's ash, and I do see tests
> > in their CVS repo. That might be worth looking into.
> >
> > http://cvsweb.netbsd.org/bsdweb.cgi/src/tests/bin/sh/
>
> I see this is the variant from Rihard Elz, so it may make sense.
> The original ash is too buggy as a reference.
>
> > > When ever I change something in bosh, I run the unit tests to verify that I did
> > > not introduce a bug. One of the unit tests is to run a configure and compare
> > > the results with the results frm a reference shell.
> > >
> > > BTW: Did you ever think about replacing dash by bosh?
> >
> > We use bash as the default /bin/sh, but users are free to replace it
> > with whatever shell they like, so long as it is reasonably
> > POSIX-compliant. Other shells are obviously less tested in Gentoo.
>
> Well, bosh has been tested to work as /bin/sh on Gentoo.
> BTW: On Solaris, bosh is faster than dash (because Solaris has a fully working
> vfork()). On Linux bosh is "only" of the same speed as dash since vfork() on
> Linux does not borrow the parents address space description but copies it.

Is that also true of clone(CLONE_VM|CLONE_VFORK)? Recent versions of
glibc use this to implement the posix_spawn() function.

Re: [gentoo-user] Per package /bin/sh selection

[Joerg Schilling]

Mike Gilbert <floppym@gentoo.org> wrote:

> > Well, bosh has been tested to work as /bin/sh on Gentoo.
> > BTW: On Solaris, bosh is faster than dash (because Solaris has a fully working
> > vfork()). On Linux bosh is "only" of the same speed as dash since vfork() on
> > Linux does not borrow the parents address space description but copies it.
>
> Is that also true of clone(CLONE_VM|CLONE_VFORK)? Recent versions of
> glibc use this to implement the posix_spawn() function.

I guess... from reading the man page.

There is a simple test to verify my clain for vfork():

bosh may easily be configured for various behavior. If you like a bosh that
does not use vfork() just call:

	cd sh
	smake clean
	smake COPTX=-DNO_VFORK

If you run tests like:

	CONFIG_SHELL=$shell $shell ./configure

in the directory "autoconf" from the schilytools:

http://sourceforge.net/projects/schilytools/files/

(first run "rm *cache"), there is a significant performance win on Solaris
for the version that uses vfork() - 20% less kernel CPU time.

On Linux, there is a too little win for the same comparison.

On Solaris, a vfork() call (check with truss -c) is takes only 1/3 of the time 
of a fork(), on Linux the win is only 30%. The total performance win for bosh
executing configure using vfork() on Solaris is 10-15%, on Linux the total win 
is only 3%.

Note that Linux copied the Solaris concept to implement a normal fork() with 
copy on write mmaps to the parent. So the only difference between fork() and 
vfork() is the handling of the address space description for the processes 
in the kernel.

I would be happy, if there could be a way to get the same vfork() performance win 
on Linux as seen on Solaris.

BTW: I forgot to mention that the unit tests for bosh are used this way:

-	first compile schilytools by running smake from top level.

-	Then chdir to "sh" and call "smake tests", see the global code
	in $SRCROOT/tests and the specific code in sh/tests

Jörg

-- 
 EMail:joerg@schily.net                    (home) Jörg Schilling D-13353 Berlin
    joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL: http://cdrecord.org/private/ http://sf.net/projects/schilytools/files/'

[gentoo-user] Re: Per package /bin/sh selection

[Ian Zimmerman]

On 2020-04-08 12:14, Mike Gilbert wrote:

> We use bash as the default /bin/sh, but users are free to replace it
> with whatever shell they like, so long as it is reasonably
> POSIX-compliant. Other shells are obviously less tested in Gentoo.
 
Are .ebuild files always interpreted with bash, or with whatever /bin/sh
points to?  I thought it was the former.

-- 
Ian

Re: [gentoo-user] Re: Per package /bin/sh selection

[Michael Orlitzky]

On 4/8/20 6:44 PM, Ian Zimmerman wrote:
> On 2020-04-08 12:14, Mike Gilbert wrote:
> 
>> We use bash as the default /bin/sh, but users are free to replace it
>> with whatever shell they like, so long as it is reasonably
>> POSIX-compliant. Other shells are obviously less tested in Gentoo.
>  
> Are .ebuild files always interpreted with bash, or with whatever /bin/sh
> points to?  I thought it was the former.
> 

Ebuilds are bash, but the ./configure scripts and makefiles that often
get run within the ebuilds use /bin/sh by default.

[gentoo-user] Re: Per package /bin/sh selection

[Ian Zimmerman]

On 2020-04-08 18:49, Michael Orlitzky wrote:

> Ebuilds are bash, but the ./configure scripts and makefiles that often
> get run within the ebuilds use /bin/sh by default.

I see, but then it is an upstream problem no?  I mean if upstreams keep
putting bashisms in their build systems they will keep doing it no
matter how clean the gentoo dash package is.

-- 
Ian

Re: [gentoo-user] Re: Per package /bin/sh selection

[Michael Orlitzky]

On 4/10/20 10:29 PM, Ian Zimmerman wrote:
> On 2020-04-08 18:49, Michael Orlitzky wrote:
> 
>> Ebuilds are bash, but the ./configure scripts and makefiles that often
>> get run within the ebuilds use /bin/sh by default.
> 
> I see, but then it is an upstream problem no?  I mean if upstreams keep
> putting bashisms in their build systems they will keep doing it no
> matter how clean the gentoo dash package is.
> 

Yup. Ideally they will fix the bashisms, but if not and they decide only
to support bash, they should still set SHELL=bash or something like that
instead of just praying that /bin/sh points there.

Re: [gentoo-user] Per package /bin/sh selection

[Mike Gilbert]

On Tue, Apr 7, 2020 at 2:48 PM Alessandro Barbieri
<lssndrbarbieri@gmail.com> wrote:
>
> I already filed bugs here:
>
> https://bugs.gentoo.org/716504
> https://bugs.gentoo.org/716496
>
> I need the workaround for the two above plus this https://bugs.gentoo.org/714094

In the future, please mention you are using dash as /bin/sh and block
bug 714092 when you file bugs about this kind of issue. The fontforge
bug report doesn't mention anything about it.