[gentoo-hardened] 2.6.27-hardened-r8: assassination

[gentoo-hardened] 2.6.27-hardened-r8: assassination

[Alex Efros]

Hi!

It's ~6am here and I'm too tired to file new bugs, I wanna sleep a little first.

While in general Gentoo "stable" branch is very stable, shit always happens.
This time is was upgrade from 2.6.26-hardened-r9 to 2.6.27-hardened-r8. 

First issue: many perl scripts (including FastCGI servers) failed to start
with segmentation fault. See http://bugs.gentoo.org/show_bug.cgi?id=261357
for details and ugly workarounds.

Second issue: apache failed to start with segmentation fault. As I said,
I'll file bugs later, but in short it's trouble with Ioncube and
ZendOptimizer. I had to switch off both to run apache. I've no idea how it
may be possible to workaround this without doing paxctl -m on apache.


For now I just rollback to previous kernel (I think that will be more
secure than paxctl -m for apache, plus I afraid new kernel may do other
nasty things too).


Resume: this upgrade kill both perl and apache. :(

-- 
			WBR, Alex.

Re: [gentoo-hardened] 2.6.27-hardened-r8: assassination

[Alex Efros]

Hi!

On Fri, Mar 06, 2009 at 05:57:18AM +0200, Alex Efros wrote:
> For now I just rollback to previous kernel (I think that will be more
> secure than paxctl -m for apache, plus I afraid new kernel may do other
> nasty things too).
> 
> 
> Resume: this upgrade kill both perl and apache. :(

And it just killed amarok on my workstation, forcing me to downgrade
kernel on workstation too.

-- 
			WBR, Alex.

Re: [gentoo-hardened] 2.6.27-hardened-r8: assassination

[pageexec]

On 6 Mar 2009 at 5:57, Alex Efros wrote:

> First issue: many perl scripts (including FastCGI servers) failed to start
> with segmentation fault. See http://bugs.gentoo.org/show_bug.cgi?id=261357
> for details and ugly workarounds.
> 
> Second issue: apache failed to start with segmentation fault. As I said,
> I'll file bugs later, but in short it's trouble with Ioncube and
> ZendOptimizer. I had to switch off both to run apache. I've no idea how it
> may be possible to workaround this without doing paxctl -m on apache.

two things i'd like you to try:

1. 2.6.28.7 and PaX alone
2. get coredumps and analyze them for the usual things, to see why the segfaults
   occured. if that doesn't point to anything, maybe try an strace as well.

Re: [gentoo-hardened] 2.6.27-hardened-r8: assassination

[Alex Efros]

Hi!

On Fri, Mar 06, 2009 at 09:15:36AM +0200, pageexec@freemail.hu wrote:
> two things i'd like you to try:
> 
> 1. 2.6.28.7 and PaX alone
> 2. get coredumps and analyze them for the usual things, to see why the segfaults
>    occured. if that doesn't point to anything, maybe try an strace as well.

Two questions:
1) Is "2.6.28.7 and PaX alone" mean hardened-sources-2.6.28 with
everything except PaX switched off, or vanilla-sources-2.6.28.7 manually
patched with latests PaX?
2) I'm perl programmer, not C. So I need more detailed instructions (list
of commands to run) how to "get coredumps and analyze them for the usual
things". Probably this info already available somewhere, so url to this
doc will be enough.

As for strace - did that, it helps me detect .so libraries (Ioncube and
ZendOptimizer) because of which apache was killed.

-- 
			WBR, Alex.

Re: [gentoo-hardened] 2.6.27-hardened-r8: assassination

[pageexec]

On 6 Mar 2009 at 17:13, Alex Efros wrote:

> Two questions:
> 1) Is "2.6.28.7 and PaX alone" mean hardened-sources-2.6.28 with
> everything except PaX switched off, or vanilla-sources-2.6.28.7 manually
> patched with latests PaX?

it's always the latter ;), i need to make sure it's a PaX problem.

> 2) I'm perl programmer, not C. So I need more detailed instructions (list
> of commands to run) how to "get coredumps and analyze them for the usual
> things". Probably this info already available somewhere, so url to this
> doc will be enough.

i mentioned them quite a few times on the list and bugzilla and the grsec forums,
here it is again. first, the coredump: you enable coredumps in your shell
(ulimit -c unlimited) then run your program that crashes. this will produce
a coredump file that you load into gdb and then issue the following gdb commands:

  bt
  x/8i $pc
  x/8x $sp
  info reg

> As for strace - did that, it helps me detect .so libraries (Ioncube and
> ZendOptimizer) because of which apache was killed.

on a second thought, i'd need the strace output regardless of the gdb analysis,
just to see how text relocations went as that's where the problem is probably.

Re: [gentoo-hardened] 2.6.27-hardened-r8: assassination

[pageexec]

On 6 Mar 2009 at 23:51, Alex Efros wrote:

> When I run apache for the first time after reboot - without strace/core,
> just to see is it crash - I got this in kernel log:
> 
>     2009-03-06_20:48:56.60108 kern.info: apache2[4621]: segfault at
>     4d554ed0 ip 4d541399 sp 594130d0 error 7 in ld-2.6.1.so[4d53a000+1a000]

ah crap, i know what it is. it's a several years old glibc bug where someone
put a certain variable into the RELRO segment but forgot that it'll be written
to later when a library with RWE GNU_STACK is loaded. the workaround is to find
that library (just extract them from strace, probably it'll be pari's library)
and run execstack -c on it.

Re: [gentoo-hardened] 2.6.27-hardened-r8: assassination

[Alex Efros]

Hi!

On Fri, Mar 06, 2009 at 07:28:17PM +0200, pageexec@freemail.hu wrote:
> it's always the latter ;), i need to make sure it's a PaX problem.

Ok. With this kernel, using pax-linux-2.6.28.7-test19.patch, I was able to
reproduce issues with apache/php/{ioncube,zendoptimizer} and perl module
Math::Pari. Amarok doesn't crash.

> i mentioned them quite a few times on the list and bugzilla and the grsec forums,
> here it is again. first, the coredump: you enable coredumps in your shell

thanks for instructions, here are results:


I've tried to recompile perl, apache and php with "debug" USE-flag enabled,
but looks like ioncube&zendoptimizer don't support php built this way.
So, only perl & apache was built with "debug" flag.

When I run apache for the first time after reboot - without strace/core,
just to see is it crash - I got this in kernel log:

    2009-03-06_20:48:56.60108 kern.info: apache2[4621]: segfault at
    4d554ed0 ip 4d541399 sp 594130d0 error 7 in ld-2.6.1.so[4d53a000+1a000]

I must note it looks very similar to errors I got previously with this
issue - segfault always was reported like "error 7 in ld-2.6.1.so".

But all next runs (under strace and with core dumps enabled) doesn't
produce any error messages in kernel log, which is quite unusual.



# strace -f apache2 -D NO_DETACH -k start -D MANUAL -D DEFLATE -D FASTCGI -D PHP5 -D SSL &>apache2.strace
# gdb
(gdb) core /core 
(no debugging symbols found)
Core was generated by `apache2 -D NO_DETACH -k start -D MANUAL -D DEFLATE -D FASTCGI -D PHP5 -D SSL'.
Program terminated with signal 11, Segmentation fault.
[New process 11835]
#0  0x4ce14399 in ?? ()
(gdb) bt
#0  0x4ce14399 in ?? ()
#1  0x4ce27000 in ?? ()
#2  0x00000ed4 in ?? ()
#3  0x00000003 in ?? ()
#4  0x00000003 in ?? ()
#5  0x00000004 in ?? ()
#6  0x00000000 in ?? ()
(gdb) x/8i $pc
0x4ce14399:     Cannot access memory at address 0x4ce14399
(gdb) x/8x $sp
0x5a681770:     0x4ce27000      0x00000ed4      0x00000003      0x00000003
0x5a681780:     0x00000004      0x00000000      0x00000001      0x4cb5a170
(gdb) info reg
eax            0xffffffff       -1
ecx            0x4ce27fc4       1289912260
edx            0xd      13
ebx            0x4ce27fc4       1289912260
esp            0x5a681770       0x5a681770
ebp            0x5a681890       0x5a681890
esi            0x4ce27000       1289908224
edi            0xed4    3796
eip            0x4ce14399       0x4ce14399
eflags         0x10286  [ PF SF IF RF ]
cs             0x73     115
ss             0x7b     123
ds             0x7b     123
es             0x7b     123
fs             0x0      0
gs             0x33     51



# vi /etc/php/apache2-php5/php.ini     ### disable ioncube
# strace -f apache2 -D NO_DETACH -k start -D MANUAL -D DEFLATE -D FASTCGI -D PHP5 -D SSL &>apache2.strace_zend
# gdb /usr/sbin/apache2 /core 
This GDB was configured as "i686-pc-linux-gnu"...
(no debugging symbols found)

warning: Can't read pathname for load map: Input/output error.
(no debugging symbols found)
Loaded symbols for /usr/sbin/apache2
...
Reading symbols from /usr/local/Zend/lib/ZendExtensionManager.so...(no debugging symbols found)...done.
Loaded symbols for /usr/local/Zend/lib/ZendExtensionManager.so

(no debugging symbols found)
Core was generated by `apache2 -D NO_DETACH -k start -D MANUAL -D DEFLATE -D FASTCGI -D PHP5 -D SSL'.
Program terminated with signal 11, Segmentation fault.
[New process 31217]
#0  0x51015399 in ?? () from /lib/ld-linux.so.2
(gdb) bt
#0  0x51015399 in ?? () from /lib/ld-linux.so.2
#1  0x51028000 in ?? ()
#2  0x00000ed4 in ?? ()
#3  0x00000003 in ?? ()
#4  0x5d5cf82c in ?? ()
#5  0x00000004 in ?? ()
#6  0x00000000 in ?? ()
(gdb) x/8i $pc
0x51015399 <free@plt+27445>:    orl    $0x7,-0xf4(%ebx)
0x510153a0 <free@plt+27452>:    mov    $0x1,%ecx
0x510153a5 <free@plt+27457>:    mov    %ecx,0x8(%esp)
0x510153a9 <free@plt+27461>:    mov    %edi,0x4(%esp)
0x510153ad <free@plt+27465>:    mov    %esi,(%esp)
0x510153b0 <free@plt+27468>:    call   0x51022e80
0x510153b5 <free@plt+27473>:    jmp    0x5101505c <free@plt+26616>
0x510153ba <free@plt+27478>:    xor    %ecx,%ecx
(gdb)  x/8x $sp
0x5d5cf800:     0x51028000      0x00000ed4      0x00000003      0x5d5cf82c
0x5d5cf810:     0x00000004      0x00000000      0x00000001      0x50d5b170
(gdb) info reg
eax            0xffffffff       -1
ecx            0x51028fc4       1359122372
edx            0xd      13
ebx            0x51028fc4       1359122372
esp            0x5d5cf800       0x5d5cf800
ebp            0x5d5cf920       0x5d5cf920
esi            0x51028000       1359118336
edi            0xed4    3796
eip            0x51015399       0x51015399 <free@plt+27445>
eflags         0x10286  [ PF SF IF RF ]
cs             0x73     115
ss             0x7b     123
ds             0x7b     123
es             0x7b     123
fs             0x0      0
gs             0x33     51



# ACCEPT_KEYWORDS=~x86 emerge -a math-pari

if I run perl without strace - I got error message in kernel log:

# perl -e 'use Math::Pari;'              
Segmentation fault (core dumped)

    2009-03-06_21:31:02.23339 kern.info: perl[17676]: segfault at 4ebd7ed0
    ip 4ebc4399 sp 58019490 error 7 in ld-2.6.1.so[4ebbd000+1a000]

if I run perl with strace - there will be no messages in kernel log

# strace -f perl -e 'use Math::Pari;' &>perl.strace
# gdb /usr/bin/perl core 
This GDB was configured as "i686-pc-linux-gnu"...
(no debugging symbols found)

warning: Can't read pathname for load map: Input/output error.
(no debugging symbols found)
Loaded symbols for /usr/bin/perl
Reading symbols from /lib/libpthread.so.0...(no debugging symbols found)...done.
Loaded symbols for /lib/libpthread.so.0
Reading symbols from /lib/libnsl.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libnsl.so.1
Reading symbols from /lib/libdl.so.2...
(no debugging symbols found)...done.
Loaded symbols for /lib/libdl.so.2
Reading symbols from /lib/libm.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/libm.so.6
Reading symbols from /lib/libcrypt.so.1...
(no debugging symbols found)...done.
Loaded symbols for /lib/libcrypt.so.1
Reading symbols from /lib/libutil.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libutil.so.1
Reading symbols from /lib/libc.so.6...
(no debugging symbols found)...done.
Loaded symbols for /lib/libc.so.6
Reading symbols from /lib/ld-linux.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/ld-linux.so.2
Reading symbols from /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/Math/Pari/Pari.so...
(no debugging symbols found)...done.
Loaded symbols for /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/Math/Pari/Pari.so
(no debugging symbols found)
Core was generated by `perl -e use Math::Pari;'.
Program terminated with signal 11, Segmentation fault.
[New process 30393]
#0  0x4fa55399 in ?? () from /lib/ld-linux.so.2
(gdb) bt
#0  0x4fa55399 in ?? () from /lib/ld-linux.so.2
#1  0x4fa68000 in ?? ()
#2  0x00000ed4 in ?? ()
#3  0x00000003 in ?? ()
#4  0x17364a75 in ?? () from /usr/bin/perl
#5  0x00000145 in ?? ()
#6  0x17426824 in ?? ()
#7  0x5a96a6a8 in ?? ()
#8  0x17301567 in ?? () from /usr/bin/perl
#9  0x17426824 in ?? ()
#10 0x00000050 in ?? ()
#11 0x173040d8 in Perl_av_undef () from /usr/bin/perl
#12 0x4fa55f4e in ?? () from /lib/ld-linux.so.2
#13 0x5a96a79c in ?? ()
#14 0x17443df8 in ?? ()
#15 0x00000000 in ?? ()
(gdb) x/8i $pc
0x4fa55399 <free@plt+27445>:    orl    $0x7,-0xf4(%ebx)
0x4fa553a0 <free@plt+27452>:    mov    $0x1,%ecx
0x4fa553a5 <free@plt+27457>:    mov    %ecx,0x8(%esp)
0x4fa553a9 <free@plt+27461>:    mov    %edi,0x4(%esp)
0x4fa553ad <free@plt+27465>:    mov    %esi,(%esp)
0x4fa553b0 <free@plt+27468>:    call   0x4fa62e80
0x4fa553b5 <free@plt+27473>:    jmp    0x4fa5505c <free@plt+26616>
0x4fa553ba <free@plt+27478>:    xor    %ecx,%ecx
(gdb) x/8x $sp
0x5a96a600:     0x4fa68000      0x00000ed4      0x00000003      0x17364a75
0x5a96a610:     0x00000145      0x17426824      0x5a96a6a8      0x17301567
(gdb) info reg
eax            0xffffffff       -1
ecx            0x4fa68fc4       1336315844
edx            0xd      13
ebx            0x4fa68fc4       1336315844
esp            0x5a96a600       0x5a96a600
ebp            0x5a96a720       0x5a96a720
esi            0x4fa68000       1336311808
edi            0xed4    3796
eip            0x4fa55399       0x4fa55399 <free@plt+27445>
eflags         0x10286  [ PF SF IF RF ]
cs             0x73     115
ss             0x7b     123
ds             0x7b     123
es             0x7b     123
fs             0x0      0
gs             0x33     51



> on a second thought, i'd need the strace output regardless of the gdb analysis,
> just to see how text relocations went as that's where the problem is probably.

http://powerman.name/tmp/apache2.strace
http://powerman.name/tmp/apache2.strace_zend
http://powerman.name/tmp/perl.strace

-- 
			WBR, Alex.

Re: [gentoo-hardened] 2.6.27-hardened-r8: assassination

[Alex Efros]

Hi!

On Fri, Mar 06, 2009 at 11:12:59PM +0200, pageexec@freemail.hu wrote:
> ah crap, i know what it is. it's a several years old glibc bug where someone
> put a certain variable into the RELRO segment but forgot that it'll be written
> to later when a library with RWE GNU_STACK is loaded. the workaround is
> to find that library (just extract them from strace, probably it'll be
> pari's library) and run execstack -c on it.

I don't have execstack command. Looks like it belong to prelink package,
but http://www.gentoo.org/doc/en/prelink-howto.xml states it's
incompatible with hardened. Because of this I decide to compile it
manually, just to get execstack command:

# emerge -f prelink
# cd /usr/src
# tar xjvf /usr/portage-distfiles/prelink-20071009.tar.bz2
# cd prelink
# ./configure && make

Now I tried your workaround:

# /usr/src/prelink/src/execstack -c /usr/lib/perl5/site_perl/5.8.8/i686-linux/auto/Math/Pari/Pari.so
# /usr/src/prelink/src/execstack -c /usr/local/ioncube/ioncube_loader_lin_5.2.so
# /usr/src/prelink/src/execstack -c /usr/local/Zend/lib/ZendExtensionManager.so
# /usr/src/prelink/src/execstack -c /usr/local/Zend/lib/ZendExtensionManager_TS.so
# /usr/src/prelink/src/execstack -c /usr/local/Zend/lib/Optimizer-3.3.0/php-5.2.x/ZendOptimizer.so

and it works!!

Is this issue will be fixed in next stable hardened-sources?

-- 
			WBR, Alex.

Re: [gentoo-hardened] 2.6.27-hardened-r8: assassination

[Ned Ludd]

On Sat, 2009-03-07 at 00:57 +0200, Alex Efros wrote:
> Hi!
> 
> On Fri, Mar 06, 2009 at 11:12:59PM +0200, pageexec@freemail.hu wrote:
> > ah crap, i know what it is. it's a several years old glibc bug where someone
> > put a certain variable into the RELRO segment but forgot that it'll be written
> > to later when a library with RWE GNU_STACK is loaded. the workaround is
> > to find that library (just extract them from strace, probably it'll be
> > pari's library) and run execstack -c on it.
> 
> I don't have execstack command. Looks like it belong to prelink package,
> but http://www.gentoo.org/doc/en/prelink-howto.xml states it's
> incompatible with hardened. Because of this I decide to compile it
> manually, just to get execstack command:
> 
> # emerge -f prelink
> # cd /usr/src
> # tar xjvf /usr/portage-distfiles/prelink-20071009.tar.bz2
> # cd prelink
> # ./configure && make
> 
> Now I tried your workaround:
> 
> # /usr/src/prelink/src/execstack -c /usr/lib/perl5/site_perl/5.8.8/i686-linux/auto/Math/Pari/Pari.so
> # /usr/src/prelink/src/execstack -c /usr/local/ioncube/ioncube_loader_lin_5.2.so
> # /usr/src/prelink/src/execstack -c /usr/local/Zend/lib/ZendExtensionManager.so
> # /usr/src/prelink/src/execstack -c /usr/local/Zend/lib/ZendExtensionManager_TS.so
> # /usr/src/prelink/src/execstack -c /usr/local/Zend/lib/Optimizer-3.3.0/php-5.2.x/ZendOptimizer.so
> 
> and it works!!
> 
> Is this issue will be fixed in next stable hardened-sources?

FYI.. PaX Team maintains the PaX kernel and has little control over what
fixes go into the "next" hardened-sources. Also seems to me a little
strange that the PaX Team would have to put a work-around in the kernel
for a bug in glibc.. Seems like glibc should be fixed vs the kernel.



-- 
Ned Ludd <solar@gentoo.org>
Gentoo Linux

Re: [gentoo-hardened] 2.6.27-hardened-r8: assassination

[Alex Efros]

Hi!

On Fri, Mar 06, 2009 at 03:25:16PM -0800, Ned Ludd wrote:
> FYI.. PaX Team maintains the PaX kernel and has little control over what
> fixes go into the "next" hardened-sources. Also seems to me a little
> strange that the PaX Team would have to put a work-around in the kernel
> for a bug in glibc.. Seems like glibc should be fixed vs the kernel.

Some changes in hardened-sources trigger this bug (which exists for years
and don't bother anybody) and kill my servers (apache and perl are
critical for normal server operation).

I'm neither security expert nor Gentoo developer so I don't pretend to
decide where/how this bug should be fixed and surely don't recommend to
add any work-around in the kernel (but that at a glance looks like one of
possible solutions because previous kernel don't trigger this bug).

Only thing I need to know - how long I've to live with this workaround: is
it will be fixed soon, or I have to prepare to redo these 'execstack -c'
commands in next years after each math-pari/zend/ioncube upgrade... and be
prepared to see other applications unexpectedly crash (on loading new
plugin, for example) because of this bug.

If we'll wait until "several years old glibc bug" will be fixed in glibc -
it probably will take few more years.


Anyway, thank you all for your work!
And for providing this workaround, of course - it's much better to have one.

-- 
			WBR, Alex.

Re: [gentoo-hardened] 2.6.27-hardened-r8: assassination

[pageexec]

On 7 Mar 2009 at 1:46, Alex Efros wrote:

> Hi!
> 
> On Fri, Mar 06, 2009 at 03:25:16PM -0800, Ned Ludd wrote:
> > FYI.. PaX Team maintains the PaX kernel and has little control over what
> > fixes go into the "next" hardened-sources. Also seems to me a little
> > strange that the PaX Team would have to put a work-around in the kernel
> > for a bug in glibc.. Seems like glibc should be fixed vs the kernel.
> 
> Some changes in hardened-sources trigger this bug (which exists for years
> and don't bother anybody) and kill my servers (apache and perl are
> critical for normal server operation).

you're both correct ;). the bug is in glibc and it got 'fixed' (see below)
and that 'fix' worked until recently when i did make a change (for security)
that exposed how wrong that fix was.

the glibc bug revolves around the well known GNU_STACK braindamage. in this
case glibc maintains an internal variable that tracks the executable status
(access rights, to be precise) of stacks. it has to be a dynamic property (vs.
something decided at process creation) as under the GNU_STACK approach apps
can load libraries at runtime that need an executable stack and glibc has to
create future stacks with that in mind (beyond mprotecting all existing ones).

now this variable happened to be put into the RELRO segment, so you can imagine
how it would segfault if an app tried to load such a library.

the 'fix' was to mprotect the variable temporarily so that it could be updated...
except this directly contradicts the definition and purpose of RELRO of course.
not to mention that they royally screwed up the ret2libc 'prevention' as well
at the same time. that's two birds with one shot for the guys at Red Hat ;).

here's the commit in question:

  http://sources.redhat.com/cgi-bin/cvsweb.cgi/libc/elf/dl-load.c.diff?r1=1.270&r2=1.271&cvsroot=glibc&f=h

yes, that dates back to 2005 and has been broken ever since.

now what i changed recently was the result of a cleanup in textrel handling.
in earlier versions PaX could allow text relocations (which circumvent
MPROTECT, so it's less than ideal of course) by parsing the ELF headers and
making decisions based on that. for reasons i forget now, i didn't put that
code into the ELF loader so it only supported one kind of ELF, that of the
kernel's bitness. so you'd get textrel processing for your 64 bit amd64
executables, but not for your 32 bit i386 userland on a 64 bit kernel. this
is what i fixed up so now it works for any userland that the kernel's ELF
loader gets compiled for.

along the way i also i implemented a long-standing item on my todo list:
enforcement of RELRO. what it means is that when the kernel detects the
final mprotect(PROT_READ) from userland on a RELRO segment, the new logic
in PaX will change the access rights so that PROT_WRITE cannot be granted
to it ever again (that's kinda the idea behind RELRO after all, it's just
not enforced by the normal kernel). and as you can guess, that enforcement
directly contradicts what the above mentioned 'fix' wants to do.

so there you have it, another little episode in the drama played by Red Hat
where the left hand doesn't seem to know what the right one was doing.

Re: [gentoo-hardened] 2.6.27-hardened-r8: assassination

[John Eckhart]

[-- Attachment #1: Type: text/plain, Size: 3872 bytes --]

On Thu, Mar 19, 2009 at 10:50 AM, <pageexec@freemail.hu> wrote:

> On 7 Mar 2009 at 1:46, Alex Efros wrote:
>
> > Hi!
> >
> > On Fri, Mar 06, 2009 at 03:25:16PM -0800, Ned Ludd wrote:
> > > FYI.. PaX Team maintains the PaX kernel and has little control over
> what
> > > fixes go into the "next" hardened-sources. Also seems to me a little
> > > strange that the PaX Team would have to put a work-around in the kernel
> > > for a bug in glibc.. Seems like glibc should be fixed vs the kernel.
> >
> > Some changes in hardened-sources trigger this bug (which exists for years
> > and don't bother anybody) and kill my servers (apache and perl are
> > critical for normal server operation).
>
> you're both correct ;). the bug is in glibc and it got 'fixed' (see below)
> and that 'fix' worked until recently when i did make a change (for
> security)
> that exposed how wrong that fix was.
>
> the glibc bug revolves around the well known GNU_STACK braindamage. in this
> case glibc maintains an internal variable that tracks the executable status
> (access rights, to be precise) of stacks. it has to be a dynamic property
> (vs.
> something decided at process creation) as under the GNU_STACK approach apps
> can load libraries at runtime that need an executable stack and glibc has
> to
> create future stacks with that in mind (beyond mprotecting all existing
> ones).
>
> now this variable happened to be put into the RELRO segment, so you can
> imagine
> how it would segfault if an app tried to load such a library.
>
> the 'fix' was to mprotect the variable temporarily so that it could be
> updated...
> except this directly contradicts the definition and purpose of RELRO of
> course.
> not to mention that they royally screwed up the ret2libc 'prevention' as
> well
> at the same time. that's two birds with one shot for the guys at Red Hat
> ;).
>
> here's the commit in question:
>
>
> http://sources.redhat.com/cgi-bin/cvsweb.cgi/libc/elf/dl-load.c.diff?r1=1.270&r2=1.271&cvsroot=glibc&f=h
>
> yes, that dates back to 2005 and has been broken ever since.
>
> now what i changed recently was the result of a cleanup in textrel
> handling.
> in earlier versions PaX could allow text relocations (which circumvent
> MPROTECT, so it's less than ideal of course) by parsing the ELF headers and
> making decisions based on that. for reasons i forget now, i didn't put that
> code into the ELF loader so it only supported one kind of ELF, that of the
> kernel's bitness. so you'd get textrel processing for your 64 bit amd64
> executables, but not for your 32 bit i386 userland on a 64 bit kernel. this
> is what i fixed up so now it works for any userland that the kernel's ELF
> loader gets compiled for.
>
> along the way i also i implemented a long-standing item on my todo list:
> enforcement of RELRO. what it means is that when the kernel detects the
> final mprotect(PROT_READ) from userland on a RELRO segment, the new logic
> in PaX will change the access rights so that PROT_WRITE cannot be granted
> to it ever again (that's kinda the idea behind RELRO after all, it's just
> not enforced by the normal kernel). and as you can guess, that enforcement
> directly contradicts what the above mentioned 'fix' wants to do.
>
> so there you have it, another little episode in the drama played by Red Hat
> where the left hand doesn't seem to know what the right one was doing.
>
>
>
It seems like we have a multiway catch22 as the fix for the kernel was
correct from both a security and a "trueness to specification" standpoint
and the fix for glibc will likely be a long time in coming. Based on that, I
would think that the best "gentoo" fix is to put the execstack call into the
ebuild (conditionally run on the hardened use flag). However, execstack is
part of the prelink, which, by nature, is not compatible with hardened. Any
suggestions how to proceed?

[-- Attachment #2: Type: text/html, Size: 4653 bytes --]

Re: [gentoo-hardened] 2.6.27-hardened-r8: assassination

[pageexec]

On 19 Mar 2009 at 12:46, John Eckhart wrote:

> It seems like we have a multiway catch22 as the fix for the kernel was
> correct from both a security and a "trueness to specification" standpoint
> and the fix for glibc will likely be a long time in coming. Based on that, I
> would think that the best "gentoo" fix is to put the execstack call into the
> ebuild (conditionally run on the hardened use flag). However, execstack is
> part of the prelink, which, by nature, is not compatible with hardened. Any
> suggestions how to proceed?

prelink is compatible with PaX/ASLR as the mmap address hint is simply ignored
there. in any case, playing the GNU_STACK games has only one logical end that
i've advocated since the beginning: ignore it for good. for glibc in this case
that means moving __stack_prot out of RELRO.

Re: [gentoo-hardened] 2.6.27-hardened-r8: assassination

[atoth]

I guess all the community members should say thank you to PaxTeam pointing
out such mistakes.

Who would be the best to push through a fix in glibc?

Regards:
Dw.
-- 
dr Tóth Attila, Radiológus, 06-20-825-8057, 06-30-5962-962
Attila Toth MD, Radiologist, +36-20-825-8057, +36-30-5962-962
-- 
dr Tóth Attila, Radiológus, 06-20-825-8057, 06-30-5962-962
Attila Toth MD, Radiologist, +36-20-825-8057, +36-30-5962-962

On Pén, Március 20, 2009 23:31, pageexec@freemail.hu wrote:
> On 19 Mar 2009 at 12:46, John Eckhart wrote:
>
>> It seems like we have a multiway catch22 as the fix for the kernel was
>> correct from both a security and a "trueness to specification"
>> standpoint
>> and the fix for glibc will likely be a long time in coming. Based on
>> that, I
>> would think that the best "gentoo" fix is to put the execstack call into
>> the
>> ebuild (conditionally run on the hardened use flag). However, execstack
>> is
>> part of the prelink, which, by nature, is not compatible with hardened.
>> Any
>> suggestions how to proceed?
>
> prelink is compatible with PaX/ASLR as the mmap address hint is simply
> ignored
> there. in any case, playing the GNU_STACK games has only one logical end
> that
> i've advocated since the beginning: ignore it for good. for glibc in this
> case
> that means moving __stack_prot out of RELRO.
>
>

Re: [gentoo-hardened] 2.6.27-hardened-r8: assassination

[Alex Efros]

Hi!

On Fri, Mar 06, 2009 at 03:25:16PM -0800, Ned Ludd wrote:
> > On Fri, Mar 06, 2009 at 11:12:59PM +0200, pageexec@freemail.hu wrote:
> > > ah crap, i know what it is. it's a several years old glibc bug where someone
> > > put a certain variable into the RELRO segment but forgot that it'll be written
> > > to later when a library with RWE GNU_STACK is loaded. the workaround is
> > > to find that library (just extract them from strace, probably it'll be
> > > pari's library) and run execstack -c on it.
> > 
> > I don't have execstack command. Looks like it belong to prelink package,
> > but http://www.gentoo.org/doc/en/prelink-howto.xml states it's
> > incompatible with hardened. Because of this I decide to compile it
> > manually, just to get execstack command:
> > 
> > # emerge -f prelink
> > # cd /usr/src
> > # tar xjvf /usr/portage-distfiles/prelink-20071009.tar.bz2
> > # cd prelink
> > # ./configure && make
> > 
> > Now I tried your workaround:
> > 
> > # /usr/src/prelink/src/execstack -c /usr/lib/perl5/site_perl/5.8.8/i686-linux/auto/Math/Pari/Pari.so
> > # /usr/src/prelink/src/execstack -c /usr/local/ioncube/ioncube_loader_lin_5.2.so
> > # /usr/src/prelink/src/execstack -c /usr/local/Zend/lib/ZendExtensionManager.so
> > # /usr/src/prelink/src/execstack -c /usr/local/Zend/lib/ZendExtensionManager_TS.so
> > # /usr/src/prelink/src/execstack -c /usr/local/Zend/lib/Optimizer-3.3.0/php-5.2.x/ZendOptimizer.so
> > 
> > and it works!!
> > 
> > Is this issue will be fixed in next stable hardened-sources?
> 
> FYI.. PaX Team maintains the PaX kernel and has little control over what
> fixes go into the "next" hardened-sources. Also seems to me a little
> strange that the PaX Team would have to put a work-around in the kernel
> for a bug in glibc.. Seems like glibc should be fixed vs the kernel.

2 years later… Just updated system, bug still exists, and I still have to
use execstack workaround for zendoptimizer and ioncube.

-- 
			WBR, Alex.

Re: [gentoo-hardened] 2.6.27-hardened-r8: assassination

[Mike Frysinger]

On Tue, Mar 8, 2011 at 1:40 PM, Alex Efros wrote:
> On Fri, Mar 06, 2009 at 03:25:16PM -0800, Ned Ludd wrote:
>> > On Fri, Mar 06, 2009 at 11:12:59PM +0200, pageexec@freemail.hu wrote:
>> > > ah crap, i know what it is. it's a several years old glibc bug where someone
>> > > put a certain variable into the RELRO segment but forgot that it'll be written
>> > > to later when a library with RWE GNU_STACK is loaded. the workaround is
>> > > to find that library (just extract them from strace, probably it'll be
>> > > pari's library) and run execstack -c on it.
>> >
>> > I don't have execstack command. Looks like it belong to prelink package,
>> > but http://www.gentoo.org/doc/en/prelink-howto.xml states it's
>> > incompatible with hardened. Because of this I decide to compile it
>> > manually, just to get execstack command:
>> >
>> > # emerge -f prelink
>> > # cd /usr/src
>> > # tar xjvf /usr/portage-distfiles/prelink-20071009.tar.bz2
>> > # cd prelink
>> > # ./configure && make
>> >
>> > Now I tried your workaround:
>> >
>> > # /usr/src/prelink/src/execstack -c /usr/lib/perl5/site_perl/5.8.8/i686-linux/auto/Math/Pari/Pari.so
>> > # /usr/src/prelink/src/execstack -c /usr/local/ioncube/ioncube_loader_lin_5.2.so
>> > # /usr/src/prelink/src/execstack -c /usr/local/Zend/lib/ZendExtensionManager.so
>> > # /usr/src/prelink/src/execstack -c /usr/local/Zend/lib/ZendExtensionManager_TS.so
>> > # /usr/src/prelink/src/execstack -c /usr/local/Zend/lib/Optimizer-3.3.0/php-5.2.x/ZendOptimizer.so
>> >
>> > and it works!!
>> >
>> > Is this issue will be fixed in next stable hardened-sources?
>>
>> FYI.. PaX Team maintains the PaX kernel and has little control over what
>> fixes go into the "next" hardened-sources. Also seems to me a little
>> strange that the PaX Team would have to put a work-around in the kernel
>> for a bug in glibc.. Seems like glibc should be fixed vs the kernel.
>
> 2 years later… Just updated system, bug still exists, and I still have to
> use execstack workaround for zendoptimizer and ioncube.

like they said, this doesnt seem to be a bug in the kernel, so the pax
source arent going to be changing

if there's a bug in glibc, an actual bug in bugs.g.o needs to be
opened with real details/patches.  otherwise, nothing is going to
change.
-mike

Re: [gentoo-hardened] 2.6.27-hardened-r8: assassination

[Alex Efros]

Hi!

On Tue, Mar 08, 2011 at 02:05:46PM -0500, Mike Frysinger wrote:
> if there's a bug in glibc, an actual bug in bugs.g.o needs to be
> opened with real details/patches.  otherwise, nothing is going to
> change.

Actually, from initial discussion I got impression this is *well* known
(at least to PaX/Hardened/Glibc developers) issue, so there unlikely any
needs in opening new bug. And I don't have patch for glibc. If anyone
think it have sense to report this to gentoo's bugzilla - I'll do it.

-- 
			WBR, Alex.

Re: [gentoo-hardened] 2.6.27-hardened-r8: assassination

[Mike Frysinger]

On Tue, Mar 8, 2011 at 2:52 PM, Alex Efros wrote:
> On Tue, Mar 08, 2011 at 02:05:46PM -0500, Mike Frysinger wrote:
>> if there's a bug in glibc, an actual bug in bugs.g.o needs to be
>> opened with real details/patches.  otherwise, nothing is going to
>> change.
>
> Actually, from initial discussion I got impression this is *well* known
> (at least to PaX/Hardened/Glibc developers) issue, so there unlikely any
> needs in opening new bug. And I don't have patch for glibc. If anyone
> think it have sense to report this to gentoo's bugzilla - I'll do it.

nothing in the e-mail you quoted said this was a kernel bug.  it did
however say "glibc bug" multiple times.
-mike

Re: [gentoo-hardened] 2.6.27-hardened-r8: assassination

[klondike]

2011/3/8 Alex Efros <powerman@powerman.name>:
> Actually, from initial discussion I got impression this is *well* known
> (at least to PaX/Hardened/Glibc developers) issue, so there unlikely any
> needs in opening new bug. And I don't have patch for glibc. If anyone
> think it have sense to report this to gentoo's bugzilla - I'll do it.

With the "Just use a supported kernel" guy maintaining glibc I doubt
this is going to change soon :(

Re: [gentoo-hardened] 2.6.27-hardened-r8: assassination

[Anthony G. Basile]

On 03/08/2011 02:05 PM, Mike Frysinger wrote:
> On Tue, Mar 8, 2011 at 1:40 PM, Alex Efros wrote:
>> On Fri, Mar 06, 2009 at 03:25:16PM -0800, Ned Ludd wrote:
>>>> On Fri, Mar 06, 2009 at 11:12:59PM +0200, pageexec@freemail.hu wrote:
>>>>> ah crap, i know what it is. it's a several years old glibc bug where someone
>>>>> put a certain variable into the RELRO segment but forgot that it'll be written
>>>>> to later when a library with RWE GNU_STACK is loaded. the workaround is
>>>>> to find that library (just extract them from strace, probably it'll be
>>>>> pari's library) and run execstack -c on it.
>>>>
>>>> I don't have execstack command. Looks like it belong to prelink package,
>>>> but http://www.gentoo.org/doc/en/prelink-howto.xml states it's
>>>> incompatible with hardened. Because of this I decide to compile it
>>>> manually, just to get execstack command:
>>>>
>>>> # emerge -f prelink
>>>> # cd /usr/src
>>>> # tar xjvf /usr/portage-distfiles/prelink-20071009.tar.bz2
>>>> # cd prelink
>>>> # ./configure && make
>>>>
>>>> Now I tried your workaround:
>>>>
>>>> # /usr/src/prelink/src/execstack -c /usr/lib/perl5/site_perl/5.8.8/i686-linux/auto/Math/Pari/Pari.so
>>>> # /usr/src/prelink/src/execstack -c /usr/local/ioncube/ioncube_loader_lin_5.2.so
>>>> # /usr/src/prelink/src/execstack -c /usr/local/Zend/lib/ZendExtensionManager.so
>>>> # /usr/src/prelink/src/execstack -c /usr/local/Zend/lib/ZendExtensionManager_TS.so
>>>> # /usr/src/prelink/src/execstack -c /usr/local/Zend/lib/Optimizer-3.3.0/php-5.2.x/ZendOptimizer.so
>>>>
>>>> and it works!!
>>>>
>>>> Is this issue will be fixed in next stable hardened-sources?
>>>
>>> FYI.. PaX Team maintains the PaX kernel and has little control over what
>>> fixes go into the "next" hardened-sources. Also seems to me a little
>>> strange that the PaX Team would have to put a work-around in the kernel
>>> for a bug in glibc.. Seems like glibc should be fixed vs the kernel.
>>
>> 2 years later… Just updated system, bug still exists, and I still have to
>> use execstack workaround for zendoptimizer and ioncube.
> 
> like they said, this doesnt seem to be a bug in the kernel, so the pax
> source arent going to be changing
> 
> if there's a bug in glibc, an actual bug in bugs.g.o needs to be
> opened with real details/patches.  otherwise, nothing is going to
> change.
> -mike

Nothing to say that Mike hasn't already said.  pipacs knows about this
but what can he do?  Good luck with upstream glibc.  Next time I speak
with pipacs I can bring it up, see if anything is changing.  I doubt it.

Take a look at [1] for a good laugh.


Ref:

[1] http://sourceware.org/bugzilla/show_bug.cgi?id=12492

-- 
Anthony G. Basile, Ph.D.
Gentoo Developer

Re: [gentoo-hardened] 2.6.27-hardened-r8: assassination

[Mike Frysinger]

On Tue, Mar 8, 2011 at 3:49 PM, Anthony G. Basile wrote:
> On 03/08/2011 02:05 PM, Mike Frysinger wrote:
>> On Tue, Mar 8, 2011 at 1:40 PM, Alex Efros wrote:
>>> On Fri, Mar 06, 2009 at 03:25:16PM -0800, Ned Ludd wrote:
>>>>> On Fri, Mar 06, 2009 at 11:12:59PM +0200, pageexec@freemail.hu wrote:
>>>>>> ah crap, i know what it is. it's a several years old glibc bug where someone
>>>>>> put a certain variable into the RELRO segment but forgot that it'll be written
>>>>>> to later when a library with RWE GNU_STACK is loaded. the workaround is
>>>>>> to find that library (just extract them from strace, probably it'll be
>>>>>> pari's library) and run execstack -c on it.
>>>>>
>>>>> I don't have execstack command. Looks like it belong to prelink package,
>>>>> but http://www.gentoo.org/doc/en/prelink-howto.xml states it's
>>>>> incompatible with hardened. Because of this I decide to compile it
>>>>> manually, just to get execstack command:
>>>>>
>>>>> # emerge -f prelink
>>>>> # cd /usr/src
>>>>> # tar xjvf /usr/portage-distfiles/prelink-20071009.tar.bz2
>>>>> # cd prelink
>>>>> # ./configure && make
>>>>>
>>>>> Now I tried your workaround:
>>>>>
>>>>> # /usr/src/prelink/src/execstack -c /usr/lib/perl5/site_perl/5.8.8/i686-linux/auto/Math/Pari/Pari.so
>>>>> # /usr/src/prelink/src/execstack -c /usr/local/ioncube/ioncube_loader_lin_5.2.so
>>>>> # /usr/src/prelink/src/execstack -c /usr/local/Zend/lib/ZendExtensionManager.so
>>>>> # /usr/src/prelink/src/execstack -c /usr/local/Zend/lib/ZendExtensionManager_TS.so
>>>>> # /usr/src/prelink/src/execstack -c /usr/local/Zend/lib/Optimizer-3.3.0/php-5.2.x/ZendOptimizer.so
>>>>>
>>>>> and it works!!
>>>>>
>>>>> Is this issue will be fixed in next stable hardened-sources?
>>>>
>>>> FYI.. PaX Team maintains the PaX kernel and has little control over what
>>>> fixes go into the "next" hardened-sources. Also seems to me a little
>>>> strange that the PaX Team would have to put a work-around in the kernel
>>>> for a bug in glibc.. Seems like glibc should be fixed vs the kernel.
>>>
>>> 2 years later… Just updated system, bug still exists, and I still have to
>>> use execstack workaround for zendoptimizer and ioncube.
>>
>> like they said, this doesnt seem to be a bug in the kernel, so the pax
>> source arent going to be changing
>>
>> if there's a bug in glibc, an actual bug in bugs.g.o needs to be
>> opened with real details/patches.  otherwise, nothing is going to
>> change.
>
> Nothing to say that Mike hasn't already said.  pipacs knows about this
> but what can he do?  Good luck with upstream glibc.  Next time I speak
> with pipacs I can bring it up, see if anything is changing.  I doubt it.

if there is a real bug in glibc that drepper is ignoring, that doesnt
mean we cant merge it into Gentoo's glibc.  hence open a bug in
bugs.g.o with the patch for me to review.
-mike

Re: [gentoo-hardened] 2.6.27-hardened-r8: assassination

[Alex Efros]

Hi!

On Tue, Mar 08, 2011 at 03:49:34PM -0500, Anthony G. Basile wrote:
> Take a look at [1] for a good laugh.

Yep, that was funny. :) BTW, if I understood correctly, with proposed
patch my apache won't segfault anymore, but zendoptimizer and ioncube libs
won't be loaded… so this isn't looks like complete solution for this issue.

-- 
			WBR, Alex.

Re: [gentoo-hardened] 2.6.27-hardened-r8: assassination

[pageexec]

On 8 Mar 2011 at 15:55, Mike Frysinger wrote:

> On Tue, Mar 8, 2011 at 3:49 PM, Anthony G. Basile wrote:
> > Nothing to say that Mike hasn't already said.  pipacs knows about this
> > but what can he do?  Good luck with upstream glibc.  Next time I speak
> > with pipacs I can bring it up, see if anything is changing.  I doubt it.
>
> if there is a real bug in glibc that drepper is ignoring, that doesnt
> mean we cant merge it into Gentoo's glibc.  hence open a bug in
> bugs.g.o with the patch for me to review.

it's not just a glibc bug, it's the whole shit concept of GNU_STACK and
all it brings with it, see my Nth explanation here:

  http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=611195#10

i can (and do) disable GNU_STACK handling in PaX, but userland is out of
my reach (well, technically i could do even that by hacking the page cache
but that's too ugly to even consider). so as far as i'm concerned, you have
the following choices in order of preference:

1. disable GNU_STACK processing (mostly in ld.so i think)
   - this should solve all current and future problems but it won't be a
     one-liner patch

2. fix at least the mprotect return value checks in ld.so
   - the patch is in the glibc bugzilla
   - this fixes only the segfaults, it won't let these app to work under PaX,
   - actually, you could modify the patch and on the first mprotect failure
     you can just not attempt the write to that relro variable (and not bother
     with the second mprotect obviously), that should then let the library load
     too (elsewhere gentoo already fixed the stack mprotect code and ignores
     *its* mprotect failures under PaX)

3. execstack -c affected libs/binaries from portage
   - at install time we can detect (and already do, iirc) RWE GNU_STACK headers
   - if such a GNU_STACK header is really needed, i.e., the given app/lib does
     need an executable stack (nested function trampolines are emulated under
     PaX so they just need EMUTRAMP enabled on the app) then disable MPROTECT
     on the app (and for libs, all apps using the given lib)
   - if the RWE GNU_STACK header is a false positive, either fix the app (if we
     have source code) or execstack -c for binaries.
   - this is again quite some work

Re: [gentoo-hardened] 2.6.27-hardened-r8: assassination

[med]

[-- Attachment #1: Type: text/plain, Size: 11 bytes --]

unsubsribe

[-- Attachment #2: Type: text/html, Size: 15 bytes --]

Re: [gentoo-hardened] 2.6.27-hardened-r8: assassination

[Anthony G. Basile]

On 03/09/2011 04:03 AM, pageexec@freemail.hu wrote:
> On 8 Mar 2011 at 15:55, Mike Frysinger wrote:
> 
>> On Tue, Mar 8, 2011 at 3:49 PM, Anthony G. Basile wrote:
>>> Nothing to say that Mike hasn't already said.  pipacs knows about this
>>> but what can he do?  Good luck with upstream glibc.  Next time I speak
>>> with pipacs I can bring it up, see if anything is changing.  I doubt it.
>>
>> if there is a real bug in glibc that drepper is ignoring, that doesnt
>> mean we cant merge it into Gentoo's glibc.  hence open a bug in
>> bugs.g.o with the patch for me to review.
> 
> it's not just a glibc bug, it's the whole shit concept of GNU_STACK and
> all it brings with it, see my Nth explanation here:
> 
>   http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=611195#10
> 
> i can (and do) disable GNU_STACK handling in PaX, but userland is out of
> my reach (well, technically i could do even that by hacking the page cache
> but that's too ugly to even consider). so as far as i'm concerned, you have
> the following choices in order of preference:
> 
> 1. disable GNU_STACK processing (mostly in ld.so i think)
>    - this should solve all current and future problems but it won't be a
>      one-liner patch
> 
> 2. fix at least the mprotect return value checks in ld.so
>    - the patch is in the glibc bugzilla
>    - this fixes only the segfaults, it won't let these app to work under PaX,
>    - actually, you could modify the patch and on the first mprotect failure
>      you can just not attempt the write to that relro variable (and not bother
>      with the second mprotect obviously), that should then let the library load
>      too (elsewhere gentoo already fixed the stack mprotect code and ignores
>      *its* mprotect failures under PaX)
> 
> 3. execstack -c affected libs/binaries from portage
>    - at install time we can detect (and already do, iirc) RWE GNU_STACK headers
>    - if such a GNU_STACK header is really needed, i.e., the given app/lib does
>      need an executable stack (nested function trampolines are emulated under
>      PaX so they just need EMUTRAMP enabled on the app) then disable MPROTECT
>      on the app (and for libs, all apps using the given lib)
>    - if the RWE GNU_STACK header is a false positive, either fix the app (if we
>      have source code) or execstack -c for binaries.
>    - this is again quite some work
> 
> 

For what its worth, I've written some proof of concept code for this, a
library which needs execstack because of a trampoline which is
dlopen'ed.  I put the tarball on my dev space:

    http://dev.gentoo.org/~blueness/pocs/trampoline-poc.tgz

If you do

	CFLAGS+="-DTRAMPOLINE" make
	make install
	make dltest

you'll hit the above problem.  If you then do

	strace ./dltest

you'll see it dies because of mprotect.

Note: there's other stuff here because I'm using this in a class I'm
teaching.

-- 
Anthony G. Basile, Ph. D.
Chair of Information Technology
D'Youville College
Buffalo, NY 14201
(716) 829-8197

Re: [gentoo-hardened] 2.6.27-hardened-r8: assassination

[Anthony G. Basile]

On 03/09/2011 04:03 AM, pageexec@freemail.hu wrote:
> On 8 Mar 2011 at 15:55, Mike Frysinger wrote:
> 
>> On Tue, Mar 8, 2011 at 3:49 PM, Anthony G. Basile wrote:
>>> Nothing to say that Mike hasn't already said.  pipacs knows about this
>>> but what can he do?  Good luck with upstream glibc.  Next time I speak
>>> with pipacs I can bring it up, see if anything is changing.  I doubt it.
>>
>> if there is a real bug in glibc that drepper is ignoring, that doesnt
>> mean we cant merge it into Gentoo's glibc.  hence open a bug in
>> bugs.g.o with the patch for me to review.
> 
> it's not just a glibc bug, it's the whole shit concept of GNU_STACK and
> all it brings with it, see my Nth explanation here:
> 
>   http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=611195#10
> 
> i can (and do) disable GNU_STACK handling in PaX, but userland is out of
> my reach (well, technically i could do even that by hacking the page cache
> but that's too ugly to even consider). so as far as i'm concerned, you have
> the following choices in order of preference:
> 
> 1. disable GNU_STACK processing (mostly in ld.so i think)
>    - this should solve all current and future problems but it won't be a
>      one-liner patch
> 
> 2. fix at least the mprotect return value checks in ld.so
>    - the patch is in the glibc bugzilla
>    - this fixes only the segfaults, it won't let these app to work under PaX,
>    - actually, you could modify the patch and on the first mprotect failure
>      you can just not attempt the write to that relro variable (and not bother
>      with the second mprotect obviously), that should then let the library load
>      too (elsewhere gentoo already fixed the stack mprotect code and ignores
>      *its* mprotect failures under PaX)
> 
> 3. execstack -c affected libs/binaries from portage
>    - at install time we can detect (and already do, iirc) RWE GNU_STACK headers
>    - if such a GNU_STACK header is really needed, i.e., the given app/lib does
>      need an executable stack (nested function trampolines are emulated under
>      PaX so they just need EMUTRAMP enabled on the app) then disable MPROTECT
>      on the app (and for libs, all apps using the given lib)
>    - if the RWE GNU_STACK header is a false positive, either fix the app (if we
>      have source code) or execstack -c for binaries.
>    - this is again quite some work
> 
> 

For what its worth, I've written some proof of concept code for this, a
library which needs execstack because of a trampoline which is
dlopen'ed.  I put the tarball on my dev space:

    http://dev.gentoo.org/~blueness/pocs/trampoline-poc.tgz

If you do

	CFLAGS+="-DTRAMPOLINE" make
	make install
	make dltest

you'll hit the above problem.  If you then do

	strace ./dltest

you'll see it dies because of mprotect.

Note: there's other stuff here because I'm using this in a class I'm
teaching.

-- 
Anthony G. Basile, Ph.D.
Gentoo Developer