[gentoo-dev] Split ELF Debug (defult or not?)

[gentoo-dev] Split ELF Debug (defult or not?)

[Ned Ludd]

Good afternoon,

probably in portage-2.0.54 a patch will be added to emit split debug
info. Having a split debug allows us to retain all the advantages of
stripping executables while gaining the ability to properly debug
executables in bfd aware programs. It's been in testing with a small
hand full of devs and works quite well, but before it's pushed in we
would like to get input from our devs & users.

Would you be willing to give up space in $ROOT/usr/lib/debug for ELF
executables by default in order to aid in better debugging by or do we
want to only emit it when a FEATURE= is defined.

Having a split debug pretty much obsoletes the need to add nostrip to
your features in order to get debug info.

Users wishing to not have debug info can simply add
INSTALL_MASK="/usr/lib/debug ${INSTALL_MASK}" to make.conf or the
environment unless we make it FEATURE based.

I'm in favor of it enabled per default but I'd like to know what you
think and why. (advantages of on/off by default etc..)

Anybody wanting to test or make use of this feature right away can grab
a copy of my prepstrip from
http://dev.gentoo.org/~solar/portage_misc/prepstrip and save it to
/usr/lib/portage/bin/prepstrip or patch portage with
http://dev.gentoo.org/~solar/patch_overlay/sys-apps/portage/portage-2.0.53_rc7-prepstrip.patch
It requires you merge pax-utils for the scanelf util.


-- 
Ned Ludd <solar@gentoo.org>
Gentoo Linux

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Split ELF Debug (defult or not?)

[Bruno]

On Saturday 26 November 2005 18:50, Ned Ludd wrote:
> Good afternoon,
>
> Would you be willing to give up space in $ROOT/usr/lib/debug for ELF
> executables by default in order to aid in better debugging by or do we
> want to only emit it when a FEATURE= is defined.
For me either way is good as long as it's well documented in the install 
handbook.
A notice for all those upgrading would be fine as well..

> Having a split debug pretty much obsoletes the need to add nostrip to
> your features in order to get debug info.

> Users wishing to not have debug info can simply add
> INSTALL_MASK="/usr/lib/debug ${INSTALL_MASK}" to make.conf or the
> environment unless we make it FEATURE based.
FEATURE based is probably the best way of doing it...

What's the advantage of splitting out the debug info to some extra location 
instead of leaving it in the original binary (maybe smaller foot-print in 
memory while the debugging info is not used), and what tools are compatible 
with this external debug info?

> I'm in favor of it enabled per default but I'd like to know what you
> think and why. (advantages of on/off by default etc..)
>
Having the debug info preserved by default makes it possible for the user to 
get debug information which may be used for bug-reporting on program crashes, 
but will probably just fill the disk of basic users who do have no interest 
in reporting bugs or trying to get more information about the cause of bugs.


Bruno
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Split ELF Debug (default or not?)

[Ned Ludd]

On Sat, 2005-11-26 at 19:30 +0100, Bruno wrote:

[snip]

> What's the advantage of splitting out the debug info to some extra location 
> instead of leaving it in the original binary (maybe smaller foot-print in 
> memory while the debugging info is not used), 

Yes exactly a stripped binary will occupy less space in RAM.


> and what tools are compatible 
> with this external debug info?

gdb being the main one.

With tools like valgrind you can simply do.
LD_LIBRARY_PATH=/usr/lib/debug valgrind --foo

ELF is cool like that.


-- 
Ned Ludd <solar@gentoo.org>
Gentoo Linux

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Split ELF Debug (defult or not?)

[Olivier Crête]

[-- Attachment #1: Type: text/plain, Size: 1334 bytes --]

On Sat, 2005-26-11 at 12:50 -0500, Ned Ludd wrote:
> I'm in favor of it enabled per default but I'd like to know what you
> think and why. (advantages of on/off by default etc..)

First, I fully support solar's patch, this feature should have been
integrated into portage months ago.

> Anybody wanting to test or make use of this feature right away can grab
> a copy of my prepstrip from
> http://dev.gentoo.org/~solar/portage_misc/prepstrip and save it to
> /usr/lib/portage/bin/prepstrip or patch portage with
> http://dev.gentoo.org/~solar/patch_overlay/sys-apps/portage/portage-2.0.53_rc7-prepstrip.patch
> It requires you merge pax-utils for the scanelf util.

Second, it's often helpful for development to also have the source code
of the libraries when debugging a program (like MSFT has done for
years). Red Hat also does that, they have built a tool to extract the
list of referenced files from the debug info itself. I've made a patch
to integrate that tool into portage and an ebuild for the tool.

Patch (apply on top of solar's patch):
http://dev.gentoo.org/~tester/prepstrip-keep-sources.patch
Fully patched prepstrip:
http://dev.gentoo.org/~tester/prepstrip

The debugedit ebuild is at:
http://dev.gentoo.org/~tester/debugedit-4.4.3-ebuild.tar.bz2


-- 
Olivier Crête
tester@gentoo.org

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

[gentoo-dev] Re: Split ELF Debug (defult or not?)

[R Hill]

Ned Ludd wrote:
> Good afternoon,
> 
> probably in portage-2.0.54 a patch will be added to emit split debug
> info. Having a split debug allows us to retain all the advantages of
> stripping executables while gaining the ability to properly debug
> executables in bfd aware programs. It's been in testing with a small
> hand full of devs and works quite well, but before it's pushed in we
> would like to get input from our devs & users.
> 
> Would you be willing to give up space in $ROOT/usr/lib/debug for ELF
> executables by default in order to aid in better debugging by or do we
> want to only emit it when a FEATURE= is defined.

How much space are we talking about?

I like on-by-default, along with a feature option to turn it off.

--de.

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Split ELF Debug (defult or not?)

[Ivan Yosifov]

On Sat, 2005-11-26 at 12:50 -0500, Ned Ludd wrote:
> Good afternoon,
> 
> probably in portage-2.0.54 a patch will be added to emit split debug
> info. Having a split debug allows us to retain all the advantages of
> stripping executables while gaining the ability to properly debug
> executables in bfd aware programs. It's been in testing with a small
> hand full of devs and works quite well, but before it's pushed in we
> would like to get input from our devs & users.
> 
> Would you be willing to give up space in $ROOT/usr/lib/debug for ELF
> executables by default in order to aid in better debugging by or do we
> want to only emit it when a FEATURE= is defined.
> 
> Having a split debug pretty much obsoletes the need to add nostrip to
> your features in order to get debug info.
> 
> Users wishing to not have debug info can simply add
> INSTALL_MASK="/usr/lib/debug ${INSTALL_MASK}" to make.conf or the
> environment unless we make it FEATURE based.
> 
> I'm in favor of it enabled per default but I'd like to know what you
> think and why. (advantages of on/off by default etc..)
> 
> Anybody wanting to test or make use of this feature right away can grab
> a copy of my prepstrip from
> http://dev.gentoo.org/~solar/portage_misc/prepstrip and save it to
> /usr/lib/portage/bin/prepstrip or patch portage with
> http://dev.gentoo.org/~solar/patch_overlay/sys-apps/portage/portage-2.0.53_rc7-prepstrip.patch
> It requires you merge pax-utils for the scanelf util.
> 
> 
> -- 
> Ned Ludd <solar@gentoo.org>
> Gentoo Linux

+1 for default

IMO it should be on by default but documented in bold in the install
guide for those who wish to disable it. It makes sense to able to debug
a problem right away rather than having to reemerge things first, to get
the debugging info. 

How much disk space should the debug info consume ?

-- 
Cheers,
Ivan Yosifov.

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Split ELF Debug (defult or not?)

[Mike Frysinger]

On Sat, Nov 26, 2005 at 12:50:30PM -0500, Ned Ludd wrote:
> Would you be willing to give up space in $ROOT/usr/lib/debug for ELF
> executables by default in order to aid in better debugging by or do we
> want to only emit it when a FEATURE= is defined.

would make more sense to have it be a FEATURE and then put the FEATRE
into our profiles

that way users can disable it via standard make.conf config rather
than having to add some weird INSTALL_MASK setting
-mike
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Split ELF Debug (defult or not?)

[Petteri Räty]

[-- Attachment #1: Type: text/plain, Size: 945 bytes --]

Ned Ludd wrote:
> Good afternoon,
> 
> Would you be willing to give up space in $ROOT/usr/lib/debug for ELF
> executables by default in order to aid in better debugging by or do we
> want to only emit it when a FEATURE= is defined.
> 
> Having a split debug pretty much obsoletes the need to add nostrip to
> your features in order to get debug info.
> 
> Users wishing to not have debug info can simply add
> INSTALL_MASK="/usr/lib/debug ${INSTALL_MASK}" to make.conf or the
> environment unless we make it FEATURE based.
> 
> I'm in favor of it enabled per default but I'd like to know what you
> think and why. (advantages of on/off by default etc..)
> 

How useful is this debug information with -fomit-frame-pointer? From
what I have read it makes debugging at least harder. I think most people
have -fomit-frame-pointer in their CFLAGS so it should not be enabled by
default if the debug info is not very useful any way.

Regards,
Petteri

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 256 bytes --]

Re: [gentoo-dev] Split ELF Debug (defult or not?)

[Luca Barbato]

Ned Ludd wrote:
>
[snip]
> 

It's great!

Make it a FEATURE default on for common profiles.

lu

-- 

Luca Barbato

Gentoo/linux Developer		Gentoo/PPC Operational Leader
http://dev.gentoo.org/~lu_zero

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Split ELF Debug (defult or not?)

[Dan Meltzer]

On 11/26/05, Petteri Räty <betelgeuse@gentoo.org> wrote:
> Ned Ludd wrote:
> > Good afternoon,
> >
> > Would you be willing to give up space in $ROOT/usr/lib/debug for ELF
> > executables by default in order to aid in better debugging by or do we
> > want to only emit it when a FEATURE= is defined.
> >
> > Having a split debug pretty much obsoletes the need to add nostrip to
> > your features in order to get debug info.
> >
> > Users wishing to not have debug info can simply add
> > INSTALL_MASK="/usr/lib/debug ${INSTALL_MASK}" to make.conf or the
> > environment unless we make it FEATURE based.
> >
> > I'm in favor of it enabled per default but I'd like to know what you
> > think and why. (advantages of on/off by default etc..)
> >
>
> How useful is this debug information with -fomit-frame-pointer? From
> what I have read it makes debugging at least harder. I think most people
> have -fomit-frame-pointer in their CFLAGS so it should not be enabled by
> default if the debug info is not very useful any way.

Well, only on x86 and other lacking arches...
>
> Regards,
> Petteri
>
>
>

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Split ELF Debug (defult or not?)

[Mart Raudsepp]

On Sat, 2005-11-26 at 19:48 -0500, Dan Meltzer wrote:
> On 11/26/05, Petteri Räty <betelgeuse@gentoo.org> wrote:
> > Ned Ludd wrote:
> > > Good afternoon,
> > >
> > > Would you be willing to give up space in $ROOT/usr/lib/debug for ELF
> > > executables by default in order to aid in better debugging by or do we
> > > want to only emit it when a FEATURE= is defined.
> > >
> > > Having a split debug pretty much obsoletes the need to add nostrip to
> > > your features in order to get debug info.
> > >
> > > Users wishing to not have debug info can simply add
> > > INSTALL_MASK="/usr/lib/debug ${INSTALL_MASK}" to make.conf or the
> > > environment unless we make it FEATURE based.
> > >
> > > I'm in favor of it enabled per default but I'd like to know what you
> > > think and why. (advantages of on/off by default etc..)
> > >
> >
> > How useful is this debug information with -fomit-frame-pointer? From
> > what I have read it makes debugging at least harder. I think most people
> > have -fomit-frame-pointer in their CFLAGS so it should not be enabled by
> > default if the debug info is not very useful any way.
> 
> Well, only on x86 and other lacking arches...

Please note that gcc4 generates location lists by default (at least on
debug builds). This should allow for debugging on x86 with
-fomit-frame-pointers. Not sure how that works in relation to split ELF
debug files.

http://gnu.paradoxical.co.uk/software/gcc/gcc-4.0/changes.html

"Location lists are now generated by default when compiling with debug
info and optimization. Location lists provide more accurate debug info
about locations of variables and they allow debugging code compiled with
-fomit-frame-pointer."

Of course gcc4 isn't even ~x86 yet... ;)

-- 
With regards,
Mart Raudsepp

Project manager of wxMUD     - http://wxmud.sourceforge.net/
Developer of wxWidgets       - http://www.wxwidgets.org/
Gtk+ port maintainer of cpaf - http://cpaf.berlios.de/

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Split ELF Debug (default or not?)

[Kevin F. Quinn]

On 26/11/2005 13:55:25, Ned Ludd (solar@gentoo.org) wrote:
> On Sat, 2005-11-26 at 19:30 +0100, Bruno wrote:
> 
> > What's the advantage of splitting out the debug info to some extra 
> > location instead of leaving it in the original binary (maybe smaller 
> > foot-print in memory while the debugging info is not used), 
> 
> Yes exactly a stripped binary will occupy less space in RAM.

I may be wrong, but I don't think this is true.  As I understand it,
debug sections aren't part of the PT_LOAD segments so the loader won't
load them into memory.

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Split ELF Debug (default or not?)

[Ned Ludd]

On Sun, 2005-11-27 at 10:53 +0100, Kevin F. Quinn wrote:
> On 26/11/2005 13:55:25, Ned Ludd (solar@gentoo.org) wrote:
> > On Sat, 2005-11-26 at 19:30 +0100, Bruno wrote:
> > 
> > > What's the advantage of splitting out the debug info to some extra 
> > > location instead of leaving it in the original binary (maybe smaller 
> > > foot-print in memory while the debugging info is not used), 
> > 
> > Yes exactly a stripped binary will occupy less space in RAM.
> 
> I may be wrong, but I don't think this is true.  As I understand it,
> debug sections aren't part of the PT_LOAD segments so the loader won't
> load them into memory.

See the objcopy manpage. Look for the --only-keep-debug section.
The entire elf has to be mmaped before individual sections are mapped.


-- 
Ned Ludd <solar@gentoo.org>
Gentoo Linux

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Split ELF Debug (default or not?)

[Ned Ludd]

On Sat, 2005-11-26 at 23:42 +0200, Petteri Räty wrote:
> Ned Ludd wrote:
> > Good afternoon,
> > 
> > Would you be willing to give up space in $ROOT/usr/lib/debug for ELF
> > executables by default in order to aid in better debugging by or do we
> > want to only emit it when a FEATURE= is defined.
> > 
> > Having a split debug pretty much obsoletes the need to add nostrip to
> > your features in order to get debug info.
> > 
> > Users wishing to not have debug info can simply add
> > INSTALL_MASK="/usr/lib/debug ${INSTALL_MASK}" to make.conf or the
> > environment unless we make it FEATURE based.
> > 
> > I'm in favor of it enabled per default but I'd like to know what you
> > think and why. (advantages of on/off by default etc..)
> > 
> 
> How useful is this debug information with -fomit-frame-pointer? From
> what I have read it makes debugging at least harder. I think most people
> have -fomit-frame-pointer in their CFLAGS so it should not be enabled by
> default if the debug info is not very useful any way.
> 
> Regards,
> Petteri

-fomit-frame-pointer does indeed make certain aspects of debugging
harder but it does not moot the idea entirely.

-- 
Ned Ludd <solar@gentoo.org>
Gentoo Linux

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: Split ELF Debug (default or not?)

[Ned Ludd]

On Sat, 2005-11-26 at 13:20 -0600, R Hill wrote:
> Ned Ludd wrote:
> > Good afternoon,
> > 
> > probably in portage-2.0.54 a patch will be added to emit split debug
> > info. Having a split debug allows us to retain all the advantages of
> > stripping executables while gaining the ability to properly debug
> > executables in bfd aware programs. It's been in testing with a small
> > hand full of devs and works quite well, but before it's pushed in we
> > would like to get input from our devs & users.
> > 
> > Would you be willing to give up space in $ROOT/usr/lib/debug for ELF
> > executables by default in order to aid in better debugging by or do we
> > want to only emit it when a FEATURE= is defined.
> 
> How much space are we talking about?

There is no fixed size here and depends on the number of packages you 
have and the CFLAGS passed to the programs you build. 
Naturally if you start building all your code with 
CFLAGS="-g3 -ggdb" your going to end up with a larger debug info.

-- 
Ned Ludd <solar@gentoo.org>
Gentoo Linux

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Split ELF Debug (defult or not?)

[Diego 'Flameeyes' Pettenò]

[-- Attachment #1: Type: text/plain, Size: 548 bytes --]

On Sunday 27 November 2005 00:10, Luca Barbato wrote:
> It's great!
> Make it a FEATURE default on for common profiles.
+1, and it would be better if the FEATURES, instead of removing the generated 
files, would disable the building of them completely, mainly because "work" 
systems with limited CPU, RAM and HDD space would probably prefer not having 
to create and store them :)

/me thinks of laptops

-- 
Diego "Flameeyes" Pettenò - http://dev.gentoo.org/~flameeyes/
Gentoo/ALT lead, Gentoo/FreeBSD, Video, AMD64, Sound, PAM, KDE

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Re: Split ELF Debug (default or not?)

[Ivan Yosifov]

On Sun, 2005-11-27 at 07:24 -0500, Ned Ludd wrote:
> On Sat, 2005-11-26 at 13:20 -0600, R Hill wrote:
> > Ned Ludd wrote:
> > > Good afternoon,
> > > 
> > > probably in portage-2.0.54 a patch will be added to emit split debug
> > > info. Having a split debug allows us to retain all the advantages of
> > > stripping executables while gaining the ability to properly debug
> > > executables in bfd aware programs. It's been in testing with a small
> > > hand full of devs and works quite well, but before it's pushed in we
> > > would like to get input from our devs & users.
> > > 
> > > Would you be willing to give up space in $ROOT/usr/lib/debug for ELF
> > > executables by default in order to aid in better debugging by or do we
> > > want to only emit it when a FEATURE= is defined.
> > 
> > How much space are we talking about?
> 
> There is no fixed size here and depends on the number of packages you 
> have and the CFLAGS passed to the programs you build. 
> Naturally if you start building all your code with 
> CFLAGS="-g3 -ggdb" your going to end up with a larger debug info.

Of course I will be compiling with CFLAGS="-g3 -ggdb" :)

The reason I don't do it now is because debug info:

1) makes binaries larger
2) makes binaries slower ( in my experience ( may have to do with 1) )

And I don't ( not sure if anyone does ) care about any non-gdb debugger.

So, can you give us a wild guess about the disk space ? How much does it
take on your system and how many packages do you have installed ?

And one more thing. For proper debugging, don't I need the source to be
present ?

-- 
Cheers,
Ivan Yosifov.

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: Split ELF Debug (default or not?)

[Edward Catmur]

On Sun, 2005-11-27 at 15:09 +0200, Ivan Yosifov wrote:
> On Sun, 2005-11-27 at 07:24 -0500, Ned Ludd wrote:
> > On Sat, 2005-11-26 at 13:20 -0600, R Hill wrote:
> > > Ned Ludd wrote:
> > > > Good afternoon,
> > > > 
> > > > probably in portage-2.0.54 a patch will be added to emit split debug
> > > > info. Having a split debug allows us to retain all the advantages of
> > > > stripping executables while gaining the ability to properly debug
> > > > executables in bfd aware programs. It's been in testing with a small
> > > > hand full of devs and works quite well, but before it's pushed in we
> > > > would like to get input from our devs & users.
> > > > 
> > > > Would you be willing to give up space in $ROOT/usr/lib/debug for ELF
> > > > executables by default in order to aid in better debugging by or do we
> > > > want to only emit it when a FEATURE= is defined.
> > > 
> > > How much space are we talking about?
> > 
> > There is no fixed size here and depends on the number of packages you 
> > have and the CFLAGS passed to the programs you build. 
> > Naturally if you start building all your code with 
> > CFLAGS="-g3 -ggdb" your going to end up with a larger debug info.
> 
> Of course I will be compiling with CFLAGS="-g3 -ggdb" :)
> 
> The reason I don't do it now is because debug info:
> 
> 1) makes binaries larger
> 2) makes binaries slower ( in my experience ( may have to do with 1) )
> 
> And I don't ( not sure if anyone does ) care about any non-gdb debugger.
> 
> So, can you give us a wild guess about the disk space ? How much does it
> take on your system and how many packages do you have installed ?
> 
> And one more thing. For proper debugging, don't I need the source to be
> present ?

I've been installing debug symbols and debug sources for a few weeks
now, using bashrc hacks.

I use "debugsymbols" and "debugsources" in FEATURES.

At a rough estimate, debug symbols with -ggdb3 will add 60% to package
sizes, and debug sources will add another 40%. So having this on by
default will necessitate a repartition for many users.

Ed Catmur

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: Split ELF Debug (default or not?)

[Ned Ludd]

On Sun, 2005-11-27 at 15:09 +0200, Ivan Yosifov wrote:
> On Sun, 2005-11-27 at 07:24 -0500, Ned Ludd wrote:
> > On Sat, 2005-11-26 at 13:20 -0600, R Hill wrote:
> > > Ned Ludd wrote:
> > > > Good afternoon,
> > > > 
> > > > probably in portage-2.0.54 a patch will be added to emit split debug
> > > > info. Having a split debug allows us to retain all the advantages of
> > > > stripping executables while gaining the ability to properly debug
> > > > executables in bfd aware programs. It's been in testing with a small
> > > > hand full of devs and works quite well, but before it's pushed in we
> > > > would like to get input from our devs & users.
> > > > 
> > > > Would you be willing to give up space in $ROOT/usr/lib/debug for ELF
> > > > executables by default in order to aid in better debugging by or do we
> > > > want to only emit it when a FEATURE= is defined.
> > > 
> > > How much space are we talking about?
> > 
> > There is no fixed size here and depends on the number of packages you 
> > have and the CFLAGS passed to the programs you build. 
> > Naturally if you start building all your code with 
> > CFLAGS="-g3 -ggdb" your going to end up with a larger debug info.
> 
> Of course I will be compiling with CFLAGS="-g3 -ggdb" :)
> 
> The reason I don't do it now is because debug info:
> 
> 1) makes binaries larger
> 2) makes binaries slower ( in my experience ( may have to do with 1) )
> 
> And I don't ( not sure if anyone does ) care about any non-gdb debugger.
> 
> So, can you give us a wild guess about the disk space ? How much does it
> take on your system and how many packages do you have installed ?

-rwxr-xr-x  1 root root 49268 Nov 19 18:00 /usr/bin/scanelf
-rw-r--r--  1 root root  5292 Nov 19
18:02 /usr/lib/debug/usr/bin/scanelf.debug


I really can't give an accurate example. Halcyon who has been testing it
merged world and he was yeilded with 18M of debug info (I have no idea
how many packages he has). ChrisWhite has also been merging a lot of
merging with it recently to test KDE stuff which is a pretty big
package. He may have some useful data that your after. The reason I
posted the patch and prepstrip here was so that people could formulate
their own opinions.

I'm also using an additional patch in my local portage system where I'm 
sub packaging the debug info and it only gets installed when I tell it 
to be installed via the emerge -K option. I use some pretty crafty 
tricks to accomplish this like dynamic ebuild creation. I do this in
order to have compressed debug info handy vs having it on the live
filesystem. This probably would not be accepted by portage mainline 
unless I do it in a more elegant way that wont conflict with the future 
portage-3.x series.

> And one more thing. For proper debugging, don't I need the source to be
> present ?

-g3 -ggdb embeds the source code in the debug info so I don't see the 
point.

-- 
Ned Ludd <solar@gentoo.org>
Gentoo Linux

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Split ELF Debug (defult or not?)

[Dan Meltzer]

On 11/27/05, Diego 'Flameeyes' Pettenò <flameeyes@gentoo.org> wrote:
> On Sunday 27 November 2005 00:10, Luca Barbato wrote:
> > It's great!
> > Make it a FEATURE default on for common profiles.
> +1, and it would be better if the FEATURES, instead of removing the generated
> files, would disable the building of them completely, mainly because "work"
> systems with limited CPU, RAM and HDD space would probably prefer not having
> to create and store them :)

Err, maybe I am incorrect, but isn't it more "work" to ungenerate them
(using strip) then to just not install them?
>
> /me thinks of laptops
>
> --
> Diego "Flameeyes" Pettenò - http://dev.gentoo.org/~flameeyes/
> Gentoo/ALT lead, Gentoo/FreeBSD, Video, AMD64, Sound, PAM, KDE
>
>
>

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Split ELF Debug (defult or not?)

[Diego 'Flameeyes' Pettenò]

[-- Attachment #1: Type: text/plain, Size: 565 bytes --]

On Sunday 27 November 2005 15:39, Dan Meltzer wrote:
> Err, maybe I am incorrect, but isn't it more "work" to ungenerate them
> (using strip) then to just not install them?
Their creation in-line of a binary is probably a simpler work (for the disk) 
than having to split them out, but I might be wrong.
For sure they'll take a bit more disk space while building, and that might be 
significant for things like OpenOffice.

-- 
Diego "Flameeyes" Pettenò - http://dev.gentoo.org/~flameeyes/
Gentoo/ALT lead, Gentoo/FreeBSD, Video, AMD64, Sound, PAM, KDE

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Split ELF Debug (defult or not?)

[Ned Ludd]

On Sun, 2005-11-27 at 09:39 -0500, Dan Meltzer wrote:
> On 11/27/05, Diego 'Flameeyes' Pettenò <flameeyes@gentoo.org> wrote:
> > On Sunday 27 November 2005 00:10, Luca Barbato wrote:
> > > It's great!
> > > Make it a FEATURE default on for common profiles.
> > +1, and it would be better if the FEATURES, instead of removing the generated
> > files, would disable the building of them completely, mainly because "work"
> > systems with limited CPU, RAM and HDD space would probably prefer not having
> > to create and store them :)
> 
> Err, maybe I am incorrect, but isn't it more "work" to ungenerate them
> (using strip) then to just not install them?

it's pretty easy to add a check to the save_elf_debug function to tell 
it to return if we decide that FEATURES=splitdebug will be the default 
and that if it's not set to simply return from the function without 
doing anything.


-- 
Ned Ludd <solar@gentoo.org>
Gentoo Linux

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Split ELF Debug (defult or not?)

[Tavis Ormandy]

[-- Attachment #1: Type: text/plain, Size: 1228 bytes --]

On Sat, Nov 26, 2005 at 12:50:30PM -0500, Ned Ludd wrote:
> I'm in favor of it enabled per default but I'd like to know what you
> think and why. (advantages of on/off by default etc..)
> 

This should definitely be enabled by default, we dont need to enable
debugging information for this to be useful, just having the symbol
table available will make getting backtraces and diagnosing problems
many times easier with little extra diskspace requirements. Gentoo is
way behind on this feature, all the other major distributions have
caught on to how useful detatched debugging symbols can be.

If we dont enable this by default, I think the users who really need it
(ie, the ones who want us to track down a bug but are unable to provide
enough debugging information to do so) probably wont have the foresight
to turn it on.

This could also be a major boon for administrators imho, if a service
is dumping core unpredicatbly this feature and enabling coredumps would
be enough to start tracking down the problem, or at least identifying
the culprit.

-- 
-------------------------------------
taviso@sdf.lonestar.org | finger me for my pgp key.
-------------------------------------------------------

[-- Attachment #2: Type: application/pgp-signature, Size: 299 bytes --]

Re: [gentoo-dev] Re: Split ELF Debug (default or not?)

[Edward Catmur]

On Sun, 2005-11-27 at 08:40 -0500, Ned Ludd wrote:
> On Sun, 2005-11-27 at 15:09 +0200, Ivan Yosifov wrote:
> > And one more thing. For proper debugging, don't I need the source to be
> > present ?
> 
> -g3 -ggdb embeds the source code in the debug info so I don't see the 
> point.

It doesn't; at least not with gcc 3.4.4. It does embed function
prototypes and macro definitions, though.

Ed Catmur

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Split ELF Debug (defult or not?)

[Dan Meltzer]

Random thought....  May be completely off base.

Could this debug info be NFS shared? assuming like computers, or would
it be different on each computer.

On 11/27/05, Tavis Ormandy <taviso@gentoo.org> wrote:
> On Sat, Nov 26, 2005 at 12:50:30PM -0500, Ned Ludd wrote:
> > I'm in favor of it enabled per default but I'd like to know what you
> > think and why. (advantages of on/off by default etc..)
> >
>
> This should definitely be enabled by default, we dont need to enable
> debugging information for this to be useful, just having the symbol
> table available will make getting backtraces and diagnosing problems
> many times easier with little extra diskspace requirements. Gentoo is
> way behind on this feature, all the other major distributions have
> caught on to how useful detatched debugging symbols can be.
>
> If we dont enable this by default, I think the users who really need it
> (ie, the ones who want us to track down a bug but are unable to provide
> enough debugging information to do so) probably wont have the foresight
> to turn it on.
>
> This could also be a major boon for administrators imho, if a service
> is dumping core unpredicatbly this feature and enabling coredumps would
> be enough to start tracking down the problem, or at least identifying
> the culprit.
>
> --
> -------------------------------------
> taviso@sdf.lonestar.org | finger me for my pgp key.
> -------------------------------------------------------
>
>
>

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Split ELF Debug (defult or not?)

[Edward Catmur]

On Sun, 2005-11-27 at 09:39 -0500, Dan Meltzer wrote:
> On 11/27/05, Diego 'Flameeyes' Pettenò <flameeyes@gentoo.org> wrote:
> > On Sunday 27 November 2005 00:10, Luca Barbato wrote:
> > > It's great!
> > > Make it a FEATURE default on for common profiles.
> > +1, and it would be better if the FEATURES, instead of removing the generated
> > files, would disable the building of them completely, mainly because "work"
> > systems with limited CPU, RAM and HDD space would probably prefer not having
> > to create and store them :)
> 
> Err, maybe I am incorrect, but isn't it more "work" to ungenerate them
> (using strip) then to just not install them?

The debug info gets built into the binary by gcc. We copy out the debug
info using objcopy --only-keep-debug, add a link with --gnu-debuglink
and then remove it from the binary using strip.

Without FEATURES=splitdebug the first two steps would be omitted, making
it exactly the same as what happens now. So, no more work.

Ed Catmur

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: Split ELF Debug (default or not?)

[Ned Ludd]

On Sun, 2005-11-27 at 15:22 +0000, Edward Catmur wrote:
> On Sun, 2005-11-27 at 08:40 -0500, Ned Ludd wrote:
> > On Sun, 2005-11-27 at 15:09 +0200, Ivan Yosifov wrote:
> > > And one more thing. For proper debugging, don't I need the source to be
> > > present ?
> > 
> > -g3 -ggdb embeds the source code in the debug info so I don't see the 
> > point.
> 
> It doesn't; at least not with gcc 3.4.4. It does embed function
> prototypes and macro definitions, though.
> 
> Ed Catmur

Eh? 


solar@simple ~ $ gcc -g3 -nopie -ggdb segfault_me.c -o segfault_me
solar@simple ~ $ paxctl -permsx segfault_me
solar@simple ~ $ gdb -q ./segfault_me
Using host libthread_db library "/lib/libthread_db.so.1".
(gdb) b main
Breakpoint 1 at 0x804853d: file segfault_me.c, line 25.
(gdb) step
The program is not being run.
(gdb) r
Starting program: /home/solar/segfault_me 

Breakpoint 1, main () at segfault_me.c:25
25		foo();
(gdb) step
foo () at segfault_me.c:17
17		int i = 0; 
(gdb) 
18		for (i = 0 ; i < 3; i++) {
(gdb) 
19			printf("Just an example %d\n", i);
(gdb) 
Just an example 0
18		for (i = 0 ; i < 3; i++) {
(gdb) 
19			printf("Just an example %d\n", i);
(gdb) 
Just an example 1
18		for (i = 0 ; i < 3; i++) {
(gdb) 
19			printf("Just an example %d\n", i);
(gdb) 
Just an example 2
18		for (i = 0 ; i < 3; i++) {
(gdb) 
21		bar();
(gdb) 
bar () at segfault_me.c:13
13		baz();
(gdb) 
baz () at segfault_me.c:9
9		atexit(break_self);
(gdb) 
10	}
(gdb) 
bar () at segfault_me.c:14
14	}
(gdb) 
foo () at segfault_me.c:22
22	}
(gdb) 
main () at segfault_me.c:26
26		return 0;
(gdb) 
27	}
(gdb) 
0x400407b6 in __libc_start_main () from /lib/libc.so.6
(gdb) 
Single stepping until exit from function __libc_start_main, 
which has no line number information.

Program received signal SIGSEGV, Segmentation fault.
0x40050681 in kill () from /lib/libc.so.6
(gdb) 

solar@simple ~ $ cat segfault_me.c 
#include <stdio.h>
#include <signal.h>

void break_self() {
	kill(getpid(), SIGSEGV);
}

int baz() {
	atexit(break_self);
}

void bar() {
	baz();
}

void foo() {
	int i; 
	for (i = 0 ; i < 3; i++) {
		printf("Just an example %d\n", i);
	}
	bar();
}

int main() {
	foo();
	return 0;
}



# While single stepping sure looks like source code to me. 

solar@simple ~ $ gcc -v 2>&1 | tail -n 1
gcc version 3.4.4 (Gentoo Hardened 3.4.4, HTB-3.4.4_pre-1.00,
ssp-3.4.4-1.0, pie-8.7.8)

-- 
Ned Ludd <solar@gentoo.org>
Gentoo Linux

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: Split ELF Debug (default or not?)

[Ned Ludd]

On Sun, 2005-11-27 at 10:44 -0500, Ned Ludd wrote:
> On Sun, 2005-11-27 at 15:22 +0000, Edward Catmur wrote:
> > On Sun, 2005-11-27 at 08:40 -0500, Ned Ludd wrote:
> > > On Sun, 2005-11-27 at 15:09 +0200, Ivan Yosifov wrote:
> > > > And one more thing. For proper debugging, don't I need the source to be
> > > > present ?
> > > 
> > > -g3 -ggdb embeds the source code in the debug info so I don't see the 
> > > point.
> > 
> > It doesn't; at least not with gcc 3.4.4. It does embed function
> > prototypes and macro definitions, though.
> > 
> > Ed Catmur
> 
> Eh? 


Never mind you were right Ed. taviso pointed out that dwarf2 doesnt 
support embedding the actual source. I was seeing the source due to me 
not having deleted the source.

Guess that is where the debugedit thing of Tester's would come in handy 
on glibc hosts.


-- 
Ned Ludd <solar@gentoo.org>
Gentoo Linux

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: Split ELF Debug (default or not?)

[Mark Loeser]

[-- Attachment #1: Type: text/plain, Size: 375 bytes --]

Ned Ludd <solar@gentoo.org> said:
> I really can't give an accurate example. Halcyon who has been testing it
> merged world and he was yeilded with 18M of debug info (I have no idea
> how many packages he has).

Just for the sake of reference, this was with 95 packages and CFLAGS="-O2 
-march=pentium4 -fomit-frame-pointer -pipe". It is essentially a stage3.

Mark

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Re: Split ELF Debug (default or not?)

[Olivier Crête]

[-- Attachment #1: Type: text/plain, Size: 692 bytes --]

On Sun, 2005-27-11 at 13:03 -0500, Mark Loeser wrote:
> Ned Ludd <solar@gentoo.org> said:
> > I really can't give an accurate example. Halcyon who has been testing it
> > merged world and he was yeilded with 18M of debug info (I have no idea
> > how many packages he has).
> 
> Just for the sake of reference, this was with 95 packages and CFLAGS="-O2 
> -march=pentium4 -fomit-frame-pointer -pipe". It is essentially a stage3.

I have been building stuff with CFLAGS="-O2 -pipe -ggdb" for a while
with my own keepdebug/keepsources patch and I have over 475 MB of debug
info and 246 MB of source code, but I have 1.5 GB inside /usr/lib ..

-- 
Olivier Crête
tester@gentoo.org

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Re: Split ELF Debug (default or not?)

[Ivan Yosifov]

On Sun, 2005-11-27 at 11:55 -0500, Ned Ludd wrote:
> On Sun, 2005-11-27 at 10:44 -0500, Ned Ludd wrote:
> > On Sun, 2005-11-27 at 15:22 +0000, Edward Catmur wrote:
> > > On Sun, 2005-11-27 at 08:40 -0500, Ned Ludd wrote:
> > > > On Sun, 2005-11-27 at 15:09 +0200, Ivan Yosifov wrote:
> > > > > And one more thing. For proper debugging, don't I need the source to be
> > > > > present ?
> > > > 
> > > > -g3 -ggdb embeds the source code in the debug info so I don't see the 
> > > > point.
> > > 
> > > It doesn't; at least not with gcc 3.4.4. It does embed function
> > > prototypes and macro definitions, though.
> > > 
> > > Ed Catmur
> > 
> > Eh? 
> 
> 
> Never mind you were right Ed. taviso pointed out that dwarf2 doesnt 
> support embedding the actual source. I was seeing the source due to me 
> not having deleted the source.
> 
> Guess that is where the debugedit thing of Tester's would come in handy 
> on glibc hosts.

What is this debugedit thing for us non-devs ? IMO portage should have
some way to keep the sources around for debugging, for the patch you are
proposing to be fully useful.

-- 
Cheers,
Ivan Yosifov.

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: Split ELF Debug (default or not?)

[Ned Ludd]

On Sun, 2005-11-27 at 23:01 +0200, Ivan Yosifov wrote:
> On Sun, 2005-11-27 at 11:55 -0500, Ned Ludd wrote:
> > On Sun, 2005-11-27 at 10:44 -0500, Ned Ludd wrote:
> > > On Sun, 2005-11-27 at 15:22 +0000, Edward Catmur wrote:
> > > > On Sun, 2005-11-27 at 08:40 -0500, Ned Ludd wrote:
> > > > > On Sun, 2005-11-27 at 15:09 +0200, Ivan Yosifov wrote:
> > > > > > And one more thing. For proper debugging, don't I need the source to be
> > > > > > present ?
> > > > > 
> > > > > -g3 -ggdb embeds the source code in the debug info so I don't see the 
> > > > > point.
> > > > 
> > > > It doesn't; at least not with gcc 3.4.4. It does embed function
> > > > prototypes and macro definitions, though.
> > > > 
> > > > Ed Catmur
> > > 
> > > Eh? 
> > 
> > 
> > Never mind you were right Ed. taviso pointed out that dwarf2 doesnt 
> > support embedding the actual source. I was seeing the source due to me 
> > not having deleted the source.
> > 
> > Guess that is where the debugedit thing of Tester's would come in handy 
> > on glibc hosts.
> 
> What is this debugedit thing for us non-devs ? IMO portage should have
> some way to keep the sources around for debugging, for the patch you are
> proposing to be fully useful.

Having the source around or not does not make the splitdebug feature any
less useful. debugedit would however enhance some aspects of debugging.
See the comments posted at the top of this thread from tester@gentoo on 
debugedit.

-- 
Ned Ludd <solar@gentoo.org>
Gentoo Linux

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Split ELF Debug (defult or not?)

[Spider (D.m.D. Lj.)]

[-- Attachment #1: Type: text/plain, Size: 1865 bytes --]

On Sat, 2005-11-26 at 12:50 -0500, Ned Ludd wrote:
> Good afternoon,
> 
> probably in portage-2.0.54 a patch will be added to emit split debug
> info. Having a split debug allows us to retain all the advantages of
> stripping executables while gaining the ability to properly debug
> executables in bfd aware programs. It's been in testing with a small
> hand full of devs and works quite well, but before it's pushed in we
> would like to get input from our devs & users.
> 
> Would you be willing to give up space in $ROOT/usr/lib/debug for ELF
> executables by default in order to aid in better debugging by or do we
> want to only emit it when a FEATURE= is defined.
> 
> Having a split debug pretty much obsoletes the need to add nostrip to
> your features in order to get debug info.
> 
> Users wishing to not have debug info can simply add
> INSTALL_MASK="/usr/lib/debug ${INSTALL_MASK}" to make.conf or the
> environment unless we make it FEATURE based.
> 
> I'm in favor of it enabled per default but I'd like to know what you
> think and why. (advantages of on/off by default etc..)
> 
> Anybody wanting to test or make use of this feature right away can grab
> a copy of my prepstrip from
> http://dev.gentoo.org/~solar/portage_misc/prepstrip and save it to
> /usr/lib/portage/bin/prepstrip or patch portage with
> http://dev.gentoo.org/~solar/patch_overlay/sys-apps/portage/portage-2.0.53_rc7-prepstrip.patch
> It requires you merge pax-utils for the scanelf util.
> 


I would certainly like this as on by default, and controlled via
INSTALL_MASK rather than a FEATURE.



(Although I'd have to rebuild my complete system, since I do have it all
with debug-symbols enabled ;)

//Spider

-- 
begin  .signature
Tortured users / Laughing in pain
See Microsoft KB Article Q265230 for more information.
end


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Split ELF Debug (defult or not?)

[Georgi Georgiev]

[-- Attachment #1: Type: text/plain, Size: 969 bytes --]

maillog: 27/11/2005-13:54:33(+0100): Diego 'Flameeyes' Pettenò types
> On Sunday 27 November 2005 00:10, Luca Barbato wrote:
> > It's great!
> > Make it a FEATURE default on for common profiles.
> +1, and it would be better if the FEATURES, instead of removing the generated 
> files, would disable the building of them completely, mainly because "work" 
> systems with limited CPU, RAM and HDD space would probably prefer not having 
> to create and store them :)

Wouldn't that break binary packages? I.e., if a binary package is built
on a system with FEATURES=-splitdebug and then merged on a system with
FEATURES=splitdebug the debug info would be missing.

-- 
\    Georgi Georgiev   \  "What a wonder is USENET; such wholesale   \
/     chutz@gg3.net    /  production of conjecture from such a       /
\  http://www.gg3.net/ \  trifling investment in fact." ale-- Carl   \
/  ------------------- /  S. Gutekunst                               /

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Split ELF Debug (defult or not?)

[Edward Catmur]

On Mon, 2005-11-28 at 10:18 +0900, Georgi Georgiev wrote:
> maillog: 27/11/2005-13:54:33(+0100): Diego 'Flameeyes' Pettenò types
> > On Sunday 27 November 2005 00:10, Luca Barbato wrote:
> > > It's great!
> > > Make it a FEATURE default on for common profiles.
> > +1, and it would be better if the FEATURES, instead of removing the generated 
> > files, would disable the building of them completely, mainly because "work" 
> > systems with limited CPU, RAM and HDD space would probably prefer not having 
> > to create and store them :)
> 
> Wouldn't that break binary packages? I.e., if a binary package is built
> on a system with FEATURES=-splitdebug and then merged on a system with
> FEATURES=splitdebug the debug info would be missing.

There's nothing to break. The only program that looks for debug symbols
is gdb; if they're not there it does what it does currently; print "(no
debugging symbols found)".

Ed Catmur

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Split ELF Debug (defult or not?)

[Paul de Vrieze]

[-- Attachment #1: Type: text/plain, Size: 1612 bytes --]

On Saturday 26 November 2005 18:50, Ned Ludd wrote:
> Good afternoon,
>
> probably in portage-2.0.54 a patch will be added to emit split debug
> info. Having a split debug allows us to retain all the advantages of
> stripping executables while gaining the ability to properly debug
> executables in bfd aware programs. It's been in testing with a small
> hand full of devs and works quite well, but before it's pushed in we
> would like to get input from our devs & users.
>
> Would you be willing to give up space in $ROOT/usr/lib/debug for ELF
> executables by default in order to aid in better debugging by or do we
> want to only emit it when a FEATURE= is defined.
>
> Having a split debug pretty much obsoletes the need to add nostrip to
> your features in order to get debug info.

If we decide to do this, I think we should also find a solution that 
ensures that the executables actually get built with the debugging 
information, and that the upstream makefiles don't strip the binaries 
before we can stop them to.

I see two things we can do to fix these problems:
- In portage mode, add a null split binary to the path, such that
  makefiles that try to split, actually don't. Easier than anything else
  as specifying --debug often also means that you get awful CFLAGS
- Make gcc-wrapper be smart about it, and inject a "-g" CFLAG into the
  command line of the actual gcc.

Of course this all should be turned on/off with a feature or useflag or 
whatever.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Re: Split ELF Debug (default or not?)

[Paul de Vrieze]

[-- Attachment #1: Type: text/plain, Size: 761 bytes --]

On Sunday 27 November 2005 22:01, Ivan Yosifov wrote:
>
> What is this debugedit thing for us non-devs ? IMO portage should have
> some way to keep the sources around for debugging, for the patch you
> are proposing to be fully useful.

It allows some hacks on the debug info. For example the debug info records 
the location where the source was when the executable was built. 
Debugedit allows one to change this to some other location. As one 
doesn't want to retain the source in /var/tmp, this is probably a good 
idea. Besides that one can use debugedit to find out which files are 
involved and put only those at the other location.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Split ELF Debug (defult or not?)

[Paul de Vrieze]

[-- Attachment #1: Type: text/plain, Size: 747 bytes --]

On Sunday 27 November 2005 16:30, Dan Meltzer wrote:
> Could this debug info be NFS shared? assuming like computers, or would
> it be different on each computer.

It is probably as shareable as normal library files are. Maybe there is 
more in common, but if the source was different, the end result will be. 
As much code contains conditionals based a.o. on the system architecture 
built on, the information would probably be incorrect if e.g. generated 
on a x86, but used on a ppc. I'm not an expert, so it might even be that 
the information is incompatible, but I wouldn't expect it to (except from 
endianess problems).

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]