[gentoo-dev] check-reqs* vs CFLAGS=-g

[gentoo-dev] check-reqs* vs CFLAGS=-g

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1359 bytes --]

Hello,

Since LLVM builds have grown in size lately, I wanted to add some of
check-reqs-r1 checks to it. However, I'm having real trouble guessing
what the correct sizes should be.

Most importantly, as bug #479356 points out, using '-g' greatly
increases the build size. My small measures show that without that
option, LLVM has peek build space consumption around:

- 400-550M without clang (depending on targets),
- 950-1200M with clang,
- 16G with clang & USE=debug (assertions, checks).

Those measures were done with '-O2', USE=-debug and single ABI build.
multilib build consumes almost twice as much.

But if we change the flags, for complete llvm+clang:

- 1.2G for -O2 (as shown above),
- 12G for -O0 -g.

With such a difference, I don't really see using one of the two values.
If I go for the lower one, '-g' users may still hit unexpected
out-of-space issues (like the bug shows). If I go for the higher one,
many users (including me) will unnecessarily hit the space constraints.

What can we do to improve this? I'm not really happy to have LLVM
ebuild analyze CFLAGS to set proper space constraints. Maybe we should
make check-reqs-r1 automatically bump the constraints by some
statistical multiplier when it detects -g?

[1]:https://bugs.gentoo.org/show_bug.cgi?id=479356

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]

[gentoo-dev] Re: check-reqs* vs CFLAGS=-g

[Duncan]

Michał Górny posted on Thu, 01 Aug 2013 13:33:48 +0200 as excerpted:

> LLVM has peek build space consumption around:
> 
> - 400-550M without clang (depending on targets),
> - 950-1200M with clang,
> - 16G with clang & USE=debug (assertions, checks).

Ouch!

Thanks for the heads-up.  I didn't realize -g/debug added THAT much!  For 
sure I'll have to keep that in mind if I ever decide to build llvm with 
debug... and the general rule in mind for building anything else with 
debug.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Re: check-reqs* vs CFLAGS=-g

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 798 bytes --]

Dnia 2013-08-02, o godz. 02:07:18
Duncan <1i5t5.duncan@cox.net> napisał(a):

> Michał Górny posted on Thu, 01 Aug 2013 13:33:48 +0200 as excerpted:
> 
> > LLVM has peek build space consumption around:
> > 
> > - 400-550M without clang (depending on targets),
> > - 950-1200M with clang,
> > - 16G with clang & USE=debug (assertions, checks).
> 
> Ouch!
> 
> Thanks for the heads-up.  I didn't realize -g/debug added THAT much!  For 
> sure I'll have to keep that in mind if I ever decide to build llvm with 
> debug... and the general rule in mind for building anything else with 
> debug.

Just to make it clear, USE=debug doesn't imply -g. It gets 16G with
-O2. Curious enough, -O0 -g gave me 14G but maybe I screwed something
up :).

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]

Re: [gentoo-dev] Re: check-reqs* vs CFLAGS=-g

[Diego Elio Pettenò]

[-- Attachment #1: Type: text/plain, Size: 1254 bytes --]

Unlikely you screwed up, -O0 makes bigger code than -O2 almost in every
case; then -g annotates it. I'm expecting -ggdb to take some few GBs more.

It'll be the same if not worse with almost all software, -g3 would make it
even worse.


Diego Elio Pettenò — Flameeyes
flameeyes@flameeyes.eu — http://blog.flameeyes.eu/


On Fri, Aug 2, 2013 at 9:05 AM, Michał Górny <mgorny@gentoo.org> wrote:

> Dnia 2013-08-02, o godz. 02:07:18
> Duncan <1i5t5.duncan@cox.net> napisał(a):
>
> > Michał Górny posted on Thu, 01 Aug 2013 13:33:48 +0200 as excerpted:
> >
> > > LLVM has peek build space consumption around:
> > >
> > > - 400-550M without clang (depending on targets),
> > > - 950-1200M with clang,
> > > - 16G with clang & USE=debug (assertions, checks).
> >
> > Ouch!
> >
> > Thanks for the heads-up.  I didn't realize -g/debug added THAT much!  For
> > sure I'll have to keep that in mind if I ever decide to build llvm with
> > debug... and the general rule in mind for building anything else with
> > debug.
>
> Just to make it clear, USE=debug doesn't imply -g. It gets 16G with
> -O2. Curious enough, -O0 -g gave me 14G but maybe I screwed something
> up :).
>
> --
> Best regards,
> Michał Górny
>

[-- Attachment #2: Type: text/html, Size: 1977 bytes --]

Re: [gentoo-dev] check-reqs* vs CFLAGS=-g

[Andreas K. Huettel]

Am Donnerstag 01 August 2013, 13:33:48 schrieb Michał Górny:
>
> - 1.2G for -O2 (as shown above),
> - 12G for -O0 -g.
> 

I thought -O0 was generally discouraged, even for debugging?!

-- 
Andreas K. Huettel
Gentoo Linux developer
kde, sci, arm, tex, printing

Re: [gentoo-dev] check-reqs* vs CFLAGS=-g

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 440 bytes --]

Dnia 2013-08-02, o godz. 14:08:46
"Andreas K. Huettel" <dilfridge@gentoo.org> napisał(a):

> Am Donnerstag 01 August 2013, 13:33:48 schrieb Michał Górny:
> >
> > - 1.2G for -O2 (as shown above),
> > - 12G for -O0 -g.
> > 
> 
> I thought -O0 was generally discouraged, even for debugging?!

Depends on what you want to achieve. It's often hard to debug when
stuff is 'optimized out' :).

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]

Re: [gentoo-dev] check-reqs* vs CFLAGS=-g

[Diego Elio Pettenò]

[-- Attachment #1: Type: text/plain, Size: 894 bytes --]

On Fri, Aug 2, 2013 at 1:08 PM, Andreas K. Huettel <dilfridge@gentoo.org>wrote:

> I thought -O0 was generally discouraged, even for debugging?!


As Michał said, it all depends on what you want to debug. I would say that
for 90% of issues you *do not* want to use -O0. Your code might not even
compile (libav for instance used to rely heavily in the DCE pass being
executed, GCC disables DCE at -O0), and even if it, you're now given a
different program altogether, so if you're looking for a crash, it might
magically disappear (memory areas get cleared out at -O0 but they might be
re-used without clearing at any other -O level).

But if you're trying to step through an algorithm, an optimized version of
the code will most likely reorder the code that you read from the sources.


Diego Elio Pettenò — Flameeyes
flameeyes@flameeyes.eu — http://blog.flameeyes.eu/

[-- Attachment #2: Type: text/html, Size: 1435 bytes --]

[gentoo-dev] Re: check-reqs* vs CFLAGS=-g

[Ryan Hill]

[-- Attachment #1: Type: text/plain, Size: 996 bytes --]

On Fri, 2 Aug 2013 13:47:10 +0100
Diego Elio Pettenò <flameeyes@flameeyes.eu> wrote:

> On Fri, Aug 2, 2013 at 1:08 PM, Andreas K. Huettel
> <dilfridge@gentoo.org>wrote:
> 
> > I thought -O0 was generally discouraged, even for debugging?!
> 
> 
> As Michał said, it all depends on what you want to debug. I would say that
> for 90% of issues you *do not* want to use -O0. Your code might not even
> compile (libav for instance used to rely heavily in the DCE pass being
> executed, GCC disables DCE at -O0), and even if it, you're now given a
> different program altogether, so if you're looking for a crash, it might
> magically disappear (memory areas get cleared out at -O0 but they might be
> re-used without clearing at any other -O level).

If you're feeling adventurous you could try -Og -g with gcc 4.8.


-- 
Ryan Hill                        psn: dirtyepic_sk
   gcc-porting/toolchain/wxwidgets @ gentoo.org

47C3 6D62 4864 0E49 8E9E  7F92 ED38 BD49 957A 8463

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

[gentoo-dev] Re: check-reqs* vs CFLAGS=-g

[Ryan Hill]

[-- Attachment #1: Type: text/plain, Size: 874 bytes --]

On Thu, 1 Aug 2013 13:33:48 +0200
Michał Górny <mgorny@gentoo.org> wrote:

> What can we do to improve this? I'm not really happy to have LLVM
> ebuild analyze CFLAGS to set proper space constraints. Maybe we should
> make check-reqs-r1 automatically bump the constraints by some
> statistical multiplier when it detects -g?

The problem with that is there is no way to determine how much larger a -g
build will be.  It varies widely across GCC versions (eg. when var-tracking was
added the size of debug info exploded, in some cases up to an order of
magnitude), and different -g/-ggdb levels.  All you can really do is warn people
they may run out of space if they're using debugging options.


-- 
Ryan Hill                        psn: dirtyepic_sk
   gcc-porting/toolchain/wxwidgets @ gentoo.org

47C3 6D62 4864 0E49 8E9E  7F92 ED38 BD49 957A 8463

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 490 bytes --]