[gentoo-dev] impending c++11 clusterfuck?

[gentoo-dev] impending c++11 clusterfuck?

[Gregory M. Turner]

I'm quoting myself from bug #566328 here.  These were off-the-cuff
remarks that got away from me and became a call-to-arms...

(In reply to Michał Górny from comment #7)
> This is never this simple. C++11 can change the ABI. So the point kinda is,
> we need to ensure that all C++ libraries in a depgraph use the same C++
> version.

This is pretty awful when you really think about it.  I feel like I'm
watching a train-wreck in super slow motion.

I'm not sure we're taking this seriously enough -- sooner or later it
seems destined to become a major clusterfuck if we don't do something
proactive about it now while the drawing-board is relatively
uncluttered.

The only thing I can think of that has this kind of two-way depgraph
magic property are the major "abi" USE_EXPAND values (multilib-build
and python-r1, in other words).

But those rely on fancy framework-generated USE-flag deps, which seem
like overkill and likely to incur unjustifiable user-experience-costs.

Perhaps a solution to this cxx11 clusterfuck can be found that works
more like perl?  By that I mean, pick your poison (respectively, your
cxx11 ABI of preference or your major perl version of choice), rely on
inbuilt portage features do the trick most of the time, and, when it
breaks, run "magically-fix-everything.sh," grab a caffeinated beverage
or three and fire up your favorite VOD client while the mess gets
magically cleaned up by robots somehow.

-gmt

Greg Turner
gmt@be-evil.net

Re: [gentoo-dev] impending c++11 clusterfuck?

[C Bergström]

On Mon, Nov 30, 2015 at 10:56 AM, Gregory M. Turner <gmt@be-evil.net> wrote:
> I'm quoting myself from bug #566328 here.  These were off-the-cuff
> remarks that got away from me and became a call-to-arms...
>
> (In reply to Michał Górny from comment #7)
>> This is never this simple. C++11 can change the ABI. So the point kinda is,
>> we need to ensure that all C++ libraries in a depgraph use the same C++
>> version.
>
> This is pretty awful when you really think about it.  I feel like I'm
> watching a train-wreck in super slow motion.
>
> I'm not sure we're taking this seriously enough -- sooner or later it
> seems destined to become a major clusterfuck if we don't do something
> proactive about it now while the drawing-board is relatively
> uncluttered.
>
> The only thing I can think of that has this kind of two-way depgraph
> magic property are the major "abi" USE_EXPAND values (multilib-build
> and python-r1, in other words).
>
> But those rely on fancy framework-generated USE-flag deps, which seem
> like overkill and likely to incur unjustifiable user-experience-costs.
>
> Perhaps a solution to this cxx11 clusterfuck can be found that works
> more like perl?  By that I mean, pick your poison (respectively, your
> cxx11 ABI of preference or your major perl version of choice), rely on
> inbuilt portage features do the trick most of the time, and, when it
> breaks, run "magically-fix-everything.sh," grab a caffeinated beverage
> or three and fire up your favorite VOD client while the mess gets
> magically cleaned up by robots somehow.

I'm not sure I fully understand the problem, but what about
a. Always define the c++11 flag
b. Not support a system with mixed (and incompatible) c++ standards

I personally don't think it seems reasonable to try to carry both
c++03 and c++11 at the same time. This especially seems like nonsense
to me in the gentoo world. gcc has or will soon switch to c++11 by
default. Packages which can't be built will start having issues
anyway.

Re: [gentoo-dev] impending c++11 clusterfuck?

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2842 bytes --]

On Sun, 29 Nov 2015 19:56:04 -0800
"Gregory M. Turner" <gmt@be-evil.net> wrote:

> I'm quoting myself from bug #566328 here.  These were off-the-cuff
> remarks that got away from me and became a call-to-arms...
> 
> (In reply to Michał Górny from comment #7)
> > This is never this simple. C++11 can change the ABI. So the point kinda is,
> > we need to ensure that all C++ libraries in a depgraph use the same C++
> > version.  
> 
> This is pretty awful when you really think about it.  I feel like I'm
> watching a train-wreck in super slow motion.

Well, it's not that bad actually. After some thinking, I figured out
they fixed most 98/11 incompatibilities around gcc 4.8/4.9, and left
only a few 'unlikely' to cause issues.

However, if one dep switches to C++11, it is quite likely to require
C++11 in its revdeps, and that's what happening with libsigc++
and other gtkmm libraries.

Plus, there's of course the classical issue of ABI incompatibility
between libstdc++ bundled with 4.9 and 5.1, and 5.2... so along with
switching g++ version, you soon start to have to rebuild random C++
libraries.

And the issue of supporting alternative C++ standard library
implementations -- like using libcxx with clang. They are of course
incompatible with GNU's ever-changing ABI.

> I'm not sure we're taking this seriously enough -- sooner or later it
> seems destined to become a major clusterfuck if we don't do something
> proactive about it now while the drawing-board is relatively
> uncluttered.
> 
> The only thing I can think of that has this kind of two-way depgraph
> magic property are the major "abi" USE_EXPAND values (multilib-build
> and python-r1, in other words).
> 
> But those rely on fancy framework-generated USE-flag deps, which seem
> like overkill and likely to incur unjustifiable user-experience-costs.

Yes, it is terrible. You end up introducing a lot of USE flags that
need to be manually switched along with gcc versions. If we start
splitting them between c++98 and c++11, we're quite likely to hit USE
flag conflicts between packages/developers which prefer one over
another.

> Perhaps a solution to this cxx11 clusterfuck can be found that works
> more like perl?  By that I mean, pick your poison (respectively, your
> cxx11 ABI of preference or your major perl version of choice), rely on
> inbuilt portage features do the trick most of the time, and, when it
> breaks, run "magically-fix-everything.sh," grab a caffeinated beverage
> or three and fire up your favorite VOD client while the mess gets
> magically cleaned up by robots somehow.

Sadly := can't help here since gcc switches occur independently of
package installs. And AFAIK revdep-rebuild doesn't help either.
-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] impending c++11 clusterfuck?

[Anthony G. Basile]

On 11/30/15 1:42 AM, Michał Górny wrote:
> On Sun, 29 Nov 2015 19:56:04 -0800
> "Gregory M. Turner" <gmt@be-evil.net> wrote:
>
>> I'm quoting myself from bug #566328 here.  These were off-the-cuff
>> remarks that got away from me and became a call-to-arms...
>>
>> (In reply to Michał Górny from comment #7)
>>> This is never this simple. C++11 can change the ABI. So the point kinda is,
>>> we need to ensure that all C++ libraries in a depgraph use the same C++
>>> version.  
>> This is pretty awful when you really think about it.  I feel like I'm
>> watching a train-wreck in super slow motion.
> Well, it's not that bad actually. After some thinking, I figured out
> they fixed most 98/11 incompatibilities around gcc 4.8/4.9, and left
> only a few 'unlikely' to cause issues.
>
> However, if one dep switches to C++11, it is quite likely to require
> C++11 in its revdeps, and that's what happening with libsigc++
> and other gtkmm libraries.
When building a package, you can't just switch between -std=gnu++98 or
c++99 or gnu++11 or c++11 since there are syntactic difference.
>
> Plus, there's of course the classical issue of ABI incompatibility
> between libstdc++ bundled with 4.9 and 5.1, and 5.2... so along with
> switching g++ version, you soon start to have to rebuild random C++
> libraries.
>
> And the issue of supporting alternative C++ standard library
> implementations -- like using libcxx with clang. They are of course
> incompatible with GNU's ever-changing ABI.
>
>> I'm not sure we're taking this seriously enough -- sooner or later it
>> seems destined to become a major clusterfuck if we don't do something
>> proactive about it now while the drawing-board is relatively
>> uncluttered.
>>
>> The only thing I can think of that has this kind of two-way depgraph
>> magic property are the major "abi" USE_EXPAND values (multilib-build
>> and python-r1, in other words).
>>
>> But those rely on fancy framework-generated USE-flag deps, which seem
>> like overkill and likely to incur unjustifiable user-experience-costs.
> Yes, it is terrible. You end up introducing a lot of USE flags that
> need to be manually switched along with gcc versions. If we start
> splitting them between c++98 and c++11, we're quite likely to hit USE
> flag conflicts between packages/developers which prefer one over
> another.

This would be a nightmare.

>
>> Perhaps a solution to this cxx11 clusterfuck can be found that works
>> more like perl?  By that I mean, pick your poison (respectively, your
>> cxx11 ABI of preference or your major perl version of choice), rely on
>> inbuilt portage features do the trick most of the time, and, when it
>> breaks, run "magically-fix-everything.sh," grab a caffeinated beverage
>> or three and fire up your favorite VOD client while the mess gets
>> magically cleaned up by robots somehow.
> Sadly := can't help here since gcc switches occur independently of
> package installs. And AFAIK revdep-rebuild doesn't help either.
You can run `revdep-rebuild -L 'libstdc\+\+\.so\.6'` to rebuild
everything that links against libstdc++.so.6.  This will rebuild a lot
of packages but will fix everything.

If we record enough information at build time (eg. gcc version or
libcxx/clang) then we can build tools that intelligently predict if
there's an abi incompatibility.  Unfortunately gcc doesn't bump soname
and/or version-info when it changes c++11 abi.  (since c++11 is
experimental and c++03/98 have stable abi, they don't want to force
rebuilds).  So we have to record the equivalent of an soname.  If we put
that information in a file like NEEDED.ELF.2 in vdb, it could be read by
utilities like magically-fix-everything.sh (a revddep-rebuild.sh for
libstdc++).

-- 
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail    : blueness@gentoo.org
GnuPG FP  : 1FED FAD9 D82C 52A5 3BAB  DC79 9384 FA6E F52D 4BBA
GnuPG ID  : F52D4BBA

Re: [gentoo-dev] impending c++11 clusterfuck?

[Mikhail Korolev]

2015-11-30 7:10 GMT+03:00 C Bergström <cbergstrom@pathscale.com>:
>
> On Mon, Nov 30, 2015 at 10:56 AM, Gregory M. Turner <gmt@be-evil.net> wrote:
> > I'm quoting myself from bug #566328 here.  These were off-the-cuff
> > remarks that got away from me and became a call-to-arms...
> >
> > (In reply to Michał Górny from comment #7)
> >> This is never this simple. C++11 can change the ABI. So the point kinda is,
> >> we need to ensure that all C++ libraries in a depgraph use the same C++
> >> version.
> >
> > This is pretty awful when you really think about it.  I feel like I'm
> > watching a train-wreck in super slow motion.
> >
> > I'm not sure we're taking this seriously enough -- sooner or later it
> > seems destined to become a major clusterfuck if we don't do something
> > proactive about it now while the drawing-board is relatively
> > uncluttered.
> >
> > The only thing I can think of that has this kind of two-way depgraph
> > magic property are the major "abi" USE_EXPAND values (multilib-build
> > and python-r1, in other words).
> >
> > But those rely on fancy framework-generated USE-flag deps, which seem
> > like overkill and likely to incur unjustifiable user-experience-costs.
> >
> > Perhaps a solution to this cxx11 clusterfuck can be found that works
> > more like perl?  By that I mean, pick your poison (respectively, your
> > cxx11 ABI of preference or your major perl version of choice), rely on
> > inbuilt portage features do the trick most of the time, and, when it
> > breaks, run "magically-fix-everything.sh," grab a caffeinated beverage
> > or three and fire up your favorite VOD client while the mess gets
> > magically cleaned up by robots somehow.
>
> I'm not sure I fully understand the problem, but what about
> a. Always define the c++11 flag
> b. Not support a system with mixed (and incompatible) c++ standards
>
> I personally don't think it seems reasonable to try to carry both
> c++03 and c++11 at the same time. This especially seems like nonsense
> to me in the gentoo world. gcc has or will soon switch to c++11 by
> default. Packages which can't be built will start having issues
> anyway.
>

But there are such packages.
```
 [ ~ ] # grep "=c++11" /etc/portage/make.conf
CXXFLAGS="${CFLAGS} -std=c++11"
 [ ~ ] # cat /etc/portage/package.env
sys-libs/libcxx clang
sys-libs/libcxxrt clang

media-gfx/blender cxxflags_std-gnupp11
dev-qt/qtwebkit:4 cxxflags_std-cpp03
kde-base/kdelibs:4 cxxflags_std-cpp03
media-video/vlc cxxflags_std-gnupp11

 [ ~ ] # ls -d1 /var/db/pkg/*/* | wc -l
1294
```

Re: [gentoo-dev] impending c++11 clusterfuck?

[Gregory M. Turner]

On Sun, Nov 29, 2015 at 10:42 PM, Michał Górny <mgorny@gentoo.org> wrote:
> On Sun, 29 Nov 2015 19:56:04 -0800
> "Gregory M. Turner" <gmt@be-evil.net> wrote:
> >  the mess gets magically cleaned up by robots somehow.
>
> Sadly := can't help here since gcc switches occur independently of
> package installs. And AFAIK revdep-rebuild doesn't help either.

Indeed... magical hand-wavey part is the only teeny-tiny flaw with my
proposal that the problem be magically, hand-wavically solved --
otherwise, it's rock-solid :)

OTOH when you're spending ten hours a week waiting for revdep-rebuild,
preserved-rebuild stops seeming like an intractable fantasy and more
like an interesting puzzle you'd might as well take a crack at
solving.

I dunno about y'all but my not-really-that-computationally-powerful
workstation has 2000 packages installed, about three-quarters of which
seem to have forked webkit (just kidding, but it sure feels that way).

emerge -e @world literally takes me two days if I'm lucky.  If I'm
not, more like a week, with my cores left mostly idling while my
productivity is thwarted by various non-webkit-compiling distractions
like using the toilet or working.

Point being, it's damn slow; I'm starting to feel like the guy waiting
for revdep-rebuild all week.  There's gotta be a way to do it.  If it
requires some encapsulation-breaking hacks to get it done (as
preserved-rebuild kinda did) it's probably well worth it.

-gmt

Greg Turner
gmt@be-evil.net

Re: [gentoo-dev] impending c++11 clusterfuck?

[Michał Górny]

Dnia 30 listopada 2015 09:07:30 CET, "Anthony G. Basile" <blueness@gentoo.org> napisał(a):
>On 11/30/15 1:42 AM, Michał Górny wrote:
>> On Sun, 29 Nov 2015 19:56:04 -0800
>> "Gregory M. Turner" <gmt@be-evil.net> wrote:
>>
>>> I'm quoting myself from bug #566328 here.  These were off-the-cuff
>>> remarks that got away from me and became a call-to-arms...
>>>
>>> (In reply to Michał Górny from comment #7)
>>>> This is never this simple. C++11 can change the ABI. So the point
>kinda is,
>>>> we need to ensure that all C++ libraries in a depgraph use the same
>C++
>>>> version.  
>>> This is pretty awful when you really think about it.  I feel like
>I'm
>>> watching a train-wreck in super slow motion.
>> Well, it's not that bad actually. After some thinking, I figured out
>> they fixed most 98/11 incompatibilities around gcc 4.8/4.9, and left
>> only a few 'unlikely' to cause issues.
>>
>> However, if one dep switches to C++11, it is quite likely to require
>> C++11 in its revdeps, and that's what happening with libsigc++
>> and other gtkmm libraries.
>When building a package, you can't just switch between -std=gnu++98 or
>c++99 or gnu++11 or c++11 since there are syntactic difference.
>>
>> Plus, there's of course the classical issue of ABI incompatibility
>> between libstdc++ bundled with 4.9 and 5.1, and 5.2... so along with
>> switching g++ version, you soon start to have to rebuild random C++
>> libraries.
>>
>> And the issue of supporting alternative C++ standard library
>> implementations -- like using libcxx with clang. They are of course
>> incompatible with GNU's ever-changing ABI.
>>
>>> I'm not sure we're taking this seriously enough -- sooner or later
>it
>>> seems destined to become a major clusterfuck if we don't do
>something
>>> proactive about it now while the drawing-board is relatively
>>> uncluttered.
>>>
>>> The only thing I can think of that has this kind of two-way depgraph
>>> magic property are the major "abi" USE_EXPAND values (multilib-build
>>> and python-r1, in other words).
>>>
>>> But those rely on fancy framework-generated USE-flag deps, which
>seem
>>> like overkill and likely to incur unjustifiable
>user-experience-costs.
>> Yes, it is terrible. You end up introducing a lot of USE flags that
>> need to be manually switched along with gcc versions. If we start
>> splitting them between c++98 and c++11, we're quite likely to hit USE
>> flag conflicts between packages/developers which prefer one over
>> another.
>
>This would be a nightmare.
>
>>
>>> Perhaps a solution to this cxx11 clusterfuck can be found that works
>>> more like perl?  By that I mean, pick your poison (respectively,
>your
>>> cxx11 ABI of preference or your major perl version of choice), rely
>on
>>> inbuilt portage features do the trick most of the time, and, when it
>>> breaks, run "magically-fix-everything.sh," grab a caffeinated
>beverage
>>> or three and fire up your favorite VOD client while the mess gets
>>> magically cleaned up by robots somehow.
>> Sadly := can't help here since gcc switches occur independently of
>> package installs. And AFAIK revdep-rebuild doesn't help either.
>You can run `revdep-rebuild -L 'libstdc\+\+\.so\.6'` to rebuild
>everything that links against libstdc++.so.6.  This will rebuild a lot
>of packages but will fix everything.
>
>If we record enough information at build time (eg. gcc version or
>libcxx/clang) then we can build tools that intelligently predict if
>there's an abi incompatibility.  Unfortunately gcc doesn't bump soname
>and/or version-info when it changes c++11 abi.  (since c++11 is
>experimental and c++03/98 have stable abi, they don't want to force
>rebuilds).  So we have to record the equivalent of an soname.  If we
>put
>that information in a file like NEEDED.ELF.2 in vdb, it could be read
>by
>utilities like magically-fix-everything.sh (a revddep-rebuild.sh for
>libstdc++).

In my case, checking CXX + library symbols (to distinguish C++ libraries) works. But most of the people believe setting CXX to a static version is a bad idea, and it's better to use implicit magic of gcc-config.

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: [gentoo-dev] impending c++11 clusterfuck?

[Michał Górny]

Dnia 30 listopada 2015 10:18:53 CET, "Gregory M. Turner" <gmt@be-evil.net> napisał(a):
>On Sun, Nov 29, 2015 at 10:42 PM, Michał Górny <mgorny@gentoo.org>
>wrote:
>> On Sun, 29 Nov 2015 19:56:04 -0800
>> "Gregory M. Turner" <gmt@be-evil.net> wrote:
>> >  the mess gets magically cleaned up by robots somehow.
>>
>> Sadly := can't help here since gcc switches occur independently of
>> package installs. And AFAIK revdep-rebuild doesn't help either.
>
>Indeed... magical hand-wavey part is the only teeny-tiny flaw with my
>proposal that the problem be magically, hand-wavically solved --
>otherwise, it's rock-solid :)
>
>OTOH when you're spending ten hours a week waiting for revdep-rebuild,
>preserved-rebuild stops seeming like an intractable fantasy and more
>like an interesting puzzle you'd might as well take a crack at
>solving.

It won't help either since SONAME doesn't change. And as would Ciaran say, it's the wrong solution to a different problem.

>
>I dunno about y'all but my not-really-that-computationally-powerful
>workstation has 2000 packages installed, about three-quarters of which
>seem to have forked webkit (just kidding, but it sure feels that way).
>
>emerge -e @world literally takes me two days if I'm lucky.  If I'm
>not, more like a week, with my cores left mostly idling while my
>productivity is thwarted by various non-webkit-compiling distractions
>like using the toilet or working.
>
>Point being, it's damn slow; I'm starting to feel like the guy waiting
>for revdep-rebuild all week.  There's gotta be a way to do it.  If it
>requires some encapsulation-breaking hacks to get it done (as
>preserved-rebuild kinda did) it's probably well worth it.
>
>-gmt
>
>Greg Turner
>gmt@be-evil.net

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: [gentoo-dev] impending c++11 clusterfuck?

[Greg Turner]

On Mon, Nov 30, 2015 at 12:07 AM, Anthony G. Basile <blueness@gentoo.org> wrote:
> If we record enough information at build time (eg. gcc version or
> libcxx/clang) then we can build tools that intelligently predict if
> there's an abi incompatibility.

Yes, exactly the type of thing I'm thinking of. As you go on to note,

> Unfortunately gcc doesn't bump soname
> and/or version-info when it changes c++11 abi.

So, maybe kludge #1 we require is some kind of made-up section header
injected into object files... let's say, hypothetically, a tuple from
the cartesian cross-product of something vaguely like:

<
compiler={gcc34,gcc47,gcc49,gcc52,clang}
arch={$(< profiles/arch.list)} or maybe {(gnu target triplets)}
abi={c++03,c++11,gnu++11,plain-ol'-c}
>

i.e.: gcc52-amd64-gnu++11. Probably the arch is redundant -- hell,
maybe the compiler too -- and could be dropped... but you get the
idea.

So then, some agent digs this out from executables, and, like you say,
caches it in the vdb somewhere.

Meanwhile, suppose there's a global eselect c++abi setting.  Once it's
changed, we could just zip through the vdb and, say, "maybe some of
these are false positives, but, to be safe, here's a nag to rebuild
these so-and-so packages, which seem to have been built against some
c++ abi other than the current one."

No depgraph required, easy to implement magically-fix-it.sh.  Might
spin its wheels a bit but how often would you change this setting?
Almost never...

Well just thinking out loud, I probably have horrible thinkos above,
but ya gotta start somewhere...

-gmt

Greg Turner
gmt@be-evil.net

Re: [gentoo-dev] impending c++11 clusterfuck?

[Greg Turner]

Actually, come to think of it, there's an even dumber, easier
approach: just have some "is it c++" heuristics like, does it
dynamically link to a libstdc++?  Did it inhert qt.eclass?  etc.  So
we have a boolean "is it probably a c++ package" heuristic boolean (or
maybe a vector of per-compiler maybe booleans).  It'd be ok to have
some false positives, and pretty bad to have false negatives.

Now for each "probably c++" package merged, just store away the global
(per-compiler?) c++abi value (for false positives this would be
meaningless noise).

Now when we eselect c++abi set some-new-value, we could just take all
those maybe meaningless -- but presumptively not
meaningful-but-incorrect -- values in the vdb, and queue up rebuild
nags for all the ones that don't match.
-gmt

Greg Turner
gmt@be-evil.net


On Mon, Nov 30, 2015 at 2:16 AM, Greg Turner <gmt@be-evil.net> wrote:
> On Mon, Nov 30, 2015 at 12:07 AM, Anthony G. Basile <blueness@gentoo.org> wrote:
>> If we record enough information at build time (eg. gcc version or
>> libcxx/clang) then we can build tools that intelligently predict if
>> there's an abi incompatibility.
>
> Yes, exactly the type of thing I'm thinking of. As you go on to note,
>
>> Unfortunately gcc doesn't bump soname
>> and/or version-info when it changes c++11 abi.
>
> So, maybe kludge #1 we require is some kind of made-up section header
> injected into object files... let's say, hypothetically, a tuple from
> the cartesian cross-product of something vaguely like:
>
> <
> compiler={gcc34,gcc47,gcc49,gcc52,clang}
> arch={$(< profiles/arch.list)} or maybe {(gnu target triplets)}
> abi={c++03,c++11,gnu++11,plain-ol'-c}
>>
>
> i.e.: gcc52-amd64-gnu++11. Probably the arch is redundant -- hell,
> maybe the compiler too -- and could be dropped... but you get the
> idea.
>
> So then, some agent digs this out from executables, and, like you say,
> caches it in the vdb somewhere.
>
> Meanwhile, suppose there's a global eselect c++abi setting.  Once it's
> changed, we could just zip through the vdb and, say, "maybe some of
> these are false positives, but, to be safe, here's a nag to rebuild
> these so-and-so packages, which seem to have been built against some
> c++ abi other than the current one."
>
> No depgraph required, easy to implement magically-fix-it.sh.  Might
> spin its wheels a bit but how often would you change this setting?
> Almost never...
>
> Well just thinking out loud, I probably have horrible thinkos above,
> but ya gotta start somewhere...
>
> -gmt
>
> Greg Turner
> gmt@be-evil.net

Re: [gentoo-dev] impending c++11 clusterfuck?

[Anthony G. Basile]

On 11/30/15 4:52 AM, Michał Górny wrote:
>
> Dnia 30 listopada 2015 09:07:30 CET, "Anthony G. Basile" <blueness@gentoo.org> napisał(a):
>> On 11/30/15 1:42 AM, Michał Górny wrote:
>>> On Sun, 29 Nov 2015 19:56:04 -0800
>>> "Gregory M. Turner" <gmt@be-evil.net> wrote:
>>>
>>>> I'm quoting myself from bug #566328 here.  These were off-the-cuff
>>>> remarks that got away from me and became a call-to-arms...
>>>>
>>>> (In reply to Michał Górny from comment #7)
>>>>> This is never this simple. C++11 can change the ABI. So the point
>> kinda is,
>>>>> we need to ensure that all C++ libraries in a depgraph use the same
>> C++
>>>>> version.  
>>>> This is pretty awful when you really think about it.  I feel like
>> I'm
>>>> watching a train-wreck in super slow motion.
>>> Well, it's not that bad actually. After some thinking, I figured out
>>> they fixed most 98/11 incompatibilities around gcc 4.8/4.9, and left
>>> only a few 'unlikely' to cause issues.
>>>
>>> However, if one dep switches to C++11, it is quite likely to require
>>> C++11 in its revdeps, and that's what happening with libsigc++
>>> and other gtkmm libraries.
>> When building a package, you can't just switch between -std=gnu++98 or
>> c++99 or gnu++11 or c++11 since there are syntactic difference.
>>> Plus, there's of course the classical issue of ABI incompatibility
>>> between libstdc++ bundled with 4.9 and 5.1, and 5.2... so along with
>>> switching g++ version, you soon start to have to rebuild random C++
>>> libraries.
>>>
>>> And the issue of supporting alternative C++ standard library
>>> implementations -- like using libcxx with clang. They are of course
>>> incompatible with GNU's ever-changing ABI.
>>>
>>>> I'm not sure we're taking this seriously enough -- sooner or later
>> it
>>>> seems destined to become a major clusterfuck if we don't do
>> something
>>>> proactive about it now while the drawing-board is relatively
>>>> uncluttered.
>>>>
>>>> The only thing I can think of that has this kind of two-way depgraph
>>>> magic property are the major "abi" USE_EXPAND values (multilib-build
>>>> and python-r1, in other words).
>>>>
>>>> But those rely on fancy framework-generated USE-flag deps, which
>> seem
>>>> like overkill and likely to incur unjustifiable
>> user-experience-costs.
>>> Yes, it is terrible. You end up introducing a lot of USE flags that
>>> need to be manually switched along with gcc versions. If we start
>>> splitting them between c++98 and c++11, we're quite likely to hit USE
>>> flag conflicts between packages/developers which prefer one over
>>> another.
>> This would be a nightmare.
>>
>>>> Perhaps a solution to this cxx11 clusterfuck can be found that works
>>>> more like perl?  By that I mean, pick your poison (respectively,
>> your
>>>> cxx11 ABI of preference or your major perl version of choice), rely
>> on
>>>> inbuilt portage features do the trick most of the time, and, when it
>>>> breaks, run "magically-fix-everything.sh," grab a caffeinated
>> beverage
>>>> or three and fire up your favorite VOD client while the mess gets
>>>> magically cleaned up by robots somehow.
>>> Sadly := can't help here since gcc switches occur independently of
>>> package installs. And AFAIK revdep-rebuild doesn't help either.
>> You can run `revdep-rebuild -L 'libstdc\+\+\.so\.6'` to rebuild
>> everything that links against libstdc++.so.6.  This will rebuild a lot
>> of packages but will fix everything.
>>
>> If we record enough information at build time (eg. gcc version or
>> libcxx/clang) then we can build tools that intelligently predict if
>> there's an abi incompatibility.  Unfortunately gcc doesn't bump soname
>> and/or version-info when it changes c++11 abi.  (since c++11 is
>> experimental and c++03/98 have stable abi, they don't want to force
>> rebuilds).  So we have to record the equivalent of an soname.  If we
>> put
>> that information in a file like NEEDED.ELF.2 in vdb, it could be read
>> by
>> utilities like magically-fix-everything.sh (a revddep-rebuild.sh for
>> libstdc++).
> In my case, checking CXX + library symbols (to distinguish C++ libraries) works. But most of the people believe setting CXX to a static version is a bad idea, and it's better to use implicit magic of gcc-config.
>

different direction: what about building with
rpath=/usr/lib/gcc/x86_64-pc-linux-gnu/4.9.3/ and then making sure that
portage respects that library file during any --depclean or
@preserved-rebuild?  i'm not sure how we'd
inject|||LDFLAGS=-Wl,-rpath=... consistantly and sanely into all c++ builds.

this would solve all problems i can see: 1) gcc-config reshuffles
/etc/ld.so.conf.d/05gcc-<tuple>.conf but rpath takes presidence, 2) the
correct library symbols are guaranteed to be there in both exe and lib. 
3) it pro-actively guards against abi mismatches when switching gcc even
for other languages like fortran, java, obj-c.|

-- 
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail    : blueness@gentoo.org
GnuPG FP  : 1FED FAD9 D82C 52A5 3BAB  DC79 9384 FA6E F52D 4BBA
GnuPG ID  : F52D4BBA

Re: [gentoo-dev] impending c++11 clusterfuck?

[Anthony G. Basile]

On 11/30/15 6:17 AM, Anthony G. Basile wrote:
> different direction: what about building with
> rpath=/usr/lib/gcc/x86_64-pc-linux-gnu/4.9.3/ and then making sure that
> portage respects that library file during any --depclean or
> @preserved-rebuild?  i'm not sure how we'd
> inject LDFLAGS=-Wl,-rpath=... consistantly and sanely into all c++ builds.
>
> this would solve all problems i can see: 1) gcc-config reshuffles
> /etc/ld.so.conf.d/05gcc-<tuple>.conf but rpath takes presidence, 2) the
> correct library symbols are guaranteed to be there in both exe and lib. 
> 3) it pro-actively guards against abi mismatches when switching gcc even
> for other languages like fortran, java, obj-c.

oh i should add that we don't support switching back and forth between
c++98 and c++11.  which is used is an upstream choice via whatever -std=
they add to the build system, otherwise we go with the compiler default.

i'm going to play with situations like foo is built using
c++{98,11}/gcc-4.{8,9} and links against libbar built using
c++{98,11}/gcc-4.{8,9} and libbas similarly built using the rpath
technique and see what kind of wierdness i might hit.  i stil think it
will work.

-- 
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail    : blueness@gentoo.org
GnuPG FP  : 1FED FAD9 D82C 52A5 3BAB  DC79 9384 FA6E F52D 4BBA
GnuPG ID  : F52D4BBA

Re: [gentoo-dev] impending c++11 clusterfuck?

[Steven Lembark]

> Perhaps a solution to this cxx11 clusterfuck can be found that works
> more like perl?

Or a "use vX" that simply causes the later version to behave like 
the earlier one? At that point a single flag/directive could keep
things at least consistently behaving as if the upgrade never happend.



-- 
Steven Lembark                                           3646 Flora Pl
Workhorse Computing                                 St Louis, MO 63110
lembark@wrkhors.com                                    +1 888 359 3508

Re: [gentoo-dev] impending c++11 clusterfuck?

[Greg Turner]

https://gcc.gnu.org/onlinedocs/libstdc++/manual/using_dual_abi.html
makes for pretty interesting reading on this topic.

https://youtu.be/Ic4j0ujWL-A looks interesting too.

-gmt
-gmt

Greg Turner
gmt@be-evil.net


On Mon, Nov 30, 2015 at 3:31 AM, Anthony G. Basile <blueness@gentoo.org> wrote:
> On 11/30/15 6:17 AM, Anthony G. Basile wrote:
>> different direction: what about building with
>> rpath=/usr/lib/gcc/x86_64-pc-linux-gnu/4.9.3/ and then making sure that
>> portage respects that library file during any --depclean or
>> @preserved-rebuild?  i'm not sure how we'd
>> inject LDFLAGS=-Wl,-rpath=... consistantly and sanely into all c++ builds.
>>
>> this would solve all problems i can see: 1) gcc-config reshuffles
>> /etc/ld.so.conf.d/05gcc-<tuple>.conf but rpath takes presidence, 2) the
>> correct library symbols are guaranteed to be there in both exe and lib.
>> 3) it pro-actively guards against abi mismatches when switching gcc even
>> for other languages like fortran, java, obj-c.
>
> oh i should add that we don't support switching back and forth between
> c++98 and c++11.  which is used is an upstream choice via whatever -std=
> they add to the build system, otherwise we go with the compiler default.
>
> i'm going to play with situations like foo is built using
> c++{98,11}/gcc-4.{8,9} and links against libbar built using
> c++{98,11}/gcc-4.{8,9} and libbas similarly built using the rpath
> technique and see what kind of wierdness i might hit.  i stil think it
> will work.
>
> --
> Anthony G. Basile, Ph.D.
> Gentoo Linux Developer [Hardened]
> E-Mail    : blueness@gentoo.org
> GnuPG FP  : 1FED FAD9 D82C 52A5 3BAB  DC79 9384 FA6E F52D 4BBA
> GnuPG ID  : F52D4BBA
>
>

Re: [gentoo-dev] impending c++11 clusterfuck?

[Greg Turner]

On Mon, Nov 30, 2015 at 10:34 AM, Greg Turner <gmt@be-evil.net> wrote:
> https://youtu.be/Ic4j0ujWL-A looks interesting too.

--> https://sourceware.org/libabigail/

-gmt

Greg Turner
gmt@be-evil.net

Re: [gentoo-dev] impending c++11 clusterfuck?

[C Bergström]

One thing to point out.. trying to detect and using vX are just hacks
for what this really is - Adding abi (C++STL/ABI) information to the
ebuilds/packages.

To extend this - what happens when you have a compiler that isn't
compatible with the system default? When the package is merged should
some information from the toolchain like g++-48-cpp03 be inserted for
the package? The only downside I could think about is a crazy world
where libs end up being installed in.

/usr/lib64/{CPP-ABI}
/usr/lib64/g++-48-cpp03/

In this case the default system compiler could be gcc5 and then an
"alt" compiler which provides a different ABI could be used based on
some "restrict" or explicit information in the ebuild. That alt
compiler would have to build all the dependencies, but as long as it
knew about /usr/lib64/g++-48-cpp03/ and not /usr/lib64/g++-52-cpp11/
things would be ok...

If you want c++11 and 03 to exist, link together and be perfect at the
same time.. I think you're asking for trouble or the impossible.

While we're solving this problem - the same sort of "ABI" may also
help other languages suffering from similar problems like python
(Which now "fixed" their problem, but it's still not a perfect world)

Nothing is to say it won't pop up again in the future (Java?)...

Re: [gentoo-dev] impending c++11 clusterfuck?

[Benda Xu]

Hi Anthony,

"Anthony G. Basile" <blueness@gentoo.org> writes:

> different direction: what about building with
> rpath=/usr/lib/gcc/x86_64-pc-linux-gnu/4.9.3/ and then making sure that
> portage respects that library file during any --depclean or
> @preserved-rebuild?  i'm not sure how we'd
> inject|||LDFLAGS=-Wl,-rpath=... consistantly and sanely into all c++ builds.

FYI, Gentoo Prefix injects "LDFLAGS=-Wl,-rpath=" with a wrapper of ld.

Benda

Re: [gentoo-dev] impending c++11 clusterfuck?

[Anthony G. Basile]

On 12/1/15 3:59 AM, Benda Xu wrote:
> Hi Anthony,
>
> "Anthony G. Basile" <blueness@gentoo.org> writes:
>
>> different direction: what about building with
>> rpath=/usr/lib/gcc/x86_64-pc-linux-gnu/4.9.3/ and then making sure that
>> portage respects that library file during any --depclean or
>> @preserved-rebuild?  i'm not sure how we'd
>> inject|||LDFLAGS=-Wl,-rpath=... consistantly and sanely into all c++ builds.
> FYI, Gentoo Prefix injects "LDFLAGS=-Wl,-rpath=" with a wrapper of ld.
>
> Benda
>

thanks. i'm getting somewhere with this.  at the very least, i think i'm
going to be able to come up with some "best practices".  we may be able
to get something in code too.

-- 
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail    : blueness@gentoo.org
GnuPG FP  : 1FED FAD9 D82C 52A5 3BAB  DC79 9384 FA6E F52D 4BBA
GnuPG ID  : F52D4BBA

Re: [gentoo-dev] impending c++11 clusterfuck?

[Michał Górny]

Dnia 30 listopada 2015 12:17:32 CET, "Anthony G. Basile" <blueness@gentoo.org> napisał(a):
>On 11/30/15 4:52 AM, Michał Górny wrote:
>>
>> Dnia 30 listopada 2015 09:07:30 CET, "Anthony G. Basile"
><blueness@gentoo.org> napisał(a):
>>> On 11/30/15 1:42 AM, Michał Górny wrote:
>>>> On Sun, 29 Nov 2015 19:56:04 -0800
>>>> "Gregory M. Turner" <gmt@be-evil.net> wrote:
>>>>
>>>>> I'm quoting myself from bug #566328 here.  These were off-the-cuff
>>>>> remarks that got away from me and became a call-to-arms...
>>>>>
>>>>> (In reply to Michał Górny from comment #7)
>>>>>> This is never this simple. C++11 can change the ABI. So the point
>>> kinda is,
>>>>>> we need to ensure that all C++ libraries in a depgraph use the
>same
>>> C++
>>>>>> version.  
>>>>> This is pretty awful when you really think about it.  I feel like
>>> I'm
>>>>> watching a train-wreck in super slow motion.
>>>> Well, it's not that bad actually. After some thinking, I figured
>out
>>>> they fixed most 98/11 incompatibilities around gcc 4.8/4.9, and
>left
>>>> only a few 'unlikely' to cause issues.
>>>>
>>>> However, if one dep switches to C++11, it is quite likely to
>require
>>>> C++11 in its revdeps, and that's what happening with libsigc++
>>>> and other gtkmm libraries.
>>> When building a package, you can't just switch between -std=gnu++98
>or
>>> c++99 or gnu++11 or c++11 since there are syntactic difference.
>>>> Plus, there's of course the classical issue of ABI incompatibility
>>>> between libstdc++ bundled with 4.9 and 5.1, and 5.2... so along
>with
>>>> switching g++ version, you soon start to have to rebuild random C++
>>>> libraries.
>>>>
>>>> And the issue of supporting alternative C++ standard library
>>>> implementations -- like using libcxx with clang. They are of course
>>>> incompatible with GNU's ever-changing ABI.
>>>>
>>>>> I'm not sure we're taking this seriously enough -- sooner or later
>>> it
>>>>> seems destined to become a major clusterfuck if we don't do
>>> something
>>>>> proactive about it now while the drawing-board is relatively
>>>>> uncluttered.
>>>>>
>>>>> The only thing I can think of that has this kind of two-way
>depgraph
>>>>> magic property are the major "abi" USE_EXPAND values
>(multilib-build
>>>>> and python-r1, in other words).
>>>>>
>>>>> But those rely on fancy framework-generated USE-flag deps, which
>>> seem
>>>>> like overkill and likely to incur unjustifiable
>>> user-experience-costs.
>>>> Yes, it is terrible. You end up introducing a lot of USE flags that
>>>> need to be manually switched along with gcc versions. If we start
>>>> splitting them between c++98 and c++11, we're quite likely to hit
>USE
>>>> flag conflicts between packages/developers which prefer one over
>>>> another.
>>> This would be a nightmare.
>>>
>>>>> Perhaps a solution to this cxx11 clusterfuck can be found that
>works
>>>>> more like perl?  By that I mean, pick your poison (respectively,
>>> your
>>>>> cxx11 ABI of preference or your major perl version of choice),
>rely
>>> on
>>>>> inbuilt portage features do the trick most of the time, and, when
>it
>>>>> breaks, run "magically-fix-everything.sh," grab a caffeinated
>>> beverage
>>>>> or three and fire up your favorite VOD client while the mess gets
>>>>> magically cleaned up by robots somehow.
>>>> Sadly := can't help here since gcc switches occur independently of
>>>> package installs. And AFAIK revdep-rebuild doesn't help either.
>>> You can run `revdep-rebuild -L 'libstdc\+\+\.so\.6'` to rebuild
>>> everything that links against libstdc++.so.6.  This will rebuild a
>lot
>>> of packages but will fix everything.
>>>
>>> If we record enough information at build time (eg. gcc version or
>>> libcxx/clang) then we can build tools that intelligently predict if
>>> there's an abi incompatibility.  Unfortunately gcc doesn't bump
>soname
>>> and/or version-info when it changes c++11 abi.  (since c++11 is
>>> experimental and c++03/98 have stable abi, they don't want to force
>>> rebuilds).  So we have to record the equivalent of an soname.  If we
>>> put
>>> that information in a file like NEEDED.ELF.2 in vdb, it could be
>read
>>> by
>>> utilities like magically-fix-everything.sh (a revddep-rebuild.sh for
>>> libstdc++).
>> In my case, checking CXX + library symbols (to distinguish C++
>libraries) works. But most of the people believe setting CXX to a
>static version is a bad idea, and it's better to use implicit magic of
>gcc-config.
>>
>
>different direction: what about building with
>rpath=/usr/lib/gcc/x86_64-pc-linux-gnu/4.9.3/ and then making sure that
>portage respects that library file during any --depclean or
>@preserved-rebuild?  i'm not sure how we'd
>inject|||LDFLAGS=-Wl,-rpath=... consistantly and sanely into all c++
>builds.
>
>this would solve all problems i can see: 1) gcc-config reshuffles
>/etc/ld.so.conf.d/05gcc-<tuple>.conf but rpath takes presidence, 2) the
>correct library symbols are guaranteed to be there in both exe and lib.
>
>3) it pro-actively guards against abi mismatches when switching gcc
>even
>for other languages like fortran, java, obj-c.|

I'm afraid any of those problems are really affecting us here. GCC maintains backwards ABI compatibility in the library, so applications will continue to work as long as runtime libstdc++ isn't older than build time.

Currently, we always force newest installed libstdc++ at runtime, and use the version matching GCC version at build time, so that works.

So the best thing your solution could give us is loading the wrong version of libstdc++ when you link to a library built against older one.
-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: [gentoo-dev] impending c++11 clusterfuck?

[Anthony G. Basile]

On 12/1/15 6:25 AM, Michał Górny wrote:
>
> Dnia 30 listopada 2015 12:17:32 CET, "Anthony G. Basile" <blueness@gentoo.org> napisał(a):
>>
>> different direction: what about building with
>> rpath=/usr/lib/gcc/x86_64-pc-linux-gnu/4.9.3/ and then making sure that
>> portage respects that library file during any --depclean or
>> @preserved-rebuild?  i'm not sure how we'd
>> inject|||LDFLAGS=-Wl,-rpath=... consistantly and sanely into all c++
>> builds.
>>
>> this would solve all problems i can see: 1) gcc-config reshuffles
>> /etc/ld.so.conf.d/05gcc-<tuple>.conf but rpath takes presidence, 2) the
>> correct library symbols are guaranteed to be there in both exe and lib.
>>
>> 3) it pro-actively guards against abi mismatches when switching gcc
>> even
>> for other languages like fortran, java, obj-c.|
> I'm afraid any of those problems are really affecting us here. GCC maintains backwards ABI compatibility in the library, so applications will continue to work as long as runtime libstdc++ isn't older than build time.
>
> Currently, we always force newest installed libstdc++ at runtime, and use the version matching GCC version at build time, so that works.
>
> So the best thing your solution could give us is loading the wrong version of libstdc++ when you link to a library built against older one.
Yeah there are two problems going on here and the libsigc++-2.6 is the
c++98 <=> 11 problem.  So on that bug we're talking about selectively
adding -std=c++11 (or possibly gnu++11) to packages.  This may get
messy.  I'm wondering if it isn't possible to just globally add
CXXFLAGS+="-std=c++11".  This should work because anything written with
c++98 will compile under c++11 (although not vice versa) although I
don't know how we'd migrate existing systems.

-- 
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail    : blueness@gentoo.org
GnuPG FP  : 1FED FAD9 D82C 52A5 3BAB  DC79 9384 FA6E F52D 4BBA
GnuPG ID  : F52D4BBA

Re: [gentoo-dev] impending c++11 clusterfuck?

[Gregory M. Turner]

On Tue, 2015-12-01 at 07:18 -0500, Anthony G. Basile wrote:
> So on that bug we're talking about selectively
> adding -std=c++11 (or possibly gnu++11) to packages.

Yes, this is the biggest real-world problem we face.  It requires an
immediate solution in the ~* branches; the affected ebuilds just dump a
bunch of gcc gobbldeygook and crash.

If I understand the generalized problem we are facing there, as a
package gets "c++11-ized", all of its reverse-BDEPs find themselves in
the following situation:

Imagine cat/foo is a library.  cat/foo-3.0 is the newest non-c++11-ized 
version (in its slot, perhaps).  Now cat/foo-3.1 comes along and from
then on, anything BDEPENDing on it must be built with -std=c++11 (I'm
ignoring the c++11 vs. gnu++11 but presumably we'll eventually need to
figure out some kind of game plan about that).

So, now, "everything" pulling in headers from  cat/foo finds itself in
this situation:

 o if it pulls in headers from <cat/foo-3.1, all is as before
 o if it pulls in headers from >=cat/foo-3.1, we must add a CFLAG

I guess the reasonable way to achieve that sort of behavior is
pkgconfig (which is not really a rock-solid solution.  First, some
packages might fail to put it in there (but, OK we just add it
ourselves, let's say).  Second, how many of cat/foo's reverse BDEPs
side-step pkgconfig?  Sometimes this is pretty common, I'm afraid.  

So, let's say lots of packages depending on cat/foo have this pkgconfig
side-stepping problem, we could theoretically write some eclass to
inject it when appropriate, and expect those side-stepping ebuilds to
consume it, no?  That seems pretty easy and hopefully wouldn't require
any of the scary ideas that have been discussed in this thread.

-gmt

Re: [gentoo-dev] impending c++11 clusterfuck?

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2491 bytes --]

On Tue, 01 Dec 2015 18:12:17 -0800
"Gregory M. Turner" <gmt@be-evil.net> wrote:

> On Tue, 2015-12-01 at 07:18 -0500, Anthony G. Basile wrote:
> > So on that bug we're talking about selectively
> > adding -std=c++11 (or possibly gnu++11) to packages.  
> 
> Yes, this is the biggest real-world problem we face.  It requires an
> immediate solution in the ~* branches; the affected ebuilds just dump a
> bunch of gcc gobbldeygook and crash.
> 
> If I understand the generalized problem we are facing there, as a
> package gets "c++11-ized", all of its reverse-BDEPs find themselves in
> the following situation:
> 
> Imagine cat/foo is a library.  cat/foo-3.0 is the newest non-c++11-ized 
> version (in its slot, perhaps).  Now cat/foo-3.1 comes along and from
> then on, anything BDEPENDing on it must be built with -std=c++11 (I'm
> ignoring the c++11 vs. gnu++11 but presumably we'll eventually need to
> figure out some kind of game plan about that).

Just to be clear, this only happened because upstream started to use
C++11 features in API. Normally libraries don't go that far, or use
#ifs to support C++03.

> So, now, "everything" pulling in headers from  cat/foo finds itself in
> this situation:
> 
>  o if it pulls in headers from <cat/foo-3.1, all is as before
>  o if it pulls in headers from >=cat/foo-3.1, we must add a CFLAG
> 
> I guess the reasonable way to achieve that sort of behavior is
> pkgconfig (which is not really a rock-solid solution.  First, some
> packages might fail to put it in there (but, OK we just add it
> ourselves, let's say).  Second, how many of cat/foo's reverse BDEPs
> side-step pkgconfig?  Sometimes this is pretty common, I'm afraid.  

pkg-config is not a solution. -std= flags don't belong there.
Furthermore, if you put -std= flags there, it is likely to override
${CXXFLAGS} and in particular make it really painful to force other
standard on ebuild's end.

> So, let's say lots of packages depending on cat/foo have this pkgconfig
> side-stepping problem, we could theoretically write some eclass to
> inject it when appropriate, and expect those side-stepping ebuilds to
> consume it, no?  That seems pretty easy and hopefully wouldn't require
> any of the scary ideas that have been discussed in this thread.

So you're saying we should write a whole eclass to do:

  append-cppflags -std=c++11

?

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] impending c++11 clusterfuck?

[Anthony G. Basile]

On 12/2/15 2:06 AM, Michał Górny wrote:
>
> So you're saying we should write a whole eclass to do:
>
>   append-cppflags -std=c++11
>
> ?
>
I think you mean append-cxxflags.  Anyhow, I had an off list discussion
with David Klaftenegger about the semantic difference between c++11 and
98 with respect to their atomics and I'm convinced that a global
-std=c++11 could get us into trouble on packages that make use of
threading.  So drop my idea of global c++11 and you'll have to catch
these on a per package basis.

-- 
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail    : blueness@gentoo.org
GnuPG FP  : 1FED FAD9 D82C 52A5 3BAB  DC79 9384 FA6E F52D 4BBA
GnuPG ID  : F52D4BBA

Re: [gentoo-dev] impending c++11 clusterfuck?

[Gregory M. Turner]

On Wed, 2015-12-02 at 08:06 +0100, Michał Górny wrote:
> On Tue, 01 Dec 2015 18:12:17 -0800
> "Gregory M. Turner" <gmt@be-evil.net> wrote:
> 
> > On Tue, 2015-12-01 at 07:18 -0500, Anthony G. Basile wrote:
> > > So on that bug we're talking about selectively
> > > adding -std=c++11 (or possibly gnu++11) to packages.  
> > 
> > Yes, this is the biggest real-world problem we face.  It requires
> > an
> > immediate solution in the ~* branches; the affected ebuilds just
> > dump a
> > bunch of gcc gobbldeygook and crash.
> > 
> > If I understand the generalized problem we are facing there, as a
> > package gets "c++11-ized", all of its reverse-BDEPs find themselves
> > in
> > the following situation:
> > 
> > Imagine cat/foo is a library.  cat/foo-3.0 is the newest non-c++11-
> > ized 
> > version (in its slot, perhaps).  Now cat/foo-3.1 comes along and
> > from
> > then on, anything BDEPENDing on it must be built with -std=c++11
> > (I'm
> > ignoring the c++11 vs. gnu++11 but presumably we'll eventually need
> > to
> > figure out some kind of game plan about that).
> 
> Just to be clear, this only happened because upstream started to use
> C++11 features in API. Normally libraries don't go that far, or use
> #ifs to support C++03.
> 
> > So, now, "everything" pulling in headers from  cat/foo finds itself
> > in
> > this situation:
> > 
> >  o if it pulls in headers from <cat/foo-3.1, all is as before
> >  o if it pulls in headers from >=cat/foo-3.1, we must add a CFLAG
> > 
> > I guess the reasonable way to achieve that sort of behavior is
> > pkgconfig (which is not really a rock-solid solution.  First, some
> > packages might fail to put it in there (but, OK we just add it
> > ourselves, let's say).  Second, how many of cat/foo's reverse BDEPs
> > side-step pkgconfig?  Sometimes this is pretty common, I'm afraid.
> >  
> 
> pkg-config is not a solution. -std= flags don't belong there.
> Furthermore, if you put -std= flags there, it is likely to override
> ${CXXFLAGS} and in particular make it really painful to force other
> standard on ebuild's end.

I see what you mean.  Maybe pkgconfig needs a new feature?  I wonder if
they are already thinking about it in pkgconfig-land.  I'll look around
on their ml/bugdb and see if I can find anything.

> > So, let's say lots of packages depending on cat/foo have this
> > pkgconfig
> > side-stepping problem, we could theoretically write some eclass to
> > inject it when appropriate, and expect those side-stepping ebuilds
> > to
> > consume it, no?  That seems pretty easy and hopefully wouldn't
> > require
> > any of the scary ideas that have been discussed in this thread.
> 
> So you're saying we should write a whole eclass to do:
> 
>   append-cppflags -std=c++11
> 
> ?

If we could just do "append-cppflags" somewhere, it'd be a non-bug as
we would have just done it.  That doesn't seem to be the case.

If I'm not mixed up, the difficulty is determining /when/ that's
appropriate.  Specifically, this c++11 requirement can bubble up
indirectly through deep dependencies or even non-DEPENDencies.  Source
file "X" just needs to #include header "Y" that #include's header "Z"
with the c++11 requirement, and "kablooey!".

So, yeah, an eclass that figures out when to append the flag (and then,
as you say, appends it) doesn't seem so crazy to me.

For example: if we can identify the dependencies, deep, hidden, or not,
that cause these failures (i.e., by waiting for bugzilla reports and
acting on them), we could pretty trivially fix them with such an
eclass.

So, just imagining one possible implementation of that, here's a
possible user-story -- which I mean, an eclass user (and therefore and
ebuild or eclass developer, but developer-story isn't a word afaik):

Bug wranglers observe that www-client/frobzilla ebuilds are crashing
due to headers, which are coming, ultimately, from net-libs/tldr. 

Assignees of the bug would:

o figure out which versions of net-libs/tldr trigger the problem
  and ensure they are selected by the hand-crafted list of
  DEPENDency specifiers that live in cxx11reqs.eclass.

o add the following code to all affected www-client/frobzilla ebuilds:

  CXX11REQ_CULPRITS=( net-libs/tldr )
  inherit cxx11reqs

This would cause the right thing to happen (CXXFLAGS augmentation) when
and only when the affected versions of tldr are installed.

Only thing left to do is to check that frobzilla doesn't strip the flag
back out, for some reason; if it did, perhaps it could add it back in,
by calling a xxx11reqs eclass API.

Note that frobzilla might not even DEPEND on tldr (not even indirectly
through several layers of DEPENDency); this would still do the trick;
the eclass could just use has_version to find out if the flag is
needed, according to a hand-crafted dependency specifier table.

I guess as far as bright ideas go, that's the best I've got, for now.

If a fully automagical solution is practical to implement, I haven't
yet imagined it.

-gmt

Re: [gentoo-dev] impending c++11 clusterfuck?

[Patrick Lauer]

On 12/02/2015 08:38 AM, Anthony G. Basile wrote:
> On 12/2/15 2:06 AM, Michał Górny wrote:
>> So you're saying we should write a whole eclass to do:
>>
>>   append-cppflags -std=c++11
>>
>> ?
>>
> I think you mean append-cxxflags.  Anyhow, I had an off list discussion
> with David Klaftenegger about the semantic difference between c++11 and
> 98 with respect to their atomics and I'm convinced that a global
> -std=c++11 could get us into trouble on packages that make use of
> threading.  So drop my idea of global c++11 and you'll have to catch
> these on a per package basis.
>
Could we use something along the lines of a use_expand'ed variable?

CXX_ABI="03"

==> DEPEND="dev-libs/boost[cxx_abi_03]"

Would need some CFLAGS injection, but like this any C++ packages can be
forced to 'same' ABI in a more reasonable fashion, and it wouldn't
require global overrides.
 

Re: [gentoo-dev] impending c++11 clusterfuck?

[Michał Górny]

Dnia 2 grudnia 2015 11:00:20 CET, "Gregory M. Turner" <gmt@be-evil.net> napisał(a):
>On Wed, 2015-12-02 at 08:06 +0100, Michał Górny wrote:
>> On Tue, 01 Dec 2015 18:12:17 -0800
>> "Gregory M. Turner" <gmt@be-evil.net> wrote:
>> 
>> > On Tue, 2015-12-01 at 07:18 -0500, Anthony G. Basile wrote:
>> > > So on that bug we're talking about selectively
>> > > adding -std=c++11 (or possibly gnu++11) to packages.  
>> > 
>> > Yes, this is the biggest real-world problem we face.  It requires
>> > an
>> > immediate solution in the ~* branches; the affected ebuilds just
>> > dump a
>> > bunch of gcc gobbldeygook and crash.
>> > 
>> > If I understand the generalized problem we are facing there, as a
>> > package gets "c++11-ized", all of its reverse-BDEPs find themselves
>> > in
>> > the following situation:
>> > 
>> > Imagine cat/foo is a library.  cat/foo-3.0 is the newest non-c++11-
>> > ized 
>> > version (in its slot, perhaps).  Now cat/foo-3.1 comes along and
>> > from
>> > then on, anything BDEPENDing on it must be built with -std=c++11
>> > (I'm
>> > ignoring the c++11 vs. gnu++11 but presumably we'll eventually need
>> > to
>> > figure out some kind of game plan about that).
>> 
>> Just to be clear, this only happened because upstream started to use
>> C++11 features in API. Normally libraries don't go that far, or use
>> #ifs to support C++03.
>> 
>> > So, now, "everything" pulling in headers from  cat/foo finds itself
>> > in
>> > this situation:
>> > 
>> >  o if it pulls in headers from <cat/foo-3.1, all is as before
>> >  o if it pulls in headers from >=cat/foo-3.1, we must add a CFLAG
>> > 
>> > I guess the reasonable way to achieve that sort of behavior is
>> > pkgconfig (which is not really a rock-solid solution.  First, some
>> > packages might fail to put it in there (but, OK we just add it
>> > ourselves, let's say).  Second, how many of cat/foo's reverse BDEPs
>> > side-step pkgconfig?  Sometimes this is pretty common, I'm afraid.
>> >  
>> 
>> pkg-config is not a solution. -std= flags don't belong there.
>> Furthermore, if you put -std= flags there, it is likely to override
>> ${CXXFLAGS} and in particular make it really painful to force other
>> standard on ebuild's end.
>
>I see what you mean.  Maybe pkgconfig needs a new feature?  I wonder if
>they are already thinking about it in pkgconfig-land.  I'll look around
>on their ml/bugdb and see if I can find anything.

Please don't. Pkg-config is already bad enough as it is, and they are known to break stuff without any concern for compatibility.

>
>> > So, let's say lots of packages depending on cat/foo have this
>> > pkgconfig
>> > side-stepping problem, we could theoretically write some eclass to
>> > inject it when appropriate, and expect those side-stepping ebuilds
>> > to
>> > consume it, no?  That seems pretty easy and hopefully wouldn't
>> > require
>> > any of the scary ideas that have been discussed in this thread.
>> 
>> So you're saying we should write a whole eclass to do:
>> 
>>   append-cppflags -std=c++11
>> 
>> ?
>
>If we could just do "append-cppflags" somewhere, it'd be a non-bug as
>we would have just done it.  That doesn't seem to be the case.
>
>If I'm not mixed up, the difficulty is determining /when/ that's
>appropriate.  Specifically, this c++11 requirement can bubble up
>indirectly through deep dependencies or even non-DEPENDencies.  Source
>file "X" just needs to #include header "Y" that #include's header "Z"
>with the c++11 requirement, and "kablooey!".
>
>So, yeah, an eclass that figures out when to append the flag (and then,
>as you say, appends it) doesn't seem so crazy to me.
>
>For example: if we can identify the dependencies, deep, hidden, or not,
>that cause these failures (i.e., by waiting for bugzilla reports and
>acting on them), we could pretty trivially fix them with such an
>eclass.
>
>So, just imagining one possible implementation of that, here's a
>possible user-story -- which I mean, an eclass user (and therefore and
>ebuild or eclass developer, but developer-story isn't a word afaik):
>
>Bug wranglers observe that www-client/frobzilla ebuilds are crashing
>due to headers, which are coming, ultimately, from net-libs/tldr. 
>
>Assignees of the bug would:
>
>o figure out which versions of net-libs/tldr trigger the problem
>  and ensure they are selected by the hand-crafted list of
>  DEPENDency specifiers that live in cxx11reqs.eclass.
>
>o add the following code to all affected www-client/frobzilla ebuilds:
>
>  CXX11REQ_CULPRITS=( net-libs/tldr )
>  inherit cxx11reqs
>
>This would cause the right thing to happen (CXXFLAGS augmentation) when
>and only when the affected versions of tldr are installed.
>
>Only thing left to do is to check that frobzilla doesn't strip the flag
>back out, for some reason; if it did, perhaps it could add it back in,
>by calling a xxx11reqs eclass API.
>
>Note that frobzilla might not even DEPEND on tldr (not even indirectly
>through several layers of DEPENDency); this would still do the trick;
>the eclass could just use has_version to find out if the flag is
>needed, according to a hand-crafted dependency specifier table.
>
>I guess as far as bright ideas go, that's the best I've got, for now.
>
>If a fully automagical solution is practical to implement, I haven't
>yet imagined it.

Don't. This is a problem upstreams need to eventually address. There is no reason to invent complex workarounds on our end when they are definitely only temporary.

>
>-gmt

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.