[gentoo-dev] How to support C++11 in libraries?

[gentoo-dev] How to support C++11 in libraries?

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2587 bytes --]

Hello, folks.

As some of you are already aware, the C++11 standard brought a few
changes to the C++ standard library. As a result, the standard library
used in C++11 mode has a different ABI than the one used in pre-C++11
mode. And this means that libraries that use some of standard C++ types
in their APIs also have different ABIs depending on the C++ standard
used to build them. This is somehow explained in [1].

As a result, if a single library suffers that, its reverse dependencies
need to be built with the same C++ standard. And then, those can force
even more dependencies and you may guess where this is going.

Basically, I've hit this with sys-devel/llvm. A user has requested lldb
support to be enabled in the ebuild [2]. Sadly, lldb requires C++11 to
be used, and this means that whole LLVM needs to become C++11 enabled.
And then, it would be at least recommended that all reverse deps become
C++11 enabled as well.

[1]:http://gcc.gnu.org/wiki/Cxx11AbiCompatibility
[2]:https://bugs.gentoo.org/show_bug.cgi?id=464354


This raises the following question: how do we want to do it? I see two
possibilities:

a) adding USE=c++11 and USE-deps to all the packages in question,

b) doing the switch via synchronous version bump and matching
dependencies.


I think that the variant a) is simpler. It goes like this:

1) we add 'sys-devel/llvm[-c++11(-)]' deps to everything that uses it,

2) we add USE=c++11 to llvm,

3) we add USE=c++11 and 'sys-devel/llvm[c++11=(-)]' deps to newest
version of everything that uses it.

The advantage is that plain users may just keep USE=c++11 disabled as
it is by default, and avoid hitting some issues with non-tree packages
and so on.

At some point, we'll probably want to remove non-C++11 support
completely. Then we could start by use.forcing the flag, changing deps
in packages and so on.


The variant b) requires much more synchronous work. In this case, we:

1) update LLVM deps to '<=' deps in everything that uses it,

2) revbump llvm and unconditionally enable C++11 in it,

3) update the reverse deps to enable C++11 and use '>=' deps on the new
version.

Of course, the same steps need to be repeated for every C++ reverse dep
of reverse deps, and libs and so on. In other words, it's a lot of work.

The problem with this solution is that as soon as user upgrades, shklee
is forced to use C++11. On the other hand, we get rid of pre-C++11
packages quite transparently, without extra work.


What are your thoughts?

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]

Re: [gentoo-dev] How to support C++11 in libraries?

["C. Bergström"]

On 12/18/13 02:54 PM, Michał Górny wrote:
> Hello, folks.
>
Hi

<snip>
> Basically, I've hit this with sys-devel/llvm. A user has requested lldb
> support to be enabled in the ebuild [2]. Sadly, lldb requires C++11 to
> be used, and this means that whole LLVM needs to become C++11 enabled.
> And then, it would be at least recommended that all reverse deps become
> C++11 enabled as well.
/*
Personally, I think lldb is pooh (bloated mess that has poor internal 
design, offers little or no logical features that can't be accomplished 
with source access+printf and takes a needlessly long time to compile as 
a result)
*/
--------
If the only driving motivation is lldb then I think this isn't worth the 
effort and I wonder what may be incompatible as a result. Long term it 
certainly should happen - I can't/won't argue or disagree with the long 
term merits, but when.. and who will do all that work..

Just a heads up that clang/llvm will (have in svn trunk) force building 
with c++11 for the next major release (6 months from now). So unless 
some 3rd party goes and backports or removes the c++11 pieces - this 
will add to the list of c++11 only software in the near future.

Re: [gentoo-dev] How to support C++11 in libraries?

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1025 bytes --]

Dnia 2013-12-18, o godz. 15:20:10
"C. Bergström" <cbergstrom@pathscale.com> napisał(a):

> If the only driving motivation is lldb then I think this isn't worth the 
> effort and I wonder what may be incompatible as a result. Long term it 
> certainly should happen - I can't/won't argue or disagree with the long 
> term merits, but when.. and who will do all that work..
> 
> Just a heads up that clang/llvm will (have in svn trunk) force building 
> with c++11 for the next major release (6 months from now). So unless 
> some 3rd party goes and backports or removes the c++11 pieces - this 
> will add to the list of c++11 only software in the near future.

Well, last time I was asked to enable C++11 in llvm I answered that
I'd delayed it because of the potential ABI incompatibility. While lldb
was what made me revisit the subject, I think it's something we will
need to handle anyway.

I'd rather find a good solution right now while we don't have to hurry.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]

Re: [gentoo-dev] How to support C++11 in libraries?

[heroxbd]

[-- Attachment #1: Type: text/plain, Size: 497 bytes --]

Hey, Michał,

Michał Górny <mgorny@gentoo.org> writes:

> a) adding USE=c++11 and USE-deps to all the packages in question,

I think it is better achieved by a (simple and stupid) global
CXXFLAGS. Adding an extra USE flag feels a little over-engineering.

Any anyway, if it is only for lldb, a piece of elog conveying a
preferred solution would suffice.

> b) doing the switch via synchronous version bump and matching
> dependencies.

This sounds tedious to maintain.

Benda

[-- Attachment #2: Type: application/pgp-signature, Size: 489 bytes --]

Re: [gentoo-dev] How to support C++11 in libraries?

[Jan Kundrát]

On Wednesday, 18 December 2013 14:58:07 CEST, heroxbd@gentoo.org wrote:
> I think it is better achieved by a (simple and stupid) global
> CXXFLAGS. Adding an extra USE flag feels a little over-engineering.

What compiler flag do you propose to use? Note that --std=c++11 will not 
work.

Cheers,
Jan

Re: [gentoo-dev] How to support C++11 in libraries?

["C. Bergström"]

On 12/18/13 11:29 PM, Jan Kundrát wrote:
> On Wednesday, 18 December 2013 14:58:07 CEST, heroxbd@gentoo.org wrote:
>> I think it is better achieved by a (simple and stupid) global
>> CXXFLAGS. Adding an extra USE flag feels a little over-engineering.
>
> What compiler flag do you propose to use? Note that --std=c++11 will 
> not work.
 From the perspective of a compiler vendor - I must ask why not?

Re: [gentoo-dev] How to support C++11 in libraries?

[Jan Kundrát]

On Wednesday, 18 December 2013 17:37:56 CEST, "C. Bergström" wrote:
>  From the perspective of a compiler vendor - I must ask why not?

There is code out there which builds fine under C++98, but fails to build 
when C++11 is enabled (as but one exmaple, have a look at [1]).

[1] https://bugs.freedesktop.org/show_bug.cgi?id=46147

Re: [gentoo-dev] How to support C++11 in libraries?

["C. Bergström"]

On 12/18/13 11:50 PM, Jan Kundrát wrote:
> On Wednesday, 18 December 2013 17:37:56 CEST, "C. Bergström" wrote:
>>  From the perspective of a compiler vendor - I must ask why not?
>
> There is code out there which builds fine under C++98, but fails to 
> build when C++11 is enabled (as but one exmaple, have a look at [1]).
If moving to C++11 - Isn't that considered just part of the work along 
the path? There's some clang tools to help with the migration, but I 
don't think anyone expects it to be zero work. The flag is just a way to 
a) enable building programs that can be built with c++11 b) flush out 
the culprits in the cases it can't be. If (b) is a bug - how else to 
find it easily?

Re: [gentoo-dev] How to support C++11 in libraries?

[Jan Kundrát]

On Wednesday, 18 December 2013 18:05:46 CEST, "C. Bergström" wrote:
> If moving to C++11 - Isn't that considered just part of the 
> work along the path? There's some clang tools to help with the 
> migration, but I don't think anyone expects it to be zero work. 
> The flag is just a way to a) enable building programs that can 
> be built with c++11 b) flush out the culprits in the cases it 
> can't be. If (b) is a bug - how else to find it easily?

This perspective is interesting (and I admit that I tend to like it) -- 
considering packages which won't build with C++11 to be buggy.

I'm worried by the cost of such a policy, though, because we would suddenly 
have to patch some unknown amount of software (and I'm pretty sure some 
upstreams would reject these patches anyway). If we were an enterprise 
distro with binary compatibility requirements, we would also have to worry 
about that and either assume that the ABI changes are non-issue in real 
world, or provide two versions of all C++ libraries. I tried to check how 
RHEL7 will deal with it, but I wasn't able to find any information about 
that. It also seems that Fedora hasn't addressed this yet, either.

Either way, it is reasonable to assume that some users would like to build 
their own software and link it with system libraries. It is not reasonable 
to force these users to build in the C++11 mode, IMHO.

Cheers,
Jan

Re: [gentoo-dev] How to support C++11 in libraries?

[Kent Fredric]

On 19 December 2013 06:33, Jan Kundrát <jkt@gentoo.org> wrote:
> I'm worried by the cost of such a policy, though, because we would suddenly
> have to patch some unknown amount of software


Given the nature that changing that CXX Flag globally for all users
could cause many packages to spontaneously fail to build, wouldn't
that imply that changing that flag would essentially be de-stabilizing
the whole tree, and a package being (arch) would no longer be an
indication of sane, tested behaviour?

This is really the perk of the USE driven process, the granular
piecemeal approach that does only as much as necessary, without
changing things that are already stable.

-- 
Kent

Re: [gentoo-dev] How to support C++11 in libraries?

["C. Bergström"]

On 12/19/13 12:47 AM, Kent Fredric wrote:
> On 19 December 2013 06:33, Jan Kundrát <jkt@gentoo.org> wrote:
>> I'm worried by the cost of such a policy, though, because we would suddenly
>> have to patch some unknown amount of software
>
> Given the nature that changing that CXX Flag globally for all users
> could cause many packages to spontaneously fail to build, wouldn't
> that imply that changing that flag would essentially be de-stabilizing
> the whole tree, and a package being (arch) would no longer be an
> indication of sane, tested behaviour?
>
> This is really the perk of the USE driven process, the granular
> piecemeal approach that does only as much as necessary, without
> changing things that are already stable.
In practice wouldn't that mean you'd have to add c++11 USE flag to every 
C++11 application and lib?

I just sent an email with some crazy thoughts - Your point is totally 
correct - this "migration" needs to happen while not breaking the whole 
tree. Logistically - what's the best way to maintain both those "ABI" at 
the same time?

"Best case" both build and you end up with a linker problem (can be 
worked around with compiler patches)
/usr/lib64/libboost.so
/usr/lib64-c++11/libboost.so

Worst case only 1 builds
this breaks down into generally 2 cases
     1. Programs/libs which have intentionally adopted c++11 and don't 
care about C++03 (clang/llvm/lldb)
     2. Programs/libs which can't be compiles with c++11 mode


/usr/lib64/libfoo.so

Re: [gentoo-dev] How to support C++11 in libraries?

[heroxbd]

"C. Bergström" <cbergstrom@pathscale.com> writes:

> "Best case" both build and you end up with a linker problem (can be
> worked around with compiler patches)
> /usr/lib64/libboost.so
> /usr/lib64-c++11/libboost.so

This is the right way to do, but as scary as our multilib where a
couple of USE flags are introduced to every package globally.

Re: [gentoo-dev] How to support C++11 in libraries?

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1578 bytes --]

Dnia 2013-12-19, o godz. 00:56:31
"C. Bergström" <cbergstrom@pathscale.com> napisał(a):

> On 12/19/13 12:47 AM, Kent Fredric wrote:
> > On 19 December 2013 06:33, Jan Kundrát <jkt@gentoo.org> wrote:
> >> I'm worried by the cost of such a policy, though, because we would suddenly
> >> have to patch some unknown amount of software
> >
> > Given the nature that changing that CXX Flag globally for all users
> > could cause many packages to spontaneously fail to build, wouldn't
> > that imply that changing that flag would essentially be de-stabilizing
> > the whole tree, and a package being (arch) would no longer be an
> > indication of sane, tested behaviour?
> >
> > This is really the perk of the USE driven process, the granular
> > piecemeal approach that does only as much as necessary, without
> > changing things that are already stable.
> In practice wouldn't that mean you'd have to add c++11 USE flag to every 
> C++11 application and lib?

No. Only the libs that change their ABI in C++11.

> "Best case" both build and you end up with a linker problem (can be 
> worked around with compiler patches)
> /usr/lib64/libboost.so
> /usr/lib64-c++11/libboost.so

What's wrong with this solution:

1. distro-specific compiler patching is wrong,

2. kinda FHS deviation, at least in spirit of lib<qual> directory.

We could go with '-L' but this is very fragile anyway. It's *very easy*
for the compiler to link the 'wrong' library due to -L/usr/lib64 being
added by some kind of foo-config.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]

Re: [gentoo-dev] How to support C++11 in libraries?

["C. Bergström"]

On 12/19/13 03:20 PM, Michał Górny wrote:
> Dnia 2013-12-19, o godz. 00:56:31
> "C. Bergström" <cbergstrom@pathscale.com> napisał(a):
>
>> On 12/19/13 12:47 AM, Kent Fredric wrote:
>>> On 19 December 2013 06:33, Jan Kundrát <jkt@gentoo.org> wrote:
>>>> I'm worried by the cost of such a policy, though, because we would suddenly
>>>> have to patch some unknown amount of software
>>> Given the nature that changing that CXX Flag globally for all users
>>> could cause many packages to spontaneously fail to build, wouldn't
>>> that imply that changing that flag would essentially be de-stabilizing
>>> the whole tree, and a package being (arch) would no longer be an
>>> indication of sane, tested behaviour?
>>>
>>> This is really the perk of the USE driven process, the granular
>>> piecemeal approach that does only as much as necessary, without
>>> changing things that are already stable.
>> In practice wouldn't that mean you'd have to add c++11 USE flag to every
>> C++11 application and lib?
> No. Only the libs that change their ABI in C++11.
>
>> "Best case" both build and you end up with a linker problem (can be
>> worked around with compiler patches)
>> /usr/lib64/libboost.so
>> /usr/lib64-c++11/libboost.so
> What's wrong with this solution:
>
> 1. distro-specific compiler patching is wrong,
Pragmatically, this needs to be upstream and should have been there 
already. Get some feedback to see if gcc people are receptive to the 
idea before testing a gentoo-only patch. If they accept it upstream - 
backport it. If they tell you f* off - get their feedback on how to deal 
with it - more below.

(this is not a gentoo only problem - this discussion should happen on a 
more global level...)

Unfortunately, it's going to be really hard to tell what will break ABI 
and what won't. I guess for ABI compatible packages 
/usr/lib64-c++11/libfoobar.so would be a symlink back to 
/usr/lib64/libfoobar.so
>
> 2. kinda FHS deviation, at least in spirit of lib<qual> directory.
>
> We could go with '-L' but this is very fragile anyway. It's *very easy*
> for the compiler to link the 'wrong' library due to -L/usr/lib64 being
> added by some kind of foo-config.
-L would likely mean you also need -nostdlib to make it work - which is 
more hacky than the above. pretty please don't do this.. pleeeeaassse

Re: [gentoo-dev] How to support C++11 in libraries?

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2715 bytes --]

Dnia 2013-12-19, o godz. 15:28:46
"C. Bergström" <cbergstrom@pathscale.com> napisał(a):

> On 12/19/13 03:20 PM, Michał Górny wrote:
> > Dnia 2013-12-19, o godz. 00:56:31
> > "C. Bergström" <cbergstrom@pathscale.com> napisał(a):
> >
> >> On 12/19/13 12:47 AM, Kent Fredric wrote:
> >>> On 19 December 2013 06:33, Jan Kundrát <jkt@gentoo.org> wrote:
> >>>> I'm worried by the cost of such a policy, though, because we would suddenly
> >>>> have to patch some unknown amount of software
> >>> Given the nature that changing that CXX Flag globally for all users
> >>> could cause many packages to spontaneously fail to build, wouldn't
> >>> that imply that changing that flag would essentially be de-stabilizing
> >>> the whole tree, and a package being (arch) would no longer be an
> >>> indication of sane, tested behaviour?
> >>>
> >>> This is really the perk of the USE driven process, the granular
> >>> piecemeal approach that does only as much as necessary, without
> >>> changing things that are already stable.
> >> In practice wouldn't that mean you'd have to add c++11 USE flag to every
> >> C++11 application and lib?
> > No. Only the libs that change their ABI in C++11.
> >
> >> "Best case" both build and you end up with a linker problem (can be
> >> worked around with compiler patches)
> >> /usr/lib64/libboost.so
> >> /usr/lib64-c++11/libboost.so
> > What's wrong with this solution:
> >
> > 1. distro-specific compiler patching is wrong,
> Pragmatically, this needs to be upstream and should have been there 
> already. Get some feedback to see if gcc people are receptive to the 
> idea before testing a gentoo-only patch. If they accept it upstream - 
> backport it. If they tell you f* off - get their feedback on how to deal 
> with it - more below.
> 
> (this is not a gentoo only problem - this discussion should happen on a 
> more global level...)

And how is this an issue to the major distributions? Binary distros can
do a simple switch with standard all-package upgrade and forget about
it. Like they usually do. Only people who built from sources have to
think about it.

> > 2. kinda FHS deviation, at least in spirit of lib<qual> directory.
> >
> > We could go with '-L' but this is very fragile anyway. It's *very easy*
> > for the compiler to link the 'wrong' library due to -L/usr/lib64 being
> > added by some kind of foo-config.
> -L would likely mean you also need -nostdlib to make it work - which is 
> more hacky than the above. pretty please don't do this.. pleeeeaassse

What? I have no idea what you're trying to accomplish but this seems
out of the scope of the problem.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]

Re: [gentoo-dev] How to support C++11 in libraries?

["C. Bergström"]

On 12/19/13 03:35 PM, Michał Górny wrote:
> Dnia 2013-12-19, o godz. 15:28:46
> "C. Bergström" <cbergstrom@pathscale.com> napisał(a):
>
>> On 12/19/13 03:20 PM, Michał Górny wrote:
>>> Dnia 2013-12-19, o godz. 00:56:31
>>> "C. Bergström" <cbergstrom@pathscale.com> napisał(a):
>>>
>>>> On 12/19/13 12:47 AM, Kent Fredric wrote:
>>>>> On 19 December 2013 06:33, Jan Kundrát <jkt@gentoo.org> wrote:
>>>>>> I'm worried by the cost of such a policy, though, because we would suddenly
>>>>>> have to patch some unknown amount of software
>>>>> Given the nature that changing that CXX Flag globally for all users
>>>>> could cause many packages to spontaneously fail to build, wouldn't
>>>>> that imply that changing that flag would essentially be de-stabilizing
>>>>> the whole tree, and a package being (arch) would no longer be an
>>>>> indication of sane, tested behaviour?
>>>>>
>>>>> This is really the perk of the USE driven process, the granular
>>>>> piecemeal approach that does only as much as necessary, without
>>>>> changing things that are already stable.
>>>> In practice wouldn't that mean you'd have to add c++11 USE flag to every
>>>> C++11 application and lib?
>>> No. Only the libs that change their ABI in C++11.
>>>
>>>> "Best case" both build and you end up with a linker problem (can be
>>>> worked around with compiler patches)
>>>> /usr/lib64/libboost.so
>>>> /usr/lib64-c++11/libboost.so
>>> What's wrong with this solution:
>>>
>>> 1. distro-specific compiler patching is wrong,
>> Pragmatically, this needs to be upstream and should have been there
>> already. Get some feedback to see if gcc people are receptive to the
>> idea before testing a gentoo-only patch. If they accept it upstream -
>> backport it. If they tell you f* off - get their feedback on how to deal
>> with it - more belo
>>
>> (this is not a gentoo only problem - this discussion should happen on a
>> more global level...)
> And how is this an issue to the major distributions? Binary distros can
> do a simple switch with standard all-package upgrade and forget about
> it. Like they usually do. Only people who built from sources have to
> think about it.
Umm..  no? Lets use a hypothetical example...

libboost.so (or any really popular lib.. Qt..) built with -std=c++11 
breaks abi

If they don't do some sort of multilib approach - they are only going to 
build it once and then any consumer of that outside the distro is stuck 
with their decision. That's probably fine in the predominately C++03 
world we have today, but for how long? I expect users on the binary 
distro just do what they have to work around the problem (go build their 
whole dependency chain from source). It didn't solve the problem - just 
made it work for distro packages and pushed it off to the user.

My -L rant would depend on the above being used - that's all

Re: [gentoo-dev] How to support C++11 in libraries?

[Jan Kundrát]

On Thursday, 19 December 2013 09:44:38 CEST, "C. Bergström" wrote:
> libboost.so (or any really popular lib.. Qt..) built with 
> -std=c++11 breaks abi

As I said, the problem is more complicated. Qt5 built with the C++11 
support does not break its ABI compared to usign the C++98 mode.

Boost is in a category of its own because they do not provide a stable ABI 
to begin with.

Jan

Re: [gentoo-dev] How to support C++11 in libraries?

[Jan Kundrát]

On Thursday, 19 December 2013 09:35:14 CEST, Michał Górny wrote:
> And how is this an issue to the major distributions? Binary distros can
> do a simple switch with standard all-package upgrade and forget about
> it. Like they usually do. Only people who built from sources have to
> think about it.

Wrong; the binary-only distributions have to ship a ton of compat-$foo 
packages, if only for compatibility with 3rd party software. At least 
that's what RHEL has been doing for ages.

Jan

Re: [gentoo-dev] How to support C++11 in libraries?

[Georg Rudoy]

2013/12/19 Michał Górny <mgorny@gentoo.org>:
>> In practice wouldn't that mean you'd have to add c++11 USE flag to every
>> C++11 application and lib?
>
> No. Only the libs that change their ABI in C++11.

Which is true for every library that exchanges quite a few STL classes
with the outer world (see [1] in your original post for the list).

-- 
  Georg Rudoy
  LeechCraft — http://leechcraft.org

Re: [gentoo-dev] How to support C++11 in libraries?

["C. Bergström"]

On 12/19/13 12:33 AM, Jan Kundrát wrote:
> On Wednesday, 18 December 2013 18:05:46 CEST, "C. Bergström" wrote:
>> If moving to C++11 - Isn't that considered just part of the work 
>> along the path? There's some clang tools to help with the migration, 
>> but I don't think anyone expects it to be zero work. The flag is just 
>> a way to a) enable building programs that can be built with c++11 b) 
>> flush out the culprits in the cases it can't be. If (b) is a bug - 
>> how else to find it easily?
>
> This perspective is interesting (and I admit that I tend to like it) 
> -- considering packages which won't build with C++11 to be buggy.
>
> I'm worried by the cost of such a policy, though, because we would 
> suddenly have to patch some unknown amount of software (and I'm pretty 
> sure some upstreams would reject these patches anyway). If we were an 
> enterprise distro with binary compatibility requirements, we would 
> also have to worry about that and either assume that the ABI changes 
> are non-issue in real world, or provide two versions of all C++ 
> libraries. I tried to check how RHEL7 will deal with it, but I wasn't 
> able to find any information about that. It also seems that Fedora 
> hasn't addressed this yet, either.
>
> Either way, it is reasonable to assume that some users would like to 
> build their own software and link it with system libraries. It is not 
> reasonable to force these users to build in the C++11 mode, IMHO.
I want my cake and eat it too!

/* Off the cuff crazy ideas and not meant to be taken too seriously */

In my mind this is almost similar to ABI (incompatibilities) between 32 
and 64bit. Why not just (ab)use the multilib approach for c++11?

/usr/lib64-c11/

It would possibly (likely) require some patching to clang/g++ when the 
--std=c++11 flag is used, but might allow the extension of a c++11 
library universe while leaving the things which are working today 
unbroken. The main problem I see with this is proliferation of more crap 
in /usr/lib* (4 variations instead of just the 2 (32vs64) we have now). 
(Personally, I'd typically build *only* 64bit versions and 32bit x86 can 
go to hell...)

To get support from the "enterprise" distros - I'd try to move the 
discussion of this problem upstream to the LSB level. They will at some 
point need to solve this same problem as well. It's not like it'll be 
gentoo specific forever. (You guys are just pioneering and ahead of the 
curve.. this isn't the same thing as -Omgfast)

With this approach - would it make sense to create a new profile?

I don't know if a USE flag would in general get pushback, but if the 
c++11 use flag was used - it would put those libs in /usr/lib64-c11/

Re: [gentoo-dev] How to support C++11 in libraries?

[Georg Rudoy]

> Either way, it is reasonable to assume that some users would like to build
> their own software and link it with system libraries. It is not reasonable
> to force these users to build in the C++11 mode, IMHO.

As far as I understand now you're just forcing users to build in C++03
mode, don't you?

-- 
  Georg Rudoy
  LeechCraft — http://leechcraft.org

Re: [gentoo-dev] How to support C++11 in libraries?

[heroxbd]

Hey Jan,

Jan Kundrát <jkt@gentoo.org> writes:

> This perspective is interesting (and I admit that I tend to like it) -- 
> considering packages which won't build with C++11 to be buggy.
>
> I'm worried by the cost of such a policy, though, because we would
> suddenly have to patch some unknown amount of software (and I'm pretty
> sure some upstreams would reject these patches anyway). If we were an
> enterprise distro with binary compatibility requirements, we would
> also have to worry about that and either assume that the ABI changes
> are non-issue in real world, or provide two versions of all C++
> libraries. I tried to check how RHEL7 will deal with it, but I wasn't
> able to find any information about that. It also seems that Fedora
> hasn't addressed this yet, either.
>
> Either way, it is reasonable to assume that some users would like to
> build their own software and link it with system libraries. It is not
> reasonable to force these users to build in the C++11 mode, IMHO.

I'd like to make an analogy to the version bump of gcc[1]. We (gentoo)
decide to support c++11 officially or not. If so, open a tracker bug to
push it globally. If not, patch lldb to support non-c++11, or leave it
up to the user to fiddle with the CXXFLAGS, where we only point the user
by to proper docs.

There is no problem to introduce a new USE flag for a new ABI. I am
concerned with too many ABIs for an average ebuild keeper to maintain.

Benda

1. https://bugs.gentoo.org/show_bug.cgi?id=gcc-4.8

Re: [gentoo-dev] How to support C++11 in libraries?

[Jan Kundrát]

On Thursday, 19 December 2013 02:41:55 CEST, heroxbd@gentoo.org wrote:
> I'd like to make an analogy to the version bump of gcc[1]. We (gentoo)
> decide to support c++11 officially or not. If so, open a tracker bug to
> push it globally. If not, patch lldb to support non-c++11, or leave it
> up to the user to fiddle with the CXXFLAGS, where we only point the user
> by to proper docs.

To be honest, I do not really see a link between the "let's bring in a new 
version of compiler, it is a bit stricter in some situations, but these 
were bugs anyway, like missing headers or unfounded assumptions about 
memcpy()" with "let's support a new version of language which produces 
object files with different ABI". Perhaps a Python 2.6 vs. Python 3.3 is a 
better analogy?

Anyway, GCC 4.8 is pretty clear that the C++11's support is still 
experimental [1] and subject to change [2]. The upstream developers have 
announced that they plan to break the ABI of the code using C++11 features 
in 4.9 [3].

Please also note that this is a very complicated problem, much more 
difficult than "code uses C++11's features -> it is incompatible". So far, 
the only known incompatibility is in the way STL is built. The impact of 
the other changes like the modified signatures of a couple of STL methods 
is not clear to me (and I did search for an ultimate answer). Even the 
document listing the breakage [4] does not provide a clear message "if you 
do $FOO, stuff breaks, but $BAR is completely safe".

For example, there is no reason for not building e.g. Qt5 with C++11 
support. In fact, limiting it to use just the C++98 features would be 
considered a regression from the perspective of a developer using Qt.

Right now, it seems that we shall wait at least for GCC 4.9 to come and for 
upstream to decide how to solve this properly.

Cheers,
Jan

[1] http://gcc.gnu.org/gcc-4.8/cxx0x_status.html
[2] https://lwn.net/Articles/552831/
[3] https://lwn.net/Articles/552750/
[4] http://gcc.gnu.org/wiki/Cxx11AbiCompatibility

Re: [gentoo-dev] How to support C++11 in libraries?

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2699 bytes --]

Dnia 2013-12-19, o godz. 09:43:40
Jan Kundrát <jkt@gentoo.org> napisał(a):

> On Thursday, 19 December 2013 02:41:55 CEST, heroxbd@gentoo.org wrote:
> > I'd like to make an analogy to the version bump of gcc[1]. We (gentoo)
> > decide to support c++11 officially or not. If so, open a tracker bug to
> > push it globally. If not, patch lldb to support non-c++11, or leave it
> > up to the user to fiddle with the CXXFLAGS, where we only point the user
> > by to proper docs.
> 
> To be honest, I do not really see a link between the "let's bring in a new 
> version of compiler, it is a bit stricter in some situations, but these 
> were bugs anyway, like missing headers or unfounded assumptions about 
> memcpy()" with "let's support a new version of language which produces 
> object files with different ABI". Perhaps a Python 2.6 vs. Python 3.3 is a 
> better analogy?

Well, it's even worse than that. I think the main difference is that
usual gcc/whatever bumps may have resulted in *one* different libstdc++
ABI. People rebuilt all their packages, world went back to normal.

The issue here is that gcc is providing two ABIs in parallel, with
a -std= switch. And this sucks pretty much...

> Anyway, GCC 4.8 is pretty clear that the C++11's support is still 
> experimental [1] and subject to change [2]. The upstream developers have 
> announced that they plan to break the ABI of the code using C++11 features 
> in 4.9 [3].

Would it be possible to have a consistent ABI for both C++03 and C++11?
The simpler changes like adding new fields can be backported quite
easily (even if it would mean having dummy fields in C++03), I have no
idea about the more complex changes.

> Right now, it seems that we shall wait at least for GCC 4.9 to come and for 
> upstream to decide how to solve this properly.

Well, if they considered the C++11 ABI in gcc-4.9 stable, we could
consider changing the default to C++11. Then, we could do our
bump/switch thing as a matter of gcc-4.9 upgrade problem.

And that brings another issue in Gentoo -- gcc-config. AFAIR this tool
is completely insane and switches libstdc++ along with gcc version.
As a result, after switching to a gcc version with different C++ ABI,
installed software gets broken. And you can't really fix it without
going through the broken-system state or some hackery.

It would be much better if the switching was done by some ebuild. We
could then use subslots to force rebuilds of stuff using libstdc++.
Well, more than that, preserved-libs would prevent disappearing old
libstdc++ from breaking stuff.

But well, that's just my wishes...

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]

Re: [gentoo-dev] How to support C++11 in libraries?

[heroxbd]

[-- Attachment #1: Type: text/plain, Size: 704 bytes --]

Michał Górny <mgorny@gentoo.org> writes:

> And that brings another issue in Gentoo -- gcc-config. AFAIR this tool
> is completely insane and switches libstdc++ along with gcc version.
> As a result, after switching to a gcc version with different C++ ABI,
> installed software gets broken. And you can't really fix it without
> going through the broken-system state or some hackery.

Not that insane. Packages linked with libstdc++ are not crucial in
Gentoo, and can be rebuilt with emerge -e @world. Although it's a bad
idea for everybody to do so, the systems without "emerge -e @world" for
two years is likely to suck anyway.

It just reflects the fact that the world is not perfect.

[-- Attachment #2: Type: application/pgp-signature, Size: 489 bytes --]

Re: [gentoo-dev] How to support C++11 in libraries?

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1292 bytes --]

Dnia 2013-12-19, o godz. 22:47:50
heroxbd@gentoo.org napisał(a):

> Michał Górny <mgorny@gentoo.org> writes:
> 
> > And that brings another issue in Gentoo -- gcc-config. AFAIR this tool
> > is completely insane and switches libstdc++ along with gcc version.
> > As a result, after switching to a gcc version with different C++ ABI,
> > installed software gets broken. And you can't really fix it without
> > going through the broken-system state or some hackery.
> 
> Not that insane. Packages linked with libstdc++ are not crucial in
> Gentoo, and can be rebuilt with emerge -e @world. Although it's a bad
> idea for everybody to do so, the systems without "emerge -e @world" for
> two years is likely to suck anyway.

I think you are getting it the other way around.

It's not 'we do not need to support C++ properly because there are no
C++ packages crucial to Gentoo'. It's rather 'we have no crucial C++
packages because C++ support in Gentoo is broken by design'. It is
a limitation, not a reason to keep stuff buggy.

Think of paludis as a good example. People who'd like to use Paludis
will end up with broken package manager from time to time. How are they
supposed to rebuild it without a working package manager?

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]

Re: [gentoo-dev] How to support C++11 in libraries?

[heroxbd]

[-- Attachment #1: Type: text/plain, Size: 335 bytes --]

Michał Górny <mgorny@gentoo.org> writes:

> Think of paludis as a good example. People who'd like to use Paludis
> will end up with broken package manager from time to time. How are they
> supposed to rebuild it without a working package manager?

Oh, I'm scared. I'll step away and watch out for such situation at all
cost.

[-- Attachment #2: Type: application/pgp-signature, Size: 489 bytes --]

Re: [gentoo-dev] How to support C++11 in libraries?

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 929 bytes --]

On Thu, 19 Dec 2013 22:47:50 +0900
heroxbd@gentoo.org wrote:
> Michał Górny <mgorny@gentoo.org> writes:
> > And that brings another issue in Gentoo -- gcc-config. AFAIR this
> > tool is completely insane and switches libstdc++ along with gcc
> > version. As a result, after switching to a gcc version with
> > different C++ ABI, installed software gets broken. And you can't
> > really fix it without going through the broken-system state or some
> > hackery.
> 
> Not that insane. Packages linked with libstdc++ are not crucial in
> Gentoo, and can be rebuilt with emerge -e @world. Although it's a bad
> idea for everybody to do so, the systems without "emerge -e @world"
> for two years is likely to suck anyway.
> 
> It just reflects the fact that the world is not perfect.

Gentoo is the only distribution that gets this wrong. It's self
inflicted, not a problem with the world.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] How to support C++11 in libraries?

[Jan Kundrát]

On Thursday, 19 December 2013 10:18:55 CEST, Michał Górny wrote:
> Would it be possible to have a consistent ABI for both C++03 and C++11?
> The simpler changes like adding new fields can be backported quite
> easily (even if it would mean having dummy fields in C++03), I have no
> idea about the more complex changes.

I don't know, but from a bystander's point of view, I surely hope that it 
will be possible. Otherwise there would be no option but providing a 
multilib-like setup for C++11, after all.

Some messages on gcc's ML indicate that there are software vendors who are 
*very* afraid of doing the SONAME change again. Given that C++11 forbids a 
refcounted std::string while libstdc++ currently use just that for its 
implementation, I suspect that the upstream developers have a very 
interesting problem to solve. (And there's much more.)

It is pretty clear to me that even the gcc people have not reach a 
consensus on how the ABI of the standard library will look like in 4.9, so 
maybe it is premature for us to talk about how to solve this. The ball is 
on their side.

> Well, if they considered the C++11 ABI in gcc-4.9 stable, we could
> consider changing the default to C++11. Then, we could do our
> bump/switch thing as a matter of gcc-4.9 upgrade problem.

To put things into perspective, *if* the ABI changes and if the new version 
is compatible between C++98 and C++11, then we're talking something very 
similar to an upgrade from GCC 3.3.

Cheers,
Jan

Re: [gentoo-dev] How to support C++11 in libraries?

[vivo75]

On 12/19/13 16:17, Jan Kundrát wrote:
> On Thursday, 19 December 2013 10:18:55 CEST, Michał Górny wrote:
>> Would it be possible to have a consistent ABI for both C++03 and C++11?
>> The simpler changes like adding new fields can be backported quite
>> easily (even if it would mean having dummy fields in C++03), I have no
>> idea about the more complex changes.
>
> I don't know, but from a bystander's point of view, I surely hope that
> it will be possible. Otherwise there would be no option but providing
> a multilib-like setup for C++11, after all.
>
> Some messages on gcc's ML indicate that there are software vendors who
> are *very* afraid of doing the SONAME change again. Given that C++11
> forbids a refcounted std::string while libstdc++ currently use just
> that for its implementation, I suspect that the upstream developers
> have a very interesting problem to solve. (And there's much more.)
>
> It is pretty clear to me that even the gcc people have not reach a
> consensus on how the ABI of the standard library will look like in
> 4.9, so maybe it is premature for us to talk about how to solve this.
> The ball is on their side.
>
>> Well, if they considered the C++11 ABI in gcc-4.9 stable, we could
>> consider changing the default to C++11. Then, we could do our
>> bump/switch thing as a matter of gcc-4.9 upgrade problem.
>
> To put things into perspective, *if* the ABI changes and if the new
> version is compatible between C++98 and C++11, then we're talking
> something very similar to an upgrade from GCC 3.3.
>
> Cheers,
> Jan
>
just a question, what would do -fabi-version=6 added to CXXFLAGS even
w/o C++11?

Re: [gentoo-dev] How to support C++11 in libraries?

[Jan Kundrát]

On Thursday, 19 December 2013 17:29:19 CEST, vivo75@gmail.com wrote:
> just a question, what would do -fabi-version=6 added to CXXFLAGS even
> w/o C++11?

I believe that -fabi-version is for "low level bits" at the level of e.g. 
identifier mangling. It cannot affect whether a std::string is refcounted 
or not, or whether a std::list contains a member for O(1) size() behavior 
-- these require modifications to the actual memory layout of the class.

Cheers,
Jan

[gentoo-dev] Re: How to support C++11 in libraries?

[Ryan Hill]

[-- Attachment #1: Type: text/plain, Size: 1179 bytes --]

On Thu, 19 Dec 2013 10:18:55 +0100
Michał Górny <mgorny@gentoo.org> wrote:

> And that brings another issue in Gentoo -- gcc-config. AFAIR this tool
> is completely insane and switches libstdc++ along with gcc version.
> As a result, after switching to a gcc version with different C++ ABI,
> installed software gets broken. And you can't really fix it without
> going through the broken-system state or some hackery.

This hasn't been true for a while now.  The latest version of libstdc++ is
always used, no matter what version is selected.  I can't remember when this
was changed but we've gone through a couple GCC stabilizations since.

I have no opinion on how to support C++-11, except that it can't be globally
enabled.  We have to support compilers that predate the standard.

I wouldn't like any pkg-config hackery like ICU tried to pull a while back, but
now that we have a version of gcc that at least understands the flag in
stable at least it wouldn't instantly break everything.


-- 
Ryan Hill                        psn: dirtyepic_sk
   gcc-porting/toolchain/wxwidgets @ gentoo.org

47C3 6D62 4864 0E49 8E9E  7F92 ED38 BD49 957A 8463

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] How to support C++11 in libraries?

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1483 bytes --]

Dnia 2013-12-18, o godz. 22:58:07
heroxbd@gentoo.org napisał(a):

> Hey, Michał,
> 
> Michał Górny <mgorny@gentoo.org> writes:
> 
> > a) adding USE=c++11 and USE-deps to all the packages in question,
> 
> I think it is better achieved by a (simple and stupid) global
> CXXFLAGS. Adding an extra USE flag feels a little over-engineering.

This is nowhere near a good solution IMO.

First of all, it doesn't give us a way of ensuring ABI compatibility.
Users switch the flags and have to rebuild all C++ packages to regain
the ABI compatibility. The system ends up borked quite easily.

Then, we don't have a good way of finding packages to rebuild. Users
could try to find out which libraries used C++ but well... it's nowhere
near good. Or they just rebuild everything...

Then, many developers just won't bother. Users will be the ones to hit
the incompatible package build failures first.

Lastly, this gives us no way of switching to C++11 completely without
modifying the compiler defaults. Even if we put '-std=c++11' into
profiles, most of the people override CXXFLAGS and won't have it.

> Any anyway, if it is only for lldb, a piece of elog conveying a
> preferred solution would suffice.

elog? I think you mean dying with CXXFLAGS that don't specify
the necessary standard. Which is kinda backwards to REQUIRED_USE...

And then, simple CXXFLAGS solution would end up breaking users'
systems...

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]

[gentoo-dev] Re: How to support C++11 in libraries?

[Martin Vaeth]

Michał Górny <mgorny@gentoo.org> wrote:
>
> Then, many developers just won't bother. Users will be the ones to hit
> the incompatible package build failures first.

Is Diego's tinderbox still available?

My feeling is that the non-downward compatbility (API-wise)
of C++11 is much smaller than e.g. the gcc upgrades 4.5->4.6
or similar ones - except for rather exotic cases or possible
name-clashes there are only very few changes and AFAIK no
regressions concerning features - at most tiny syntax changes.

If one could verify by sandbox that only a few packages are actually
involved and can easily be fixed then one could make -std=c++11 the
compiler default (perhaps it is necessary to disable some c++11 specific
warnings by default in addition to avoid broken behaviour with -Werror
which unfortunately is still used by some projects).

Concerning binary blobs, one would need a list first ...

Re: [gentoo-dev] How to support C++11 in libraries?

[heroxbd]

[-- Attachment #1: Type: text/plain, Size: 1855 bytes --]

Michał Górny <mgorny@gentoo.org> writes:

> This is nowhere near a good solution IMO.
>
> First of all, it doesn't give us a way of ensuring ABI compatibility.
> Users switch the flags and have to rebuild all C++ packages to regain
> the ABI compatibility. The system ends up borked quite easily.
>
> Then, we don't have a good way of finding packages to rebuild. Users
> could try to find out which libraries used C++ but well... it's nowhere
> near good. Or they just rebuild everything...
>
> Then, many developers just won't bother. Users will be the ones to hit
> the incompatible package build failures first.
>
> Lastly, this gives us no way of switching to C++11 completely without
> modifying the compiler defaults. Even if we put '-std=c++11' into
> profiles, most of the people override CXXFLAGS and won't have it.
>
>> Any anyway, if it is only for lldb, a piece of elog conveying a
>> preferred solution would suffice.
>
> elog? I think you mean dying with CXXFLAGS that don't specify
> the necessary standard. Which is kinda backwards to REQUIRED_USE...
>
> And then, simple CXXFLAGS solution would end up breaking users'
> systems...

Michał, I am totally agree with you. This approach will leave lots of
dirty tricks to the users. Therefore this is a decision between whether
the devs or the users do this heavy lift.

If the the reason is only lldb and less than 10 other ebuilds, I feel it
not worth the develop and maintenace time. And if you have only met with
this problem twice, I suggest playing with it by the simplest solution
(via CXXFLAGS) for a while to avoid early optimization.

This is just my honest view on simple vs complex. Given the expertise
you hold in the realm of ABI, introducing a new ABI to maintain might
not be a big deal to you. Then I understand.

Cheers,
Benda

[-- Attachment #2: Type: application/pgp-signature, Size: 489 bytes --]

Re: [gentoo-dev] How to support C++11 in libraries?

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 18/12/13 02:54 AM, Michał Górny wrote:
> Hello, folks.
> 
> As some of you are already aware, the C++11 standard brought a few
>  changes to the C++ standard library. As a result, the standard 
> library used in C++11 mode has a different ABI than the one used
> in pre-C++11 mode. And this means that libraries that use some of 
> standard C++ types in their APIs also have different ABIs
> depending on the C++ standard used to build them. This is somehow
> explained in [1].
> 
> As a result, if a single library suffers that, its reverse 
> dependencies need to be built with the same C++ standard. And
> then, those can force even more dependencies and you may guess
> where this is going.
> 
> Basically, I've hit this with sys-devel/llvm. A user has requested 
> lldb support to be enabled in the ebuild [2]. Sadly, lldb requires 
> C++11 to be used, and this means that whole LLVM needs to become 
> C++11 enabled. And then, it would be at least recommended that all 
> reverse deps become C++11 enabled as well.
> 
> [1]:http://gcc.gnu.org/wiki/Cxx11AbiCompatibility 
> [2]:https://bugs.gentoo.org/show_bug.cgi?id=464354
> 
> 
> This raises the following question: how do we want to do it? I see 
> two possibilities:
> 
> a) adding USE=c++11 and USE-deps to all the packages in question,
> 
> b) doing the switch via synchronous version bump and matching 
> dependencies.
> 
> 
> I think that the variant a) is simpler. It goes like this:
> 
> 1) we add 'sys-devel/llvm[-c++11(-)]' deps to everything that uses 
> it,
> 
> 2) we add USE=c++11 to llvm,
> 
> 3) we add USE=c++11 and 'sys-devel/llvm[c++11=(-)]' deps to newest
>  version of everything that uses it.
> 
> The advantage is that plain users may just keep USE=c++11 disabled 
> as it is by default, and avoid hitting some issues with non-tree 
> packages and so on.
> 
> At some point, we'll probably want to remove non-C++11 support 
> completely. Then we could start by use.forcing the flag, changing 
> deps in packages and so on.
> 
> 


I think variant A is probably a better solution all around, not least
because this type of choice is almost a profile-level decision and a
global USE flag setting seems an appropriate way to ensure it works.
Plus, the use flag method allows those that want to fiddle with it
per-package to be able to do so while still guaranteeing the deptree
is properly synchronized as per the version.

variant B does offer the nice ability to just force it and therefore
not give end-users this new flag to worry about, but I forsee that we
will have conflicts with upgrading/downgrading package between the
pre- and post-C++11 border.  If we were a versioned distro this would
be the way to go, but....



-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iF0EAREIAAYFAlKxsOsACgkQ2ugaI38ACPAW7wD49p8JqopRARN4h6lB8+Z5bXs2
VB8Gb0wPMJouAuwpaAD8CDAa4L7w9KP9QXXaSnJrojdBmk9bZQ0GDJL26/5sqf4=
=01b1
-----END PGP SIGNATURE-----

Re: [gentoo-dev] How to support C++11 in libraries?

[Anthony G. Basile]

On 12/18/2013 02:54 AM, Michał Górny wrote:
>
> The problem with this solution is that as soon as user upgrades, shklee
> is forced to use C++11. On the other hand, we get rid of pre-C++11
> packages quite transparently, without extra work.
>
>
> What are your thoughts?
>
I assume with variant a you will be able to go back and forth between  
C++11 abi and pre-C++11.  If so, please adopt variant a.

-- 
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail    : blueness@gentoo.org
GnuPG FP  : 1FED FAD9 D82C 52A5 3BAB  DC79 9384 FA6E F52D 4BBA
GnuPG ID  : F52D4BBA

Re: [gentoo-dev] How to support C++11 in libraries?

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 18/12/13 01:10 PM, Anthony G. Basile wrote:
> On 12/18/2013 02:54 AM, Michał Górny wrote:
>> 
>> The problem with this solution is that as soon as user upgrades,
>> shklee is forced to use C++11. On the other hand, we get rid of
>> pre-C++11 packages quite transparently, without extra work.
>> 
>> 
>> What are your thoughts?
>> 
> I assume with variant a you will be able to go back and forth
> between C++11 abi and pre-C++11.  If so, please adopt variant a.
> 

It would, but unless it's done in a way similar to multilib's
abi_{x86,amd64,x32} (as was mentioned in the "crazy idea" post), only
one version would be installed at a time.

(that is, unless I misread the original proposal and both a generic
and a c++11 version would be installed if the flag was enabled -- i
can see the utility to that solution, but I am not a fan of it; better
to convert and patch when it becomes necessary)

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iF4EAREIAAYFAlKx6UwACgkQ2ugaI38ACPC26wD/cS7sN4eK67blugKVq/fUDbEv
50PtDy8xdgBp0tFowZwA/3GrPaROmN5XOUf8nW1tuoeuoD5oVH3nIGV/5AotXSZR
=FkC9
-----END PGP SIGNATURE-----

Re: [gentoo-dev] How to support C++11 in libraries?

[Sven Eden]

[-- Attachment #1: Type: text/plain, Size: 1550 bytes --]

Am Mittwoch, 18. Dezember 2013, 08:54:47 schrieb Michał Górny:
> This raises the following question: how do we want to do it? I see two
> possibilities:
> 
> a) adding USE=c++11 and USE-deps to all the packages in question,
> 
> b) doing the switch via synchronous version bump and matching
> dependencies.
>
(snip)
> 
> What are your thoughts?

I have already switched to C++11 on all my projects ages ago. It offers a lot, 
and the incompatibilities are rare at best.

C++11 is the current standard with the next being worked on already.
What is the rationale for staying with C++03 or (worse) C++98 in the first 
place? Nothing is gained. Only the need to fix what becomes broken.

( And a lot breakage can occur from using something like
#include <tr1/cstdint>
Instead of the C++11
#include <cstdint>
... Breakages that are trivial to fix ... )

And as C++11 is stricter in many places, I consider a program failing to 
compile with -std=c++11 to be broken and buggy per se, and needing to be fixed 
asap anyway. (I consider any C program failing to compile with g++ broken as 
well, allthough the cases might be more complex.)

So I'd go the reverse way. Make CXXFLAGS="-std=c++11" the default, and only  
override this for packages that do fishy stuff and break with it.

But of course this must be tested thoroughly first. I have some extra time to 
spare in the first three weeks of january, and I've already planned to do an 
'emerge -e @world' with c++11 as a global default anyway.


Cheers

Sven

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] How to support C++11 in libraries?

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1179 bytes --]

Dnia 2013-12-19, o godz. 09:58:25
Sven Eden <sven.eden@gmx.de> napisał(a):

> Am Mittwoch, 18. Dezember 2013, 08:54:47 schrieb Michał Górny:
> > This raises the following question: how do we want to do it? I see two
> > possibilities:
> > 
> > a) adding USE=c++11 and USE-deps to all the packages in question,
> > 
> > b) doing the switch via synchronous version bump and matching
> > dependencies.
> >
> (snip)
> > 
> > What are your thoughts?
> 
> I have already switched to C++11 on all my projects ages ago. It offers a lot, 
> and the incompatibilities are rare at best.
> 
> C++11 is the current standard with the next being worked on already.
> What is the rationale for staying with C++03 or (worse) C++98 in the first 
> place? Nothing is gained. Only the need to fix what becomes broken.

I can agree with that but we need a way to get a smooth transition.

> So I'd go the reverse way. Make CXXFLAGS="-std=c++11" the default, and only  
> override this for packages that do fishy stuff and break with it.

How can we do that? I think the only possibility is to patch gcc
and change the default...

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]

Re: [gentoo-dev] How to support C++11 in libraries?

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 19/12/13 04:07 AM, Michał Górny wrote:
> Dnia 2013-12-19, o godz. 09:58:25 Sven Eden <sven.eden@gmx.de>
> napisał(a):
> 
>> So I'd go the reverse way. Make CXXFLAGS="-std=c++11" the
>> default, and only override this for packages that do fishy stuff
>> and break with it.
> 
> How can we do that? I think the only possibility is to patch gcc 
> and change the default...
> 

A change in profiles?  14.0/* adds that to the default CXXFLAGS in
base, new stage3's etc are all rolled with this.  We recommend
migration to 14.0 profile and have a check somewhere about
"-std=c++11" missing from CXXFLAGS in case it's overridden in
make.conf, so users put it in place?

Now just a quick question about this; is an emerge -e @world going to
be necessary to make end-user systems work after such a change?  it's
sounding like it would be ....







-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iF4EAREIAAYFAlKzCf0ACgkQ2ugaI38ACPBBoAD/Y/e01CuaFf/40HfZMvGoknZg
oK9k5kX5HPCB30xNTYUA/jzg6mfTL1h6RYSgitKUQ8un3ewJTV9Nybmgr3nuvxr2
=SBQP
-----END PGP SIGNATURE-----

Re: [gentoo-dev] How to support C++11 in libraries?

[Jan Kundrát]

On Thursday, 19 December 2013 16:00:13 CEST, Ian Stakenvicius wrote:
> A change in profiles?  14.0/* adds that to the default CXXFLAGS in
> base, new stage3's etc are all rolled with this.  We recommend
> migration to 14.0 profile and have a check somewhere about
> "-std=c++11" missing from CXXFLAGS in case it's overridden in
> make.conf, so users put it in place?

Before you invest any more time in this, please understand that C++98 and 
C++11 are source-incompatible. There is no way to expect that a package 
builds fine when you throw -std=c++11 on it. And even if you patched them 
all, you are breaking an unknown number of 3rd party software over which 
you have exactly zero control.

Also note that as of gcc 4.8, the C++11 support is still labeled as 
experimental and upstream developers announced they will introduce ABI 
breaks in future.

With kind regards,
Jan

Re: [gentoo-dev] How to support C++11 in libraries?

[Anthony G. Basile]

On 12/19/2013 10:23 AM, Jan Kundrát wrote:
> On Thursday, 19 December 2013 16:00:13 CEST, Ian Stakenvicius wrote:
>> A change in profiles?  14.0/* adds that to the default CXXFLAGS in
>> base, new stage3's etc are all rolled with this.  We recommend
>> migration to 14.0 profile and have a check somewhere about
>> "-std=c++11" missing from CXXFLAGS in case it's overridden in
>> make.conf, so users put it in place?
>
> Before you invest any more time in this, please understand that C++98 
> and C++11 are source-incompatible. There is no way to expect that a 
> package builds fine when you throw -std=c++11 on it. And even if you 
> patched them all, you are breaking an unknown number of 3rd party 
> software over which you have exactly zero control.
>
> Also note that as of gcc 4.8, the C++11 support is still labeled as 
> experimental and upstream developers announced they will introduce ABI 
> breaks in future.
>
> With kind regards,
> Jan
>
>
I would look to gcc-4.9 for C++11.  By that point many upstream 
providers will start to feel the pressure and patch for us.

-- 
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail    : blueness@gentoo.org
GnuPG FP  : 1FED FAD9 D82C 52A5 3BAB  DC79 9384 FA6E F52D 4BBA
GnuPG ID  : F52D4BBA

[gentoo-dev] Re: How to support C++11 in libraries?

[Martin Vaeth]

Jan Kundrát <jkt@gentoo.org> wrote:
>
> Before you invest any more time in this, please understand that C++98 and
> C++11 are source-incompatible.

The question is what impact this theoretical incompatibility in a few
corner cases has in practice.

> There is no way to expect that a package builds fine when you
> throw -std=c++11 on it.

Yes, but the same is true for any gcc upgrade.
I repeat that numbers are necessary: If practice shows that there is only
a few packages in the tree needing a few trivial patches then the same can
be assumed about 3rd party software. The situation is rather different if
it turns out that almost nothing runs without severe patches.
Nobody can know the answer without actually trying.
However, I would be very surprised if the latter is true:
The example with string reference-counters which you gave is IMHO typical;
one would really need to write strange code to make it work *with* reference
counters but break without. Hard to believe that this happens in practice.
What *will* happen in practice is that the execution speed changes (probably
getting slower, but there might also be exceptions).

Re: [gentoo-dev] How to support C++11 in libraries?

[Jan Kundrát]

On Friday, 20 December 2013 10:00:43 CEST, Martin Vaeth wrote:
> The example with string reference-counters which you gave is IMHO 
typical;
> one would really need to write strange code to make it work *with* 
reference
> counters but break without. Hard to believe that this happens in 
practice.
> What *will* happen in practice is that the execution speed changes 
(probably
> getting slower, but there might also be exceptions).

You have not considered the implications of the updated requirements. With 
std::string, this might be hard to understand within all the layers of 
template wrapping, but consider std::list instead.

The "old" (C++98/C++03) std::list is implemented by containing exactly one 
member, the struct _List_node_base. This struct has exactly two pointers 
inside, one for the next item and one for the last. This layout cannot be 
changed without breaking the binary compatibility; it is effectively made 
public because GCC's standard library does not use the PIMPL idiom.

Now, this particular layout (which we just established cannot be changed 
without breaking the ABI) means that std::list::size() has O(n) time cost 
simply because it has to traverse the whole list to compute the number of 
items. The C++11 standard, however, mandates the time complexity to be 
O(1). This means that there will be a very visible change, at least for 
std::list. I won't speculate on how the upstream is going to solve this, 
but I do not expect that the end result will allow linking a translation 
unit built for C++98 by GCC <= 4.8 with one built for C++11 by the new 
compiler.

Jan

[gentoo-dev] Re: How to support C++11 in libraries?

[Martin Vaeth]

Jan Kundrát <jkt@gentoo.org> wrote:
> On Friday, 20 December 2013 10:00:43 CEST, Martin Vaeth wrote:
>> The example with string reference-counters which you gave is IMHO
>> typical;
>
> You have not considered the implications of the updated requirements

It seems you are changing the topic: We were talking about
downward-compatibility of source code. There is no doubt that
the C++11 requirements need a new ABI:

> without breaking the ABI [...]

> but I do not expect that the end result will allow linking a translation
> unit built for C++98 by GCC <= 4.8 with one built for C++11 by the new
> compiler.

That's why it might be a good idea to translate with C++11 by default.
No old units <-> no problem ;)
Again: It is clear that this route is possible only if the number
of packages breaking with such a default is small, and fixes are simple.
Which needs to be examined by "experiment" and not by theoretical
considerations.

Re: [gentoo-dev] How to support C++11 in libraries?

[Sven Eden]

[-- Attachment #1: Type: text/plain, Size: 2504 bytes --]

Am Donnerstag, 19. Dezember 2013, 16:23:08 schrieb Jan Kundrát:
> On Thursday, 19 December 2013 16:00:13 CEST, Ian Stakenvicius wrote:
> > A change in profiles?  14.0/* adds that to the default CXXFLAGS in
> > base, new stage3's etc are all rolled with this.  We recommend
> > migration to 14.0 profile and have a check somewhere about
> > "-std=c++11" missing from CXXFLAGS in case it's overridden in
> > make.conf, so users put it in place?
> 
> Before you invest any more time in this, please understand that C++98 and
> C++11 are source-incompatible. There is no way to expect that a package
> builds fine when you throw -std=c++11 on it. And even if you patched them
> all, you are breaking an unknown number of 3rd party software over which
> you have exactly zero control.

No. If you do something against the standard that is working due to lack of 
control when compiling with -std=c++98, then your source code is severly 
broken. Most developers will use C++03 (plus tr1) anyway, won't they?

And no, the languages are _not_ "source-incompatible". That would be a 
scandal!

But if you have your C++98/03 code, and do what most developers do and let 
your console be flooded with warnings you ignore, you must not be surprised, 
if the compilation fails when you decide to throw
"-std=c++11 -Wall -Wextra -Wpedantic -fsanitize=address -fsanitize=thread"
with gcc 4.8.2 at it.

There is absolutely no reason to expect a compilation to fail with C++11, if 
it went well with C++03 and "-Wall -Wextra -pedantic".

If you try to outsmart your compiler, it will get it's revenge very soon and 
very hard.

And according to [1] it goes even further:
Quote:
> If you use C++11 then in general you can combine C++11 code built with GCC
> 4.X and C++03 code built with any GCC, but there is not the same guarantee
> that you can combine with C++11 code built with GCC 4.Y or GCC 4.Z, because
> the C++11 features are not all stable yet (e.g. for GCC 4.9 I'm about to add
> a new virtual function to a base class in <future>.) This is why C++11
> support is still labelled "experimental", because it would be worse to claim
> it's stable and then have to break the ABI.

So basically C++11 <-> C++03 is no problem at all (unless you *export* certain 
symbols [2]), but combining C++11 from different gcc versions is nowhere 
guaranteed to work.

Cheers

Sven


[1] : https://lwn.net/Articles/552831/
[2] : http://gcc.gnu.org/wiki/Cxx11AbiCompatibility

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] How to support C++11 in libraries?

[Jan Kundrát]

On Friday, 20 December 2013 12:56:43 CEST, Sven Eden wrote:
> And no, the languages are _not_ "source-incompatible". That would be a 
> scandal!

You might argue about this, but that doesn't change these facts. This is 
absolutely valid C++98 program:

jkt@svist ~ $ cat foo.cpp 
int main() {
    auto int foo;
    return 0;
}
jkt@svist ~ $ g++ -std=c++98 -pedantic foo.cpp
jkt@svist ~ $ echo $?
0

...which will *not* build under the C++11 mode:

jkt@svist ~ $ g++ -std=c++0x foo.cpp
foo.cpp: In function ‘int main()’:
foo.cpp:2:14: error: two or more data types in declaration of ‘foo’

Yes, -Wc++0x-compat warns about this, yes, it's included in -Wall, but it 
does not change the fact that there *is* code out there which does conform 
to C++98 standard, does *not* try to "outsmart the compiler", and which 
will not build in the C++11 mode. Do we really have to be having this 
discussion?

Cheers,
Jan

Re: [gentoo-dev] How to support C++11 in libraries?

[Georg Rudoy]

2013/12/20 Jan Kundrát <jkt@gentoo.org>:
> On Friday, 20 December 2013 12:56:43 CEST, Sven Eden wrote:
>>
>> And no, the languages are _not_ "source-incompatible". That would be a
>> scandal!
>
>
> You might argue about this, but that doesn't change these facts. This is
> absolutely valid C++98 program:
>
> jkt@svist ~ $ cat foo.cpp int main() {
>    auto int foo;
>    return 0;
> }
> jkt@svist ~ $ g++ -std=c++98 -pedantic foo.cpp
> jkt@svist ~ $ echo $?
> 0
>
> ...which will *not* build under the C++11 mode:
>
> jkt@svist ~ $ g++ -std=c++0x foo.cpp
> foo.cpp: In function ‘int main()’:
> foo.cpp:2:14: error: two or more data types in declaration of ‘foo’
>
> Yes, -Wc++0x-compat warns about this, yes, it's included in -Wall, but it
> does not change the fact that there *is* code out there which does conform
> to C++98 standard, does *not* try to "outsmart the compiler", and which will
> not build in the C++11 mode. Do we really have to be having this discussion?

The C++ Committee considered this exact case in relation to the new
meaning of `auto` and decided that such code doesn't really exist in
the wild. You won't hit auto-related issues in almost all packages in
Portage I guess.

There are more obscure cases of incompatibility though, with more
obscure error messages, like with autogenerated move ctors and the
likes. I've hitted it myself a couple of times in more or less complex
template code, but can't think of an example off the top of my head
unfortunately.

-- 
  Georg Rudoy
  LeechCraft — http://leechcraft.org

[gentoo-dev] Re: How to support C++11 in libraries?

[Martin Vaeth]

Jan Kundrát <jkt@gentoo.org> wrote:
>> And no, the languages are _not_ "source-incompatible". That would be a 
>> scandal!
>
> You might argue about this, but that doesn't change these facts.

I think nobody had doubts that *theoretical* such examples can
be constructed (I even mentioned the case of name collission on
which your example builds). The question is how often does this
occur in *real-world* projects.

> does not change the fact that there *is* code out there

Your example is not "code out there". I would bet that the
name collission case hits less than 1% of existing projects.
If name collissions would be the only case, it would not even be
worth to discuss these things. The more restrictive syntax for
string concatenation (mentioned in some bug posted in this thread)
is a more realistic issue, but at least in this case, the project
has already fixed the problem.

Again: Numbers are needed; somebody (preferrably somebody with a fast
machine, so I am out ;) ) has to try to compile w/test the whole
~x86/~amd64 tree with CXXFLAGS=-std=c++11, and only then one can
seriously discuss how "source-incompatible" the languages really are.

My guess is still that you will observe less problems than with
a minor gcc upgrade, but it is only a guess, of course.

Re: [gentoo-dev] How to support C++11 in libraries?

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 20/12/13 06:56 AM, Sven Eden wrote:
> 
> So basically C++11 <-> C++03 is no problem at all (unless you
> *export* certain symbols [2]), but combining C++11 from different
> gcc versions is nowhere guaranteed to work.
> 
> Cheers
> 
> Sven
> 
> 
> [1] : https://lwn.net/Articles/552831/ [2] :
> http://gcc.gnu.org/wiki/Cxx11AbiCompatibility
> 


Aha... so what we should probably be doing then is filtering out
- --std=c++11 until gcc-4.9 or whatever version is released, that will
standardize things, so that we don't end up with systems that have a
mix-and-match.

And probably alert users using earlier versions of gcc that if they
enable --std=c++11, they should expect to 'emerge -e @world' on any
compiler switch.


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iF4EAREIAAYFAlK0XdEACgkQ2ugaI38ACPB2WgEAtnLeonyTFCF5cMEIi0kSIHZ/
RZcgjzRbojT3YejvMmkA/2v/qC7Cy58QAgz7oEC5z+KvPUVBJ79Ana0+rrPoq9TM
=pn2S
-----END PGP SIGNATURE-----

Re: [gentoo-dev] How to support C++11 in libraries?

[Anthony G. Basile]

On 12/19/2013 10:00 AM, Ian Stakenvicius wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> On 19/12/13 04:07 AM, Michał Górny wrote:
>> Dnia 2013-12-19, o godz. 09:58:25 Sven Eden <sven.eden@gmx.de>
>> napisał(a):
>>
>>> So I'd go the reverse way. Make CXXFLAGS="-std=c++11" the
>>> default, and only override this for packages that do fishy stuff
>>> and break with it.
>> How can we do that? I think the only possibility is to patch gcc
>> and change the default...
>>
> A change in profiles?  14.0/* adds that to the default CXXFLAGS in
> base, new stage3's etc are all rolled with this.  We recommend
> migration to 14.0 profile and have a check somewhere about
> "-std=c++11" missing from CXXFLAGS in case it's overridden in
> make.conf, so users put it in place?

If we are going to make -std=c++11 the default, I would do it in the gcc 
spec files and then override it with CXXFLAGS if USE=-c++11.

-- 
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail    : blueness@gentoo.org
GnuPG FP  : 1FED FAD9 D82C 52A5 3BAB  DC79 9384 FA6E F52D 4BBA
GnuPG ID  : F52D4BBA