[gentoo-portage-dev] Pre RFC on RFC: Add compiler information to exported a Package Manger's Cached Information.

[gentoo-portage-dev] Pre RFC on RFC: Add compiler information to exported a Package Manger's Cached Information.

[Anthony G. Basile]

Hi everyone,

I'm not sure about the following RFC which I was thinking of sending out 
to gentoo-dev@ but I think I want to bounce off the portage developers 
first for opinions.  So I guess this is a Pre-RFC to an RFC.


RFC: Add compiler information to exported a Package Manger's Cached 
Information.

Hi everyone,

I'd like to propose adding compiler information to portage's VDB on a 
per package basis so that we record the version of the compiler a 
package is built with.  The motivation and details of what is to be 
cached and exported (as per GLEP 64) I spell out below.  The 
implementation specific are left to the package maintainers.

Currently we cache dynamic linkage information about a package's 
linkable and executable objects so we can obtain a dependency graph 
connecting shared objects to the shared objects and executables that 
consume them.  This is useful for deciding what packages need to be 
rebuilt when a library package is updated that breaks backwards 
compatibility.

Recently the portage team expanded the cached linkage information to 
include multilib ABI identifiers to distinguish between different 
executable ABIs present on a system. [1,2]   Since the consumer of a 
library can only link against a library file of the same ABI, this 
information is necessary to properly construct the dependencies.

I propose extending this to include the ABI information for C++ 
libraries.  While C++ ABIs are of a different quality than the 
executable ABIs of a multilib system, the dependency issue is the same: 
the consumer of a library can only link against a library file of the 
same ABI.  Differences in C++ ABIs arrise because there is no guaranteed 
compatibility between shared and executable objects built under 
different C++ standards. [3]  Also, there is no guaranteed ABI 
compatibility even with the same standard when built with different 
versions of GCC differing in minor bumps. [4]  Since in Gentoo we allow 
users to supply their own CFLAGS/CXXFLAGS, thus specifying the C++ 
standard via the -std= flag, and because we allow them to switch between 
version of compiler and even between different compilers, we introduce 
the possibility of breakage due to differing C++ ABIs.

While changes in ABI are normally reflected in changes in the SONAME of 
the library, GCC upstream is not willing to do so for libstdc++ for 
other design reasons [4].  So to identify possible mismatches in C++ 
ABIs, it is necessary to record any user supplied CFLAGS/CXXFLAGS which 
change the default C++ standard  as well as the compiler used and its 
version.  The flags are already cached in VDB and can be compare to the 
compiler's default c++ standard obtained from `$CC -x c++ -E -P - <<< 
__cplusplus`.  Thus only the compiler and its version need to be added 
to VDB cache.  This should be done on a per package bases in 
VDB/<category>/<package>/COMPILER.  The contents of this file are to be 
parsed from the output of the first line of `$CC --version` of the 
compiler used to build the package.  The format should be as follows:

<compiler name> <version> <optional information>

eg. "clang 3.3" or "gcc 4.8.3 (Gentoo Hardened 4.8.3 p1.1, pie-0.5.9)"

Per GLEP 64, this information should be made available for utilities to 
help identify C++ ABI mismatches.

Finally, a limitation of the above should be noted.  Since the 
CFLAGS/CXXFLAGS cached are only those supplied by the user, it does not 
cover situations where the package build system or ebuild supply their 
own -std= flag.  Since this information cannot and should not be cached 
by the package manager, utilities used to find any mismatches in C++ ABI 
must provide for this intelligence.


Refs.
[1] https://bugs.gentoo.org/show_bug.cgi?id=534206.
[2] 
http://cgit.gentooexperimental.org/proj/portage.git/commit/?id=f1c1b8a77eebf7713b32e5f9945690f60f4f46de
[3] This can lead to breakage between libraries and their consumers. 
For example,
see https://bugs.gentoo.org/show_bug.cgi?id=513386.
[4] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61758.  "It is totally 
unsupported
(and unlikely to work) to mix C++11 code built with GCC 4.x and 4.y, for 
any x!=y"
The same incompatibilities may be introduced by clang as well.



-- 
Anthony G. Basile, Ph. D.
Chair of Information Technology
D'Youville College
Buffalo, NY 14201
(716) 829-8197

Re: [gentoo-portage-dev] Pre RFC on RFC: Add compiler information to exported a Package Manger's Cached Information.

[Zac Medico]

On 02/21/2015 08:14 AM, Anthony G. Basile wrote:
> Finally, a limitation of the above should be noted.  Since the
> CFLAGS/CXXFLAGS cached are only those supplied by the user, it does not
> cover situations where the package build system or ebuild supply their
> own -std= flag.  Since this information cannot and should not be cached
> by the package manager, utilities used to find any mismatches in C++ ABI
> must provide for this intelligence.

I wonder if it's feasible to force the -rpath option when using the
-std= flag, and force the real/canonical libstc++.so path into the
rpath. There's some mention of -rpath here:

http://tldp.org/HOWTO/Program-Library-HOWTO/shared-libraries.html#AEN95

Have the -rpath information in the binaries should make it easy to
correctly resolve the library dependencies.
-- 
Thanks,
Zac

Re: [gentoo-portage-dev] Pre RFC on RFC: Add compiler information to exported a Package Manger's Cached Information.

[Zac Medico]

On 02/21/2015 12:18 PM, Zac Medico wrote:
> On 02/21/2015 08:14 AM, Anthony G. Basile wrote:
>> Finally, a limitation of the above should be noted.  Since the
>> CFLAGS/CXXFLAGS cached are only those supplied by the user, it does not
>> cover situations where the package build system or ebuild supply their
>> own -std= flag.  Since this information cannot and should not be cached
>> by the package manager, utilities used to find any mismatches in C++ ABI
>> must provide for this intelligence.
> 
> I wonder if it's feasible to force the -rpath option when using the
> -std= flag, and force the real/canonical libstc++.so path into the
> rpath. There's some mention of -rpath here:
> 
> http://tldp.org/HOWTO/Program-Library-HOWTO/shared-libraries.html#AEN95
> 
> Have the -rpath information in the binaries should make it easy to
> correctly resolve the library dependencies.

If we put the real/canonical libstdc++.so path in the DT_NEEDED section,
then it will automatically work with existing soname dependency support.
-- 
Thanks,
Zac

Re: [gentoo-portage-dev] Pre RFC on RFC: Add compiler information to exported a Package Manger's Cached Information.

[Zac Medico]

On 02/21/2015 10:22 PM, Zac Medico wrote:
> If we put the real/canonical libstdc++.so path in the DT_NEEDED section,
> then it will automatically work with existing soname dependency support.

Actually, we'd also have to add a way for you to put the full path of
the libstdc++.so in PROVIDES. For example:

PROVIDES_ABSOLUTE="/usr/lib/gcc/*/*/libstdc++.so.6"
-- 
Thanks,
Zac

Re: [gentoo-portage-dev] Pre RFC on RFC: Add compiler information to exported a Package Manger's Cached Information.

[Anthony G. Basile]

On 02/22/15 01:30, Zac Medico wrote:
> On 02/21/2015 10:22 PM, Zac Medico wrote:
>> If we put the real/canonical libstdc++.so path in the DT_NEEDED section,
>> then it will automatically work with existing soname dependency support.
>
> Actually, we'd also have to add a way for you to put the full path of
> the libstdc++.so in PROVIDES. For example:
>
> PROVIDES_ABSOLUTE="/usr/lib/gcc/*/*/libstdc++.so.6"
>

I guess I don't understand how this would work exactly.  What if someone 
has gcc-4.8.3.  Builds library libfoo.so which uses c++.  Then upgrades 
to gcc-4.9, removes 4.8 and then tries to build bar which is also 
written in c++ and links against libfoo.so.  We would have mismatching 
abis.  How would this catch it and trigger the correct rebuilds?

Unless I'm misunderstanding your *'s in that line.  Are you using 
PROVIDES_ABSOLUTE as a way of recording what version of the compiler 
libfoo.so was build with?  So that you'd have a line that says libfoo.so 
links against /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libstdc++.so, so 
that parsing that line gives 4.8.3?

Also if you had the absolute path in VDB somewhere, like in PROVIDES, 
then you don't need it in the elf's rpath which would make me feel better.

-- 
Anthony G. Basile, Ph. D.
Chair of Information Technology
D'Youville College
Buffalo, NY 14201
(716) 829-8197

Re: [gentoo-portage-dev] Pre RFC on RFC: Add compiler information to exported a Package Manger's Cached Information.

[Anthony G. Basile]

On 02/25/15 14:51, Anthony G. Basile wrote:
> On 02/22/15 01:30, Zac Medico wrote:
>> On 02/21/2015 10:22 PM, Zac Medico wrote:
>>> If we put the real/canonical libstdc++.so path in the DT_NEEDED section,
>>> then it will automatically work with existing soname dependency support.
>>
>> Actually, we'd also have to add a way for you to put the full path of
>> the libstdc++.so in PROVIDES. For example:
>>
>> PROVIDES_ABSOLUTE="/usr/lib/gcc/*/*/libstdc++.so.6"
>>
>
> I guess I don't understand how this would work exactly.  What if someone
> has gcc-4.8.3.  Builds library libfoo.so which uses c++.  Then upgrades
> to gcc-4.9, removes 4.8 and then tries to build bar which is also
> written in c++ and links against libfoo.so.  We would have mismatching
> abis.  How would this catch it and trigger the correct rebuilds?
>
> Unless I'm misunderstanding your *'s in that line.  Are you using
> PROVIDES_ABSOLUTE as a way of recording what version of the compiler
> libfoo.so was build with?  So that you'd have a line that says libfoo.so
> links against /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libstdc++.so, so
> that parsing that line gives 4.8.3?
>
> Also if you had the absolute path in VDB somewhere, like in PROVIDES,
> then you don't need it in the elf's rpath which would make me feel better.
>

Actually no, you'd still need rpath for the elf itslef otherwise you can 
still link against the wrong version of libstdc++.so.  Note in my 
following example that even though I build test.cpp with 4.7.3 I still 
wind up linking aginast 4.8.3.


yellow tmp # cat test.cpp
#include <iostream>
using namespace std;
int main() { cout << "hello owrld" << endl ; }

yellow tmp # gcc-config -l
  [1] x86_64-pc-linux-gnu-4.7.3 *
  [2] x86_64-pc-linux-gnu-4.7.3-hardenednopie
  [3] x86_64-pc-linux-gnu-4.7.3-hardenednopiessp
  [4] x86_64-pc-linux-gnu-4.7.3-hardenednossp
  [5] x86_64-pc-linux-gnu-4.7.3-vanilla
  [6] x86_64-pc-linux-gnu-4.8.3
  [7] x86_64-pc-linux-gnu-4.8.3-hardenednopie
  [8] x86_64-pc-linux-gnu-4.8.3-hardenednopiessp
  [9] x86_64-pc-linux-gnu-4.8.3-hardenednossp
  [10] x86_64-pc-linux-gnu-4.8.3-vanilla

yellow tmp # g++ -o go test.cpp

yellow tmp # ldd go
	linux-vdso.so.1 (0x0000033f63717000)
	libstdc++.so.6 => /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libstdc++.so.6 
(0x0000033f631cf000)
	libm.so.6 => /lib64/libm.so.6 (0x0000033f62ecb000)
	libgcc_s.so.1 => /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libgcc_s.so.1 
(0x0000033f62cb4000)
	libc.so.6 => /lib64/libc.so.6 (0x0000033f628f8000)
	/lib64/ld-linux-x86-64.so.2 (0x0000033f634f6000)

yellow tmp # g++ -Wl,-rpath,/usr/lib/gcc/x86_64-pc-linux-gnu/4.7.3/ -o 
go test.cpp

yellow tmp # ldd go

	linux-vdso.so.1 (0x0000036035212000)
	libstdc++.so.6 => /usr/lib/gcc/x86_64-pc-linux-gnu/4.7.3/libstdc++.so.6 
(0x0000036034ccf000)
	libm.so.6 => /lib64/libm.so.6 (0x00000360349cb000)
	libgcc_s.so.1 => /usr/lib/gcc/x86_64-pc-linux-gnu/4.7.3/libgcc_s.so.1 
(0x00000360347b4000)
	libc.so.6 => /lib64/libc.so.6 (0x00000360343f8000)
	/lib64/ld-linux-x86-64.so.2 (0x0000036034ff1000)


-- 
Anthony G. Basile, Ph. D.
Chair of Information Technology
D'Youville College
Buffalo, NY 14201
(716) 829-8197

Re: [gentoo-portage-dev] Pre RFC on RFC: Add compiler information to exported a Package Manger's Cached Information.

[Zac Medico]

On 02/25/2015 12:01 PM, Anthony G. Basile wrote:
> On 02/25/15 14:51, Anthony G. Basile wrote:
>> On 02/22/15 01:30, Zac Medico wrote:
>>> On 02/21/2015 10:22 PM, Zac Medico wrote:
>>>> If we put the real/canonical libstdc++.so path in the DT_NEEDED
>>>> section,
>>>> then it will automatically work with existing soname dependency
>>>> support.
>>>
>>> Actually, we'd also have to add a way for you to put the full path of
>>> the libstdc++.so in PROVIDES. For example:
>>>
>>> PROVIDES_ABSOLUTE="/usr/lib/gcc/*/*/libstdc++.so.6"
>>>
>>
>> I guess I don't understand how this would work exactly.  What if someone
>> has gcc-4.8.3.  Builds library libfoo.so which uses c++.  Then upgrades
>> to gcc-4.9, removes 4.8 and then tries to build bar which is also
>> written in c++ and links against libfoo.so.  We would have mismatching
>> abis.  How would this catch it and trigger the correct rebuilds?
>>
>> Unless I'm misunderstanding your *'s in that line.  Are you using
>> PROVIDES_ABSOLUTE as a way of recording what version of the compiler
>> libfoo.so was build with?  So that you'd have a line that says libfoo.so
>> links against /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libstdc++.so, so
>> that parsing that line gives 4.8.3?
>>
>> Also if you had the absolute path in VDB somewhere, like in PROVIDES,
>> then you don't need it in the elf's rpath which would make me feel
>> better.
>>
> 
> Actually no, you'd still need rpath for the elf itslef otherwise you can
> still link against the wrong version of libstdc++.so.  Note in my
> following example that even though I build test.cpp with 4.7.3 I still
> wind up linking aginast 4.8.3.

If DT_NEEDED contains the absolute libstdc++.so path, it's guaranteed to
link against the correct version, regardless of rpath.
-- 
Thanks,
Zac

Re: [gentoo-portage-dev] Pre RFC on RFC: Add compiler information to exported a Package Manger's Cached Information.

[Anthony G. Basile]

On 02/25/15 15:38, Zac Medico wrote:
> On 02/25/2015 12:01 PM, Anthony G. Basile wrote:
>> On 02/25/15 14:51, Anthony G. Basile wrote:
>>> On 02/22/15 01:30, Zac Medico wrote:
>>>> On 02/21/2015 10:22 PM, Zac Medico wrote:
>>>>> If we put the real/canonical libstdc++.so path in the DT_NEEDED
>>>>> section,
>>>>> then it will automatically work with existing soname dependency
>>>>> support.
>>>>
>>>> Actually, we'd also have to add a way for you to put the full path of
>>>> the libstdc++.so in PROVIDES. For example:
>>>>
>>>> PROVIDES_ABSOLUTE="/usr/lib/gcc/*/*/libstdc++.so.6"
>>>>
>>>
>>> I guess I don't understand how this would work exactly.  What if someone
>>> has gcc-4.8.3.  Builds library libfoo.so which uses c++.  Then upgrades
>>> to gcc-4.9, removes 4.8 and then tries to build bar which is also
>>> written in c++ and links against libfoo.so.  We would have mismatching
>>> abis.  How would this catch it and trigger the correct rebuilds?
>>>
>>> Unless I'm misunderstanding your *'s in that line.  Are you using
>>> PROVIDES_ABSOLUTE as a way of recording what version of the compiler
>>> libfoo.so was build with?  So that you'd have a line that says libfoo.so
>>> links against /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libstdc++.so, so
>>> that parsing that line gives 4.8.3?
>>>
>>> Also if you had the absolute path in VDB somewhere, like in PROVIDES,
>>> then you don't need it in the elf's rpath which would make me feel
>>> better.
>>>
>>
>> Actually no, you'd still need rpath for the elf itslef otherwise you can
>> still link against the wrong version of libstdc++.so.  Note in my
>> following example that even though I build test.cpp with 4.7.3 I still
>> wind up linking aginast 4.8.3.
>
> If DT_NEEDED contains the absolute libstdc++.so path, it's guaranteed to
> link against the correct version, regardless of rpath.
>

How do you get DT_NEEDED to the absolute libstdc++.so path when building?

-- 
Anthony G. Basile, Ph. D.
Chair of Information Technology
D'Youville College
Buffalo, NY 14201
(716) 829-8197

Re: [gentoo-portage-dev] Pre RFC on RFC: Add compiler information to exported a Package Manger's Cached Information.

[Zac Medico]

On 02/25/2015 03:41 PM, Anthony G. Basile wrote:
> On 02/25/15 15:38, Zac Medico wrote:
>> On 02/25/2015 12:01 PM, Anthony G. Basile wrote:
>>> On 02/25/15 14:51, Anthony G. Basile wrote:
>>>> On 02/22/15 01:30, Zac Medico wrote:
>>>>> On 02/21/2015 10:22 PM, Zac Medico wrote:
>>>>>> If we put the real/canonical libstdc++.so path in the DT_NEEDED
>>>>>> section,
>>>>>> then it will automatically work with existing soname dependency
>>>>>> support.
>>>>>
>>>>> Actually, we'd also have to add a way for you to put the full path of
>>>>> the libstdc++.so in PROVIDES. For example:
>>>>>
>>>>> PROVIDES_ABSOLUTE="/usr/lib/gcc/*/*/libstdc++.so.6"
>>>>>
>>>>
>>>> I guess I don't understand how this would work exactly.  What if
>>>> someone
>>>> has gcc-4.8.3.  Builds library libfoo.so which uses c++.  Then upgrades
>>>> to gcc-4.9, removes 4.8 and then tries to build bar which is also
>>>> written in c++ and links against libfoo.so.  We would have mismatching
>>>> abis.  How would this catch it and trigger the correct rebuilds?
>>>>
>>>> Unless I'm misunderstanding your *'s in that line.  Are you using
>>>> PROVIDES_ABSOLUTE as a way of recording what version of the compiler
>>>> libfoo.so was build with?  So that you'd have a line that says
>>>> libfoo.so
>>>> links against /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libstdc++.so, so
>>>> that parsing that line gives 4.8.3?
>>>>
>>>> Also if you had the absolute path in VDB somewhere, like in PROVIDES,
>>>> then you don't need it in the elf's rpath which would make me feel
>>>> better.
>>>>
>>>
>>> Actually no, you'd still need rpath for the elf itslef otherwise you can
>>> still link against the wrong version of libstdc++.so.  Note in my
>>> following example that even though I build test.cpp with 4.7.3 I still
>>> wind up linking aginast 4.8.3.
>>
>> If DT_NEEDED contains the absolute libstdc++.so path, it's guaranteed to
>> link against the correct version, regardless of rpath.
>>
> 
> How do you get DT_NEEDED to the absolute libstdc++.so path when building?

I'm not sure how we're going to accomplish that yet, but it should be
feasible. Ideally, the build system would support it. The worst case
would be that we would have to patch the DT_NEEDED sections after
src_install.
-- 
Thanks,
Zac

Re: [gentoo-portage-dev] Pre RFC on RFC: Add compiler information to exported a Package Manger's Cached Information.

[Zac Medico]

On 02/25/2015 04:07 PM, Zac Medico wrote:
> On 02/25/2015 03:41 PM, Anthony G. Basile wrote:
>> On 02/25/15 15:38, Zac Medico wrote:
>>> On 02/25/2015 12:01 PM, Anthony G. Basile wrote:
>>>> On 02/25/15 14:51, Anthony G. Basile wrote:
>>>>> On 02/22/15 01:30, Zac Medico wrote:
>>>>>> On 02/21/2015 10:22 PM, Zac Medico wrote:
>>>>>>> If we put the real/canonical libstdc++.so path in the DT_NEEDED
>>>>>>> section,
>>>>>>> then it will automatically work with existing soname dependency
>>>>>>> support.
>>>>>>
>>>>>> Actually, we'd also have to add a way for you to put the full path of
>>>>>> the libstdc++.so in PROVIDES. For example:
>>>>>>
>>>>>> PROVIDES_ABSOLUTE="/usr/lib/gcc/*/*/libstdc++.so.6"
>>>>>>
>>>>>
>>>>> I guess I don't understand how this would work exactly.  What if
>>>>> someone
>>>>> has gcc-4.8.3.  Builds library libfoo.so which uses c++.  Then upgrades
>>>>> to gcc-4.9, removes 4.8 and then tries to build bar which is also
>>>>> written in c++ and links against libfoo.so.  We would have mismatching
>>>>> abis.  How would this catch it and trigger the correct rebuilds?
>>>>>
>>>>> Unless I'm misunderstanding your *'s in that line.  Are you using
>>>>> PROVIDES_ABSOLUTE as a way of recording what version of the compiler
>>>>> libfoo.so was build with?  So that you'd have a line that says
>>>>> libfoo.so
>>>>> links against /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libstdc++.so, so
>>>>> that parsing that line gives 4.8.3?
>>>>>
>>>>> Also if you had the absolute path in VDB somewhere, like in PROVIDES,
>>>>> then you don't need it in the elf's rpath which would make me feel
>>>>> better.
>>>>>
>>>>
>>>> Actually no, you'd still need rpath for the elf itslef otherwise you can
>>>> still link against the wrong version of libstdc++.so.  Note in my
>>>> following example that even though I build test.cpp with 4.7.3 I still
>>>> wind up linking aginast 4.8.3.
>>>
>>> If DT_NEEDED contains the absolute libstdc++.so path, it's guaranteed to
>>> link against the correct version, regardless of rpath.
>>>
>>
>> How do you get DT_NEEDED to the absolute libstdc++.so path when building?
> 
> I'm not sure how we're going to accomplish that yet, but it should be
> feasible. Ideally, the build system would support it. The worst case
> would be that we would have to patch the DT_NEEDED sections after
> src_install.

Using patchelf built from git [1], I tried to patch the DT_NEEDED of a
"hello world" program:
$ cat hello.cpp
#include <iostream>
using namespace std;
int main() { cout << "Hello World!" << endl ; }
$ g++ -o hello hello.cpp
$ scanelf -qF '%a;%p;%S;%r;%n' hello
EM_X86_64;hello;;;libstdc++.so.6,libm.so.6,libgcc_s.so.1,libc.so.6
$ ldd hello
        linux-vdso.so.1 (0x00007fff5992a000)
        libstdc++.so.6 =>
/usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libstdc++.so.6 (0x00007f38f6a82000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f38f6783000)
        libgcc_s.so.1 =>
/usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libgcc_s.so.1 (0x00007f38f656d000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f38f61c7000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f38f6d86000)
$ ./hello
Hello World!
$ patchelf --replace-needed libstdc++.so.6
/usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libstdc++.so.6 hello
$ scanelf -qF '%a;%p;%S;%r;%n' hello
EM_X86_64;hello;;;/usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libstdc++.so.6,libm.so.6,libgcc_s.so.1,libc.so.6
$ ldd hello
Inconsistency detected by ld.so: dl-version.c: 224:
_dl_check_map_versions: Assertion `needed != ((void *)0)' failed!
$ ./hello
Inconsistency detected by ld.so: dl-version.c: 224:
_dl_check_map_versions: Assertion `needed != ((void *)0)' failed!

Apparently the ld.so is not capable of resolving absolute paths in
DT_NEEDED. Maybe it's more practical to go with -rpath instead.

[1] https://github.com/NixOS/patchelf
-- 
Thanks,
Zac

Re: [gentoo-portage-dev] Pre RFC on RFC: Add compiler information to exported a Package Manger's Cached Information.

[Zac Medico]

On 02/25/2015 12:01 PM, Anthony G. Basile wrote:
> On 02/25/15 14:51, Anthony G. Basile wrote:
>> On 02/22/15 01:30, Zac Medico wrote:
>>> On 02/21/2015 10:22 PM, Zac Medico wrote:
>>>> If we put the real/canonical libstdc++.so path in the DT_NEEDED
>>>> section,
>>>> then it will automatically work with existing soname dependency
>>>> support.
>>>
>>> Actually, we'd also have to add a way for you to put the full path of
>>> the libstdc++.so in PROVIDES. For example:
>>>
>>> PROVIDES_ABSOLUTE="/usr/lib/gcc/*/*/libstdc++.so.6"
>>>
>>
>> I guess I don't understand how this would work exactly.  What if someone
>> has gcc-4.8.3.  Builds library libfoo.so which uses c++.  Then upgrades
>> to gcc-4.9, removes 4.8 and then tries to build bar which is also
>> written in c++ and links against libfoo.so.  We would have mismatching
>> abis.  How would this catch it and trigger the correct rebuilds?
>>
>> Unless I'm misunderstanding your *'s in that line.  Are you using
>> PROVIDES_ABSOLUTE as a way of recording what version of the compiler
>> libfoo.so was build with?  So that you'd have a line that says libfoo.so
>> links against /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libstdc++.so, so
>> that parsing that line gives 4.8.3?
>>
>> Also if you had the absolute path in VDB somewhere, like in PROVIDES,
>> then you don't need it in the elf's rpath which would make me feel
>> better.
>>
> 
> Actually no, you'd still need rpath for the elf itslef otherwise you can
> still link against the wrong version of libstdc++.so.  Note in my
> following example that even though I build test.cpp with 4.7.3 I still
> wind up linking aginast 4.8.3.
> 
> 
> yellow tmp # cat test.cpp
> #include <iostream>
> using namespace std;
> int main() { cout << "hello owrld" << endl ; }
> 
> yellow tmp # gcc-config -l
>  [1] x86_64-pc-linux-gnu-4.7.3 *
>  [2] x86_64-pc-linux-gnu-4.7.3-hardenednopie
>  [3] x86_64-pc-linux-gnu-4.7.3-hardenednopiessp
>  [4] x86_64-pc-linux-gnu-4.7.3-hardenednossp
>  [5] x86_64-pc-linux-gnu-4.7.3-vanilla
>  [6] x86_64-pc-linux-gnu-4.8.3
>  [7] x86_64-pc-linux-gnu-4.8.3-hardenednopie
>  [8] x86_64-pc-linux-gnu-4.8.3-hardenednopiessp
>  [9] x86_64-pc-linux-gnu-4.8.3-hardenednossp
>  [10] x86_64-pc-linux-gnu-4.8.3-vanilla
> 
> yellow tmp # g++ -o go test.cpp
> 
> yellow tmp # ldd go
>     linux-vdso.so.1 (0x0000033f63717000)
>     libstdc++.so.6 =>
> /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libstdc++.so.6 (0x0000033f631cf000)
>     libm.so.6 => /lib64/libm.so.6 (0x0000033f62ecb000)
>     libgcc_s.so.1 =>
> /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libgcc_s.so.1 (0x0000033f62cb4000)
>     libc.so.6 => /lib64/libc.so.6 (0x0000033f628f8000)
>     /lib64/ld-linux-x86-64.so.2 (0x0000033f634f6000)
> 
> yellow tmp # g++ -Wl,-rpath,/usr/lib/gcc/x86_64-pc-linux-gnu/4.7.3/ -o
> go test.cpp
> 
> yellow tmp # ldd go
> 
>     linux-vdso.so.1 (0x0000036035212000)
>     libstdc++.so.6 =>
> /usr/lib/gcc/x86_64-pc-linux-gnu/4.7.3/libstdc++.so.6 (0x0000036034ccf000)
>     libm.so.6 => /lib64/libm.so.6 (0x00000360349cb000)
>     libgcc_s.so.1 =>
> /usr/lib/gcc/x86_64-pc-linux-gnu/4.7.3/libgcc_s.so.1 (0x00000360347b4000)
>     libc.so.6 => /lib64/libc.so.6 (0x00000360343f8000)
>     /lib64/ld-linux-x86-64.so.2 (0x0000036034ff1000)
> 
> 

If we use -rpath, then we need to decide how to encode the rpath
information for use in soname dependency resolution. For example, we can
introduce a PROVIDES_RPATH variable that the gcc ebuild will use to
export paths for use in soname dependency resolution. For libstdc++
consumers, we can introduce a corresponding REQUIRES_RPATH variable.

Instead of requiring ebuilds to explicitly set a REQUIRES_RPATH
variable, we might instead extract the information from DT_RPATH and
DT_RUNPATH sections. However, the result would probably need to be
subject to filtering by something like REQUIRES_EXCLUDE. Maybe it's
better to have explicit REQUIRES_RPATH settings.
-- 
Thanks,
Zac

Re: [gentoo-portage-dev] Pre RFC on RFC: Add compiler information to exported a Package Manger's Cached Information.

[Zac Medico]

On 02/25/2015 11:51 AM, Anthony G. Basile wrote:
> On 02/22/15 01:30, Zac Medico wrote:
>> On 02/21/2015 10:22 PM, Zac Medico wrote:
>>> If we put the real/canonical libstdc++.so path in the DT_NEEDED section,
>>> then it will automatically work with existing soname dependency support.
>>
>> Actually, we'd also have to add a way for you to put the full path of
>> the libstdc++.so in PROVIDES. For example:
>>
>> PROVIDES_ABSOLUTE="/usr/lib/gcc/*/*/libstdc++.so.6"
>>
> 
> I guess I don't understand how this would work exactly.  What if someone
> has gcc-4.8.3.  Builds library libfoo.so which uses c++.  Then upgrades
> to gcc-4.9, removes 4.8 and then tries to build bar which is also
> written in c++ and links against libfoo.so.  We would have mismatching
> abis.  How would this catch it and trigger the correct rebuilds?

If the absolute libstdc++ path is recorded in DT_NEEDED, then libfoo.so
built against gcc-4.8 will break as soon as gcc-4.8 is unmerged. It's
easy to see that a rebuild is needed, because the DT_NEEDED data in
NEEDED.ELF.2 shows that libfoo.so is linked against gcc-4.8. The
relevant part of the DT_NEEDED data is also recorded in REQUIRES (which
is available during soname dependency resolution for binary packages).

> Unless I'm misunderstanding your *'s in that line.  Are you using
> PROVIDES_ABSOLUTE as a way of recording what version of the compiler
> libfoo.so was build with? 

No, it's for the gcc ebuild, in order to indicate that the absolute
libstdc++ path should be included in PROVIDES, for the purposes of
soname dependency resolution.

> So that you'd have a line that says libfoo.so
> links against /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libstdc++.so, so
> that parsing that line gives 4.8.3?

No, this dependency information would propagate via DT_NEEDED. Any
builds that use the -std= flag would be responsible for ensuring that
the absolute libstdc++ path is recorded in DT_NEEDED, rather than the
plain libstdc++.so.6 soname.

> Also if you had the absolute path in VDB somewhere, like in PROVIDES,
> then you don't need it in the elf's rpath which would make me feel better.

Yeah, the absolute path of libstdc++ will be recorded in gcc's PROVIDES,
thanks to the ebuild setting the PROVIDES_ABSOLUTE variable. The
absolute path of libstdc++ that libfoo.so links against will be recorded
in both NEEDED.ELF.2 and REQUIRES. There's no need for rpath.
-- 
Thanks,
Zac