[gentoo-musl] [GSoC] _GNU_SOURCE in C++

[gentoo-musl] [GSoC] _GNU_SOURCE in C++

[Lei Zhang]

Hi,

So far the toughest issue with building LLVM against musl is that,
LLVM redefines a bunch of non-POSIX symbols, including fopen64,
stat64, etc, inside a C++ namespace. The namespace prevents them from
clashing with glibc's definitions; but unfortunately, musl defines
these symbols as macros, and macros don't respect C++ namespace, thus
the clash.

Here's how it looks in musl:

#if defined(_LARGEFILE64_SOURCE) || defined(_GNU_SOURCE)
#define tmpfile64 tmpfile
#define fopen64 fopen
...
#endif

Actually these definitions are protected by _GNU_SOURCE, and LLVM
*only* define this macro when glibc is present. So theoretically LLVM
should play nice with musl. But an unfortunate truth is that,
_GNU_SOURCE is unconditionally defined by g++ and clang++ when
compiling any C++ code on Linux, rendering this protection useless.

I did some googling, and it turns out that libstdc++ heavily relies on
_GNU_SOURCE, so g++ always enable this macro for libstdc++ to function
correctly. I guess clang++ does the same thing just for compatibility
with g++.

I think the incompatibility between LLVM and musl is probably neither
side's fault, but the compiler's. Anyway, at least one of the three
should be fixed.

Thoughts?

Lei

Re: [gentoo-musl] [GSoC] _GNU_SOURCE in C++

[Felix Janda]

Lei Zhang wrote:
> Hi,
> 
> So far the toughest issue with building LLVM against musl is that,
> LLVM redefines a bunch of non-POSIX symbols, including fopen64,
> stat64, etc, inside a C++ namespace. The namespace prevents them from
> clashing with glibc's definitions; but unfortunately, musl defines
> these symbols as macros, and macros don't respect C++ namespace, thus
> the clash.
> 
> Here's how it looks in musl:
> 
> #if defined(_LARGEFILE64_SOURCE) || defined(_GNU_SOURCE)
> #define tmpfile64 tmpfile
> #define fopen64 fopen
> ...
> #endif
> 
> Actually these definitions are protected by _GNU_SOURCE, and LLVM
> *only* define this macro when glibc is present. So theoretically LLVM
> should play nice with musl. But an unfortunate truth is that,
> _GNU_SOURCE is unconditionally defined by g++ and clang++ when
> compiling any C++ code on Linux, rendering this protection useless.
> 
> I did some googling, and it turns out that libstdc++ heavily relies on
> _GNU_SOURCE, so g++ always enable this macro for libstdc++ to function
> correctly. I guess clang++ does the same thing just for compatibility
> with g++.
> 
> I think the incompatibility between LLVM and musl is probably neither
> side's fault, but the compiler's. Anyway, at least one of the three
> should be fixed.
> 
> Thoughts?

Take a look at the discussion at

http://www.openwall.com/lists/musl/2014/04/15/3

Felix

Re: [gentoo-musl] [GSoC] _GNU_SOURCE in C++

[Lei Zhang]

2016-06-14 16:10 GMT+08:00 Felix Janda <felix.janda@posteo.de>:
> Take a look at the discussion at
>
> http://www.openwall.com/lists/musl/2014/04/15/3

And some view from the LLVM side:

http://article.gmane.org/gmane.comp.compilers.clang.devel/46390

IMHO, the real problem is the abuse of _GNU_SOURCE by C++ compiler.
The decisions made by LLVM and musl both look reasonable to me.


Lei

Re: [gentoo-musl] [GSoC] _GNU_SOURCE in C++

[Lei Zhang]

2016-06-14 17:12 GMT+08:00 Luca Barbato <lu_zero@gentoo.org>:
> On 14/06/16 10:28, Lei Zhang wrote:
>> 2016-06-14 16:10 GMT+08:00 Felix Janda <felix.janda@posteo.de>:
>>> Take a look at the discussion at
>>>
>>> http://www.openwall.com/lists/musl/2014/04/15/3
>>
>> And some view from the LLVM side:
>>
>> http://article.gmane.org/gmane.comp.compilers.clang.devel/46390
>>
>> IMHO, the real problem is the abuse of _GNU_SOURCE by C++ compiler.
>> The decisions made by LLVM and musl both look reasonable to me.
>>
>
> -U_GNU_SOURCE would make you get through the problem for the time being?
>
> Given you aren't using libstdc++ in that case, assuming libcxx does not
> have the need probably might be the easiest workaround.

Unfortunately libc++ depends on this macro as well:

http://lists.llvm.org/pipermail/cfe-dev/2016-June/049447.html

I'm imagining some ugly hacks on LLVM now...


Lei

Re: [gentoo-musl] [GSoC] _GNU_SOURCE in C++

[Lei Zhang]

2016-06-14 17:28 GMT+08:00 Luca Barbato <lu_zero@gentoo.org>:
> On 14/06/16 11:17, Lei Zhang wrote:
>> 2016-06-14 17:12 GMT+08:00 Luca Barbato <lu_zero@gentoo.org>:
>>> On 14/06/16 10:28, Lei Zhang wrote:
>>>> 2016-06-14 16:10 GMT+08:00 Felix Janda <felix.janda@posteo.de>:
>>>>> Take a look at the discussion at
>>>>>
>>>>> http://www.openwall.com/lists/musl/2014/04/15/3
>>>>
>>>> And some view from the LLVM side:
>>>>
>>>> http://article.gmane.org/gmane.comp.compilers.clang.devel/46390
>>>>
>>>> IMHO, the real problem is the abuse of _GNU_SOURCE by C++ compiler.
>>>> The decisions made by LLVM and musl both look reasonable to me.
>>>>
>>>
>>> -U_GNU_SOURCE would make you get through the problem for the time being?
>>>
>>> Given you aren't using libstdc++ in that case, assuming libcxx does not
>>> have the need probably might be the easiest workaround.
>>
>> Unfortunately libc++ depends on this macro as well:
>>
>> http://lists.llvm.org/pipermail/cfe-dev/2016-June/049447.html
>>
>> I'm imagining some ugly hacks on LLVM now...
>>
>
> Please try to prepare a list and blog about this problem since possibly
> needs additional awareness.
>
> If the symbols needed aren't GNU-only there are other macros to expose
> them (and the llvm people just went the lazy route to get them).

I agree. It seems libstdc++ is also taking the lazy route, and someone
intended to fix it:

https://lwn.net/Articles/590504/

but I see no progress so far.


Lei

Re: [gentoo-musl] [GSoC] _GNU_SOURCE in C++

[Lei Zhang]

2016-06-14 19:18 GMT+08:00 Luca Barbato <lu_zero@gentoo.org>:
> On 14/06/16 11:35, Lei Zhang wrote:
>> I agree. It seems libstdc++ is also taking the lazy route, and someone
>> intended to fix it:
>>
>> https://lwn.net/Articles/590504/
>>
>> but I see no progress so far.
>>
>
> make -k and let's try to extract the list of missing symbols from it.

A quick test with libc++:

aligned_alloc
atoll
_Exit
fwide
fwprintf
fwscanf
isblank
iswblank
llabs
lldiv
lldiv_t
snprintf
strtof
strtold
strtoll
strtoull
swprintf
swscanf
vfscanf
vfwprintf
vfwscanf
vscanf
vsnprintf
vsscanf
vswprintf
vswscanf
wcstof
wcstold
wcstoll


Lei

Re: [gentoo-musl] [GSoC] _GNU_SOURCE in C++

[Lei Zhang]

This issue is probably not as simple as I thought. Supposing C++
header iostream depends on some non-standard functions from stdio.h,
it may look like:

// iostream
#define _BSD_SOURCE
#include <stdio.h>
#undef _BSD_SOURCE
...

But if a user writes some code like:

// user code
#include <stdio.h>
#include <iostream> // stdio.h already expanded before this point
...

then iostream never has the chance to get those non-standard symbols,
*unless* the compiler predefines _BSD_SOURCE for it. So simply
patching libc++/libstdc++ doesn't work; you have to carefully choose
which visibility macros to predefine in the compiler. And predefining
_GNU_SOURCE is just the safest bet.

I don't see any easy fix for this at the moment :(


Lei

2016-06-14 20:03 GMT+08:00 Luca Barbato <lu_zero@gentoo.org>:
> On 14/06/16 13:48, Lei Zhang wrote:
>> 2016-06-14 19:18 GMT+08:00 Luca Barbato <lu_zero@gentoo.org>:
>>> On 14/06/16 11:35, Lei Zhang wrote:
>>>> I agree. It seems libstdc++ is also taking the lazy route, and someone
>>>> intended to fix it:
>>>>
>>>> https://lwn.net/Articles/590504/
>>>>
>>>> but I see no progress so far.
>>>>
>>>
>>> make -k and let's try to extract the list of missing symbols from it.
>>
>> A quick test with libc++:
>>
>> aligned_alloc
>
> aligned_alloc(): _ISOC11_SOURCE
>
>> atoll
>
> atoll():
>            _BSD_SOURCE || _SVID_SOURCE || _XOPEN_SOURCE >= 600 ||
> _ISOC99_SOURCE || _POSIX_C_SOURCE >= 200112L;
>
>> _Exit
>
> _Exit():
>            _XOPEN_SOURCE >= 600 || _ISOC99_SOURCE || _POSIX_C_SOURCE >=
> 200112L;
>
>> fwide
>
> fwide():
>            _XOPEN_SOURCE >= 500 || _ISOC99_SOURCE ||
>            _POSIX_C_SOURCE >= 200112L;
>
> etc etc etc
>
> try to figure out which defines correctly enable those and send patches
> to be precise =)
>
> lu
>
>
>

Re: [gentoo-musl] [GSoC] _GNU_SOURCE in C++

[Lei Zhang]

[-- Attachment #1: Type: text/plain, Size: 1016 bytes --]

2016-06-15 17:36 GMT+08:00 Luca Barbato <lu_zero@gentoo.org>:
> On 15/06/16 03:45, Lei Zhang wrote:
>> This issue is probably not as simple as I thought. Supposing C++
>> header iostream depends on some non-standard functions from stdio.h,
>> it may look like:
>>
>> // iostream
>> #define _BSD_SOURCE
>> #include <stdio.h>
>> #undef _BSD_SOURCE
>> ...
>>
>> But if a user writes some code like:
>>
>
> Makes a mistake.
>
> You should not mix C headers and C++ headers like that =) There is a
> reason why you have <cstdio> and such =)
>
> The correct visibility macros can be extracted by reading the man pages
> for each of those functions.
>
> Please make the full list of them, once it is known we can discuss where
> to put them.

Attached is what I've got so far.

Some of them are not documented in man pages, so I checked their
declarations in musl's headers. Functions strtold_l, strtoll_l and
strtoull_l are not available in musl, but libc++ is smart enough to
not use them when linked against musl.


Lei

[-- Attachment #2: libcxx-nonstd-funcs.txt --]
[-- Type: text/plain, Size: 1019 bytes --]

# _ISOC11_SOURCE
aligned_alloc

# _BSD_SOURCE | _SVID_SOURCE | _ISOC99_SOURCE | _XOPEN_SOURCE >= 600 \
# | _POSIX_C_SOURCE >= 200112L
atoll           
strtoll        
strtoull      

# _ISOC99_SOURCE | _XOPEN_SOURCE >= 600 | _POSIX_C_SOURCE >= 200112L
_Exit     
isblank   
iswblank  
llabs     
lldiv     
strtof    
strtold   
vfscanf   
vscanf    
vsscanf   

# _ISOC99_SOURCE | _XOPEN_SOURCE >= 500 | _POSIX_C_SOURCE >= 200112L
fwide         
fwprintf      
fwscanf       
swprintf      
vfwprintf     
vswprintf     

# _BSD_SOURCE | _ISOC99_SOURCE | _XOPEN_SOURCE >= 500 \
# | _POSIX_C_SOURCE >= 200112L
snprintf      
vsnprintf    

# _XOPEN_SOURCE >= 700
uselocale

# _GNU_SOURCE | _BSD_SOURCE
vasprintf

# _BSD_SOURCE | _SVID_SOURCE | _XOPEN_SOURCE
isascii

# _POSIX_SOURCE | _POSIX_C_SOURCE | _XOPEN_SOURCE | _GNU_SOURCE | _BSD_SOURCE
iswalpha_l
iswblank_l
iswcntrl_l
iswdigit_l
iswlower_l
iswprint_l
iswpunct_l
iswspace_l
iswupper_l
iswxdigit_l

# _GNU_SOURCE (absent in musl)
strtold_l
strtoll_l
strtoull_l

Re: [gentoo-musl] [GSoC] _GNU_SOURCE in C++

[Lei Zhang]

2016-06-16 17:43 GMT+08:00 Luca Barbato <lu_zero@gentoo.org>:
> On 16/06/16 05:36, Lei Zhang wrote:
>> Attached is what I've got so far.
>>
>> Some of them are not documented in man pages, so I checked their
>> declarations in musl's headers. Functions strtold_l, strtoll_l and
>> strtoull_l are not available in musl, but libc++ is smart enough to
>> not use them when linked against musl.

I made a mistake. Actually strtold_l *is* available in musl, but
hidden by _GNU_SOURCE.

> beside vasprintf, the rest is covered by _XOPEN_SOURCE 700
>
> Might be worth trying to do -U_GNU_SOURCES -D_XOPEN_SOURCE=700 and see
> how it goes.

So far vasprintf and strtold_l are not covered. vasprintf can be
covered with -D_BSD_SOURCE. But _BSD_SOURCE is deprecated since glibc
2.20; I don't know if this has any impact on musl.

> How vasprintf is used?

It's used by <locale>. The code is kinda hard to read; I can't tell
exactly what it's doing...


Lei

Re: [gentoo-musl] [GSoC] _GNU_SOURCE in C++

[Lei Zhang]

2016-06-16 21:13 GMT+08:00 Luca Barbato <lu_zero@gentoo.org>:
> On 16/06/16 14:47, Lei Zhang wrote:
>> 2016-06-16 17:43 GMT+08:00 Luca Barbato <lu_zero@gentoo.org>:
>>> On 16/06/16 05:36, Lei Zhang wrote:
>>>> Attached is what I've got so far.
>>>>
>>>> Some of them are not documented in man pages, so I checked their
>>>> declarations in musl's headers. Functions strtold_l, strtoll_l and
>>>> strtoull_l are not available in musl, but libc++ is smart enough to
>>>> not use them when linked against musl.
>>
>> I made a mistake. Actually strtold_l *is* available in musl, but
>> hidden by _GNU_SOURCE.
>
> Not hidden by _BSD_SOURCE as well?

No, only by _GNU_SOURCE.

>>> beside vasprintf, the rest is covered by _XOPEN_SOURCE 700
>>>
>>> Might be worth trying to do -U_GNU_SOURCES -D_XOPEN_SOURCE=700 and see
>>> how it goes.
>>
>> So far vasprintf and strtold_l are not covered. vasprintf can be
>> covered with -D_BSD_SOURCE. But _BSD_SOURCE is deprecated since glibc
>> 2.20; I don't know if this has any impact on musl.
>
> It should not and -D_BSD_SOURCE should be fine for musl.

I just found that vasprintf is only protected by _GNU_SOURCE in glibc,
which means in this case _BSD_SOURCE only works for musl.

>>> How vasprintf is used?
>>
>> It's used by <locale>. The code is kinda hard to read; I can't tell
>> exactly what it's doing...
>
> same for the _l variants I guess.

So what now? So far strtold_l still needs _GNU_SOURCE in musl;
strtold_l, strtoll_l, strtoull_l and vasprintf all need _GNU_SOURCE in
glibc.


Lei

Re: [gentoo-musl] [GSoC] _GNU_SOURCE in C++

[Lei Zhang]

2016-06-16 21:42 GMT+08:00 Lei Zhang <zhanglei.april@gmail.com>:
> 2016-06-16 21:13 GMT+08:00 Luca Barbato <lu_zero@gentoo.org>:
>> On 16/06/16 14:47, Lei Zhang wrote:
>>> 2016-06-16 17:43 GMT+08:00 Luca Barbato <lu_zero@gentoo.org>:
>>>> On 16/06/16 05:36, Lei Zhang wrote:
>>>>> Attached is what I've got so far.
>>>>>
>>>>> Some of them are not documented in man pages, so I checked their
>>>>> declarations in musl's headers. Functions strtold_l, strtoll_l and
>>>>> strtoull_l are not available in musl, but libc++ is smart enough to
>>>>> not use them when linked against musl.
>>>
>>> I made a mistake. Actually strtold_l *is* available in musl, but
>>> hidden by _GNU_SOURCE.
>>
>> Not hidden by _BSD_SOURCE as well?
>
> No, only by _GNU_SOURCE.
>
>>>> beside vasprintf, the rest is covered by _XOPEN_SOURCE 700
>>>>
>>>> Might be worth trying to do -U_GNU_SOURCES -D_XOPEN_SOURCE=700 and see
>>>> how it goes.
>>>
>>> So far vasprintf and strtold_l are not covered. vasprintf can be
>>> covered with -D_BSD_SOURCE. But _BSD_SOURCE is deprecated since glibc
>>> 2.20; I don't know if this has any impact on musl.
>>
>> It should not and -D_BSD_SOURCE should be fine for musl.
>
> I just found that vasprintf is only protected by _GNU_SOURCE in glibc,
> which means in this case _BSD_SOURCE only works for musl.
>
>>>> How vasprintf is used?
>>>
>>> It's used by <locale>. The code is kinda hard to read; I can't tell
>>> exactly what it's doing...
>>
>> same for the _l variants I guess.
>
> So what now? So far strtold_l still needs _GNU_SOURCE in musl;
> strtold_l, strtoll_l, strtoull_l and vasprintf all need _GNU_SOURCE in
> glibc.

To sum it up:

* As for musl, "-U_GNU_SOURCES -U_GNU_SOURCES=700 -D_BSD_SOURCE"
covers every symbol needed by libc++ except for strtold_l.

* As for glibc, "-U_GNU_SOURCE -D_ISOC11_SOURCE -D_DEFAULT_SOURCE"
covers everything except for strtold_l, strtoll_l, strtoull_l and
vasprintf.


Lei

Re: [gentoo-musl] [GSoC] _GNU_SOURCE in C++

[Lei Zhang]

2016-06-17 19:12 GMT+08:00 Lei Zhang <zhanglei.april@gmail.com>:
> 2016-06-16 21:42 GMT+08:00 Lei Zhang <zhanglei.april@gmail.com>:
>> 2016-06-16 21:13 GMT+08:00 Luca Barbato <lu_zero@gentoo.org>:
>>> On 16/06/16 14:47, Lei Zhang wrote:
>>>> 2016-06-16 17:43 GMT+08:00 Luca Barbato <lu_zero@gentoo.org>:
>>>>> On 16/06/16 05:36, Lei Zhang wrote:
>>>>>> Attached is what I've got so far.
>>>>>>
>>>>>> Some of them are not documented in man pages, so I checked their
>>>>>> declarations in musl's headers. Functions strtold_l, strtoll_l and
>>>>>> strtoull_l are not available in musl, but libc++ is smart enough to
>>>>>> not use them when linked against musl.
>>>>
>>>> I made a mistake. Actually strtold_l *is* available in musl, but
>>>> hidden by _GNU_SOURCE.
>>>
>>> Not hidden by _BSD_SOURCE as well?
>>
>> No, only by _GNU_SOURCE.
>>
>>>>> beside vasprintf, the rest is covered by _XOPEN_SOURCE 700
>>>>>
>>>>> Might be worth trying to do -U_GNU_SOURCES -D_XOPEN_SOURCE=700 and see
>>>>> how it goes.
>>>>
>>>> So far vasprintf and strtold_l are not covered. vasprintf can be
>>>> covered with -D_BSD_SOURCE. But _BSD_SOURCE is deprecated since glibc
>>>> 2.20; I don't know if this has any impact on musl.
>>>
>>> It should not and -D_BSD_SOURCE should be fine for musl.
>>
>> I just found that vasprintf is only protected by _GNU_SOURCE in glibc,
>> which means in this case _BSD_SOURCE only works for musl.
>>
>>>>> How vasprintf is used?
>>>>
>>>> It's used by <locale>. The code is kinda hard to read; I can't tell
>>>> exactly what it's doing...
>>>
>>> same for the _l variants I guess.
>>
>> So what now? So far strtold_l still needs _GNU_SOURCE in musl;
>> strtold_l, strtoll_l, strtoull_l and vasprintf all need _GNU_SOURCE in
>> glibc.
>
> To sum it up:
>
> * As for musl, "-U_GNU_SOURCES -U_GNU_SOURCES=700 -D_BSD_SOURCE"
> covers every symbol needed by libc++ except for strtold_l.

Sorry, it should be "-U_GNU_SOURCE -D_XOPEN_SOURCE=700 -D_BSD_SOURCE"

Lei

Re: [gentoo-musl] [GSoC] _GNU_SOURCE in C++

[Lei Zhang]

2016-06-17 19:25 GMT+08:00 Luca Barbato <lu_zero@gentoo.org>:
> On 17/06/16 13:12, Lei Zhang wrote:
>> 2016-06-16 21:42 GMT+08:00 Lei Zhang <zhanglei.april@gmail.com>:
>>> 2016-06-16 21:13 GMT+08:00 Luca Barbato <lu_zero@gentoo.org>:
>>>> On 16/06/16 14:47, Lei Zhang wrote:
>>>>> 2016-06-16 17:43 GMT+08:00 Luca Barbato <lu_zero@gentoo.org>:
>>>>>> On 16/06/16 05:36, Lei Zhang wrote:
>>>>>>> Attached is what I've got so far.
>>>>>>>
>>>>>>> Some of them are not documented in man pages, so I checked their
>>>>>>> declarations in musl's headers. Functions strtold_l, strtoll_l and
>>>>>>> strtoull_l are not available in musl, but libc++ is smart enough to
>>>>>>> not use them when linked against musl.
>>>>>
>>>>> I made a mistake. Actually strtold_l *is* available in musl, but
>>>>> hidden by _GNU_SOURCE.
>>>>
>>>> Not hidden by _BSD_SOURCE as well?
>>>
>>> No, only by _GNU_SOURCE.
>>>
>>>>>> beside vasprintf, the rest is covered by _XOPEN_SOURCE 700
>>>>>>
>>>>>> Might be worth trying to do -U_GNU_SOURCES -D_XOPEN_SOURCE=700 and see
>>>>>> how it goes.
>>>>>
>>>>> So far vasprintf and strtold_l are not covered. vasprintf can be
>>>>> covered with -D_BSD_SOURCE. But _BSD_SOURCE is deprecated since glibc
>>>>> 2.20; I don't know if this has any impact on musl.
>>>>
>>>> It should not and -D_BSD_SOURCE should be fine for musl.
>>>
>>> I just found that vasprintf is only protected by _GNU_SOURCE in glibc,
>>> which means in this case _BSD_SOURCE only works for musl.
>>>
>>>>>> How vasprintf is used?
>>>>>
>>>>> It's used by <locale>. The code is kinda hard to read; I can't tell
>>>>> exactly what it's doing...
>>>>
>>>> same for the _l variants I guess.
>>>
>>> So what now? So far strtold_l still needs _GNU_SOURCE in musl;
>>> strtold_l, strtoll_l, strtoull_l and vasprintf all need _GNU_SOURCE in
>>> glibc.
>>
>> To sum it up:
>>
>> * As for musl, "-U_GNU_SOURCES -U_GNU_SOURCES=700 -D_BSD_SOURCE"
>> covers every symbol needed by libc++ except for strtold_l.
>>
>> * As for glibc, "-U_GNU_SOURCE -D_ISOC11_SOURCE -D_DEFAULT_SOURCE"
>> covers everything except for strtold_l, strtoll_l, strtoull_l and
>> vasprintf.
>
>
> Sounds almost good, now some configure/cmake patches to put that in and
> some #define _GNU_SOURCES /#undef in selected places might get it
> working completely...

After putting about eight "#define _GNU_SOURCE"s in various *.cpp
files, it seems to work now :)

I'll see if I can reduce the number of "#define"s needed.

Lei

Re: [gentoo-musl] [GSoC] _GNU_SOURCE in C++

[Lei Zhang]

2016-06-17 20:44 GMT+08:00 Lei Zhang <zhanglei.april@gmail.com>:
> After putting about eight "#define _GNU_SOURCE"s in various *.cpp
> files, it seems to work now :)

I was wrong; it doesn't work... I still can't figure out a good solution :(

Take glibc's strtoll_l for example: it's hidden by _GNU_SOURCE in
stdlib.h and used exclusively by libc++'s header <locale>; but
defining _GNU_SOURCE in <locale> could be too late, since stdlib.h
might be included prior to <locale>.

Though glibc's stdlib.h is only explicitly included in <cstdlib>,
defining _GNU_SOURCE in <cstdlib> doesn't work either. It turns out
strtoll_l (and others) is not directly protected by _GNU_SOURCE, but
by __USE_GNU, which is in turn defined in features.h according to
_GNU_SOURCE. That means "#define _GNU_SOURCE" must appear before the
inclusion of features.h, and it's not very clear where "#undef
_GNU_SOURCE" should be put.

Any suggestions?


Lei

Re: [gentoo-musl] [GSoC] _GNU_SOURCE in C++

[Lei Zhang]

2016-06-21 12:54 GMT+08:00 Lei Zhang <zhanglei.april@gmail.com>:
> 2016-06-17 20:44 GMT+08:00 Lei Zhang <zhanglei.april@gmail.com>:
>> After putting about eight "#define _GNU_SOURCE"s in various *.cpp
>> files, it seems to work now :)
>
> I was wrong; it doesn't work... I still can't figure out a good solution :(
>
> Take glibc's strtoll_l for example: it's hidden by _GNU_SOURCE in
> stdlib.h and used exclusively by libc++'s header <locale>; but
> defining _GNU_SOURCE in <locale> could be too late, since stdlib.h
> might be included prior to <locale>.
>
> Though glibc's stdlib.h is only explicitly included in <cstdlib>,
> defining _GNU_SOURCE in <cstdlib> doesn't work either. It turns out
> strtoll_l (and others) is not directly protected by _GNU_SOURCE, but
> by __USE_GNU, which is in turn defined in features.h according to
> _GNU_SOURCE. That means "#define _GNU_SOURCE" must appear before the
> inclusion of features.h, and it's not very clear where "#undef
> _GNU_SOURCE" should be put.
>
> Any suggestions?

The only feasible solution I can think of is to wrap every inclusion of C header

Re: [gentoo-musl] [GSoC] _GNU_SOURCE in C++

[Lei Zhang]

2016-06-21 12:54 GMT+08:00 Lei Zhang <zhanglei.april@gmail.com>:
> 2016-06-17 20:44 GMT+08:00 Lei Zhang <zhanglei.april@gmail.com>:
>> After putting about eight "#define _GNU_SOURCE"s in various *.cpp
>> files, it seems to work now :)
>
> I was wrong; it doesn't work... I still can't figure out a good solution :(
>
> Take glibc's strtoll_l for example: it's hidden by _GNU_SOURCE in
> stdlib.h and used exclusively by libc++'s header <locale>; but
> defining _GNU_SOURCE in <locale> could be too late, since stdlib.h
> might be included prior to <locale>.
>
> Though glibc's stdlib.h is only explicitly included in <cstdlib>,
> defining _GNU_SOURCE in <cstdlib> doesn't work either. It turns out
> strtoll_l (and others) is not directly protected by _GNU_SOURCE, but
> by __USE_GNU, which is in turn defined in features.h according to
> _GNU_SOURCE. That means "#define _GNU_SOURCE" must appear before the
> inclusion of features.h, and it's not very clear where "#undef
> _GNU_SOURCE" should be put.
>
> Any suggestions?

So far the only feasible solution I can think of is to wrap every
inclusion of C headers in libc++ with a pair of "#define _GNU_SOURCE"
and "#undef _GNU_SOURCE", thus making sure _GNU_SOURCE is defined
before any implicitly inclusion of features.h. It is cumbersome, but
works. I haven't really figured out if this solution is 100%
error-proof.

P.S.: sorry for my previous mis-sent message.


Lei

Re: [gentoo-musl] [GSoC] _GNU_SOURCE in C++

[Lei Zhang]

2016-06-22 21:54 GMT+08:00 Lei Zhang <zhanglei.april@gmail.com>:
> 2016-06-21 12:54 GMT+08:00 Lei Zhang <zhanglei.april@gmail.com>:
>> 2016-06-17 20:44 GMT+08:00 Lei Zhang <zhanglei.april@gmail.com>:
>>> After putting about eight "#define _GNU_SOURCE"s in various *.cpp
>>> files, it seems to work now :)
>>
>> I was wrong; it doesn't work... I still can't figure out a good solution :(
>>
>> Take glibc's strtoll_l for example: it's hidden by _GNU_SOURCE in
>> stdlib.h and used exclusively by libc++'s header <locale>; but
>> defining _GNU_SOURCE in <locale> could be too late, since stdlib.h
>> might be included prior to <locale>.
>>
>> Though glibc's stdlib.h is only explicitly included in <cstdlib>,
>> defining _GNU_SOURCE in <cstdlib> doesn't work either. It turns out
>> strtoll_l (and others) is not directly protected by _GNU_SOURCE, but
>> by __USE_GNU, which is in turn defined in features.h according to
>> _GNU_SOURCE. That means "#define _GNU_SOURCE" must appear before the
>> inclusion of features.h, and it's not very clear where "#undef
>> _GNU_SOURCE" should be put.
>>
>> Any suggestions?
>
> So far the only feasible solution I can think of is to wrap every
> inclusion of C headers in libc++ with a pair of "#define _GNU_SOURCE"
> and "#undef _GNU_SOURCE", thus making sure _GNU_SOURCE is defined
> before any implicitly inclusion of features.h. It is cumbersome, but
> works. I haven't really figured out if this solution is 100%
> error-proof.

As an after-thought, I don't this justifies as a *real* solution, as
it still exposes nearly all the hidden symbols to user program; it
just enables us to compile LLVM against musl...


Lei

Re: [gentoo-musl] [GSoC] _GNU_SOURCE in C++

[Felix Janda]

Lei Zhang wrote:
> 2016-06-14 16:10 GMT+08:00 Felix Janda <felix.janda@posteo.de>:
> > Take a look at the discussion at
> >
> > http://www.openwall.com/lists/musl/2014/04/15/3
> 
> And some view from the LLVM side:
> 
> http://article.gmane.org/gmane.comp.compilers.clang.devel/46390

Thanks for the pointer.

http://www.openwall.com/lists/musl/2014/04/15/5

gives the reasons why musl is using macros and cannot easily use
symbol aliases (as suggested in the mail you've linked to).

Several questions on the thread on the llvm list have been left
unanswered. I think it would make sense to reopen the discussion.

> IMHO, the real problem is the abuse of _GNU_SOURCE by C++ compiler.
> The decisions made by LLVM and musl both look reasonable to me.

That's a real problem, but I guess also a difficult one to fix.

Felix

Re: [gentoo-musl] [GSoC] _GNU_SOURCE in C++

[Lei Zhang]

2016-06-15 11:38 GMT+08:00 Aric Belsito <lluixhi@gmail.com>:
> Hi Lei,
>
> Out of curiousity, have you seen the patches that voidlinux is using for
> LLVM+Clang or the ones that I'm hosting for gentoo at
> https://github.com/lluixhi/musl-extras/tree/master/sys-devel/llvm ?
> Just curious.

Not before you mention it. I just took a look, and they seem to
largely overlap with what I'm trying to do. So far I've seen a bunch
of patches by different people trying to do the same thing... I really
hope I can finally push them upstream this summer :)


Lei