[gentoo-user] memset_s

[gentoo-user] memset_s

[Jorge Almeida]

I'm trying to use memset_s() but the system (glibc?) doesn't know
about it. I also tried to compile against musl, same result.

There's precious little info about memset_s in the net. Does it exist
at all? No man page.

(https://www.cs.helsinki.fi/group/boi2016/doc/cppreference/reference/en.cppreference.com/w/c/string/byte/memset.html)

What I tried:

#include <unistd.h>
#include <stdlib.h>
#include <stdint.h>
#define __STDC_WANT_LIB_EXT1__  1
#include <string.h>
#include <stdio.h>

int main(int argc, char** argv){
#ifndef __STDC_LIB_EXT1__
printf("CRAP\n");
#else
printf("COOL\n");
#endif
}

Compiled with -std=c11

You can guess what the output is.

Someone using it?

Jorge Almeida

Re: [gentoo-user] memset_s

[Marc Joliet]

[-- Attachment #1: Type: text/plain, Size: 1341 bytes --]

Am Freitag, 10. November 2017, 10:54:53 CET schrieb Jorge Almeida:
> I'm trying to use memset_s() but the system (glibc?) doesn't know
> about it. I also tried to compile against musl, same result.
> 
> There's precious little info about memset_s in the net. Does it exist
> at all? No man page.
> 
> (https://www.cs.helsinki.fi/group/boi2016/doc/cppreference/reference/en.cppr
> eference.com/w/c/string/byte/memset.html)
> 
> What I tried:
> 
> #include <unistd.h>
> #include <stdlib.h>
> #include <stdint.h>
> #define __STDC_WANT_LIB_EXT1__  1
> #include <string.h>
> #include <stdio.h>
> 
> int main(int argc, char** argv){
> #ifndef __STDC_LIB_EXT1__
> printf("CRAP\n");
> #else
> printf("COOL\n");
> #endif
> }
> 
> Compiled with -std=c11
> 
> You can guess what the output is.
> 
> Someone using it?
> 
> Jorge Almeida

It seems as though it is simply not implemented, I found a variety of links  
that all support this:

https://stackoverflow.com/a/40162721

https://stackoverflow.com/questions/38322363/when-will-the-safe-string-functions-of-c11-be-part-of-glibc

https://gcc.gnu.org/wiki/C11Status (which states that Annex K is not 
implemented)

http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1967.htm

HTH
-- 
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] memset_s

[Jorge Almeida]

On Fri, Nov 10, 2017 at 10:52 AM, Marc Joliet <marcec@gmx.de> wrote:
> Am Freitag, 10. November 2017, 10:54:53 CET schrieb Jorge Almeida:
>> I'm trying to use memset_s() but the system (glibc?) doesn't know
>> about it. I also tried to compile against musl, same result.
>>


> It seems as though it is simply not implemented, I found a variety of links
> that all support this:
>
> https://stackoverflow.com/a/40162721
>
> https://stackoverflow.com/questions/38322363/when-will-the-safe-string-functions-of-c11-be-part-of-glibc
>
> https://gcc.gnu.org/wiki/C11Status (which states that Annex K is not
> implemented)
>
> http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1967.htm
>
OK, thanks. The last link even suggests that Annex K should be
deprecated. I suppose this people don't care about security at all.

Of course, what would really solve the optimize-into-oblivion problem
is a pragma that when invoked on a particular block of code (maybe
only a function definition) would tell the compiler to do what the
programmer says rather than viewing a function as a kind of black box.

Jorge Almeida

Re: [gentoo-user] memset_s

[R0b0t1]

Hello,

On Fri, Nov 10, 2017 at 5:34 AM, Jorge Almeida <jjalmeida@gmail.com> wrote:
> On Fri, Nov 10, 2017 at 10:52 AM, Marc Joliet <marcec@gmx.de> wrote:
>> Am Freitag, 10. November 2017, 10:54:53 CET schrieb Jorge Almeida:
>>> I'm trying to use memset_s() but the system (glibc?) doesn't know
>>> about it. I also tried to compile against musl, same result.
>>>
>
>
>> It seems as though it is simply not implemented, I found a variety of links
>> that all support this:
>>
>> https://stackoverflow.com/a/40162721
>>
>> https://stackoverflow.com/questions/38322363/when-will-the-safe-string-functions-of-c11-be-part-of-glibc
>>
>> https://gcc.gnu.org/wiki/C11Status (which states that Annex K is not
>> implemented)
>>
>> http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1967.htm
>>
> OK, thanks. The last link even suggests that Annex K should be
> deprecated. I suppose this people don't care about security at all.
>

I'm having trouble finding the article again, but these functions look
very similar to Microsoft's extensions to the C standard. There is a
good case to be made that they are counterproductive.

> Of course, what would really solve the optimize-into-oblivion problem
> is a pragma that when invoked on a particular block of code (maybe
> only a function definition) would tell the compiler to do what the
> programmer says rather than viewing a function as a kind of black box.
>

This would probably be useful. It may be wise to reimplement important
functionality.

Cheers,
     R0b0t1

Re: [gentoo-user] memset_s

[Alexander Kapshuk]

[-- Attachment #1: Type: text/plain, Size: 1795 bytes --]

On Fri, Nov 10, 2017 at 6:25 PM, R0b0t1 <r030t1@gmail.com> wrote:

> Hello,
>
> On Fri, Nov 10, 2017 at 5:34 AM, Jorge Almeida <jjalmeida@gmail.com>
> wrote:
> > On Fri, Nov 10, 2017 at 10:52 AM, Marc Joliet <marcec@gmx.de> wrote:
> >> Am Freitag, 10. November 2017, 10:54:53 CET schrieb Jorge Almeida:
> >>> I'm trying to use memset_s() but the system (glibc?) doesn't know
> >>> about it. I also tried to compile against musl, same result.
> >>>
> >
> >
> >> It seems as though it is simply not implemented, I found a variety of
> links
> >> that all support this:
> >>
> >> https://stackoverflow.com/a/40162721
> >>
> >> https://stackoverflow.com/questions/38322363/when-will-
> the-safe-string-functions-of-c11-be-part-of-glibc
> >>
> >> https://gcc.gnu.org/wiki/C11Status (which states that Annex K is not
> >> implemented)
> >>
> >> http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1967.htm
> >>
> > OK, thanks. The last link even suggests that Annex K should be
> > deprecated. I suppose this people don't care about security at all.
> >
>
> I'm having trouble finding the article again, but these functions look
> very similar to Microsoft's extensions to the C standard. There is a
> good case to be made that they are counterproductive.
>
> > Of course, what would really solve the optimize-into-oblivion problem
> > is a pragma that when invoked on a particular block of code (maybe
> > only a function definition) would tell the compiler to do what the
> > programmer says rather than viewing a function as a kind of black box.
> >
>
> This would probably be useful. It may be wise to reimplement important
> functionality.
>
> Cheers,
>      R0b0t1
>
>
It's also a part of NetBSD's libc used by Apple.
https://opensource.apple.com/source/Libc/Libc-1044.1.2/string/NetBSD/memset_s.c

[-- Attachment #2: Type: text/html, Size: 3185 bytes --]

Re: [gentoo-user] memset_s

[Jorge Almeida]

On Fri, Nov 10, 2017 at 4:25 PM, R0b0t1 <r030t1@gmail.com> wrote:
> Hello,
>

>
> I'm having trouble finding the article again, but these functions look
> very similar to Microsoft's extensions to the C standard. There is a
> good case to be made that they are counterproductive.

Yes, it looks like it. No wonder, if it's MS inspired. But what I care
about is the fact that it's not optimized away, not the boundaries
checking stuff. It's hard to believe that it is practically impossible
to clean up a buffer, unless one is willing to forego all
optimizations:

http://www.daemonology.net/blog/2014-09-04-how-to-zero-a-buffer.html


>> Of course, what would really solve the optimize-into-oblivion problem
>> is a pragma that when invoked on a particular block of code (maybe
>> only a function definition) would tell the compiler to do what the
>> programmer says rather than viewing a function as a kind of black box.
>>
>
> This would probably be useful. It may be wise to reimplement important
> functionality.
>
No idea how difficult it would be to implement, of course. There might
even exist a C keyword for that. After all, the C standard states the
"as-if" rule, it might as well establish such an exception.

Cheers

Jorge

Re: [gentoo-user] memset_s

[R0b0t1]

On Fri, Nov 10, 2017 at 2:09 PM, Jorge Almeida <jjalmeida@gmail.com> wrote:
> On Fri, Nov 10, 2017 at 4:25 PM, R0b0t1 <r030t1@gmail.com> wrote:
>> Hello,
>>
>
>>
>> I'm having trouble finding the article again, but these functions look
>> very similar to Microsoft's extensions to the C standard. There is a
>> good case to be made that they are counterproductive.
>
> Yes, it looks like it. No wonder, if it's MS inspired. But what I care
> about is the fact that it's not optimized away, not the boundaries
> checking stuff. It's hard to believe that it is practically impossible
> to clean up a buffer, unless one is willing to forego all
> optimizations:
>
> http://www.daemonology.net/blog/2014-09-04-how-to-zero-a-buffer.html
>

I really think there is a deeper issue here then, which is that the
compiler takes a lot of liberties when translating a program
description into machine code. There have been suggestions made that
this makes very nearly all compilers unsuitable for high reliability
purposes. Cryptographic or user security code is likely a candidate
for the label "high reliability."

To further explain why the additions are counterproductive: the
programmer still has to remember to use them. It is just as likely
that the programmer will forget to use memset_s properly as any of the
other functions in string.h (possibly by forgetting to sanitize input
i.e. the memory segment boundaries).

>
>>> Of course, what would really solve the optimize-into-oblivion problem
>>> is a pragma that when invoked on a particular block of code (maybe
>>> only a function definition) would tell the compiler to do what the
>>> programmer says rather than viewing a function as a kind of black box.
>>>
>>
>> This would probably be useful. It may be wise to reimplement important
>> functionality.
>>
> No idea how difficult it would be to implement, of course. There might
> even exist a C keyword for that. After all, the C standard states the
> "as-if" rule, it might as well establish such an exception.
>

Sorry, I misrepresented what I meant. I meant to suggest
reimplementing, apart from a standard library, any critical code. This
is generally recommended against but unless there is a hand-tuned
version that has been guaranteed to work around quirks in your
compiler, you are now the person who has to write and maintain that
hand-tuned version.

If you don't mind I might post this concern to the GCC mailing list,
or you can take it up if you want.

Cheers,
     R0b0t1

Re: [gentoo-user] memset_s

[Jorge Almeida]

On Fri, Nov 10, 2017 at 11:19 PM, R0b0t1 <r030t1@gmail.com> wrote:
> On Fri, Nov 10, 2017 at 2:09 PM, Jorge Almeida <jjalmeida@gmail.com> wrote:
>> On Fri, Nov 10, 2017 at 4:25 PM, R0b0t1 <r030t1@gmail.com> wrote:
>
>>
>> http://www.daemonology.net/blog/2014-09-04-how-to-zero-a-buffer.html
>>
>
> I really think there is a deeper issue here then, which is that the
> compiler takes a lot of liberties when translating a program
> description into machine code. There have been suggestions made that
> this makes very nearly all compilers unsuitable for high reliability
> purposes. Cryptographic or user security code is likely a candidate
> for the label "high reliability."

Yes, the html page above has a link to a 2nd part, where (if I
understood correctly) it is concluded that currently there is no real
solution: even if the compiler does what it is told to, it may copy
data around, and the programmer has no control whatsoever over the
fate of such data.

>
> To further explain why the additions are counterproductive: the
> programmer still has to remember to use them. It is just as likely
> that the programmer will forget to use memset_s properly as any of the
> other functions in string.h (possibly by forgetting to sanitize input
> i.e. the memory segment boundaries).
>
Well, most programmers probably won't care about this stuff anyway,
and people who deal with cryptography tend to be more cautious than
average. But I'm not really making a case for safe versions of known
functions. After all, the usual functions do fine for most
applications. memset() would be enough to clear RAM with sensitive
data if we had a pragma (or equivalent) to convince the compiler to
not ignore it (I mean a pragma to invoke on a particular function
definition when the programmer  feels that a black box behaviour is
undesirable). Of course, solving the problem of the compiler copying
stuff around might be harder nut to crack.
>

>
> If you don't mind I might post this concern to the GCC mailing list,
> or you can take it up if you want.

Please do. I'm strictly amateur league, and you'll do a better job.
>
Cheers

Jorge

Re: [gentoo-user] memset_s

[Mart Raudsepp]

On L, 2017-11-11 at 00:10 +0000, Jorge Almeida wrote:
> Well, most programmers probably won't care about this stuff anyway,
> and people who deal with cryptography tend to be more cautious than
> average. But I'm not really making a case for safe versions of known
> functions. After all, the usual functions do fine for most
> applications. memset() would be enough to clear RAM with sensitive
> data if we had a pragma (or equivalent) to convince the compiler to
> not ignore it (I mean a pragma to invoke on a particular function
> definition when the programmer  feels that a black box behaviour is
> undesirable). Of course, solving the problem of the compiler copying
> stuff around might be harder nut to crack.

Sounds like you want explicit_bzero from libbsd?

Re: [gentoo-user] memset_s

[Jorge Almeida]

On Sun, Nov 12, 2017 at 7:03 PM, Mart Raudsepp <leio@gentoo.org> wrote:
> On L, 2017-11-11 at 00:10 +0000, Jorge Almeida wrote:
>> Well, most programmers probably won't care about this stuff anyway,
>> and people who deal with cryptography tend to be more cautious than
>> average. But I'm not really making a case for safe versions of known
>> functions. After all, the usual functions do fine for most
>> applications. memset() would be enough to clear RAM with sensitive
>> data if we had a pragma (or equivalent) to convince the compiler to
>> not ignore it (I mean a pragma to invoke on a particular function
>> definition when the programmer  feels that a black box behaviour is
>> undesirable). Of course, solving the problem of the compiler copying
>> stuff around might be harder nut to crack.
>
> Sounds like you want explicit_bzero from libbsd?
>
According to their man page, yes. I'll have to [try to] check the
source. I wonder how they do it? Even the volatile modifier doesn't
solve the problem, according to the link in previous post.

Anyway, there are probably circumstances where not optimizing a
particular function would be useful. Zeroing sensitive data is just
the first that comes to mind. (For example, what if you prefer
overwriting your sensitive data with random bytes?)

Jorge

[gentoo-user] Re: memset_s

[Nikos Chantziaras]

On 13/11/17 09:17, Jorge Almeida wrote:
> On Sun, Nov 12, 2017 at 7:03 PM, Mart Raudsepp <leio@gentoo.org> wrote:
>> On L, 2017-11-11 at 00:10 +0000, Jorge Almeida wrote:
>>> Well, most programmers probably won't care about this stuff anyway,
>>> and people who deal with cryptography tend to be more cautious than
>>> average. But I'm not really making a case for safe versions of known
>>> functions. After all, the usual functions do fine for most
>>> applications. memset() would be enough to clear RAM with sensitive
>>> data if we had a pragma (or equivalent) to convince the compiler to
>>> not ignore it (I mean a pragma to invoke on a particular function
>>> definition when the programmer  feels that a black box behaviour is
>>> undesirable). Of course, solving the problem of the compiler copying
>>> stuff around might be harder nut to crack.
>>
>> Sounds like you want explicit_bzero from libbsd?
>>
> According to their man page, yes. I'll have to [try to] check the
> source. I wonder how they do it? Even the volatile modifier doesn't
> solve the problem, according to the link in previous post.

explicit_bzero() is available in glibc. It's in <string.h>.

Re: [gentoo-user] Re: memset_s

[Jorge Almeida]

On Mon, Nov 13, 2017 at 2:44 AM, Nikos Chantziaras <realnc@gmail.com> wrote:

>>>
>>> Sounds like you want explicit_bzero from libbsd?
>>>
>> According to their man page, yes. I'll have to [try to] check the
>> source. I wonder how they do it? Even the volatile modifier doesn't
>> solve the problem, according to the link in previous post.
>
>
> explicit_bzero() is available in glibc. It's in <string.h>.
>
>
OK, thanks. dietlibc also has it, musl doesn't.

Jorge Almeida

Re: [gentoo-user] Re: memset_s

[Mart Raudsepp]

On E, 2017-11-13 at 12:44 +0200, Nikos Chantziaras wrote:
> On 13/11/17 09:17, Jorge Almeida wrote:
> > 
> > On Sun, Nov 12, 2017 at 7:03 PM, Mart Raudsepp <leio@gentoo.org>
> > wrote:
> > > 
> > > On L, 2017-11-11 at 00:10 +0000, Jorge Almeida wrote:
> > > > 
> > > > Well, most programmers probably won't care about this stuff
> > > > anyway,
> > > > and people who deal with cryptography tend to be more cautious
> > > > than
> > > > average. But I'm not really making a case for safe versions of
> > > > known
> > > > functions. After all, the usual functions do fine for most
> > > > applications. memset() would be enough to clear RAM with
> > > > sensitive
> > > > data if we had a pragma (or equivalent) to convince the
> > > > compiler to
> > > > not ignore it (I mean a pragma to invoke on a particular
> > > > function
> > > > definition when the programmer  feels that a black box
> > > > behaviour is
> > > > undesirable). Of course, solving the problem of the compiler
> > > > copying
> > > > stuff around might be harder nut to crack.
> > > Sounds like you want explicit_bzero from libbsd?
> > > 
> > According to their man page, yes. I'll have to [try to] check the
> > source. I wonder how they do it? Even the volatile modifier doesn't
> > solve the problem, according to the link in previous post.
> explicit_bzero() is available in glibc. It's in <string.h>.

Interesting. Some Xorg stuff is using libbsd explicitly, but probably
since before glibc gained this. This is new since glibc-2.25.

How they do it you can find out from the source code. In libbsd case, I
saw a weak linked (do-nothing) function call after memset, so the
compiler can't know if that weakly linked function isn't getting
replaced with something that might do something with the zeroed memory,
and thus can't optimize it out. Though I looked at an older libbsd.

[gentoo-user] Re: memset_s

[Nikos Chantziaras]

On 13/11/17 13:38, Mart Raudsepp wrote:
> On E, 2017-11-13 at 12:44 +0200, Nikos Chantziaras wrote:
>> explicit_bzero() is available in glibc. It's in <string.h>.
> 
> Interesting. Some Xorg stuff is using libbsd explicitly, but probably
> since before glibc gained this. This is new since glibc-2.25.

Oops, you're right. Didn't notice this. 2.25 is very recent. So from a 
portability point of view, it might be a good idea to stick to libbsd, 
or at least have a configure test for it.

Re: [gentoo-user] memset_s

[Jorge Almeida]

On Fri, Nov 10, 2017 at 12:09 PM, Jorge Almeida <jjalmeida@gmail.com> wrote:

> http://www.daemonology.net/blog/2014-09-04-how-to-zero-a-buffer.html
>
>
>>> Of course, what would really solve the optimize-into-oblivion problem
>>> is a pragma that when invoked on a particular block of code (maybe
>>> only a function definition) would tell the compiler to do what the
>>> programmer says rather than viewing a function as a kind of black box.
>>>
>>

It seems a solution exists with gcc:

https://stackoverflow.com/questions/2219829/how-to-prevent-gcc-optimizing-some-statements-in-c

The last reply:

void __attribute__((optimize("O0"))) foo(unsigned char data) {
    // unmodifiable compiler code
}

Any comments, anyone? Yes, it's gcc, but IMO this should be in the
language itself. Am I right to assume this is a poorly known feature
of gcc?
It allows, for example,  to replace sensitive data by random bytes,
existing system callls like memset() or getrandom() can be used as
they are, no reimplementation needed.

Jorge Almeida

Re: [gentoo-user] memset_s

[R0b0t1]

On Tue, Nov 14, 2017 at 11:36 AM, Jorge Almeida <jjalmeida@gmail.com> wrote:
> On Fri, Nov 10, 2017 at 12:09 PM, Jorge Almeida <jjalmeida@gmail.com> wrote:
>
>> http://www.daemonology.net/blog/2014-09-04-how-to-zero-a-buffer.html
>>
>>
>>>> Of course, what would really solve the optimize-into-oblivion problem
>>>> is a pragma that when invoked on a particular block of code (maybe
>>>> only a function definition) would tell the compiler to do what the
>>>> programmer says rather than viewing a function as a kind of black box.
>>>>
>>>
>
> It seems a solution exists with gcc:
>
> https://stackoverflow.com/questions/2219829/how-to-prevent-gcc-optimizing-some-statements-in-c
>
> The last reply:
>
> void __attribute__((optimize("O0"))) foo(unsigned char data) {
>     // unmodifiable compiler code
> }
>
> Any comments, anyone? Yes, it's gcc, but IMO this should be in the
> language itself. Am I right to assume this is a poorly known feature
> of gcc?
> It allows, for example,  to replace sensitive data by random bytes,
> existing system callls like memset() or getrandom() can be used as
> they are, no reimplementation needed.
>

Very interesting. I imagine the opinion of the standards committee
would be that the variability in code generation precludes a standard
interface to optimization controls. This might seem unusual, but
languages with a very controlling standard (like Java or C#) are a new
concept.

What I am wondering about is if C code which uses
__attribute__((optimize(...))) is against Gentoo package standards and
would have to be removed from the Portage tree.

Cheers,
     R0b0t1

Re: [gentoo-user] memset_s

[Jorge Almeida]

On Tue, Nov 14, 2017 at 8:42 PM, R0b0t1 <r030t1@gmail.com> wrote:
> On Tue, Nov 14, 2017 at 11:36 AM, Jorge Almeida <jjalmeida@gmail.com> wrote:
>> On Fri, Nov 10, 2017 at 12:09 PM, Jorge Almeida <jjalmeida@gmail.com> wrote:
>>
>>> http://www.daemonology.net/blog/2014-09-04-how-to-zero-a-buffer.html
>>>
>>>
>>>>> Of course, what would really solve the optimize-into-oblivion problem
>>>>> is a pragma that when invoked on a particular block of code (maybe
>>>>> only a function definition) would tell the compiler to do what the
>>>>> programmer says rather than viewing a function as a kind of black box.
>>>>>
>>>>
>>
>> It seems a solution exists with gcc:
>>
>> https://stackoverflow.com/questions/2219829/how-to-prevent-gcc-optimizing-some-statements-in-c
>>
>> The last reply:
>>
>> void __attribute__((optimize("O0"))) foo(unsigned char data) {
>>     // unmodifiable compiler code
>> }
>>
>> Any comments, anyone? Yes, it's gcc, but IMO this should be in the
>> language itself. Am I right to assume this is a poorly known feature
>> of gcc?
>> It allows, for example,  to replace sensitive data by random bytes,
>> existing system callls like memset() or getrandom() can be used as
>> they are, no reimplementation needed.
>>
>
> Very interesting. I imagine the opinion of the standards committee
> would be that the variability in code generation precludes a standard
> interface to optimization controls. This might seem unusual, but
> languages with a very controlling standard (like Java or C#) are a new
> concept.

Well, we'll have to stick to gcc (or other compilers with the same
feature). OTOH, boldness doesn't seem to be the comittee's most
salient feature.

>
> What I am wondering about is if C code which uses
> __attribute__((optimize(...))) is against Gentoo package standards and
> would have to be removed from the Portage tree.
>


You can set your optimization preferences in make.conf, and still an
ebuild will override them if deemed unsafe. What would be the
difference?

Cheers

Jorge

[gentoo-user] Re: memset_s

[Grant Edwards]

On 2017-11-15, R0b0t1 <r030t1@gmail.com> wrote:

> What I am wondering about is if C code which uses
> __attribute__((optimize(...))) is against Gentoo package standards and
> would have to be removed from the Portage tree.

Huh?

Gentoo enforces standards for the source code of packages?

"They" review the source code for the Linux kernel, Gnome, KDE, Qt,
Chrome, Firefox, GCC, and 24670 thousand other packages and make sure
they all follow Gentoo coding standards?

-- 
Grant Edwards               grant.b.edwards        Yow! Sometime in 1993
                                  at               NANCY SINATRA will lead a
                              gmail.com            BLOODLESS COUP on GUAM!!

Re: [gentoo-user] Re: memset_s

[R0b0t1]

On Wed, Nov 15, 2017 at 1:22 AM, Jorge Almeida <jjalmeida@gmail.com> wrote:
> On Tue, Nov 14, 2017 at 8:42 PM, R0b0t1 <r030t1@gmail.com> wrote:
>> What I am wondering about is if C code which uses
>> __attribute__((optimize(...))) is against Gentoo package standards and
>> would have to be removed from the Portage tree.
>>
>
>
> You can set your optimization preferences in make.conf, and still an
> ebuild will override them if deemed unsafe. What would be the
> difference?
>

Ebuilds are not supposed to do this, so if you file a bug report
citing that ebuild changes will be made (eventually?) to work around
it.


On Wed, Nov 15, 2017 at 9:28 AM, Grant Edwards
<grant.b.edwards@gmail.com> wrote:
> On 2017-11-15, R0b0t1 <r030t1@gmail.com> wrote:
>
>> What I am wondering about is if C code which uses
>> __attribute__((optimize(...))) is against Gentoo package standards and
>> would have to be removed from the Portage tree.
>
> Huh?
>
> Gentoo enforces standards for the source code of packages?
>
> "They" review the source code for the Linux kernel, Gnome, KDE, Qt,
> Chrome, Firefox, GCC, and 24670 thousand other packages and make sure
> they all follow Gentoo coding standards?
>

To be consistent they would have to. Why I bring it up is that a
number of optimizations in eix were removed due to the logic I gave
above, despite there being no way to enable them without setting "-O3"
globally.

Cheers,
     R0b0t1

Re: [gentoo-user] Re: memset_s

[R0b0t1]

Apologies for the double post,

On Wed, Nov 15, 2017 at 9:41 AM, R0b0t1 <r030t1@gmail.com> wrote:
> On Wed, Nov 15, 2017 at 1:22 AM, Jorge Almeida <jjalmeida@gmail.com> wrote:
>> On Tue, Nov 14, 2017 at 8:42 PM, R0b0t1 <r030t1@gmail.com> wrote:
>>> What I am wondering about is if C code which uses
>>> __attribute__((optimize(...))) is against Gentoo package standards and
>>> would have to be removed from the Portage tree.
>>>
>>
>>
>> You can set your optimization preferences in make.conf, and still an
>> ebuild will override them if deemed unsafe. What would be the
>> difference?
>>
>
> Ebuilds are not supposed to do this, so if you file a bug report
> citing that ebuild changes will be made (eventually?) to work around
> it.
>
>
> On Wed, Nov 15, 2017 at 9:28 AM, Grant Edwards
> <grant.b.edwards@gmail.com> wrote:
>> On 2017-11-15, R0b0t1 <r030t1@gmail.com> wrote:
>>
>>> What I am wondering about is if C code which uses
>>> __attribute__((optimize(...))) is against Gentoo package standards and
>>> would have to be removed from the Portage tree.
>>
>> Huh?
>>
>> Gentoo enforces standards for the source code of packages?
>>
>> "They" review the source code for the Linux kernel, Gnome, KDE, Qt,
>> Chrome, Firefox, GCC, and 24670 thousand other packages and make sure
>> they all follow Gentoo coding standards?
>>
>
> To be consistent they would have to. Why I bring it up is that a
> number of optimizations in eix were removed due to the logic I gave
> above, despite there being no way to enable them without setting "-O3"
> globally.
>
> Cheers,
>      R0b0t1

https://bugs.gentoo.org/632315

Cheers,
     R0b0t1

[gentoo-user] Re: memset_s

[Grant Edwards]

On 2017-11-15, R0b0t1 <r030t1@gmail.com> wrote:
> Apologies for the double post,
>
> On Wed, Nov 15, 2017 at 9:41 AM, R0b0t1 <r030t1@gmail.com> wrote:
>> On Wed, Nov 15, 2017 at 1:22 AM, Jorge Almeida <jjalmeida@gmail.com> wrote:
>>> On Tue, Nov 14, 2017 at 8:42 PM, R0b0t1 <r030t1@gmail.com> wrote:
>>>> What I am wondering about is if C code which uses
>>>> __attribute__((optimize(...))) is against Gentoo package standards and
>>>> would have to be removed from the Portage tree.
>>>>
>>>
>>>
>>> You can set your optimization preferences in make.conf, and still an
>>> ebuild will override them if deemed unsafe. What would be the
>>> difference?
>>>
>>
>> Ebuilds are not supposed to do this, so if you file a bug report
>> citing that ebuild changes will be made (eventually?) to work around
>> it.
>>
>>
>> On Wed, Nov 15, 2017 at 9:28 AM, Grant Edwards
>> <grant.b.edwards@gmail.com> wrote:
>>> On 2017-11-15, R0b0t1 <r030t1@gmail.com> wrote:
>>>
>>>> What I am wondering about is if C code which uses
>>>> __attribute__((optimize(...))) is against Gentoo package standards and
>>>> would have to be removed from the Portage tree.
>>>
>>> Huh?
>>>
>>> Gentoo enforces standards for the source code of packages?
>>>
>>> "They" review the source code for the Linux kernel, Gnome, KDE, Qt,
>>> Chrome, Firefox, GCC, and 24670 thousand other packages and make sure
>>> they all follow Gentoo coding standards?
>>>
>>
>> To be consistent they would have to. Why I bring it up is that a
>> number of optimizations in eix were removed due to the logic I gave
>> above, despite there being no way to enable them without setting "-O3"
>> globally.
>>
>> Cheers,
>>      R0b0t1
>
> https://bugs.gentoo.org/632315

I don't see how that's relevent.  That bug is about use flags and
ebuild stuff, not about the C code inside a package's source files.

-- 
Grant Edwards               grant.b.edwards        Yow! Half a mind is a
                                  at               terrible thing to waste!
                              gmail.com            

Re: [gentoo-user] Re: memset_s

[R0b0t1]

On Wed, Nov 15, 2017 at 3:42 PM, Grant Edwards
<grant.b.edwards@gmail.com> wrote:
> On 2017-11-15, R0b0t1 <r030t1@gmail.com> wrote:
>> Apologies for the double post,
>>
>> On Wed, Nov 15, 2017 at 9:41 AM, R0b0t1 <r030t1@gmail.com> wrote:
>>> On Wed, Nov 15, 2017 at 1:22 AM, Jorge Almeida <jjalmeida@gmail.com> wrote:
>>>> On Tue, Nov 14, 2017 at 8:42 PM, R0b0t1 <r030t1@gmail.com> wrote:
>>>>> What I am wondering about is if C code which uses
>>>>> __attribute__((optimize(...))) is against Gentoo package standards and
>>>>> would have to be removed from the Portage tree.
>>>>>
>>>>
>>>>
>>>> You can set your optimization preferences in make.conf, and still an
>>>> ebuild will override them if deemed unsafe. What would be the
>>>> difference?
>>>>
>>>
>>> Ebuilds are not supposed to do this, so if you file a bug report
>>> citing that ebuild changes will be made (eventually?) to work around
>>> it.
>>>
>>>
>>> On Wed, Nov 15, 2017 at 9:28 AM, Grant Edwards
>>> <grant.b.edwards@gmail.com> wrote:
>>>> On 2017-11-15, R0b0t1 <r030t1@gmail.com> wrote:
>>>>
>>>>> What I am wondering about is if C code which uses
>>>>> __attribute__((optimize(...))) is against Gentoo package standards and
>>>>> would have to be removed from the Portage tree.
>>>>
>>>> Huh?
>>>>
>>>> Gentoo enforces standards for the source code of packages?
>>>>
>>>> "They" review the source code for the Linux kernel, Gnome, KDE, Qt,
>>>> Chrome, Firefox, GCC, and 24670 thousand other packages and make sure
>>>> they all follow Gentoo coding standards?
>>>>
>>>
>>> To be consistent they would have to. Why I bring it up is that a
>>> number of optimizations in eix were removed due to the logic I gave
>>> above, despite there being no way to enable them without setting "-O3"
>>> globally.
>>>
>>> Cheers,
>>>      R0b0t1
>>
>> https://bugs.gentoo.org/632315
>
> I don't see how that's relevent.  That bug is about use flags and
> ebuild stuff, not about the C code inside a package's source files.
>

Right, but the reason that it is not allowed in ebuilds (or at least
in eix's case) was some sense of purism - despite the optimizations
being behind a useflag at the package level, someone determined this
was improper.

Applying this logic consistently, any package which uses the
"optimize" GCC attribute would be unsuitable for the main portage
tree.

If this doesn't make sense, that is exactly my point. Sorry for going
off on a tangent, I didn't expect any follow-up posts on it.

Cheers,
     R0b0t1

Re: [gentoo-user] Re: memset_s

[Jorge Almeida]

On Wed, Nov 15, 2017 at 7:41 AM, R0b0t1 <r030t1@gmail.com> wrote:
> On Wed, Nov 15, 2017 at 1:22 AM, Jorge Almeida <jjalmeida@gmail.com> wrote:
>> On Tue, Nov 14, 2017 at 8:42 PM, R0b0t1 <r030t1@gmail.com> wrote:

>> You can set your optimization preferences in make.conf, and still an
>> ebuild will override them if deemed unsafe. What would be the
>> difference?
>>
>
> Ebuilds are not supposed to do this, so if you file a bug report
> citing that ebuild changes will be made (eventually?) to work around
> it.
>
Are you really sure? I was under the impression that ebuilds did it,
and I find that natural. I didn't view that as a bug at all.

Cheers

Jorge

Re: [gentoo-user] Re: memset_s

[Michael Orlitzky]

On 11/15/2017 10:50 AM, Jorge Almeida wrote:
>
> Are you really sure? I was under the impression that ebuilds did it,
> and I find that natural. I didn't view that as a bug at all.


On principle, most things in the tree should respect whatever CFLAGS,
LDFLAGS, etc. you ask for. The problems resulting from that are
predictable: if you add "-ffast-math" to your CFLAGS and then start
getting crashes; well, you asked for it. We're not going to fix that for
you in every ebuild.

On the other hand, there are packages where some optimizations fail due
to a compiler bug or something else that is not by design. Some packages
optimize their critical path in assembly, and the resulting code doesn't
play well with otherwise-reasonable CFLAGS. In cases like those, it's
often simpler to disable the problematic flag in the ebuild rather than
have a thousand people do the same thing locally.

[gentoo-user] Re: memset_s

[Nikos Chantziaras]

On 14/11/17 19:36, Jorge Almeida wrote:
> On Fri, Nov 10, 2017 at 12:09 PM, Jorge Almeida <jjalmeida@gmail.com> wrote:
> 
>> http://www.daemonology.net/blog/2014-09-04-how-to-zero-a-buffer.html
>>
>>
>>>> Of course, what would really solve the optimize-into-oblivion problem
>>>> is a pragma that when invoked on a particular block of code (maybe
>>>> only a function definition) would tell the compiler to do what the
>>>> programmer says rather than viewing a function as a kind of black box.
>>>>
>>>
> 
> It seems a solution exists with gcc:
> 
> https://stackoverflow.com/questions/2219829/how-to-prevent-gcc-optimizing-some-statements-in-c
> 
> The last reply:
> 
> void __attribute__((optimize("O0"))) foo(unsigned char data) {
>      // unmodifiable compiler code
> }
> 
> Any comments, anyone?
Unless you look at the assembly output, you can't be sure. Some 
optimization is done even at -O0.

I'd stick to using explicit_bzero() which is safe regardless of compiler 
vendor *and* version.

Re: [gentoo-user] Re: memset_s

[Jorge Almeida]

On Wed, Nov 15, 2017 at 12:54 AM, Nikos Chantziaras <realnc@gmail.com> wrote:
> On 14/11/17 19:36, Jorge Almeida wrote:
>>
>> On Fri, Nov 10, 2017 at 12:09 PM, Jorge Almeida <jjalmeida@gmail.com>
>> wrote:
>>

>
> Unless you look at the assembly output, you can't be sure. Some optimization
> is done even at -O0.
>
> I'd stick to using explicit_bzero() which is safe regardless of compiler
> vendor *and* version.
>
But what about overwriting with random bytes? Having "explicit-*"
versions of whatnot seems madness. BTW, I checked the assemby,
memset() is there. But there should be way to tell the compiler "do
what I say".

Jorge

[gentoo-user] Re: memset_s

[Nikos Chantziaras]

On 15/11/17 11:05, Jorge Almeida wrote:
> On Wed, Nov 15, 2017 at 12:54 AM, Nikos Chantziaras <realnc@gmail.com> wrote:
>> On 14/11/17 19:36, Jorge Almeida wrote:
>>>
>>> On Fri, Nov 10, 2017 at 12:09 PM, Jorge Almeida <jjalmeida@gmail.com>
>>> wrote:
>>>
> 
>>
>> Unless you look at the assembly output, you can't be sure. Some optimization
>> is done even at -O0.
>>
>> I'd stick to using explicit_bzero() which is safe regardless of compiler
>> vendor *and* version.
>>
> But what about overwriting with random bytes? Having "explicit-*"
> versions of whatnot seems madness. BTW, I checked the assemby,
> memset() is there. But there should be way to tell the compiler "do
> what I say".

Writing random bytes isn't going to help. If the compiler can deduce 
that the random bytes aren't being used and that not writing anything 
will not change the observable behavior of the code, the write might not 
happen. And even if you do use the random bytes (like writing them to 
/dev/null,) the write might still not happen. The compiler might have 
marked another memory location as "hot" (in cache terms) and safe to 
write to, and use that instead.

The only sure way to do this I can think of, is to require the caller to 
use volatile and implement your own memset(). This is obviously prone to 
error. To fix that, you could enforce your own volatile pointer with 
something like:

   typedef struct passwd_t {
       volatile char* data;
   } passwd_t;

   passwd_t* alloc_passwd(int len);
   void free_passwd(passwd_t* passwd);

However, since explicit_bzero() is something several other people have 
come up as the solution to the problem, it's what I recommend. It's been 
tested already by other people on multiple platforms and compilers. Have 
a configure check for it, and if not found, try again with -lbsd. Or 
have a configure switch for it. These tests are easy to do with CMake or 
Autoconf.

Trying to reinvent the wheel, especially when it comes to security, 
doesn't sound like a good idea. It's easy to get it wrong.

[gentoo-user] Re: memset_s

[Nikos Chantziaras]

On 10/11/17 11:54, Jorge Almeida wrote:
> I'm trying to use memset_s() but the system (glibc?) doesn't know
> about it. I also tried to compile against musl, same result.
> 
> There's precious little info about memset_s in the net. Does it exist
> at all? No man page.

$ grep -r memset_s /usr/include

Nope. Doesn't exist.