[gentoo-dev] Evaluating a new malloc()

[gentoo-dev] Evaluating a new malloc()

[Richard Yao]

[-- Attachment #1: Type: text/plain, Size: 2044 bytes --]

The Blender project found some fairly remarkable discrepancies between
what their software actually used and what glibc's ptmalloc allocated:

http://www.sintel.org/development/memory-jemalloc/

Results such as these led Blender and others (e.g. Chrome/Chromium,
Firefox, Thunderbird) to bundle private versions of jemalloc. This
bundling situation violates our policy against bundled libraries. The
maintainers could just patch their software to link to libjemalloc.
However, it might make more sense to evaluate jemalloc as a
distribution-wide replacement for glibc's ptmalloc.

It appears that run the following commands on amd64 to see how jemalloc
works when used system-wide:

echo dev-libs/jemalloc ~amd64 >> /etc/portage/package.accept_keywords
emerge dev-libs/jemalloc
echo /usr/lib64/libjemalloc.so.1 >> /etc/ld.so.preload

I did this on my system, verified that jemalloc was being loaded with
strace and rebooted. So far, sudo is broken and anything that is 32-bit
will have the loader print an error about the library not being
preloaded. On a related note, setuid binaries will obey
/etc/ld.so.preload, even though they ignore LD_PRELOAD.

There is a chance that patching jemalloc directly into glibc would solve
the sudo issue, but I imagine that sudo is doing something strange,
which likely merits attention. For now, I am using su as a substitute
until I look into why sudo is broken.

Unless a significant issue is found in jemalloc itself, I do not see any
reason to continue using glibc's ptmalloc over jemalloc. As far as I
know, FreeBSD, NetBSD, Facebook and others are using jemalloc, so I
expect that no significant issues will be found. jemalloc will still
needs testing before we can consider it. Doing good tests is hard, so I
would like to crowd source ideas on the appropriate way to test a malloc
implementation from the list. My expectation is that people on the list
will collectively think of better ideas than I could on my own.

With that said, what do people think?


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 900 bytes --]

Re: [gentoo-dev] Evaluating a new malloc()

[Diego Elio Pettenò]

[-- Attachment #1: Type: text/plain, Size: 563 bytes --]

On 26/02/2013 14:33, Richard Yao wrote:
> Results such as these led Blender and others (e.g. Chrome/Chromium,
> Firefox, Thunderbird) to bundle private versions of jemalloc. This
> bundling situation violates our policy against bundled libraries. The
> maintainers could just patch their software to link to libjemalloc.
> However, it might make more sense to evaluate jemalloc as a
> distribution-wide replacement for glibc's ptmalloc.

Short answer: no.

-- 
Diego Elio Pettenò — Flameeyes
flameeyes@flameeyes.eu — http://blog.flameeyes.eu/


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 555 bytes --]

Re: [gentoo-dev] Evaluating a new malloc()

[Richard Yao]

[-- Attachment #1: Type: text/plain, Size: 1156 bytes --]

On 02/26/2013 08:35 AM, Diego Elio Pettenò wrote:
> On 26/02/2013 14:33, Richard Yao wrote:
>> Results such as these led Blender and others (e.g. Chrome/Chromium,
>> Firefox, Thunderbird) to bundle private versions of jemalloc. This
>> bundling situation violates our policy against bundled libraries. The
>> maintainers could just patch their software to link to libjemalloc.
>> However, it might make more sense to evaluate jemalloc as a
>> distribution-wide replacement for glibc's ptmalloc.
> 
> Short answer: no.
> 

Would you elaborate on this? From a brief chat in IRC, you mentioned
Valgrind, but it looks like that issue has been solved:

http://blog.mozilla.org/jseward/2012/06/05/valgrind-now-supports-jemalloc-builds-directly/

jemalloc probably should be merged at glibc upstream, but I have zero
contact with glibc development, my Google searches have been fruitless
and I want to know what other people think about this. I think it would
be wrong to try to go to upstream with the idea if people downstream
don't like the idea. Not to mention, glibc is not the only libc in
Gentoo and this applies to each of them.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 900 bytes --]

Re: [gentoo-dev] Evaluating a new malloc()

[Alexander Berntsen]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 26/02/13 14:33, Richard Yao wrote:
> The Blender project found some fairly remarkable discrepancies
> between what their software actually used and what glibc's ptmalloc
> allocated
Have they filed a bug?
- -- 
Alexander
alexander@plaimi.net
http://plaimi.net/~alexander
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iF4EAREIAAYFAlEsvS4ACgkQRtClrXBQc7WLLQD/Sp6ypPcg5UPGErRRF4lqd9Ad
g1bbdXGbDKQ2igbIOcgA/12jT8eBq3z07QgMisOGAL5IexKtfcRL80UDHqEEUaBy
=3bY4
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Evaluating a new malloc()

[Richard Yao]

[-- Attachment #1: Type: text/plain, Size: 315 bytes --]

On 02/26/2013 08:48 AM, Alexander Berntsen wrote:
> On 26/02/13 14:33, Richard Yao wrote:
>> The Blender project found some fairly remarkable discrepancies
>> between what their software actually used and what glibc's ptmalloc
>> allocated
> Have they filed a bug?
> 

I don't know. You should ask them.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 900 bytes --]

Re: [gentoo-dev] Evaluating a new malloc()

[Florian Philipp]

[-- Attachment #1: Type: text/plain, Size: 479 bytes --]

Am 26.02.2013 14:52, schrieb Richard Yao:
> On 02/26/2013 08:48 AM, Alexander Berntsen wrote:
>> On 26/02/13 14:33, Richard Yao wrote:
>>> The Blender project found some fairly remarkable discrepancies
>>> between what their software actually used and what glibc's ptmalloc
>>> allocated
>> Have they filed a bug?
>>
> 
> I don't know. You should ask them.
> 

Probably related to
http://sourceware.org/bugzilla/show_bug.cgi?id=11261

Regards,
Florian Philipp


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 263 bytes --]

Re: [gentoo-dev] Evaluating a new malloc()

[Alec Warner]

On Tue, Feb 26, 2013 at 5:33 AM, Richard Yao <ryao@gentoo.org> wrote:
> The Blender project found some fairly remarkable discrepancies between
> what their software actually used and what glibc's ptmalloc allocated:
>
> http://www.sintel.org/development/memory-jemalloc/
>
> Results such as these led Blender and others (e.g. Chrome/Chromium,
> Firefox, Thunderbird) to bundle private versions of jemalloc. This
> bundling situation violates our policy against bundled libraries. The
> maintainers could just patch their software to link to libjemalloc.
> However, it might make more sense to evaluate jemalloc as a
> distribution-wide replacement for glibc's ptmalloc.

So I'm confused a bit. The glibc malloc does perform poorly in some
edge cases. Lots of folks use an optimized allocator. That doesn't
mean that the glibc one is necessarily bad or worth replacing.

In terms of 'following Gentoo policy.' We encourage packages to be as
close to upstream as possible. I cannot fathom why when you basically
find a performance bug in malloc, you start a thread on the list about
replacing it; as opposed to filing a bug against glibc or emailing the
glibc lists and asking them about it.

>
> It appears that run the following commands on amd64 to see how jemalloc
> works when used system-wide:
>
> echo dev-libs/jemalloc ~amd64 >> /etc/portage/package.accept_keywords
> emerge dev-libs/jemalloc
> echo /usr/lib64/libjemalloc.so.1 >> /etc/ld.so.preload

Well certainly we would never deploy an ld preload hack. If you want
to test jemalloc, why don't you actually recompile the system against
libjemalloc (patched glibc)?

>
> I did this on my system, verified that jemalloc was being loaded with
> strace and rebooted. So far, sudo is broken and anything that is 32-bit
> will have the loader print an error about the library not being
> preloaded. On a related note, setuid binaries will obey
> /etc/ld.so.preload, even though they ignore LD_PRELOAD.
>
> There is a chance that patching jemalloc directly into glibc would solve
> the sudo issue, but I imagine that sudo is doing something strange,
> which likely merits attention. For now, I am using su as a substitute
> until I look into why sudo is broken.
>
> Unless a significant issue is found in jemalloc itself, I do not see any
> reason to continue using glibc's ptmalloc over jemalloc. As far as I
> know, FreeBSD, NetBSD, Facebook and others are using jemalloc, so I
> expect that no significant issues will be found. jemalloc will still
> needs testing before we can consider it. Doing good tests is hard, so I
> would like to crowd source ideas on the appropriate way to test a malloc
> implementation from the list. My expectation is that people on the list
> will collectively think of better ideas than I could on my own.
>
> With that said, what do people think?
>

I see a *HUGE* reason. glibc ships with ptmalloc. If you think they
should use jemalloc, talk to them. Don't just do it in Gentoo.

Re: [gentoo-dev] Evaluating a new malloc()

[Mike Gilbert]

On Tue, Feb 26, 2013 at 8:33 AM, Richard Yao <ryao@gentoo.org> wrote:
> Unless a significant issue is found in jemalloc itself, I do not see any
> reason to continue using glibc's ptmalloc over jemalloc. As far as I
> know, FreeBSD, NetBSD, Facebook and others are using jemalloc, so I
> expect that no significant issues will be found. jemalloc will still
> needs testing before we can consider it. Doing good tests is hard, so I
> would like to crowd source ideas on the appropriate way to test a malloc
> implementation from the list. My expectation is that people on the list
> will collectively think of better ideas than I could on my own.
>
> With that said, what do people think?
>

Just want to point out a problem I have personally encountered with
another alternate malloc (tcmalloc). Essentially, when you combine
libGL.so from nvidia-drivers with tcmalloc, the Google Chrome web
browser fails to properly close its worker processes.

https://bugs.gentoo.org/show_bug.cgi?id=413637

This thread on the Chromium developer list goes into more detail:

https://groups.google.com/a/chromium.org/forum/#!msg/chromium-dev/uLf5l669dCk/MXXfJfIvDF0J

The main concern I would have is that upstream developers (like
nvidia) are unlikely to test their software with an alternate malloc
implementation, so we could end up running into some very subtle bugs.
Are the saved cycles really worth it?

Re: [gentoo-dev] Evaluating a new malloc()

[Rich Freeman]

On Tue, Feb 26, 2013 at 11:35 AM, Alec Warner <antarus@gentoo.org> wrote:
> I see a *HUGE* reason. glibc ships with ptmalloc. If you think they
> should use jemalloc, talk to them. Don't just do it in Gentoo.

Certainly I think it would be far more productive to talk to the glibc
maintainers first.

However, nothing prevents anybody from creating a Gentoo package with
an alternative glibc implementation, patchset, whatever, assuming they
are willing to maintain it.  It really is no different from having an
alternative udev implementation.  Gentoo is about choice.

Now, whether it ever becomes the /default/ choice is another matter
entirely.  Nobody can prevent people from experimenting - Gentoo
developers are permitted to waste their time if desired.  :)

Rich

Re: [gentoo-dev] Evaluating a new malloc()

[Matt Turner]

On Tue, Feb 26, 2013 at 8:35 AM, Alec Warner <antarus@gentoo.org> wrote:
> In terms of 'following Gentoo policy.' We encourage packages to be as
> close to upstream as possible. I cannot fathom why when you basically
> find a performance bug in malloc, you start a thread on the list about
> replacing it; as opposed to filing a bug against glibc or emailing the
> glibc lists and asking them about it.

We may not understand it, but it does completely fit ryao's typical
modus operandi.

Re: [gentoo-dev] Evaluating a new malloc()

[Maciej Mrozowski]

[-- Attachment #1: Type: text/plain, Size: 812 bytes --]

On Tuesday 26 of February 2013 11:44:31 Rich Freeman wrote:
> On Tue, Feb 26, 2013 at 11:35 AM, Alec Warner <antarus@gentoo.org> wrote:
> > I see a *HUGE* reason. glibc ships with ptmalloc. If you think they
> > should use jemalloc, talk to them. Don't just do it in Gentoo.
> 
> Certainly I think it would be far more productive to talk to the glibc
> maintainers first.

You mean productive like below? ;)

http://sourceware.org/bugzilla/show_bug.cgi?id=11261

Ulrich Drepper:
"Stop reopening.  There is a solution for people who are stupid enough to 
create too many threads.  No implementation will be perfect for everyone.  The 
glibc implementation is tuned for reasonable programs and will run much faster 
than any other I tested."

Merge of jemalloc upstream is likely never going to happen.

regards
MM

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Evaluating a new malloc()

[Jeff Horelick]

[-- Attachment #1: Type: text/plain, Size: 1081 bytes --]

On 26 February 2013 16:37, Maciej Mrozowski <reavertm@gmail.com> wrote:

> On Tuesday 26 of February 2013 11:44:31 Rich Freeman wrote:
> > On Tue, Feb 26, 2013 at 11:35 AM, Alec Warner <antarus@gentoo.org>
> wrote:
> > > I see a *HUGE* reason. glibc ships with ptmalloc. If you think they
> > > should use jemalloc, talk to them. Don't just do it in Gentoo.
> >
> > Certainly I think it would be far more productive to talk to the glibc
> > maintainers first.
>
> You mean productive like below? ;)
>
> http://sourceware.org/bugzilla/show_bug.cgi?id=11261
>
> Ulrich Drepper:
> "Stop reopening.  There is a solution for people who are stupid enough to
> create too many threads.  No implementation will be perfect for everyone.
>  The
> glibc implementation is tuned for reasonable programs and will run much
> faster
> than any other I tested."
>
> Merge of jemalloc upstream is likely never going to happen.
>
> regards
> MM



It could happen. Ulrich Drepper is no longer in charge of Glibc and he's
barely involved in the development at all recently from what i've
heard/seen.

[-- Attachment #2: Type: text/html, Size: 1639 bytes --]

Re: [gentoo-dev] Evaluating a new malloc()

[Matt Turner]

On Tue, Feb 26, 2013 at 1:37 PM, Maciej Mrozowski <reavertm@gmail.com> wrote:
> On Tuesday 26 of February 2013 11:44:31 Rich Freeman wrote:
>> On Tue, Feb 26, 2013 at 11:35 AM, Alec Warner <antarus@gentoo.org> wrote:
>> > I see a *HUGE* reason. glibc ships with ptmalloc. If you think they
>> > should use jemalloc, talk to them. Don't just do it in Gentoo.
>>
>> Certainly I think it would be far more productive to talk to the glibc
>> maintainers first.
>
> You mean productive like below? ;)
>
> http://sourceware.org/bugzilla/show_bug.cgi?id=11261
>
> Ulrich Drepper:
> "Stop reopening.  There is a solution for people who are stupid enough to
> create too many threads.  No implementation will be perfect for everyone.  The
> glibc implementation is tuned for reasonable programs and will run much faster
> than any other I tested."

Drepper is no longer around. Upstream glibc is really friendly now,
probably in an attempt to throw off the image you rightly had.

> Merge of jemalloc upstream is likely never going to happen.

Indeed, but not because of Drepper, but rather because GNU projects
require copyright assignment for non-trivial contributions and I
highly doubt that the jemalloc developers who put it under the BSD
license are going to be okay with relicensing to LGPLv3+ and assigning
copyright on their work to the FSF.

Re: [gentoo-dev] Evaluating a new malloc()

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 627 bytes --]

On Tue, Feb 26, 2013 at 11:10:09AM -0800, Matt Turner wrote:
> On Tue, Feb 26, 2013 at 8:35 AM, Alec Warner <antarus@gentoo.org> wrote:
> > In terms of 'following Gentoo policy.' We encourage packages to be as
> > close to upstream as possible. I cannot fathom why when you basically
> > find a performance bug in malloc, you start a thread on the list about
> > replacing it; as opposed to filing a bug against glibc or emailing the
> > glibc lists and asking them about it.
> 
> We may not understand it, but it does completely fit ryao's typical
> modus operandi.

I must second this unfortunately.

William


[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

[gentoo-dev] Re: Evaluating a new malloc()

[Ryan Hill]

[-- Attachment #1: Type: text/plain, Size: 534 bytes --]

On Tue, 26 Feb 2013 08:33:44 -0500
Richard Yao <ryao@gentoo.org> wrote:

> With that said, what do people think?

I think I see a lot of our upstream bug reports being closed as
invalid/unsupported.  I think that if upstreams wanted to use jemalloc they
would just do so.  If they don't then obviously what they have is working fine
for them.


-- 
gcc-porting
toolchain, wxwidgets            learn a language baby, it's that kind of place
@ gentoo.org                   where low card is hunger and high card is taste

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] Re: Evaluating a new malloc()

[Peter Stuge]

[-- Attachment #1: Type: text/plain, Size: 404 bytes --]

Ryan Hill wrote:
> I think I see a lot of our upstream bug reports being closed as
> invalid/unsupported.  I think that if upstreams wanted to use
> jemalloc they would just do so.  If they don't then obviously
> what they have is working fine for them.

It can make sense to try further discussion, but that is sometimes
best done by someone other than whoever was involved in the bug
report.


//Peter

[-- Attachment #2: Type: application/pgp-signature, Size: 190 bytes --]

Re: [gentoo-dev] Evaluating a new malloc()

[Sergei Trofimovich]

[-- Attachment #1: Type: text/plain, Size: 1516 bytes --]

On Tue, 26 Feb 2013 08:33:44 -0500
Richard Yao <ryao@gentoo.org> wrote:

> The Blender project found some fairly remarkable discrepancies between
> what their software actually used and what glibc's ptmalloc allocated:
> 
> http://www.sintel.org/development/memory-jemalloc/
> 
> Results such as these led Blender and others (e.g. Chrome/Chromium,
> Firefox, Thunderbird) to bundle private versions of jemalloc.

Thre real disease is Windows support for all the mentioned apps.
glibc is not that bad.

I suspect
    export MALLOC_CHECK_=0
and
    CFLAGS=-fno-stack-protector
would solve all those bookkeeping "issues" glibc dumps on you
by default.

malloc() (likely free()) performance dependency for an app
means writing custom allocators. IMHO it's a per-app problem,
not system-wide one.

Do you have degradation example for your use cases? (a sample
program for real workload) Fixing known deficiencies is always
simpler than switching known-to-work base.

> This bundling situation violates our policy against bundled libraries. The
> maintainers could just patch their software to link to libjemalloc.
> However, it might make more sense to evaluate jemalloc as a
> distribution-wide replacement for glibc's ptmalloc.

It would not solve bundling problem, would it?

> Unless a significant issue is found in jemalloc itself, I do not see any
> reason to continue using glibc's ptmalloc over jemalloc.

How about "that core piece of libc works fine"?

-- 

  Sergei

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Evaluating a new malloc()

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 965 bytes --]

On Tuesday 26 February 2013 11:44:31 Rich Freeman wrote:
> On Tue, Feb 26, 2013 at 11:35 AM, Alec Warner wrote:
> > I see a *HUGE* reason. glibc ships with ptmalloc. If you think they
> > should use jemalloc, talk to them. Don't just do it in Gentoo.
> 
> Certainly I think it would be far more productive to talk to the glibc
> maintainers first.
> 
> However, nothing prevents anybody from creating a Gentoo package with
> an alternative glibc implementation, patchset, whatever, assuming they
> are willing to maintain it.  It really is no different from having an
> alternative udev implementation.  Gentoo is about choice.
> 
> Now, whether it ever becomes the /default/ choice is another matter
> entirely.  Nobody can prevent people from experimenting - Gentoo
> developers are permitted to waste their time if desired.  :)

i'm not interested in maintaining it in sys-libs/glibc.  so that should 
summarize that particular aspect.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]