[gentoo-dev] glibc-2.17: nscd is optional

[gentoo-dev] glibc-2.17: nscd is optional

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 186 bytes --]

rough poll: how many people actually care about nscd ?  i'm making it into a 
USE flag for glibc-2.17 and it's easiest for me to do IUSE=nscd which means 
it'd default to off.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] glibc-2.17: nscd is optional

[Michael Weber]

On 12/30/2012 02:24 AM, Mike Frysinger wrote:
> rough poll: how many people actually care about nscd ? 
I use it for some pam_ldap machines

> ... it'd default to off.
fine with me, I'll turn it on / need it for `ls -l /home` not taking ages.

-- 
Michael Weber
Gentoo Developer
web: https://xmw.de/
mailto: Michael Weber <xmw@gentoo.org>

[gentoo-dev] Re: glibc-2.17: nscd is optional

[Duncan]

Michael Weber posted on Sun, 30 Dec 2012 02:29:40 +0100 as excerpted:

> On 12/30/2012 02:24 AM, Mike Frysinger wrote:
>> rough poll: how many people actually care about nscd ?
> I use it for some pam_ldap machines
> 
>> ... it'd default to off.
> fine with me, I'll turn it on / need it for `ls -l /home` not taking
> ages.

Fine here too.  I don't use it, but I've made the case before that 
gentoo's not a babysitter, and IMO, people not checking USE flag changes 
before an emerge and adjusting their config accordingly... get to keep 
whatever pieces they might end up with.  Given that, even if I did use it 
I'd be a hypocrite if I now tried to make a big deal about a USE flag 
change of this nature.

That said, as with the last case where big USE flag changes came up, 
let's use some wisdom and use the news feature for what it was designed 
for, to notify people about this sort of thing ahead of time. =:^)  
(Examples I've seen recently of people's emerge output noting X news 
items unread aside.  If they can't even read NEWS items... and still 
choose to use gentoo... well there's that saying about leading a horse to 
water... <shrug>)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] glibc-2.17: nscd is optional

[Alec Warner]

On Sat, Dec 29, 2012 at 5:29 PM, Michael Weber <xmw@gentoo.org> wrote:
> On 12/30/2012 02:24 AM, Mike Frysinger wrote:
>> rough poll: how many people actually care about nscd ?
> I use it for some pam_ldap machines
>
>> ... it'd default to off.
> fine with me, I'll turn it on / need it for `ls -l /home` not taking ages.

You can use nsscache or similar programs for this specifically.

-A

>
> --
> Michael Weber
> Gentoo Developer
> web: https://xmw.de/
> mailto: Michael Weber <xmw@gentoo.org>
>

Re: [gentoo-dev] glibc-2.17: nscd is optional

["Paweł Hajdan, Jr."]

[-- Attachment #1: Type: text/plain, Size: 409 bytes --]

On 12/29/12 5:24 PM, Mike Frysinger wrote:
> rough poll: how many people actually care about nscd ?  i'm making it into a 
> USE flag for glibc-2.17 and it's easiest for me to do IUSE=nscd which means 
> it'd default to off.

If you want, you can still enable the nscd USE flag in the profile so it
is on by default.

I'm fine either way, just wanted to note the technical possibility.

Paweł



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 203 bytes --]

Re: [gentoo-dev] glibc-2.17: nscd is optional

[Alec Warner]

On Sat, Dec 29, 2012 at 5:24 PM, Mike Frysinger <vapier@gentoo.org> wrote:
> rough poll: how many people actually care about nscd ?  i'm making it into a
> USE flag for glibc-2.17 and it's easiest for me to do IUSE=nscd which means
> it'd default to off.
> -mike

To be fair, in my large enterprise deployment we have turned nscd off
for years. However my guess is that this is cargo cult from 2002-esque
bugs in nscd; I'm curious why we would turn it off by default?

-A

Re: [gentoo-dev] glibc-2.17: nscd is optional

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 783 bytes --]

On Sunday 30 December 2012 03:46:01 Alec Warner wrote:
> On Sat, Dec 29, 2012 at 5:24 PM, Mike Frysinger <vapier@gentoo.org> wrote:
> > rough poll: how many people actually care about nscd ?  i'm making it
> > into a USE flag for glibc-2.17 and it's easiest for me to do IUSE=nscd
> > which means it'd default to off.
> 
> To be fair, in my large enterprise deployment we have turned nscd off
> for years. However my guess is that this is cargo cult from 2002-esque
> bugs in nscd; I'm curious why we would turn it off by default?

because i suspect it's dead weight for most people, especially desktop users.  
for people doing server deployments, i also expect those who know about nscd 
in the first place and actually want it can handle enabling it manually.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] glibc-2.17: nscd is optional

[Maxim Kammerer]

On Sun, Dec 30, 2012 at 7:58 PM, Mike Frysinger <vapier@gentoo.org> wrote:
> because i suspect it's dead weight for most people, especially desktop users.

Desktop users would probably benefit more from glibc databases, if they worked:
https://bugs.gentoo.org/show_bug.cgi?id=432020
http://sourceware.org/bugzilla/show_bug.cgi?id=14498

-- 
Maxim Kammerer
Liberté Linux: http://dee.su/liberte

Re: [gentoo-dev] glibc-2.17: nscd is optional

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 471 bytes --]

On Sunday 30 December 2012 13:47:40 Maxim Kammerer wrote:
> On Sun, Dec 30, 2012 at 7:58 PM, Mike Frysinger wrote:
> > because i suspect it's dead weight for most people, especially desktop
> > users.
> 
> Desktop users would probably benefit more from glibc databases, if they
> worked: https://bugs.gentoo.org/show_bug.cgi?id=432020
> http://sourceware.org/bugzilla/show_bug.cgi?id=14498

(1) i doubt it and (2) this isn't a discussion of bugs in nscd
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] glibc-2.17: nscd is optional

[Maxim Kammerer]

On Mon, Dec 31, 2012 at 10:45 PM, Mike Frysinger <vapier@gentoo.org> wrote:
> On Sunday 30 December 2012 13:47:40 Maxim Kammerer wrote:
>>
>> Desktop users would probably benefit more from glibc databases, if they
>> worked: https://bugs.gentoo.org/show_bug.cgi?id=432020
>> http://sourceware.org/bugzilla/show_bug.cgi?id=14498
>
> (1) i doubt it and (2) this isn't a discussion of bugs in nscd

It's relevant to the discussion, because glibc databases are an
alternative to running nscd. As a desktop user, I run nscd even on a
desktop without YP/NIS+, because I don't like the idea of glibc
functions parsing plain text files on each related query. I would use
glibc databases instead of nscd if that was a reliable option.

-- 
Maxim Kammerer
Liberté Linux: http://dee.su/liberte

Re: [gentoo-dev] glibc-2.17: nscd is optional

[Alec Warner]

On Mon, Dec 31, 2012 at 3:23 PM, Maxim Kammerer <mk@dee.su> wrote:
> On Mon, Dec 31, 2012 at 10:45 PM, Mike Frysinger <vapier@gentoo.org> wrote:
>> On Sunday 30 December 2012 13:47:40 Maxim Kammerer wrote:
>>>
>>> Desktop users would probably benefit more from glibc databases, if they
>>> worked: https://bugs.gentoo.org/show_bug.cgi?id=432020
>>> http://sourceware.org/bugzilla/show_bug.cgi?id=14498
>>
>> (1) i doubt it and (2) this isn't a discussion of bugs in nscd
>
> It's relevant to the discussion, because glibc databases are an
> alternative to running nscd. As a desktop user, I run nscd even on a
> desktop without YP/NIS+, because I don't like the idea of glibc
> functions parsing plain text files on each related query. I would use
> glibc databases instead of nscd if that was a reliable option.

flatfile lookups are 2-4ms with hot cache. How much faster is the db
option? The flatfiles are not *that* big. People who have /etc/passwd
(or other entity datastores) that are 4 megs in size are already not
using flatfiles if they have any sense. This is rare in the 'typical'
desktop world though.

-A

>
> --
> Maxim Kammerer
> Liberté Linux: http://dee.su/liberte
>

Re: [gentoo-dev] glibc-2.17: nscd is optional

[Maxim Kammerer]

On Tue, Jan 1, 2013 at 2:10 AM, Alec Warner <antarus@gentoo.org> wrote:
> flatfile lookups are 2-4ms with hot cache. How much faster is the db
> option?

I guess it depends on the implementation and how close is the system's
operational situation to an ideal one (whether swap started thrashing,
etc.). A DB is the proper solution that can be improved if necessary
(e.g., keeping often-used parts in RAM). Filesystem where it resides
can be offered hardware with lower seek time or better cache. But I
agree that it is easy to rationalize bad solutions. I don't like
waiting on an "ls -l" in addition to the system not being responsive
due to some other reason, though. But maybe I am expecting too much,
with even PolKit delegating each query to a full-blown Javascript
library nowadays.

-- 
Maxim Kammerer
Liberté Linux: http://dee.su/liberte

Re: [gentoo-dev] glibc-2.17: nscd is optional

[Doug Goldstein]

On Mon, Dec 31, 2012 at 11:13 PM, Maxim Kammerer <mk@dee.su> wrote:
> On Tue, Jan 1, 2013 at 2:10 AM, Alec Warner <antarus@gentoo.org> wrote:
>> flatfile lookups are 2-4ms with hot cache. How much faster is the db
>> option?
>
> I guess it depends on the implementation and how close is the system's
> operational situation to an ideal one (whether swap started thrashing,
> etc.). A DB is the proper solution that can be improved if necessary
> (e.g., keeping often-used parts in RAM). Filesystem where it resides
> can be offered hardware with lower seek time or better cache. But I
> agree that it is easy to rationalize bad solutions. I don't like
> waiting on an "ls -l" in addition to the system not being responsive
> due to some other reason, though. But maybe I am expecting too much,
> with even PolKit delegating each query to a full-blown Javascript
> library nowadays.
>

You realize that files are cached in RAM right? There's a page cache
and pages are ejected when the system needs that RAM for something
else and they're ejected in an LRU fashion. More than likely those
pages are always in cache. I say pages very liberally here because
most of the files we're dealing with are less than 4096 bytes (yep,
I'm making that assumption) so its really 1 page per file. The result
is that the request for the data (assuming mmap here) is handled by
just doing a bounds/range check and converting the virtual address to
the physical address the data is wired in. The time required to parse
the average GNOME single user desktop machine (I've got 44 users and
69 groups on that box) is likely smaller than the overhead of a DB.

-- 
Doug Goldstein

Re: [gentoo-dev] glibc-2.17: nscd is optional

[Maxim Kammerer]

On Tue, Jan 1, 2013 at 10:49 AM, Doug Goldstein <cardoe@gentoo.org> wrote:
> You realize that files are cached in RAM right?

Yes, I know how operating systems work.

> More than likely those pages are always in cache.

Did you read my reply at all? You are assuming ideal conditions
(enough free RAM), for a specific kind of desktop (low seek time for
root filesystem is one assumption), where the solution you are relying
upon is a generic one, and will fail under high load. I prefer
removing potential problems instead of relying on optimal behavior and
having to figure what went wrong down the road.

> The time required to parse
> the average GNOME single user desktop machine (I've got 44 users and
> 69 groups on that box) is likely smaller than the overhead of a DB.

No, since the DB can have frequent pages locked into memory. Should I
also ask: “you realize that not all DBs are MySQL and Oracle, right”?

I think this branch of discussion became pretty off-topic, so I
suggest stopping it. I just wanted people to know about the optional
glibc database functionality, which is a nice alternative for those of
us that are used to nscd with NIS+, and which doesn't work at the
moment (so maybe someone feels like figuring it out on the glibc bug
opened by vapier). I certainly have no desire to read condescending
replies. If I wanted a flamewar, I would have probably mentioned that
glibc uses /var/db for the database, which is not FHS-compliant.

-- 
Maxim Kammerer
Liberté Linux: http://dee.su/liberte

Re: [gentoo-dev] glibc-2.17: nscd is optional

[Alec Warner]

On Tue, Jan 1, 2013 at 2:24 AM, Maxim Kammerer <mk@dee.su> wrote:
> On Tue, Jan 1, 2013 at 10:49 AM, Doug Goldstein <cardoe@gentoo.org> wrote:
>> You realize that files are cached in RAM right?
>
> Yes, I know how operating systems work.
>
>> More than likely those pages are always in cache.
>
> Did you read my reply at all? You are assuming ideal conditions
> (enough free RAM), for a specific kind of desktop (low seek time for
> root filesystem is one assumption), where the solution you are relying
> upon is a generic one, and will fail under high load. I prefer
> removing potential problems instead of relying on optimal behavior and
> having to figure what went wrong down the road.

There are tons of built-in assumptions regarding system state on this
thread. I believe the argument being offered is that for the vast
majority of desktop users, the default upstream approach of flatfiles
serves the common use case fine. If you think the majority of desktop
users are using more than one machine, or NIS+ or anything
complicated, then we already disagree on the base case ;)

>
>> The time required to parse
>> the average GNOME single user desktop machine (I've got 44 users and
>> 69 groups on that box) is likely smaller than the overhead of a DB.
>
> No, since the DB can have frequent pages locked into memory. Should I
> also ask: “you realize that not all DBs are MySQL and Oracle, right”?
>
> I think this branch of discussion became pretty off-topic, so I
> suggest stopping it. I just wanted people to know about the optional
> glibc database functionality, which is a nice alternative for those of
> us that are used to nscd with NIS+, and which doesn't work at the
> moment (so maybe someone feels like figuring it out on the glibc bug
> opened by vapier). I certainly have no desire to read condescending
> replies. If I wanted a flamewar, I would have probably mentioned that
> glibc uses /var/db for the database, which is not FHS-compliant.

There are a ton of nss modules users can enable if they so choose.

>
> --
> Maxim Kammerer
> Liberté Linux: http://dee.su/liberte
>

Re: [gentoo-dev] glibc-2.17: nscd is optional

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 931 bytes --]

On Monday 31 December 2012 18:23:06 Maxim Kammerer wrote:
> On Mon, Dec 31, 2012 at 10:45 PM, Mike Frysinger wrote:
> > On Sunday 30 December 2012 13:47:40 Maxim Kammerer wrote:
> >> Desktop users would probably benefit more from glibc databases, if they
> >> worked: https://bugs.gentoo.org/show_bug.cgi?id=432020
> >> http://sourceware.org/bugzilla/show_bug.cgi?id=14498
> > 
> > (1) i doubt it and (2) this isn't a discussion of bugs in nscd
> 
> It's relevant to the discussion, because glibc databases are an
> alternative to running nscd. As a desktop user, I run nscd even on a
> desktop without YP/NIS+, because I don't like the idea of glibc
> functions parsing plain text files on each related query. I would use
> glibc databases instead of nscd if that was a reliable option.

one uncommonly used feature on the desktop vs another uncommonly used feature 
on the desktop doesn't really add much
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] glibc-2.17: nscd is optional

[Diego Elio Pettenò]

[-- Attachment #1: Type: text/plain, Size: 483 bytes --]

On 30/12/2012 02:24, Mike Frysinger wrote:
> rough poll: how many people actually care about nscd ?  i'm making it into a 
> USE flag for glibc-2.17 and it's easiest for me to do IUSE=nscd which means 
> it'd default to off.

I use it on my servers, but I'm fine with turning it on myself.
You still need to know to start it up anyway, so it should be fine with
a default-off.

-- 
Diego Elio Pettenò — Flameeyes
flameeyes@flameeyes.eu — http://blog.flameeyes.eu/


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 553 bytes --]

Re: [gentoo-dev] glibc-2.17: nscd is optional

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 308 bytes --]

On Saturday 29 December 2012 20:24:16 Mike Frysinger wrote:
> rough poll: how many people actually care about nscd ?  i'm making it into
> a USE flag for glibc-2.17 and it's easiest for me to do IUSE=nscd which
> means it'd default to off.

sounds like people can handle this, so i've added it to 2.17
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]