[gentoo-dev] pam: thoughts on modernizing pam_limits configuration that Gentoo ships with

[gentoo-dev] pam: thoughts on modernizing pam_limits configuration that Gentoo ships with

[Piotr Karbowski]

Hi,

I'd like to touch base on the topic of pam_limits and the defaults that 
we ended up with in Gentoo.

Currently on default system installation without any modification to 
/etc/security/limits.{conf,d/*} user will end up with limit o 1024 of 
file descriptors and 4096 limit of threads.

Most of limits.conf makes a lot of sense on systems that are meant to be 
used remotely by many people simultaneously and have much less of 
importance on single user desktop systems.

Recently I've been haunted by random and and not reproducible failures 
of random applications during runs with ffmpeg's libsvtav1 integration. 
Having a 4 instances of ffmpeg with was enough to get me random 
failures, terminals not to open, up until I actually seen in shell 
'cannot allocate resource' while trying to run a script.

Turns out, I was running over the limit of 4096 threads, as nproc 
somewhat suggests this is limit of processes, it is actually a limit of 
PIDs, and every thread gets its own pid.

I have strong opinion on this one, that users that runs multi users 
systems will be well aware of the need to tune limits.conf of 
pam_limits, while the users that will actually suffer are the regular 
Joe that just uses Gentoo as a single user system.

What I'd like to do is to bump the limits.conf we ship with pam to following

     * hard nproc 16384
     * soft nproc 16384
     * hard nofile 16384
     * soft nofile 16384

Those are still reasonable defaults that are much more suitable the 
modern systems. I can only see benefits in it and am unable to think 
about the potential drawbacks of bumping *defaults*.

Any thoughts?

Unless there's strong opposition to not bump those 1024/4096 current 
defaults, I'd like to bump those limits. Normally I'd create a bug and 
assign it to maintainer, however our sys-libs/pam maintainer has not 
been seen in last half a year, so I'd end up joining as co-maintainer 
there in the result.

-- Piotr.

Re: [gentoo-dev] pam: thoughts on modernizing pam_limits configuration that Gentoo ships with

[Sam James]

[-- Attachment #1: Type: text/plain, Size: 750 bytes --]



> On 11 Dec 2022, at 08:28, Piotr Karbowski <slashbeast@gentoo.org> wrote:
> 
> Hi,
> 
> I'd like to touch base on the topic of pam_limits and the defaults that we ended up with in Gentoo.
> 
> [...]
> 
> Any thoughts?
> 
> Unless there's strong opposition to not bump those 1024/4096 current defaults, I'd like to bump those limits. Normally I'd create a bug and assign it to maintainer, however our sys-libs/pam maintainer has not been seen in last half a year, so I'd end up joining as co-maintainer there in the result.
> 

You should still file a bug for two reasons:
1. Paper trail
2. sys-auth/pambase has another maintainer who *is* active :)

As for the question in your post, I'll have a think. Thanks!

Best,
sam


[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 358 bytes --]

Re: [gentoo-dev] pam: thoughts on modernizing pam_limits configuration that Gentoo ships with

[Piotr Karbowski]

Hi,

On 11/12/2022 13.46, Sam James wrote:
> You should still file a bug for two reasons:
> 1. Paper trail
> 2. sys-auth/pambase has another maintainer who*is*  active :)
> 
> As for the question in your post, I'll have a think. Thanks!

I am not against creating a bug, I do see it however as inefficient in 
this very case.

I do know however if I were to create bug and assign it to current 
single pam maintainer, it would hardly get noticed by other people, 
greatly reducing the feedback. Changing defaults will affect vast 
majority of Gentoo users, so gentoo-dev ml was the place I choosen.

I didn't realized that you are maintainer of pambase, though the 
/etc/security/limits.conf belongs to sys-libs/pam that has only zlogene.

I did mail zlogene regarding another package recently and got no 
response, thus pam itself seems maintainer-needed to me, and because of 
it a candidate for me to join as another maintainer and do the changes. 
I do not think it would make much sense for me to then create bug for 
myself to make this change of defaults, paper trail would be the commit 
subject and message itself, or perhaps I did not understood of what you 
meant here by paper trail, would appreciate clarification.

It would be great if you were interested in joining pam itself, and I am 
interested to also joining pambase, since those are so co-dependent it 
does not make sense to have split maintainers there.

-- Piotr.

Re: [gentoo-dev] pam: thoughts on modernizing pam_limits configuration that Gentoo ships with

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 1548 bytes --]

Please do file a bug tracking this proposal, and reference the
discussion thread.

On Sun, Dec 11, 2022 at 09:28:14AM +0100, Piotr Karbowski wrote:
> What I'd like to do is to bump the limits.conf we ship with pam to
> following
> 
>      * hard nproc 16384
>      * soft nproc 16384
>      * hard nofile 16384
>      * soft nofile 16384
>
> Those are still reasonable defaults that are much more suitable the 
> modern systems. I can only see benefits in it and am unable to think 
> about the potential drawbacks of bumping *defaults*.
Drawbacks:
- The "*" would apply it to all users on a system, not just the
  interactive ones, and reduce overall security posture.
- Does this also need a sysctl change for raising fs.file-max?

With those in mind, how can we deploy these defaults for interactive
users, while still trying to maintain the good security posture overall?

- Is using "@users" instead of "*" good enough? (I think yes)
- Should it be limited to shiny logins on X or should it also take
  effect via remote logins? (conceptually yes, but I don't see a way to
  do it today within the scope of only pam_limits**)


** The closest other solution I can find is using a distinct limits.conf
for interactive logins, selected via pam.d trickery, and I don't like
that proposal.

-- 
Robin Hugh Johnson
Gentoo Linux: Dev, Infra Lead, Foundation Treasurer
E-Mail   : robbat2@gentoo.org
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1113 bytes --]

Re: [gentoo-dev] pam: thoughts on modernizing pam_limits configuration that Gentoo ships with

[Piotr Karbowski]

Hi,

On 12/12/2022 06.52, Robin H. Johnson wrote:
> Please do file a bug tracking this proposal, and reference the
> discussion thread.
> 
> On Sun, Dec 11, 2022 at 09:28:14AM +0100, Piotr Karbowski wrote:
>> What I'd like to do is to bump the limits.conf we ship with pam to
>> following
>>
>>       * hard nproc 16384
>>       * soft nproc 16384
>>       * hard nofile 16384
>>       * soft nofile 16384
>>
>> Those are still reasonable defaults that are much more suitable the
>> modern systems. I can only see benefits in it and am unable to think
>> about the potential drawbacks of bumping *defaults*.
> Drawbacks:
> - The "*" would apply it to all users on a system, not just the
>    interactive ones, and reduce overall security posture.
> - Does this also need a sysctl change for raising fs.file-max?
> 
> With those in mind, how can we deploy these defaults for interactive
> users, while still trying to maintain the good security posture overall?
> 
> - Is using "@users" instead of "*" good enough? (I think yes)
> - Should it be limited to shiny logins on X or should it also take
>    effect via remote logins? (conceptually yes, but I don't see a way to
>    do it today within the scope of only pam_limits**)
> 
> 
> ** The closest other solution I can find is using a distinct limits.conf
> for interactive logins, selected via pam.d trickery, and I don't like
> that proposal.

Since both you and Sam requested bug[1], so be it -- though I still find 
it excessive and I do not remember any other case where discussion about 
change in package were tracked in bug, I just hope it will not branch 
discussion to be in two places, navigating it would be difficult.

Looks like I have some backtracking to do. I pulled off latest stage3 
and seems like the limits.conf there have no entries by default, I do 
however have the nproc limit there on 2 old gentoo systems dating back 
into 2009, perhaps at some point limits.conf have it and I do not 
remember adding it there, so either it could be default at some point in 
time, or I added it and forgot, with the later being more likely. 
Apologies for confusion.

Regardless I'd like to continue the discussion about the new Gentoo's 
defaults.

Which makes the current defaults being inherited from kernel, though pid 
1 to all the children, which are

For the 32bit x86:

     limit    soft     hard
     nproc    64095    64095
     nofile   1024     4096

And for the 64bit x86

     limit    soft     hard
     nproc    256819   256819
     nofile   1024     4096


The fs.file-max does not need any change, on 32bit x86 it's 1636118 and 
on 64bit x86 it's 6574089

What I propose here is to significantly bump the limit of open files per 
user, and limit the number of PIDs per user to 16k. If anything, the 
current defaults allow you to make a DoS by forkbomb, the current 
defaults are neither secure nor make sense in 2022. Linux kernel is full 
of defaults that beg to be updated, among others, vm.swappiness makes 
absolute no sense in its current defaults either.

As for the remote logins, local logins and X sessions -- I see no value 
in having different limits across those.

[1] https://bugs.gentoo.org/885589

-- Piotr.

Re: [gentoo-dev] pam: thoughts on modernizing pam_limits configuration that Gentoo ships with

[Sam James]

[-- Attachment #1: Type: text/plain, Size: 2029 bytes --]



> On 12 Dec 2022, at 21:55, Piotr Karbowski <slashbeast@gentoo.org> wrote:
> 
> Hi,
> 
> On 12/12/2022 06.52, Robin H. Johnson wrote:
>> Please do file a bug tracking this proposal, and reference the
>> discussion thread.
>> On Sun, Dec 11, 2022 at 09:28:14AM +0100, Piotr Karbowski wrote:
>>> What I'd like to do is to bump the limits.conf we ship with pam to
>>> following
>>> 
>>>      * hard nproc 16384
>>>      * soft nproc 16384
>>>      * hard nofile 16384
>>>      * soft nofile 16384
>>> 
>>> Those are still reasonable defaults that are much more suitable the
>>> modern systems. I can only see benefits in it and am unable to think
>>> about the potential drawbacks of bumping *defaults*.
>> Drawbacks:
>> - The "*" would apply it to all users on a system, not just the
>>   interactive ones, and reduce overall security posture.
>> - Does this also need a sysctl change for raising fs.file-max?
>> With those in mind, how can we deploy these defaults for interactive
>> users, while still trying to maintain the good security posture overall?
>> - Is using "@users" instead of "*" good enough? (I think yes)
>> - Should it be limited to shiny logins on X or should it also take
>>   effect via remote logins? (conceptually yes, but I don't see a way to
>>   do it today within the scope of only pam_limits**)
>> ** The closest other solution I can find is using a distinct limits.conf
>> for interactive logins, selected via pam.d trickery, and I don't like
>> that proposal.
> 
> Since both you and Sam requested bug[1], so be it -- though I still find it excessive and I do not remember any other case where discussion about change in package were tracked in bug, I just hope it will not branch discussion to be in two places, navigating it would be difficult.
> 

It's unusual to have discussion about a single package on the mailing lists. I tend to keep an eye on PAM
bugs because I maintained pambase.

Bugs are the primary method of discussing changes to packages.

[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 358 bytes --]

Re: [gentoo-dev] pam: thoughts on modernizing pam_limits configuration that Gentoo ships with

[Piotr Karbowski]

On 12/12/2022 23.06, Sam James wrote:
> It's unusual to have discussion about a single package on the mailing lists. I tend to keep an eye on PAM
> bugs because I maintained pambase.
> 
> Bugs are the primary method of discussing changes to packages.

You really came strong on this one. I did explain why it went to mailing 
list, that very few people would notice bug on undeclared 
maintainer-needed package, unlike mailing list, assigning it to zlogene 
and hoping for few people to catch it up, yet you still zealously 
challenge it.

I feel really burnt out from this exchange and I see that you already 
base-system'd the sys-libs/pam tactically preventing me joining and 
introducing changes, last time I wanted join base-system there was push 
back and I was informed that only invited members can join. Do your 
thing Sam, I will step back now and will take note to throw ideas into 
void of bugzilla next time.

-- Piotr.

Re: [gentoo-dev] pam: thoughts on modernizing pam_limits configuration that Gentoo ships with

[Sam James]

[-- Attachment #1: Type: text/plain, Size: 1439 bytes --]



> On 12 Dec 2022, at 22:26, Piotr Karbowski <slashbeast@gentoo.org> wrote:
> 
> On 12/12/2022 23.06, Sam James wrote:
>> It's unusual to have discussion about a single package on the mailing lists. I tend to keep an eye on PAM
>> bugs because I maintained pambase.
>> Bugs are the primary method of discussing changes to packages.
> 
> You really came strong on this one. I did explain why it went to mailing list, that very few people would notice bug on undeclared maintainer-needed package, unlike mailing list, assigning it to zlogene and hoping for few people to catch it up, yet you still zealously challenge it.
> 

I'm not trying to challenge or come on strong on anything. I'm just saying it was unexpected?

> I feel really burnt out from this exchange and I see that you already base-system'd the sys-libs/pam tactically preventing me joining and introducing changes, last time I wanted join base-system there was push back and I was informed that only invited members can join. Do your thing Sam, I will step back now and will take note to throw ideas into void of bugzilla next time.
> 

There is nothing tactical here, we discussed it in #gentoo-base, and you're free to ask to join if you want?

(It was not instead of you doing anything, it was just something I'd been planning on anyway, but
prompted by your email like the commit message said.)

I definitely never gave you pushback like that.


[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 358 bytes --]

Re: [gentoo-dev] pam: thoughts on modernizing pam_limits configuration that Gentoo ships with

[John Helmert III]

[-- Attachment #1: Type: text/plain, Size: 1549 bytes --]

On Mon, Dec 12, 2022 at 11:26:32PM +0100, Piotr Karbowski wrote:
> On 12/12/2022 23.06, Sam James wrote:
> > It's unusual to have discussion about a single package on the mailing lists. I tend to keep an eye on PAM
> > bugs because I maintained pambase.
> > 
> > Bugs are the primary method of discussing changes to packages.
> 
> You really came strong on this one. I did explain why it went to mailing 
> list, that very few people would notice bug on undeclared 
> maintainer-needed package, unlike mailing list, assigning it to zlogene 
> and hoping for few people to catch it up, yet you still zealously 
> challenge it.

You did explain, somebody else still disagreed, I don't think that
should be taken personally?

> I feel really burnt out from this exchange and I see that you already 
> base-system'd the sys-libs/pam tactically preventing me joining and 
> introducing changes, last time I wanted join base-system there was push 
> back and I was informed that only invited members can join. Do your 
> thing Sam, I will step back now and will take note to throw ideas into 
> void of bugzilla next time.

pam is a package which is pretty obviously in-scope for the Base
System project, even from a third party perspective. Based on Sam's
most recent reply, that doesn't like it was a mechanism to prevent you
from contributing? And since you seem to have taken it that way, why?
If that were the case, I'd think the Base lead (sam) would've told
you, and he hasn't (quite the opposite!).

> -- Piotr.
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [gentoo-dev] pam: thoughts on modernizing pam_limits configuration that Gentoo ships with

[Joonas Niilola]

[-- Attachment #1.1: Type: text/plain, Size: 1203 bytes --]

On 13.12.2022 0.26, Piotr Karbowski wrote:
> On 12/12/2022 23.06, Sam James wrote:
>> It's unusual to have discussion about a single package on the mailing
>> lists. I tend to keep an eye on PAM
>> bugs because I maintained pambase.
>>
>> Bugs are the primary method of discussing changes to packages.
> 
> You really came strong on this one. I did explain why it went to mailing
> list, that very few people would notice bug on undeclared
> maintainer-needed package, unlike mailing list, assigning it to zlogene
> and hoping for few people to catch it up, yet you still zealously
> challenge it.

I see value in having both, this mailing list discussion AND a bug. It
was indeed a great initiative to open the discussion here, since as you
said the main maintainer is AWOL and pam is a critical package so this
needs attention, but the fix should now be finished in a bug IMHO.

Once you make the changing commit you can reference a bug and it'll show
relevant history data for the reason. It's much harder and annoying
trying to locate the "why was this ever changed?" from a mailing list,
months or years after, when you can just find a commit and a linked bug.

-- juippis

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 618 bytes --]