[gentoo-dev] Idea: User centric kernel configuration

[gentoo-dev] Idea: User centric kernel configuration

[Gerion Entrup]

[-- Attachment #1: Type: text/plain, Size: 1206 bytes --]

Hi,

the Linux kernel has _a lot of_ configuration options, way too many to configure them by hand.
Many of these configuration options seem to be kernel centric (they deactivate a specific compilation unit, they deactivate specific source code parts).
However, my main configuration requirements as a user are mostly user space program or feature centric. I want to activate kernel support for a specific feature, e.g. docker support or cryptsetup support.

This requires a mapping between user oriented "features" and the kernel internal configuration options. This mapping exists already at the Gentoo wiki [e.g. 1, 2, 3].
Gentoo also provides (via gentoo-sources) a set of patches that adds some meta configuration options to enable some of exactly that user specific features (Systemd, OpenRC and Portage support).

Do you think that it is useful and feasible to combine these two mechanisms?
A possible way could be to automatically extract the kernel config flags from the wiki pages and map them to Kconfig options.

Best regards,
Gerion

[1] https://wiki.gentoo.org/wiki/Docker#Kernel
[2] https://wiki.gentoo.org/wiki/Dm-crypt#Kernel_Configuration
[3] https://wiki.gentoo.org/wiki/AMDGPU#Kernel



[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: [gentoo-dev] Idea: User centric kernel configuration

[Peter Stuge]

Hi,

Gerion Entrup wrote:
> the Linux kernel has _a lot of_ configuration options, way too many to configure them by hand.

I actually disagree strongly with that; I think it's important to
actively decide what kernels include, and I routinely do, but of
course not everyone will. I've made sure to include a kernel build
when teaching systems administration courses and would again.

As the kernel becomes more complex the threshold for the first
configuration also rises, but it's still completely possible to learn
what you need in order to successfully configure your own kernel.
I guess it's on the order of a weekend project given some basic
understanding of computer architecture and programming.


> This requires a mapping between user oriented "features" and the kernel
> internal configuration options.

So the challenge here is that the kernel is disjoint from user space,
and while the kernel API remains stable over time consumer requirements
such as "docker" or "cryptsetup" will mean different things for
different versions of particular user space software.


> Do you think that it is useful and feasible to combine these two mechanisms?

AFAIK there's no generic method for formal kernel requirements in user
space packages and there's also no sanctioned method for quering
kernel capabilities. The only thing available is /proc/config if that
was enabled in the kernel build, and there are of course reasons to
leave it out, and it only applies to the particular running kernel,
e.g. useless for cross-compilation. There, it would be possible to
read the kernel configuration file if the kernel source code is
available when the userspace package is being built, but that's not
guaranteed.

In Gentoo, linux-info.eclass provides linux_config_exists() to do all
of this, but order to become a widespread success there would have to
be one method for upstreams to maintain these requirements as part of
their packages, rather than forcing the burden on package maintainers
to repeat the same detective task in every single distribution.

I think it would be very useful to create something generic for that,
but that's certainly no small task.

And realistically I only see it succeeding if Linux Foundation decides
to push it onto the world.


> A possible way could be to automatically extract the kernel config
> flags from the wiki pages and map them to Kconfig options.

At very best that will only be valid for some particular point in time,
like current CONFIG_CHECK in ebuilds using linux_config_exists() are
only valid for particular package versions. At worst it's plain wrong
because the requirements have to be reverse-engineered downstream.


//Peter

Re: [gentoo-dev] Idea: User centric kernel configuration

[Gerion Entrup]

[-- Attachment #1: Type: text/plain, Size: 5482 bytes --]

Am Donnerstag, 11. März 2021, 17:41:45 CET schrieb Peter Stuge:
> Hi,
> 
> Gerion Entrup wrote:
> > the Linux kernel has _a lot of_ configuration options, way too many to configure them by hand.
> 
> I actually disagree strongly with that; I think it's important to
> actively decide what kernels include, and I routinely do, but of
> course not everyone will. I've made sure to include a kernel build
> when teaching systems administration courses and would again.
> 
> As the kernel becomes more complex the threshold for the first
> configuration also rises, but it's still completely possible to learn
> what you need in order to successfully configure your own kernel.
> I guess it's on the order of a weekend project given some basic
> understanding of computer architecture and programming.

I think, we mean two different things here. I for myself also configured
my own kernel(s) for years. I also actively teach students to do so.
However, I have never looked into all 18000 configuration options. I
don't understand them all (by far).

Actually, most of the times, I do a `make localmodconfig`, click through
various subsystems and activate what I think that it is useful or sounds
nice. Then, to update a kernel, I use `make oldconfig` and answer the
questions as good as I can (taking the default otherwise).

If my kernel is not capable to do something (for example run docker
containers), I take a look in the Gentoo Wiki and "copy" the options
into my own config.

But for me, this is not an informed decision. Of course, I learn
something in this process, maybe also more than a precompiled kernel
user, but I have by far not created a minimal config or even begin
to understand all subsystems and different configuration options.

I'm sure that this is similar to most Gentoo users.

BTW, here is a project to create a minimal config (kind of "ideal"
config) based on a "golden run" [1]. However, I have not compared such
a config with my own homegrown config.


> > This requires a mapping between user oriented "features" and the kernel
> > internal configuration options.
> 
> So the challenge here is that the kernel is disjoint from user space,
> and while the kernel API remains stable over time consumer requirements
> such as "docker" or "cryptsetup" will mean different things for
> different versions of particular user space software.
> 
> 
> > Do you think that it is useful and feasible to combine these two mechanisms?
> 
> AFAIK there's no generic method for formal kernel requirements in user
> space packages and there's also no sanctioned method for quering
> kernel capabilities. The only thing available is /proc/config if that
> was enabled in the kernel build, and there are of course reasons to
> leave it out, and it only applies to the particular running kernel,
> e.g. useless for cross-compilation. There, it would be possible to
> read the kernel configuration file if the kernel source code is
> available when the userspace package is being built, but that's not
> guaranteed.
> 
> In Gentoo, linux-info.eclass provides linux_config_exists() to do all
> of this, but order to become a widespread success there would have to
> be one method for upstreams to maintain these requirements as part of
> their packages, rather than forcing the burden on package maintainers
> to repeat the same detective task in every single distribution.
> 
> I think it would be very useful to create something generic for that,
> but that's certainly no small task.
> 
> And realistically I only see it succeeding if Linux Foundation decides
> to push it onto the world.

Sorry, I may have expressed this not clearly. I guess, we actually have
the same opinion here. I don't want to do this mapping automatically. I
don't want a "fully automatic" config or touch any (user space) ebuilds.

My idea is to patch Kconfig (as part of gentoo-sources) to provide the
same "features" (i.e. mappings) that are already present in the Gentoo
wiki.

For example this could result in this installation description for Docker
(within the Wiki):
"To enable kernel support for Docker, enable
'Gentoo Linux -> Support for user space programs -> Docker'"

This Kconfig flag than depends on all options that are needed for proper
Docker support (as already described in the Wiki) like cgroups etc..

This would allow users to configure there kernels the same way as before
but with some additional convenience shortcuts.

Of course, this is a high maintenance burden for the kernel package
maintainers. Therefore, this mechanism maybe can be automated:
Grep the wiki pages for kernel config snippets and automatically
condense that in a Kconfig readable configuration option.


> > A possible way could be to automatically extract the kernel config
> > flags from the wiki pages and map them to Kconfig options.
> 
> At very best that will only be valid for some particular point in time,
> like current CONFIG_CHECK in ebuilds using linux_config_exists() are
> only valid for particular package versions. At worst it's plain wrong
> because the requirements have to be reverse-engineered downstream.

Of course, the already existing kernel config snippets in the Wiki are
best effort. My idea is to combine these snippets directly with Kconfig.


Best regards,
Gerion

[1] https://vamos.informatik.uni-erlangen.de/trac/undertaker/wiki/UndertakerTailor

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: [gentoo-dev] Idea: User centric kernel configuration

[Mike Pagano]

On 3/12/21 11:34 AM, Gerion Entrup wrote:
> Am Donnerstag, 11. März 2021, 17:41:45 CET schrieb Peter Stuge:
>> Hi,
>>
>> Gerion Entrup wrote:
>>> the Linux kernel has _a lot of_ configuration options, way too many to configure them by hand.
>>
>> I actually disagree strongly with that; I think it's important to
>> actively decide what kernels include, and I routinely do, but of
>> course not everyone will. I've made sure to include a kernel build
>> when teaching systems administration courses and would again.
>>
>> As the kernel becomes more complex the threshold for the first
>> configuration also rises, but it's still completely possible to learn
>> what you need in order to successfully configure your own kernel.
>> I guess it's on the order of a weekend project given some basic
>> understanding of computer architecture and programming.
> 
> I think, we mean two different things here. I for myself also configured
> my own kernel(s) for years. I also actively teach students to do so.
> However, I have never looked into all 18000 configuration options. I
> don't understand them all (by far).
> 
> Actually, most of the times, I do a `make localmodconfig`, click through
> various subsystems and activate what I think that it is useful or sounds
> nice. Then, to update a kernel, I use `make oldconfig` and answer the
> questions as good as I can (taking the default otherwise).
> 
> If my kernel is not capable to do something (for example run docker
> containers), I take a look in the Gentoo Wiki and "copy" the options
> into my own config.
> 
> But for me, this is not an informed decision. Of course, I learn
> something in this process, maybe also more than a precompiled kernel
> user, but I have by far not created a minimal config or even begin
> to understand all subsystems and different configuration options.
> 
> I'm sure that this is similar to most Gentoo users.
> 
> BTW, here is a project to create a minimal config (kind of "ideal"
> config) based on a "golden run" [1]. However, I have not compared such
> a config with my own homegrown config.
> 
> 
>>> This requires a mapping between user oriented "features" and the kernel
>>> internal configuration options.
>>
>> So the challenge here is that the kernel is disjoint from user space,
>> and while the kernel API remains stable over time consumer requirements
>> such as "docker" or "cryptsetup" will mean different things for
>> different versions of particular user space software.
>>
>>
>>> Do you think that it is useful and feasible to combine these two mechanisms?
>>
>> AFAIK there's no generic method for formal kernel requirements in user
>> space packages and there's also no sanctioned method for quering
>> kernel capabilities. The only thing available is /proc/config if that
>> was enabled in the kernel build, and there are of course reasons to
>> leave it out, and it only applies to the particular running kernel,
>> e.g. useless for cross-compilation. There, it would be possible to
>> read the kernel configuration file if the kernel source code is
>> available when the userspace package is being built, but that's not
>> guaranteed.
>>
>> In Gentoo, linux-info.eclass provides linux_config_exists() to do all
>> of this, but order to become a widespread success there would have to
>> be one method for upstreams to maintain these requirements as part of
>> their packages, rather than forcing the burden on package maintainers
>> to repeat the same detective task in every single distribution.
>>
>> I think it would be very useful to create something generic for that,
>> but that's certainly no small task.
>>
>> And realistically I only see it succeeding if Linux Foundation decides
>> to push it onto the world.
> 
> Sorry, I may have expressed this not clearly. I guess, we actually have
> the same opinion here. I don't want to do this mapping automatically. I
> don't want a "fully automatic" config or touch any (user space) ebuilds.
> 
> My idea is to patch Kconfig (as part of gentoo-sources) to provide the
> same "features" (i.e. mappings) that are already present in the Gentoo
> wiki.
> 
> For example this could result in this installation description for Docker
> (within the Wiki):
> "To enable kernel support for Docker, enable
> 'Gentoo Linux -> Support for user space programs -> Docker'"
> 
> This Kconfig flag than depends on all options that are needed for proper
> Docker support (as already described in the Wiki) like cgroups etc..
> 
> This would allow users to configure there kernels the same way as before
> but with some additional convenience shortcuts.
> 
> Of course, this is a high maintenance burden for the kernel package
> maintainers. Therefore, this mechanism maybe can be automated:
> Grep the wiki pages for kernel config snippets and automatically
> condense that in a Kconfig readable configuration option.
> 
> 
>>> A possible way could be to automatically extract the kernel config
>>> flags from the wiki pages and map them to Kconfig options.
>>
>> At very best that will only be valid for some particular point in time,
>> like current CONFIG_CHECK in ebuilds using linux_config_exists() are
>> only valid for particular package versions. At worst it's plain wrong
>> because the requirements have to be reverse-engineered downstream.
> 
> Of course, the already existing kernel config snippets in the Wiki are
> best effort. My idea is to combine these snippets directly with Kconfig.
> 
> 
> Best regards,
> Gerion
> 
> [1] https://vamos.informatik.uni-erlangen.de/trac/undertaker/wiki/UndertakerTailor
> 


Hello,

If I am reading this correctly, you are looking for something like we 
have for systemd and openrc where you select something and we preselect 
a bunch of required/suggested kernel configs options.

I'm happy to add this to gentoo-sources for Docker based on the 
expertise of the gentoo developer individuals in the project.

Probably a bug to track this would be the best way for me to implement.

Or am I totally off base on your request?

Mike

Re: [gentoo-dev] Idea: User centric kernel configuration

[Gerion Entrup]

[-- Attachment #1: Type: text/plain, Size: 6674 bytes --]

Am Freitag, 12. März 2021, 17:40:02 CET schrieb Mike Pagano:
> 
> On 3/12/21 11:34 AM, Gerion Entrup wrote:
> > Am Donnerstag, 11. März 2021, 17:41:45 CET schrieb Peter Stuge:
> >> Hi,
> >>
> >> Gerion Entrup wrote:
> >>> the Linux kernel has _a lot of_ configuration options, way too many to configure them by hand.
> >>
> >> I actually disagree strongly with that; I think it's important to
> >> actively decide what kernels include, and I routinely do, but of
> >> course not everyone will. I've made sure to include a kernel build
> >> when teaching systems administration courses and would again.
> >>
> >> As the kernel becomes more complex the threshold for the first
> >> configuration also rises, but it's still completely possible to learn
> >> what you need in order to successfully configure your own kernel.
> >> I guess it's on the order of a weekend project given some basic
> >> understanding of computer architecture and programming.
> > 
> > I think, we mean two different things here. I for myself also configured
> > my own kernel(s) for years. I also actively teach students to do so.
> > However, I have never looked into all 18000 configuration options. I
> > don't understand them all (by far).
> > 
> > Actually, most of the times, I do a `make localmodconfig`, click through
> > various subsystems and activate what I think that it is useful or sounds
> > nice. Then, to update a kernel, I use `make oldconfig` and answer the
> > questions as good as I can (taking the default otherwise).
> > 
> > If my kernel is not capable to do something (for example run docker
> > containers), I take a look in the Gentoo Wiki and "copy" the options
> > into my own config.
> > 
> > But for me, this is not an informed decision. Of course, I learn
> > something in this process, maybe also more than a precompiled kernel
> > user, but I have by far not created a minimal config or even begin
> > to understand all subsystems and different configuration options.
> > 
> > I'm sure that this is similar to most Gentoo users.
> > 
> > BTW, here is a project to create a minimal config (kind of "ideal"
> > config) based on a "golden run" [1]. However, I have not compared such
> > a config with my own homegrown config.
> > 
> > 
> >>> This requires a mapping between user oriented "features" and the kernel
> >>> internal configuration options.
> >>
> >> So the challenge here is that the kernel is disjoint from user space,
> >> and while the kernel API remains stable over time consumer requirements
> >> such as "docker" or "cryptsetup" will mean different things for
> >> different versions of particular user space software.
> >>
> >>
> >>> Do you think that it is useful and feasible to combine these two mechanisms?
> >>
> >> AFAIK there's no generic method for formal kernel requirements in user
> >> space packages and there's also no sanctioned method for quering
> >> kernel capabilities. The only thing available is /proc/config if that
> >> was enabled in the kernel build, and there are of course reasons to
> >> leave it out, and it only applies to the particular running kernel,
> >> e.g. useless for cross-compilation. There, it would be possible to
> >> read the kernel configuration file if the kernel source code is
> >> available when the userspace package is being built, but that's not
> >> guaranteed.
> >>
> >> In Gentoo, linux-info.eclass provides linux_config_exists() to do all
> >> of this, but order to become a widespread success there would have to
> >> be one method for upstreams to maintain these requirements as part of
> >> their packages, rather than forcing the burden on package maintainers
> >> to repeat the same detective task in every single distribution.
> >>
> >> I think it would be very useful to create something generic for that,
> >> but that's certainly no small task.
> >>
> >> And realistically I only see it succeeding if Linux Foundation decides
> >> to push it onto the world.
> > 
> > Sorry, I may have expressed this not clearly. I guess, we actually have
> > the same opinion here. I don't want to do this mapping automatically. I
> > don't want a "fully automatic" config or touch any (user space) ebuilds.
> > 
> > My idea is to patch Kconfig (as part of gentoo-sources) to provide the
> > same "features" (i.e. mappings) that are already present in the Gentoo
> > wiki.
> > 
> > For example this could result in this installation description for Docker
> > (within the Wiki):
> > "To enable kernel support for Docker, enable
> > 'Gentoo Linux -> Support for user space programs -> Docker'"
> > 
> > This Kconfig flag than depends on all options that are needed for proper
> > Docker support (as already described in the Wiki) like cgroups etc..
> > 
> > This would allow users to configure there kernels the same way as before
> > but with some additional convenience shortcuts.
> > 
> > Of course, this is a high maintenance burden for the kernel package
> > maintainers. Therefore, this mechanism maybe can be automated:
> > Grep the wiki pages for kernel config snippets and automatically
> > condense that in a Kconfig readable configuration option.
> > 
> > 
> >>> A possible way could be to automatically extract the kernel config
> >>> flags from the wiki pages and map them to Kconfig options.
> >>
> >> At very best that will only be valid for some particular point in time,
> >> like current CONFIG_CHECK in ebuilds using linux_config_exists() are
> >> only valid for particular package versions. At worst it's plain wrong
> >> because the requirements have to be reverse-engineered downstream.
> > 
> > Of course, the already existing kernel config snippets in the Wiki are
> > best effort. My idea is to combine these snippets directly with Kconfig.
> > 
> > 
> > Best regards,
> > Gerion
> > 
> > [1] https://vamos.informatik.uni-erlangen.de/trac/undertaker/wiki/UndertakerTailor
> > 
> 
> 
> Hello,
> 
> If I am reading this correctly, you are looking for something like we 
> have for systemd and openrc where you select something and we preselect 
> a bunch of required/suggested kernel configs options.
> 
> I'm happy to add this to gentoo-sources for Docker based on the 
> expertise of the gentoo developer individuals in the project.
> 
> Probably a bug to track this would be the best way for me to implement.
> 
> Or am I totally off base on your request?

You are right, but Docker is just an example. Ideally, this would exist
for every wiki page that explains which kernel config options need to be
set to make XY possible.

Gerion

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: [gentoo-dev] Idea: User centric kernel configuration

[Mike Pagano]

On 3/12/21 1:04 PM, Gerion Entrup wrote:
> Am Freitag, 12. März 2021, 17:40:02 CET schrieb Mike Pagano:
>>
>> On 3/12/21 11:34 AM, Gerion Entrup wrote:
>>> Am Donnerstag, 11. März 2021, 17:41:45 CET schrieb Peter Stuge:
>>>> Hi,
>>>>
>>>> Gerion Entrup wrote:
>>>>> the Linux kernel has _a lot of_ configuration options, way too many to configure them by hand.
>>>>
>>>> I actually disagree strongly with that; I think it's important to
>>>> actively decide what kernels include, and I routinely do, but of
>>>> course not everyone will. I've made sure to include a kernel build
>>>> when teaching systems administration courses and would again.
>>>>
>>>> As the kernel becomes more complex the threshold for the first
>>>> configuration also rises, but it's still completely possible to learn
>>>> what you need in order to successfully configure your own kernel.
>>>> I guess it's on the order of a weekend project given some basic
>>>> understanding of computer architecture and programming.
>>>
>>> I think, we mean two different things here. I for myself also configured
>>> my own kernel(s) for years. I also actively teach students to do so.
>>> However, I have never looked into all 18000 configuration options. I
>>> don't understand them all (by far).
>>>
>>> Actually, most of the times, I do a `make localmodconfig`, click through
>>> various subsystems and activate what I think that it is useful or sounds
>>> nice. Then, to update a kernel, I use `make oldconfig` and answer the
>>> questions as good as I can (taking the default otherwise).
>>>
>>> If my kernel is not capable to do something (for example run docker
>>> containers), I take a look in the Gentoo Wiki and "copy" the options
>>> into my own config.
>>>
>>> But for me, this is not an informed decision. Of course, I learn
>>> something in this process, maybe also more than a precompiled kernel
>>> user, but I have by far not created a minimal config or even begin
>>> to understand all subsystems and different configuration options.
>>>
>>> I'm sure that this is similar to most Gentoo users.
>>>
>>> BTW, here is a project to create a minimal config (kind of "ideal"
>>> config) based on a "golden run" [1]. However, I have not compared such
>>> a config with my own homegrown config.
>>>
>>>
>>>>> This requires a mapping between user oriented "features" and the kernel
>>>>> internal configuration options.
>>>>
>>>> So the challenge here is that the kernel is disjoint from user space,
>>>> and while the kernel API remains stable over time consumer requirements
>>>> such as "docker" or "cryptsetup" will mean different things for
>>>> different versions of particular user space software.
>>>>
>>>>
>>>>> Do you think that it is useful and feasible to combine these two mechanisms?
>>>>
>>>> AFAIK there's no generic method for formal kernel requirements in user
>>>> space packages and there's also no sanctioned method for quering
>>>> kernel capabilities. The only thing available is /proc/config if that
>>>> was enabled in the kernel build, and there are of course reasons to
>>>> leave it out, and it only applies to the particular running kernel,
>>>> e.g. useless for cross-compilation. There, it would be possible to
>>>> read the kernel configuration file if the kernel source code is
>>>> available when the userspace package is being built, but that's not
>>>> guaranteed.
>>>>
>>>> In Gentoo, linux-info.eclass provides linux_config_exists() to do all
>>>> of this, but order to become a widespread success there would have to
>>>> be one method for upstreams to maintain these requirements as part of
>>>> their packages, rather than forcing the burden on package maintainers
>>>> to repeat the same detective task in every single distribution.
>>>>
>>>> I think it would be very useful to create something generic for that,
>>>> but that's certainly no small task.
>>>>
>>>> And realistically I only see it succeeding if Linux Foundation decides
>>>> to push it onto the world.
>>>
>>> Sorry, I may have expressed this not clearly. I guess, we actually have
>>> the same opinion here. I don't want to do this mapping automatically. I
>>> don't want a "fully automatic" config or touch any (user space) ebuilds.
>>>
>>> My idea is to patch Kconfig (as part of gentoo-sources) to provide the
>>> same "features" (i.e. mappings) that are already present in the Gentoo
>>> wiki.
>>>
>>> For example this could result in this installation description for Docker
>>> (within the Wiki):
>>> "To enable kernel support for Docker, enable
>>> 'Gentoo Linux -> Support for user space programs -> Docker'"
>>>
>>> This Kconfig flag than depends on all options that are needed for proper
>>> Docker support (as already described in the Wiki) like cgroups etc..
>>>
>>> This would allow users to configure there kernels the same way as before
>>> but with some additional convenience shortcuts.
>>>
>>> Of course, this is a high maintenance burden for the kernel package
>>> maintainers. Therefore, this mechanism maybe can be automated:
>>> Grep the wiki pages for kernel config snippets and automatically
>>> condense that in a Kconfig readable configuration option.
>>>
>>>
>>>>> A possible way could be to automatically extract the kernel config
>>>>> flags from the wiki pages and map them to Kconfig options.
>>>>
>>>> At very best that will only be valid for some particular point in time,
>>>> like current CONFIG_CHECK in ebuilds using linux_config_exists() are
>>>> only valid for particular package versions. At worst it's plain wrong
>>>> because the requirements have to be reverse-engineered downstream.
>>>
>>> Of course, the already existing kernel config snippets in the Wiki are
>>> best effort. My idea is to combine these snippets directly with Kconfig.
>>>
>>>
>>> Best regards,
>>> Gerion
>>>
>>> [1] https://vamos.informatik.uni-erlangen.de/trac/undertaker/wiki/UndertakerTailor
>>>
>>
>>
>> Hello,
>>
>> If I am reading this correctly, you are looking for something like we
>> have for systemd and openrc where you select something and we preselect
>> a bunch of required/suggested kernel configs options.
>>
>> I'm happy to add this to gentoo-sources for Docker based on the
>> expertise of the gentoo developer individuals in the project.
>>
>> Probably a bug to track this would be the best way for me to implement.
>>
>> Or am I totally off base on your request?
> 
> You are right, but Docker is just an example. Ideally, this would exist
> for every wiki page that explains which kernel config options need to be
> set to make XY possible.
> 
> Gerion
> 

Specific Request -> Bug -> Discussion -> Implementation -> Release

I think we can move past generics and get specific.

Re: [gentoo-dev] Idea: User centric kernel configuration

[Hans de Graaff]

[-- Attachment #1: Type: text/plain, Size: 420 bytes --]

On Fri, 2021-03-12 at 11:40 -0500, Mike Pagano wrote:
> 
> have for systemd and openrc where you select something and we preselect
> a bunch of required/suggested kernel configs options.

Another good candidate for this mechanism would be the recommendations
on
https://kernsec.org/wiki/index.php/Kernel_Self_Protection_Project/Recommended_Settings
perhaps linked to a hardened USE flag and/or profile.

Hans

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [gentoo-dev] Idea: User centric kernel configuration

[Gerion Entrup]

[-- Attachment #1: Type: text/plain, Size: 7254 bytes --]

Am Freitag, 12. März 2021, 19:28:51 CET schrieb Mike Pagano:
> 
> On 3/12/21 1:04 PM, Gerion Entrup wrote:
> > Am Freitag, 12. März 2021, 17:40:02 CET schrieb Mike Pagano:
> >>
> >> On 3/12/21 11:34 AM, Gerion Entrup wrote:
> >>> Am Donnerstag, 11. März 2021, 17:41:45 CET schrieb Peter Stuge:
> >>>> Hi,
> >>>>
> >>>> Gerion Entrup wrote:
> >>>>> the Linux kernel has _a lot of_ configuration options, way too many to configure them by hand.
> >>>>
> >>>> I actually disagree strongly with that; I think it's important to
> >>>> actively decide what kernels include, and I routinely do, but of
> >>>> course not everyone will. I've made sure to include a kernel build
> >>>> when teaching systems administration courses and would again.
> >>>>
> >>>> As the kernel becomes more complex the threshold for the first
> >>>> configuration also rises, but it's still completely possible to learn
> >>>> what you need in order to successfully configure your own kernel.
> >>>> I guess it's on the order of a weekend project given some basic
> >>>> understanding of computer architecture and programming.
> >>>
> >>> I think, we mean two different things here. I for myself also configured
> >>> my own kernel(s) for years. I also actively teach students to do so.
> >>> However, I have never looked into all 18000 configuration options. I
> >>> don't understand them all (by far).
> >>>
> >>> Actually, most of the times, I do a `make localmodconfig`, click through
> >>> various subsystems and activate what I think that it is useful or sounds
> >>> nice. Then, to update a kernel, I use `make oldconfig` and answer the
> >>> questions as good as I can (taking the default otherwise).
> >>>
> >>> If my kernel is not capable to do something (for example run docker
> >>> containers), I take a look in the Gentoo Wiki and "copy" the options
> >>> into my own config.
> >>>
> >>> But for me, this is not an informed decision. Of course, I learn
> >>> something in this process, maybe also more than a precompiled kernel
> >>> user, but I have by far not created a minimal config or even begin
> >>> to understand all subsystems and different configuration options.
> >>>
> >>> I'm sure that this is similar to most Gentoo users.
> >>>
> >>> BTW, here is a project to create a minimal config (kind of "ideal"
> >>> config) based on a "golden run" [1]. However, I have not compared such
> >>> a config with my own homegrown config.
> >>>
> >>>
> >>>>> This requires a mapping between user oriented "features" and the kernel
> >>>>> internal configuration options.
> >>>>
> >>>> So the challenge here is that the kernel is disjoint from user space,
> >>>> and while the kernel API remains stable over time consumer requirements
> >>>> such as "docker" or "cryptsetup" will mean different things for
> >>>> different versions of particular user space software.
> >>>>
> >>>>
> >>>>> Do you think that it is useful and feasible to combine these two mechanisms?
> >>>>
> >>>> AFAIK there's no generic method for formal kernel requirements in user
> >>>> space packages and there's also no sanctioned method for quering
> >>>> kernel capabilities. The only thing available is /proc/config if that
> >>>> was enabled in the kernel build, and there are of course reasons to
> >>>> leave it out, and it only applies to the particular running kernel,
> >>>> e.g. useless for cross-compilation. There, it would be possible to
> >>>> read the kernel configuration file if the kernel source code is
> >>>> available when the userspace package is being built, but that's not
> >>>> guaranteed.
> >>>>
> >>>> In Gentoo, linux-info.eclass provides linux_config_exists() to do all
> >>>> of this, but order to become a widespread success there would have to
> >>>> be one method for upstreams to maintain these requirements as part of
> >>>> their packages, rather than forcing the burden on package maintainers
> >>>> to repeat the same detective task in every single distribution.
> >>>>
> >>>> I think it would be very useful to create something generic for that,
> >>>> but that's certainly no small task.
> >>>>
> >>>> And realistically I only see it succeeding if Linux Foundation decides
> >>>> to push it onto the world.
> >>>
> >>> Sorry, I may have expressed this not clearly. I guess, we actually have
> >>> the same opinion here. I don't want to do this mapping automatically. I
> >>> don't want a "fully automatic" config or touch any (user space) ebuilds.
> >>>
> >>> My idea is to patch Kconfig (as part of gentoo-sources) to provide the
> >>> same "features" (i.e. mappings) that are already present in the Gentoo
> >>> wiki.
> >>>
> >>> For example this could result in this installation description for Docker
> >>> (within the Wiki):
> >>> "To enable kernel support for Docker, enable
> >>> 'Gentoo Linux -> Support for user space programs -> Docker'"
> >>>
> >>> This Kconfig flag than depends on all options that are needed for proper
> >>> Docker support (as already described in the Wiki) like cgroups etc..
> >>>
> >>> This would allow users to configure there kernels the same way as before
> >>> but with some additional convenience shortcuts.
> >>>
> >>> Of course, this is a high maintenance burden for the kernel package
> >>> maintainers. Therefore, this mechanism maybe can be automated:
> >>> Grep the wiki pages for kernel config snippets and automatically
> >>> condense that in a Kconfig readable configuration option.
> >>>
> >>>
> >>>>> A possible way could be to automatically extract the kernel config
> >>>>> flags from the wiki pages and map them to Kconfig options.
> >>>>
> >>>> At very best that will only be valid for some particular point in time,
> >>>> like current CONFIG_CHECK in ebuilds using linux_config_exists() are
> >>>> only valid for particular package versions. At worst it's plain wrong
> >>>> because the requirements have to be reverse-engineered downstream.
> >>>
> >>> Of course, the already existing kernel config snippets in the Wiki are
> >>> best effort. My idea is to combine these snippets directly with Kconfig.
> >>>
> >>>
> >>> Best regards,
> >>> Gerion
> >>>
> >>> [1] https://vamos.informatik.uni-erlangen.de/trac/undertaker/wiki/UndertakerTailor
> >>>
> >>
> >>
> >> Hello,
> >>
> >> If I am reading this correctly, you are looking for something like we
> >> have for systemd and openrc where you select something and we preselect
> >> a bunch of required/suggested kernel configs options.
> >>
> >> I'm happy to add this to gentoo-sources for Docker based on the
> >> expertise of the gentoo developer individuals in the project.
> >>
> >> Probably a bug to track this would be the best way for me to implement.
> >>
> >> Or am I totally off base on your request?
> > 
> > You are right, but Docker is just an example. Ideally, this would exist
> > for every wiki page that explains which kernel config options need to be
> > set to make XY possible.
> > 
> > Gerion
> > 
> 
> Specific Request -> Bug -> Discussion -> Implementation -> Release
> 
> I think we can move past generics and get specific.

Done: https://bugs.gentoo.org/775956

Gerion

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 659 bytes --]