[gentoo-dev] rfc: generic "container" keyword for init scripts

[gentoo-dev] rfc: generic "container" keyword for init scripts

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 872 bytes --]

All,

it has been proposed to me that we create a "container" keyword for
OpenRC init scripts, which would take the place of the various keywords
we have currently for separate container types, such as lxc,
systemd-nspawn, etc (basically the values RC_SYS can be set to).
We would keep all of the current keywords for backward compatibility,
but they would all become synonyms for container.
The auto detection code would be updated to test for all supported
container types at once.

The down side of this would be that we would lose the ability to pick
specific containers that a service could not run in; it would have to
run in no containers or all containers.

I don't know of specific services which can run in one container but not
in another. However, if these types of services do exist, we can't make
this change.

Let me know what you think.

Thanks,

William


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] rfc: generic "container" keyword for init scripts

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 1928 bytes --]

A quick grep in the tree shows the following existing cases (wrapped for line
length).

$ git grep -E \
 'keyword .*(lxc|systemd-nspawn|vserver|prefix|openvz|uml|jail|xen)'
sys-apps/kmod/files/kmod-static-nodes-r1:	
 keyword -lxc -systemd-nspawn
sys-fs/eudev/files/udev-postmount:	
 keyword -vserver -lxc
sys-fs/zfs/files/zfs-0.6.1-gentoo-openrc-dependencies.patch: 	
 keyword -lxc -openvz -prefix -vserver
sys-fs/zfs/files/zfs-0.6.5-fix-openrc-scripts.patch: 	
 keyword -lxc -openvz -prefix -vserver
sys-power/nut/files/nut.powerfail.initd:		
 keyword -jail -lxc -openvz -prefix -uml -vserver -xenu -timeout

The only one of those I can speak knowledgeably about is sys-power/nut.
That script signals the UPS to cut the power, and is intended to be the very
very last shutdown script ever called on a box (so it has a sleep forever bit
at the end). 

Does it make sense in a container? That depends on how the container is
configured... it's got the right details for the container parent's UPS, then
you certainly don't want it to fire. If it had details to tell the parent to
just terminate it, then it would be correct.

Rather than replacing all of the system-specific keywords, can we make
'container' into an alias that expands to the full list of known container
types? That way, if there is an init script that is specific to some container
type, it would still be usable.

One hypothetical case that does come to mind is some container-type-specific setup script.
Say there is 'foocontainerFS' and it requires special mount parameters during
the startup of the container, or the container wants some special daemon to be
running... both of these cases could use:
keywords -containers foocontainer

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Infrastructure Lead, Foundation Trustee
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 445 bytes --]

Re: [gentoo-dev] rfc: generic "container" keyword for init scripts

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 1621 bytes --]

On Tue, Dec 01, 2015 at 12:24:05AM +0000, Robin H. Johnson wrote:
> A quick grep in the tree shows the following existing cases (wrapped for line
> length).
> 
> $ git grep -E \
>  'keyword .*(lxc|systemd-nspawn|vserver|prefix|openvz|uml|jail|xen)'
> sys-apps/kmod/files/kmod-static-nodes-r1:	
>  keyword -lxc -systemd-nspawn
> sys-fs/eudev/files/udev-postmount:	
>  keyword -vserver -lxc
> sys-fs/zfs/files/zfs-0.6.1-gentoo-openrc-dependencies.patch: 	
>  keyword -lxc -openvz -prefix -vserver
> sys-fs/zfs/files/zfs-0.6.5-fix-openrc-scripts.patch: 	
>  keyword -lxc -openvz -prefix -vserver
> sys-power/nut/files/nut.powerfail.initd:		
>  keyword -jail -lxc -openvz -prefix -uml -vserver -xenu -timeout
> 
> The only one of those I can speak knowledgeably about is sys-power/nut.
> That script signals the UPS to cut the power, and is intended to be the very
> very last shutdown script ever called on a box (so it has a sleep forever bit
> at the end). 
> 
> Does it make sense in a container? That depends on how the container is
> configured... it's got the right details for the container parent's UPS, then
> you certainly don't want it to fire. If it had details to tell the parent to
> just terminate it, then it would be correct.
> 
> Rather than replacing all of the system-specific keywords, can we make
> 'container' into an alias that expands to the full list of known container
> types? That way, if there is an init script that is specific to some container
> type, it would still be usable.

This is the approach I would personally be more comfortable with.

William


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] rfc: generic "container" keyword for init scripts

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 1046 bytes --]

On Tue, Dec 01, 2015 at 09:11:45AM -0600, William Hubbs wrote:
> On Tue, Dec 01, 2015 at 12:24:05AM +0000, Robin H. Johnson wrote:
> > Rather than replacing all of the system-specific keywords, can we make
> > 'container' into an alias that expands to the full list of known container
> > types? That way, if there is an init script that is specific to some container
> > type, it would still be usable.
> 
> This is the approach I would personally be more comfortable with.

Thinking about this more, there definitely is a concern imo.

The "container" keyword, being generic, would have its meaning expanded
to cover new container systems as they come along. This means if a
service script has the keyword "-container" it will not work under any
current or future container systems. On the other hand, adding
"container" means it will work under all of them.

The more I think about this, I think fine-grained control rather than
just using a single keyword for all containers is the better option.

Thoughts?

William


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] rfc: generic "container" keyword for init scripts

[Zac Medico]

On 12/01/2015 08:50 AM, William Hubbs wrote:
> The "container" keyword, being generic, would have its meaning expanded
> to cover new container systems as they come along. This means if a
> service script has the keyword "-container" it will not work under any
> current or future container systems. On the other hand, adding
> "container" means it will work under all of them.

If it's similar to how license groups work, then there is room for doing
things like "-@container docker" which means no containers except
docker, or "@container -docker" which means all containers except
docker. Keyword groups can be implemented using a simple expansion
mechanism, just like license groups.

> The more I think about this, I think fine-grained control rather than
> just using a single keyword for all containers is the better option.

Keyword groups, as described above, result in a flexible combination of
both approaches.
-- 
Thanks,
Zac

Re: [gentoo-dev] rfc: generic "container" keyword for init scripts

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 874 bytes --]

On Tue, Dec 01, 2015 at 09:00:55AM -0800, Zac Medico wrote:
> On 12/01/2015 08:50 AM, William Hubbs wrote:
> > The "container" keyword, being generic, would have its meaning expanded
> > to cover new container systems as they come along. This means if a
> > service script has the keyword "-container" it will not work under any
> > current or future container systems. On the other hand, adding
> > "container" means it will work under all of them.
> 
> If it's similar to how license groups work, then there is room for doing
> things like "-@container docker" which means no containers except
> docker, or "@container -docker" which means all containers except
> docker. Keyword groups can be implemented using a simple expansion
> mechanism, just like license groups.
 
Keyword groups are an interesting idea; I'll think about this approach.

William


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] rfc: generic "container" keyword for init scripts

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 1212 bytes --]

On Tue, Dec 01, 2015 at 11:26:19AM -0600, William Hubbs wrote:
> On Tue, Dec 01, 2015 at 09:00:55AM -0800, Zac Medico wrote:
> > On 12/01/2015 08:50 AM, William Hubbs wrote:
> > > The "container" keyword, being generic, would have its meaning expanded
> > > to cover new container systems as they come along. This means if a
> > > service script has the keyword "-container" it will not work under any
> > > current or future container systems. On the other hand, adding
> > > "container" means it will work under all of them.
> > 
> > If it's similar to how license groups work, then there is room for doing
> > things like "-@container docker" which means no containers except
> > docker, or "@container -docker" which means all containers except
> > docker. Keyword groups can be implemented using a simple expansion
> > mechanism, just like license groups.
>  
> Keyword groups are an interesting idea; I'll think about this approach.

One question that comes to mind is,, who defines which keywords go in
each group?

The containers or virtualization systems themselves can be autodetected
so they are the same everywhere, but keyword groups are not able to be
detected.

William


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] rfc: generic "container" keyword for init scripts

[Zac Medico]

On 12/01/2015 10:19 AM, William Hubbs wrote:
> On Tue, Dec 01, 2015 at 11:26:19AM -0600, William Hubbs wrote:
>> On Tue, Dec 01, 2015 at 09:00:55AM -0800, Zac Medico wrote:
>>> On 12/01/2015 08:50 AM, William Hubbs wrote:
>>>> The "container" keyword, being generic, would have its meaning expanded
>>>> to cover new container systems as they come along. This means if a
>>>> service script has the keyword "-container" it will not work under any
>>>> current or future container systems. On the other hand, adding
>>>> "container" means it will work under all of them.
>>>
>>> If it's similar to how license groups work, then there is room for doing
>>> things like "-@container docker" which means no containers except
>>> docker, or "@container -docker" which means all containers except
>>> docker. Keyword groups can be implemented using a simple expansion
>>> mechanism, just like license groups.
>>  
>> Keyword groups are an interesting idea; I'll think about this approach.
> 
> One question that comes to mind is,, who defines which keywords go in
> each group?
> 
> The containers or virtualization systems themselves can be autodetected
> so they are the same everywhere, but keyword groups are not able to be
> detected.

The groups should go in a configuration file somewhere. For example,
they could be defined in rc.conf with a variable setting like this:

rc_keyword_group_container="docker lxc openvz vserver"
-- 
Thanks,
Zac

Re: [gentoo-dev] rfc: generic "container" keyword for init scripts

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 2024 bytes --]

On Tue, Dec 01, 2015 at 11:21:49AM -0800, Zac Medico wrote:
> On 12/01/2015 10:19 AM, William Hubbs wrote:
> > On Tue, Dec 01, 2015 at 11:26:19AM -0600, William Hubbs wrote:
> >> On Tue, Dec 01, 2015 at 09:00:55AM -0800, Zac Medico wrote:
> >>> On 12/01/2015 08:50 AM, William Hubbs wrote:
> >>>> The "container" keyword, being generic, would have its meaning expanded
> >>>> to cover new container systems as they come along. This means if a
> >>>> service script has the keyword "-container" it will not work under any
> >>>> current or future container systems. On the other hand, adding
> >>>> "container" means it will work under all of them.
> >>>
> >>> If it's similar to how license groups work, then there is room for doing
> >>> things like "-@container docker" which means no containers except
> >>> docker, or "@container -docker" which means all containers except
> >>> docker. Keyword groups can be implemented using a simple expansion
> >>> mechanism, just like license groups.
> >>  
> >> Keyword groups are an interesting idea; I'll think about this approach.
> > 
> > One question that comes to mind is,, who defines which keywords go in
> > each group?
> > 
> > The containers or virtualization systems themselves can be autodetected
> > so they are the same everywhere, but keyword groups are not able to be
> > detected.
> 
> The groups should go in a configuration file somewhere. For example,
> they could be defined in rc.conf with a variable setting like this:
> 
> rc_keyword_group_container="docker lxc openvz vserver"

If we keep the keywords as they are, I am starting to think we can just
add a container keyword that would be valid for all of the containers;
that way a script could be set up to work for no containers or for
specific ones like we are talking about.

OpenRC currently doesn't do any detection of virtual machines, so maybe
that could be another expansion of this -- add a keyword "vm" and
keywords for the virtual machines.

William


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] rfc: generic "container" keyword for init scripts

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 2615 bytes --]

On Tue, Dec 01, 2015 at 03:28:44PM -0600, William Hubbs wrote:
> On Tue, Dec 01, 2015 at 11:21:49AM -0800, Zac Medico wrote:
> > On 12/01/2015 10:19 AM, William Hubbs wrote:
> > > On Tue, Dec 01, 2015 at 11:26:19AM -0600, William Hubbs wrote:
> > >> On Tue, Dec 01, 2015 at 09:00:55AM -0800, Zac Medico wrote:
> > >>> On 12/01/2015 08:50 AM, William Hubbs wrote:
> > >>>> The "container" keyword, being generic, would have its meaning expanded
> > >>>> to cover new container systems as they come along. This means if a
> > >>>> service script has the keyword "-container" it will not work under any
> > >>>> current or future container systems. On the other hand, adding
> > >>>> "container" means it will work under all of them.
> > >>>
> > >>> If it's similar to how license groups work, then there is room for doing
> > >>> things like "-@container docker" which means no containers except
> > >>> docker, or "@container -docker" which means all containers except
> > >>> docker. Keyword groups can be implemented using a simple expansion
> > >>> mechanism, just like license groups.
> > >>  
> > >> Keyword groups are an interesting idea; I'll think about this approach.
> > > 
> > > One question that comes to mind is,, who defines which keywords go in
> > > each group?
> > > 
> > > The containers or virtualization systems themselves can be autodetected
> > > so they are the same everywhere, but keyword groups are not able to be
> > > detected.
> > 
> > The groups should go in a configuration file somewhere. For example,
> > they could be defined in rc.conf with a variable setting like this:
> > 
> > rc_keyword_group_container="docker lxc openvz vserver"
> 
> If we keep the keywords as they are, I am starting to think we can just
> add a container keyword that would be valid for all of the containers;
> that way a script could be set up to work for no containers or for
> specific ones like we are talking about.
 
Here is where my thoughts are on this now. There will be a
"-containers" keyword, which will expand to list all of the containers
 we can detect on the operating system OpenRC is running on -- for
 example, on Linux it might be:

 "-docker -lxc -openvz -rkt -systemd-nspawn -vserver"

 So any time you list -containers as a keyword it would expand
 appropriately.

> OpenRC currently doesn't do any detection of virtual machines, so maybe
> that could be another expansion of this -- add a keyword "vm" and
> keywords for the virtual machines.
 
 VM's will be a separate project that I will look at later.

Thoughts?

William

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] rfc: generic "container" keyword for init scripts

[Zac Medico]

On 12/02/2015 03:49 PM, William Hubbs wrote:
> On Tue, Dec 01, 2015 at 03:28:44PM -0600, William Hubbs wrote:
> Here is where my thoughts are on this now. There will be a
> "-containers" keyword, which will expand to list all of the containers
>  we can detect on the operating system OpenRC is running on -- for
>  example, on Linux it might be:
> 
>  "-docker -lxc -openvz -rkt -systemd-nspawn -vserver"
> 
>  So any time you list -containers as a keyword it would expand
>  appropriately.

Sounds good.

>> OpenRC currently doesn't do any detection of virtual machines, so maybe
>> that could be another expansion of this -- add a keyword "vm" and
>> keywords for the virtual machines.
>  
>  VM's will be a separate project that I will look at later.
> 
> Thoughts?

How about a bare-metal keyword?
-- 
Thanks,
Zac

Re: [gentoo-dev] rfc: generic "container" keyword for init scripts

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 1244 bytes --]

On Wed, Dec 02, 2015 at 10:20:33PM -0800, Zac Medico wrote:
> On 12/02/2015 03:49 PM, William Hubbs wrote:
> > On Tue, Dec 01, 2015 at 03:28:44PM -0600, William Hubbs wrote:
> > Here is where my thoughts are on this now. There will be a
> > "-containers" keyword, which will expand to list all of the containers
> >  we can detect on the operating system OpenRC is running on -- for
> >  example, on Linux it might be:
> > 
> >  "-docker -lxc -openvz -rkt -systemd-nspawn -vserver"
> > 
> >  So any time you list -containers as a keyword it would expand
> >  appropriately.
> 
> Sounds good.
> 
> >> OpenRC currently doesn't do any detection of virtual machines, so maybe
> >> that could be another expansion of this -- add a keyword "vm" and
> >> keywords for the virtual machines.
> >  
> >  VM's will be a separate project that I will look at later.
> > 
> > Thoughts?
> 
> How about a bare-metal keyword?

In OpenRC, the keywords block services from running in the listed
environments, so once I get the containers and vms implemented, bare
metal would be something like:

keyword -containers -vms

the idea being that if it isn't a container or vm that I know how to
detect, it is bare metal.

William


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]