[gentoo-user] openrc-run for containers

[gentoo-user] openrc-run for containers

[Damo]

[-- Attachment #1: Type: text/plain, Size: 1253 bytes --]

Hi,

I've been running docker containers for a while, where I pass
"--restart=always" into the run command, so the containers restart
automatically after reboot. I want to have more control over the startup
order of the containers, ie integrate into openrc start/stop and put into
different runlevels.

I've had mixed success so far. I would be interested if someone else has
working solution. My runlevels look something like this:

rl100
  container.registry
rl90
  container.auth
  container.router
boot
 ...

FYI, i've found systemd is doing it nicely, where systemctl start/stop
<CONTAINER> works as I would expect. I see a hardcoded dependency into the
container PID in the unit file (podman in this case):

[root@]# cat /usr/lib/systemd/system/container-libvirt-exporter.service

#

[Unit]
Description=Podman container-libvirtd-exporter.service
Documentation=man:podman-generate-systemd(1)

[Service]
Restart=always
ExecStart=/usr/bin/podman start libvirtd-exporter
ExecStop=/usr/bin/podman stop -t 10 libvirtd-exporter
KillMode=none
Type=forking
PIDFile=/var/run/containers/storage/overlay-containers/9037e389e61ed01eb5dfce16fa750b6f0f01827a67640e4748e6527bbfcb6276/userdata/conmon.pid

[Install]
WantedBy=multi-user.target


Kind regards,
Damo

[-- Attachment #2: Type: text/html, Size: 1534 bytes --]

[gentoo-user] Re: openrc-run for containers

[Damo]

[-- Attachment #1: Type: text/plain, Size: 5333 bytes --]

Hi,

I've got the integration working how I imagined. It is using s6-svscan to
manage the containers, hopefully not abusing something. General idea is to
have supervision control over containers, ie to allow start up in a given
runlevel, have the ability to manage dependency start between given
containers and start/stop using openrc (referenced
https://wiki.gentoo.org/wiki/S6).

Assuming containers are already running, ie given instance of whoami
container:

*docker run -p 80:80 -d -it --name whoami traefik/whoami*

The "run" script, /var/svc.d/whoami/run:

#!/bin/execlineb -P
exec docker start -a whoami

The "finish" script, /var/svc.d/whoami/finish


*#!/bin/execlineb -Ps6-permafailon 60 1 2 exit*

The init.d, conf.d. Cat /etc/conf.d/container.whoami:

*INSTANCE=whoami*

Cat /etc/init.d/container:












*#!/sbin/openrc-rundescription="A supervised test service with a
logger"supervisor=s6s6_service_path=/run/openrc/s6-scan/${INSTANCE}depend()
{   need s6-svscan}stop_pre() {  docker stop ${INSTANCE}}*

Finally, [openrc-run, ln -s /etc/init.d/container
/etc/init.d/container.whoami] /etc/initd.d/container.whoami start, stop
work as expected (docker ps |grep whoami does not return anything, after
running "/etc/init.d/container.whoami stop"):

List containers
root@ # *docker ps |grep whoami*
68bd2ed585ed   traefik/whoami
  "/whoami"                35 minutes ago   Up 34 minutes
0.0.0.0:80->80/tcp         whoami

root@ # *./container.whoami stop*
container.whoami       |whoami
container.whoami       | * Stopping container.whoami ...

 [ ok ]

root@ # *docker ps |grep whoami*

root@ #* ./container.whoami start*
container.whoami       | * Starting container.whoami ...

 [ ok ]
root@ # docker ps |grep whoami
68bd2ed585ed   traefik/whoami
  "/whoami"                35 minutes ago   Up 3 seconds
 0.0.0.0:80->80/tcp         whoami

root@ # *ps xf -o pid,ppid,pgrp,euser,args*
  PID  PPID  PGRP EUSER    COMMAND
21056     1 21056 root     /bin/s6-svscan /run/openrc/s6-scan
21058 21056 21056 root      \_ s6-supervise whoami/log
21059 21056 21056 root      \_ s6-supervise whoami
27584 21059 27584 root      |   \_ docker start -a whoami

Similar to above, using s6-svstat:

root@ # *s6-svstat /run/openrc/s6-scan/whoami*
up (pid 27584) 752 seconds

root@ # *./container.whoami stop*
container.whoami       |whoami
container.whoami       | * Stopping container.whoami ...

 [ ok ]

root@ #* s6-svstat /run/openrc/s6-scan/whoami*
down (exitcode 2) 1 seconds, normally up, ready 1 seconds

root@ #* ./container.whoami start*
container.whoami       | * Starting container.whoami ...

 [ ok ]
root@h003 /e/init.d # s6-svstat /run/openrc/s6-scan/whoami
up (pid 6722) 3 seconds

The goal of all this has been to incorporate containers into
/etc/runlevels. I am not sure if there is a better way, have I missed
something by not using s6-overlay or the like? I am not familiar enough to
know.

An outstanding issue, in the s6 run script, i would like to parametrize the
instance name, I don't know how to do it, as it is currently hard coded:

The "run" script, /var/svc.d/whoami/run:

#!/bin/execlineb -P
exec docker start -a *whoami*

But once that is done, then all running containers could be incorporated by
updating the conf.d for INSTANCE name, templating a /var/svc.d/<INSTANCE>
folder and linking to /etc/init.d/container, ie:

/etc/init.d/container.whoami
/etc/init.d/container.cadvisor
/etc/init.d/container.traefik

[put under control of s6]
/etc/runlevels/20/s6-svscan
...

This is kind of asymmetric, ie container start is in s6, whereas stop
[docker] is in openrc, but I am not seeing a different way, the goal is to
have robust services running. When system boots, s6-scan will start all the
containers automatically, but then further operations, ie for things like
manual failover etc, is possible using standard platform openrc -
effectively docker - start/stop commands.


kind regards




On Thu, May 13, 2021 at 5:17 AM Damo <dhatchett2@gmail.com> wrote:

> Hi,
>
> I've been running docker containers for a while, where I pass
> "--restart=always" into the run command, so the containers restart
> automatically after reboot. I want to have more control over the startup
> order of the containers, ie integrate into openrc start/stop and put into
> different runlevels.
>
> I've had mixed success so far. I would be interested if someone else has
> working solution. My runlevels look something like this:
>
> rl100
>   container.registry
> rl90
>   container.auth
>   container.router
> boot
>  ...
>
> FYI, i've found systemd is doing it nicely, where systemctl start/stop
> <CONTAINER> works as I would expect. I see a hardcoded dependency into the
> container PID in the unit file (podman in this case):
>
> [root@]# cat /usr/lib/systemd/system/container-libvirt-exporter.service
>
> #
>
> [Unit]
> Description=Podman container-libvirtd-exporter.service
> Documentation=man:podman-generate-systemd(1)
>
> [Service]
> Restart=always
> ExecStart=/usr/bin/podman start libvirtd-exporter
> ExecStop=/usr/bin/podman stop -t 10 libvirtd-exporter
> KillMode=none
> Type=forking
>
> PIDFile=/var/run/containers/storage/overlay-containers/9037e389e61ed01eb5dfce16fa750b6f0f01827a67640e4748e6527bbfcb6276/userdata/conmon.pid
>
> [Install]
> WantedBy=multi-user.target
>
>
> Kind regards,
> Damo
>
>

[-- Attachment #2: Type: text/html, Size: 7732 bytes --]

[gentoo-user] Re: openrc-run for containers

[Damo]

[-- Attachment #1: Type: text/plain, Size: 3367 bytes --]

Correction: the *s6_service_path* parameter in the parent init.d service,
/etc/init.d/container, needs to be changed from
/run/openrc/s6-scan/${INSTANCE} to /var/svc.d/${INSTANCE}





*#!/sbin/openrc-rundescription="A supervised test service with a
logger"supervisor=s6**s6_service_path=/var/svc.d/${INSTANCE}*

*depend() {*
*   need s6-svscan*
*}*

*stop_pre() {*
*  docker stop ${INSTANCE}*
*}*

NB; In runlevels, dont need to include s6-svscan. Only need to include the
service to start (s6-svscan will start as a service dependency).

Further info (go from runlevel 90 > 100 > back to 90):

root@/etc/runlevels #
* ls -ld node/*lrwxrwxrwx 1 root root 24 May 25 06:17 100/container.whoami
-> /etc/init.d/container.*whoami
*lrwxrwxrwx 1 root root 11 Jan 15 04:32 100/90 -> ../90*


*root@/etc/runlevels # openrc 100container.*whoami*      |/var/svc.d/*whoami
*container.*whoami*      | * Starting container.*whoami* ...*






*
                                                  [ ok ]root@h003
/e/runlevels # docker ps -aCONTAINER ID   IMAGE
                             COMMAND                  CREATED        STATUS
                       PORTS                      NAMES68bd2ed585ed
traefik/whoami                                                 "/whoami"
             25 hours ago   Up 1 minute                 0.0.0.0:80->80/tcp
        whoamiroot@/e/runlevels # openrc 90container.*whoami*
 |/var/svc.d/*whoami
*container.*whoami*      |*whoami
*container.*whoami*     | * Stopping container.*whoami* ...

                                [ ok ]*





*root@/etc/runlevels # docker ps -aCONTAINER ID   IMAGE
                                     COMMAND                  CREATED
 STATUS                        PORTS                      NAMES68bd2ed585ed
  traefik/whoami                                                 "/whoami"
               25 hours ago   Exited (143) 8 seconds ago
0.0.0.0:80->80/tcp         whoamiroot@/etc/runlevels # *


regs,

On Thu, May 13, 2021 at 5:17 AM Damo <dhatchett2@gmail.com> wrote:

> Hi,
>
> I've been running docker containers for a while, where I pass
> "--restart=always" into the run command, so the containers restart
> automatically after reboot. I want to have more control over the startup
> order of the containers, ie integrate into openrc start/stop and put into
> different runlevels.
>
> I've had mixed success so far. I would be interested if someone else has
> working solution. My runlevels look something like this:
>
> rl100
>   container.registry
> rl90
>   container.auth
>   container.router
> boot
>  ...
>
> FYI, i've found systemd is doing it nicely, where systemctl start/stop
> <CONTAINER> works as I would expect. I see a hardcoded dependency into the
> container PID in the unit file (podman in this case):
>
> [root@]# cat /usr/lib/systemd/system/container-libvirt-exporter.service
>
> #
>
> [Unit]
> Description=Podman container-libvirtd-exporter.service
> Documentation=man:podman-generate-systemd(1)
>
> [Service]
> Restart=always
> ExecStart=/usr/bin/podman start libvirtd-exporter
> ExecStop=/usr/bin/podman stop -t 10 libvirtd-exporter
> KillMode=none
> Type=forking
>
> PIDFile=/var/run/containers/storage/overlay-containers/9037e389e61ed01eb5dfce16fa750b6f0f01827a67640e4748e6527bbfcb6276/userdata/conmon.pid
>
> [Install]
> WantedBy=multi-user.target
>
>
> Kind regards,
> Damo
>
>

[-- Attachment #2: Type: text/html, Size: 4975 bytes --]