From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <gentoo-user+bounces-196039-garchives=archives.gentoo.org@lists.gentoo.org> Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id 9399C1382C5 for <garchives@archives.gentoo.org>; Sun, 23 May 2021 21:16:29 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id EAB31E0829; Sun, 23 May 2021 21:16:19 +0000 (UTC) Received: from mail-vs1-xe32.google.com (mail-vs1-xe32.google.com [IPv6:2607:f8b0:4864:20::e32]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id 6B2A1E081A for <gentoo-user@lists.gentoo.org>; Sun, 23 May 2021 21:16:19 +0000 (UTC) Received: by mail-vs1-xe32.google.com with SMTP id i29so7506454vsr.11 for <gentoo-user@lists.gentoo.org>; Sun, 23 May 2021 14:16:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=havMG/H5r+qPUtO88/nA67zxO9dmZz2cucvLEbliZsg=; b=AIvw0xmwUBuV8Wb41IL/b+t4oj7qtC2hfFfPfSkHsLMPWDSYlwWqt0TVUnk8F2QTZT U2jsgaUMuy+mDI+EVxxT4ikCXPbUHcyjbT47qAZjG4eMODQQM3xeEK7aTiSx+BDHO+U1 pPriHAaG+Ki2BAcZA2hVJw6tOCLyO58XJ46cxBvLLN6eM1Jgkaw6VhWnAWFGiqyH5oPw 8SbNbJFG014z3zy1ttJAx5imdhNoFLi6JBpYyA2a4eK/GoBA9j4Ki6FPiD5MYeYXEqGa yQCSbA9/01oq7nXmjR9KjNPNAQLb3UY9VnKPH3leg47jRoq314wO7Z9Q2uDxVTZX4tWT Ofvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=havMG/H5r+qPUtO88/nA67zxO9dmZz2cucvLEbliZsg=; b=eakJmZTlwZZbS1B3atebVUFV+k3IJVtNQsA+iTxMDrz6WY54ixn0tZr1kVtOvI8/af mjrOk24ppYgkBupS9Jf0mi3R7hcFfVJeaGz+G4fwy1mnGIglcnJ4RGA9wWQgMSkGEDL4 gfGfQdYJRBjUZfvpVh5hWwDBn+uSklW+uqNUw3aEV2tPR9z7xI5H6exkIwAafXTJUk7o mch2W3Y037VbjVYo8M1wQXP8ZThVUceH3d5L+7sXcvi4GhydIvd2reRv9WA4wEKXDA96 z6eUPl2lrZmYlD8Nd6gOcodmwLkJxiJwp/EAzxAah0ABLgNYr5EKAYSaaUyYYhPn1cQV JTjw== X-Gm-Message-State: AOAM533HHOi/hxDsavP814j6ONuthSCQXjmVvTvDo9HiJ5ThevgBumLC tNisoQ4WCaK3xXrGRsO/ZDv9otbRqcmguAXqiyhiKqo6JkQ= X-Google-Smtp-Source: ABdhPJxV2jv0IOFDS9aXezJSWsri+Miz2lcmZD6JKGBwCAUnEdfSEi7EjpsLA7fJ2lsz9odLXgzb6eoxWSWkobeveck= X-Received: by 2002:a67:2f89:: with SMTP id v131mr16920477vsv.16.1621804578161; Sun, 23 May 2021 14:16:18 -0700 (PDT) Precedence: bulk List-Post: <mailto:gentoo-user@lists.gentoo.org> List-Help: <mailto:gentoo-user+help@lists.gentoo.org> List-Unsubscribe: <mailto:gentoo-user+unsubscribe@lists.gentoo.org> List-Subscribe: <mailto:gentoo-user+subscribe@lists.gentoo.org> List-Id: Gentoo Linux mail <gentoo-user.gentoo.org> X-BeenThere: gentoo-user@lists.gentoo.org Reply-to: gentoo-user@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 References: <CAE5cDqNHt-UhOBz3ptggcR6syQ-LJs1ef4GNjAHvRmnn=s+dxw@mail.gmail.com> In-Reply-To: <CAE5cDqNHt-UhOBz3ptggcR6syQ-LJs1ef4GNjAHvRmnn=s+dxw@mail.gmail.com> From: Damo <dhatchett2@gmail.com> Date: Mon, 24 May 2021 07:15:34 +1000 Message-ID: <CAE5cDqOp9y36_r+jrDHZM93YWQzOeykVdsi9LTgZViCAfG=Vrg@mail.gmail.com> Subject: [gentoo-user] Re: openrc-run for containers To: gentoo-user@lists.gentoo.org Content-Type: multipart/alternative; boundary="000000000000c6cb2905c305cdac" X-Archives-Salt: 9bf9dc4e-bcc1-44f2-978b-05c4e9daf4cb X-Archives-Hash: c427d148c19a39d515d1a1b3dfc64675 --000000000000c6cb2905c305cdac Content-Type: text/plain; charset="UTF-8" Hi, I've got the integration working how I imagined. It is using s6-svscan to manage the containers, hopefully not abusing something. General idea is to have supervision control over containers, ie to allow start up in a given runlevel, have the ability to manage dependency start between given containers and start/stop using openrc (referenced https://wiki.gentoo.org/wiki/S6). Assuming containers are already running, ie given instance of whoami container: *docker run -p 80:80 -d -it --name whoami traefik/whoami* The "run" script, /var/svc.d/whoami/run: #!/bin/execlineb -P exec docker start -a whoami The "finish" script, /var/svc.d/whoami/finish *#!/bin/execlineb -Ps6-permafailon 60 1 2 exit* The init.d, conf.d. Cat /etc/conf.d/container.whoami: *INSTANCE=whoami* Cat /etc/init.d/container: *#!/sbin/openrc-rundescription="A supervised test service with a logger"supervisor=s6s6_service_path=/run/openrc/s6-scan/${INSTANCE}depend() { need s6-svscan}stop_pre() { docker stop ${INSTANCE}}* Finally, [openrc-run, ln -s /etc/init.d/container /etc/init.d/container.whoami] /etc/initd.d/container.whoami start, stop work as expected (docker ps |grep whoami does not return anything, after running "/etc/init.d/container.whoami stop"): List containers root@ # *docker ps |grep whoami* 68bd2ed585ed traefik/whoami "/whoami" 35 minutes ago Up 34 minutes 0.0.0.0:80->80/tcp whoami root@ # *./container.whoami stop* container.whoami |whoami container.whoami | * Stopping container.whoami ... [ ok ] root@ # *docker ps |grep whoami* root@ #* ./container.whoami start* container.whoami | * Starting container.whoami ... [ ok ] root@ # docker ps |grep whoami 68bd2ed585ed traefik/whoami "/whoami" 35 minutes ago Up 3 seconds 0.0.0.0:80->80/tcp whoami root@ # *ps xf -o pid,ppid,pgrp,euser,args* PID PPID PGRP EUSER COMMAND 21056 1 21056 root /bin/s6-svscan /run/openrc/s6-scan 21058 21056 21056 root \_ s6-supervise whoami/log 21059 21056 21056 root \_ s6-supervise whoami 27584 21059 27584 root | \_ docker start -a whoami Similar to above, using s6-svstat: root@ # *s6-svstat /run/openrc/s6-scan/whoami* up (pid 27584) 752 seconds root@ # *./container.whoami stop* container.whoami |whoami container.whoami | * Stopping container.whoami ... [ ok ] root@ #* s6-svstat /run/openrc/s6-scan/whoami* down (exitcode 2) 1 seconds, normally up, ready 1 seconds root@ #* ./container.whoami start* container.whoami | * Starting container.whoami ... [ ok ] root@h003 /e/init.d # s6-svstat /run/openrc/s6-scan/whoami up (pid 6722) 3 seconds The goal of all this has been to incorporate containers into /etc/runlevels. I am not sure if there is a better way, have I missed something by not using s6-overlay or the like? I am not familiar enough to know. An outstanding issue, in the s6 run script, i would like to parametrize the instance name, I don't know how to do it, as it is currently hard coded: The "run" script, /var/svc.d/whoami/run: #!/bin/execlineb -P exec docker start -a *whoami* But once that is done, then all running containers could be incorporated by updating the conf.d for INSTANCE name, templating a /var/svc.d/<INSTANCE> folder and linking to /etc/init.d/container, ie: /etc/init.d/container.whoami /etc/init.d/container.cadvisor /etc/init.d/container.traefik [put under control of s6] /etc/runlevels/20/s6-svscan ... This is kind of asymmetric, ie container start is in s6, whereas stop [docker] is in openrc, but I am not seeing a different way, the goal is to have robust services running. When system boots, s6-scan will start all the containers automatically, but then further operations, ie for things like manual failover etc, is possible using standard platform openrc - effectively docker - start/stop commands. kind regards On Thu, May 13, 2021 at 5:17 AM Damo <dhatchett2@gmail.com> wrote: > Hi, > > I've been running docker containers for a while, where I pass > "--restart=always" into the run command, so the containers restart > automatically after reboot. I want to have more control over the startup > order of the containers, ie integrate into openrc start/stop and put into > different runlevels. > > I've had mixed success so far. I would be interested if someone else has > working solution. My runlevels look something like this: > > rl100 > container.registry > rl90 > container.auth > container.router > boot > ... > > FYI, i've found systemd is doing it nicely, where systemctl start/stop > <CONTAINER> works as I would expect. I see a hardcoded dependency into the > container PID in the unit file (podman in this case): > > [root@]# cat /usr/lib/systemd/system/container-libvirt-exporter.service > > # > > [Unit] > Description=Podman container-libvirtd-exporter.service > Documentation=man:podman-generate-systemd(1) > > [Service] > Restart=always > ExecStart=/usr/bin/podman start libvirtd-exporter > ExecStop=/usr/bin/podman stop -t 10 libvirtd-exporter > KillMode=none > Type=forking > > PIDFile=/var/run/containers/storage/overlay-containers/9037e389e61ed01eb5dfce16fa750b6f0f01827a67640e4748e6527bbfcb6276/userdata/conmon.pid > > [Install] > WantedBy=multi-user.target > > > Kind regards, > Damo > > --000000000000c6cb2905c305cdac Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr">Hi,<div><br></div><div>I've got the integration workin= g how I imagined. It is using s6-svscan to manage the containers, hopefully= not abusing something. General idea is to have supervision control over co= ntainers, ie to allow start up in a given runlevel, have the ability to man= age dependency start between given containers and start/stop using openrc (= referenced <a href=3D"https://wiki.gentoo.org/wiki/S6">https://wiki.gentoo.= org/wiki/S6</a>).<div><br></div><div>Assuming containers are already runnin= g, ie given instance of whoami container:<div><br></div><div><i>docker run = -p 80:80 -d -it --name whoami traefik/whoami</i><br></div><div><br></div><d= iv>The "run" script, /var/svc.d/whoami/run:</div><div><br>#!/bin/= execlineb -P<br>exec docker start -a whoami<br></div><div><br></div><div>Th= e "finish" script, /var/svc.d/whoami/finish</div><div><br></div><= div><i>#!/bin/execlineb -P<br>s6-permafailon 60 1 2 exit</i><br></div><div>= <br></div><div>The init.d, conf.d. Cat /etc/conf.d/container.whoami:</div><= div><br></div><div><i>INSTANCE=3Dwhoami</i></div><div><br></div><div>Cat /e= tc/init.d/container:</div><div><br></div><div><i>#!/sbin/openrc-run<br>desc= ription=3D"A supervised test service with a logger"<br>supervisor= =3Ds6<br>s6_service_path=3D/run/openrc/s6-scan/${INSTANCE}<br><br>depend() = {<br>=C2=A0 =C2=A0need s6-svscan<br>}<br><br>stop_pre() {<br>=C2=A0 docker = stop ${INSTANCE}<br>}</i><br></div><div><br></div><div>Finally, [openrc-run= , ln -s /etc/init.d/container /etc/init.d/container.whoami] /etc/initd.d/co= ntainer.whoami start, stop work as expected (docker ps |grep whoami does no= t return anything, after running "/etc/init.d/container.whoami stop&qu= ot;):</div><div><br></div><div>List containers<br>root@ # <i>docker ps |gre= p whoami</i><br>68bd2ed585ed =C2=A0 traefik/whoami =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 "= ;/whoami" =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A035 mi= nutes ago =C2=A0 Up 34 minutes =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0.0.0.0:8= 0->80/tcp =C2=A0 =C2=A0 =C2=A0 =C2=A0 whoami<br></div><div>=C2=A0 <br>ro= ot@ # <i>./container.whoami stop</i><br>container.whoami =C2=A0 =C2=A0 =C2= =A0 |whoami<br>container.whoami =C2=A0 =C2=A0 =C2=A0 | * Stopping container= .whoami ... =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[ ok ]</div><div><br>ro= ot@ # <i>docker ps |grep whoami</i></div><div><br>root@ #<i> ./container.wh= oami start</i><br>container.whoami =C2=A0 =C2=A0 =C2=A0 | * Starting contai= ner.whoami ... =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[ ok ]<br>root@ # do= cker ps |grep whoami<br>68bd2ed585ed =C2=A0 traefik/whoami =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 "/whoami" =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A035 minutes ago =C2=A0 Up 3 seconds =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A00.0.0.0:80->80/tcp =C2=A0 =C2=A0 =C2=A0 =C2=A0 whoami<br></div= ><div><br></div>root@ #<i>=C2=A0</i><i>ps xf -o pid,ppid,pgrp,euser,args</i= ><br>=C2=A0 PID =C2=A0PPID =C2=A0PGRP EUSER =C2=A0 =C2=A0COMMAND<br>21056 = =C2=A0 =C2=A0 1 21056 root =C2=A0 =C2=A0 /bin/s6-svscan /run/openrc/s6-scan= <br>21058 21056 21056 root =C2=A0 =C2=A0 =C2=A0\_ s6-supervise whoami/log<b= r>21059 21056 21056 root =C2=A0 =C2=A0 =C2=A0\_ s6-supervise whoami<br>2758= 4 21059 27584 root =C2=A0 =C2=A0 =C2=A0| =C2=A0 \_ docker start -a whoami<b= r><div><br></div><div>Similar to above, using s6-svstat:</div><div><br></di= v><div>root@ # <i>s6-svstat /run/openrc/s6-scan/whoami</i><br>up (pid 27584= ) 752 seconds</div><div><br>root@ # <i>./container.whoami stop</i><br>conta= iner.whoami =C2=A0 =C2=A0 =C2=A0 |whoami<br>container.whoami =C2=A0 =C2=A0 = =C2=A0 | * Stopping container.whoami ... =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0[ ok ]</div><div><br>root@ #<i> s6-svstat /run/openrc/s6-scan/whoami<= /i><br>down (exitcode 2) 1 seconds, normally up, ready 1 seconds</div><div>= <br>root@ #<i> ./container.whoami start</i><br>container.whoami =C2=A0 =C2= =A0 =C2=A0 | * Starting container.whoami ... =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0[ ok ]<br>root@h003 /e/init.d # s6-svstat /run/openrc/s6-scan/= whoami<br>up (pid 6722) 3 seconds</div><div>=C2=A0<br></div><div>The goal o= f all this has been to incorporate containers into /etc/runlevels. I am not= sure if there is a better way, have I missed something by not using s6-ove= rlay or the like? I am not familiar enough to know.=C2=A0<br></div><div><br= ></div><div>An outstanding issue, in the s6 run script, i would like to par= ametrize the instance name, I=C2=A0don't know how to do it, as it is cu= rrently hard coded:</div><div><br></div><div><div>The "run" scrip= t, /var/svc.d/whoami/run:</div><div><br>#!/bin/execlineb -P<br>exec docker = start -a <b>whoami</b></div></div><div><b><br></b></div><div>But once that = is done, then all running containers could be incorporated by updating the = conf.d for INSTANCE name, templating a /var/svc.d/<INSTANCE> folder a= nd linking to /etc/init.d/container, ie:</div><div><br></div><div>/etc/init= .d/container.whoami</div><div>/etc/init.d/container.cadvisor<br></div><div>= /etc/init.d/container.traefik<br></div><div><br></div><div>[put under contr= ol of s6]</div><div>/etc/runlevels/20/s6-svscan</div><div>...</div><div><br= ></div><div>This is kind of asymmetric, ie container start is in s6, wherea= s stop [docker] is in openrc, but I am not seeing a different way, the goal= is to have robust services running. When system boots, s6-scan will start = all the containers automatically, but then further operations, ie for thing= s like manual failover etc, is possible using standard platform openrc - ef= fectively docker - start/stop commands.</div><div><br></div><div><br></div>= <div>kind regards</div><div><br></div><div><br></div><div><br></div></div><= /div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_a= ttr">On Thu, May 13, 2021 at 5:17 AM Damo <<a href=3D"mailto:dhatchett2@= gmail.com">dhatchett2@gmail.com</a>> wrote:<br></div><blockquote class= =3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rg= b(204,204,204);padding-left:1ex"><div dir=3D"ltr">Hi,<div><br></div><div>I&= #39;ve been running docker containers for a while, where I pass "--res= tart=3Dalways" into the run command, so the containers restart automat= ically after reboot. I want to have more control over the startup order of = the containers, ie integrate into openrc start/stop and put into different = runlevels.</div><div><br></div><div>I've had mixed success so far. I wo= uld be interested if someone else has working solution. My runlevels look s= omething like this:<br><br>rl100</div><div>=C2=A0 container.registry<br>rl9= 0<br>=C2=A0 container.auth<br>=C2=A0 container.router=C2=A0=C2=A0<br></div>= <div>boot</div><div>=C2=A0...</div><div><br></div><div>FYI, i've found = systemd is doing it nicely, where systemctl start/stop <CONTAINER> wo= rks as I would expect. I see a hardcoded dependency into the container PID = in the unit file (podman in this case):</div><div><br></div><div>[root@]# c= at /usr/lib/systemd/system/container-libvirt-exporter.service<br><br>#<br><= br>[Unit]<br>Description=3DPodman container-libvirtd-exporter.service<br>Do= cumentation=3Dman:podman-generate-systemd(1)<br><br>[Service]<br>Restart=3D= always<br>ExecStart=3D/usr/bin/podman start libvirtd-exporter<br>ExecStop= =3D/usr/bin/podman stop -t 10 libvirtd-exporter<br>KillMode=3Dnone<br>Type= =3Dforking<br>PIDFile=3D/var/run/containers/storage/overlay-containers/9037= e389e61ed01eb5dfce16fa750b6f0f01827a67640e4748e6527bbfcb6276/userdata/conmo= n.pid<br><br>[Install]<br>WantedBy=3Dmulti-user.target<br><br><br>Kind rega= rds,</div><div>Damo<br><br></div></div> </blockquote></div> --000000000000c6cb2905c305cdac--