From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id 06F3F138350 for ; Mon, 3 Feb 2020 03:54:42 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 1CE67E0841; Mon, 3 Feb 2020 03:54:41 +0000 (UTC) Received: from mail-lf1-x12d.google.com (mail-lf1-x12d.google.com [IPv6:2a00:1450:4864:20::12d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id 99E83E0841 for ; Mon, 3 Feb 2020 03:54:40 +0000 (UTC) Received: by mail-lf1-x12d.google.com with SMTP id c23so8647997lfi.7 for ; Sun, 02 Feb 2020 19:54:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=3fB27R9vz9vw8Gvuu0D0gOz8N1y/sig6/0YqaroRVn4=; b=GD1Q8fSUEHCHWz/jaeW9a2LZhvwJ4ax4r2Dh4n9zmad8vTO23ga/aaWO/1gqIlw8PJ 0KsfoL62oxhKVH41sffmUWiAsB4zBX5heqGyxSHClvuPJzfQL744qadpFaTY/IFCh61N yPnoLbMFfxIQE8457YBuagVaD4gPjoi1YI3Ew8XxULdbuWX3KVZ47Jor/PWv6Am6vXR7 UxMTsbh7L18UKhFtegIqhGb/eMNztUvsM4C62nkox0bbmq9qgoXyNIdCDt8aEMs02P0u qpqKqgQHgzFbasRTVBdMg7jPStVdheNy39BWaslwA3UCCSXczjbqgwrrVYEW/I3dByyE i/XA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=3fB27R9vz9vw8Gvuu0D0gOz8N1y/sig6/0YqaroRVn4=; b=QSDNjgJNc9ZA6DLj4uZI7Hy9S4WF8VYnwpt8vx0aRH24W3wQaPre814V8sXACgHUcz cQall2uz2YsYBtQB5FbvDzGFfjG99K/JyP7dfl2+Uo8D9D/ploYp6OkUMR62rw+8XHfv Z2P5Afqt51zeXTl8+C80HOqrAzIMWHBE78v0on2qcFZ0W/pLCKs9ghiAEjuCr19usoFX 0qby5bs7cPAF7JEY3BMdyJ0xm9hSlI7Ypqt3tLibqvA3bb1qufEGILHafuRl7LAkyiCe hrarAhWh2uuvfrb9kAYIAKvs8RH3TQfevqDlGp0OReaI5wZXcqS2AHMQ0ZdTDZfESavX rluQ== X-Gm-Message-State: APjAAAX4gSKiGCct+CD0cfZIzG45zLawxhP3qyQFxI1noefnM9MUdU8c O5tEMylnRlQh0iNHq1qGjzVTTZdsjBkKKQk1DQB+YU7N X-Google-Smtp-Source: APXvYqyvfmkqhK8qeg04W0OxHHPyZr0OJlnnHaH9to0wDE8TsXHjyYywp/7FANqErMHlrxxbBvIG/4+ZqaGaDshB+fY= X-Received: by 2002:a19:f00d:: with SMTP id p13mr10797278lfc.37.1580702078232; Sun, 02 Feb 2020 19:54:38 -0800 (PST) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-alt@lists.gentoo.org Reply-to: gentoo-alt@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 References: <874kwwduzd.fsf@robot.e.airelinux.org> <20200202120752.GR1169@gentoo.org> In-Reply-To: <20200202120752.GR1169@gentoo.org> From: Sam Pfeiffer Date: Mon, 3 Feb 2020 14:54:26 +1100 Message-ID: Subject: Re: [gentoo-alt] Allow non-interactive bootstrap to be stopped at arbitrary stages To: gentoo-alt@lists.gentoo.org Content-Type: multipart/alternative; boundary="000000000000de6a06059da3e1c1" X-Archives-Salt: ae2b5007-9902-413c-9983-5d28cd38b0e5 X-Archives-Hash: 585694dc45945489c75ec662d9606d65 --000000000000de6a06059da3e1c1 Content-Type: text/plain; charset="UTF-8" Hey Fabian, Thanks for chipping in. I do think it's useful and user-friendly to be able to control the target of the interactive/noninteractive script. I proposed in my first email in the thread something similar with the $STOP_AFTER_STAGE variable, even though I was capturing it from the commandline, and as I understand you propose, it would probably be better to capture it from an environment variable (it would actually be more in-line with other existing variables). Benda, I'm sorry for the noise I'm causing. I just want to improve the experience for the next person that tries to use this amazing project by taking advantage of already implemented stuff. And also just having this discussion is good to have more information archived to google about Gentoo Prefix and its bootstrap process. I think. On Sun, 2 Feb 2020 at 23:08, Fabian Groffen wrote: > What I gather from this discussion is that the bootstrap script is > complex, and grew more complexity over the years. > > Back in the early days, there was just documentation, one needed to > follow. Since this frequently needed an update, and people kept using > older copies, at some point the bootstrap-prefix.sh script was > introduced. > > This script followed the documentation to the letter. It was an exact > replica of the instructions in the bootstrap document. This included > the notions of 3 stages and a world recompile. > > Today, the purpose of the stages have blurred, and seem more like > batches in which to do things. Some of the initial intentions still > stand out, e.g. that stage1 is there to provide a working portage > environment with some minimal efforts. Minimal got redefined as soon as > we had to install python and more and more tools seemed necesary (sed, > path, awk, etc.). > > Now, it seems you need to cut time and sort of bootstrap up-to a stage. > Would it be an option to simply add a(nother) var that will control the > interactive target in terms of how far it will run along. Normal resume > strategies it employs already would simply allow a next run to run along > further. > > I think above can be done easily with a few lines of code. > > Let me know what you think. > > Thanks, > Fabian > > On 23-01-2020 13:31:05 +1100, Sam Pfeiffer wrote: > > Grobian, any input about this? > > > > On Thu, Jan 16, 2020, 17:39 Sam Pfeiffer <[1]sammypfeiffer@gmail.com> > wrote: > > > > > Hello Benda, > > > > > To be more specific... To easily go thru it I made a gist with the > latest > > > version of bootstrap-prefix.sh and I'll be pointing to specific lines > (this > > > also helps me recap why I did this, as I remember using stage{1,2,3} > didn't > > > work for me when I first came into this): > > > > > When you execute bootstrap-prefix.sh without arguments (other than the > > > EPREFIX, ./bootstrap-prefix.sh /MY/EPREFIX) it calls this line: > > > > > [2] > https://gist.github.com/awesomebytes/ba4df9c6ef3ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L3033 > > > > > Calling bootstrap_interactive > > > > > If you give an extra argument (./bootstrap-prefix.sh /MY/EPREFIX > > > [noninteractive, stage{1,2,3}]) it calls this line: > > > [3] > https://gist.github.com/awesomebytes/ba4df9c6ef3ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L3082 > > > > > bootstrap_${TODO#non} || exit 1 > > > > > Which, on the noninteractive case, goes to the same place than no > arguments > > > (bootstrap_interactive). But in the stage{1,2,3} calls directly the > > > bootstrap_stageX function. > > > > > The thing is, noninteractive does some checks, finds resources and > sets up > > > environment variables like: > > > > > * Check for bad flags: > > > [4] > https://gist.github.com/awesomebytes/ba4df9c6ef3ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L2200-L2247 > > > * We find a gcc: > > > [5] > https://gist.github.com/awesomebytes/ba4df9c6ef3ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L2273-L2433 > > > * We get a cpu count and adjust -j for Make: > > > [6] > https://gist.github.com/awesomebytes/ba4df9c6ef3ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L2443-L2501 > > > * Check if we deal with multilib systems: > > > [7] > https://gist.github.com/awesomebytes/ba4df9c6ef3ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L2503-L2576 > > > * Check if we are boostrapping from a Gentoo system: > > > [8] > https://gist.github.com/awesomebytes/ba4df9c6ef3ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L2578-L2608 > > > * Set our EPREFIX: > > > [9] > https://gist.github.com/awesomebytes/ba4df9c6ef3ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L2610-L2681 > > > > > Which is summarised at the end by setting the environment variables > EPREFIX, > > > CHOST, PATH, MAKEOPTS. > > > > > And then finally calls bootstrap_stage1_log, which runs > bootstrap_stage1 > > > itself. > > > > > [10] > https://gist.github.com/awesomebytes/ba4df9c6ef3ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L2724-L2726 > > > > > Then, we compare with calling bootstrap-prefix.sh EPREFIX stage1... > > > > > Calls bootstrap_stage1(): > > > [11] > https://gist.github.com/awesomebytes/ba4df9c6ef3ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L1355 > > > > > And as you can see it won't do any of the extra work done in > > > bootstrap_interactive(). > > > > > I'm not that versed in this script, so I may be missing something easy > to > > > overcome this (or I don't know enough about what is going on), but > there is no > > > separate function that does everything in bootstrap_interactive without > > > calling bootstrap_stage1. And bootstrap_stage1 does not do all that > setup that > > > bootstrap_interactive does. So that's how I came to my proposal. > > > > > If there is some patch to enable that setup from bootstrap_interactive > with > > > bootstrap_stage1, that would be a good alternative, I guess. I still > like the > > > fact that with my approach you are just saying "bootstrap everything > until > > > this step" instead of needing to manually do bootstrap_setup (imaginary > > > function I just named) then bootstrap_stage1, then bootstrap_stage2, > then > > > bootstrap_stage3, and then no argument for the last emerge -e system, > to get > > > to a specific bootstrap stage, but that's just commodity. > > > > > Let me know if I'm wrong somewhere or you think of any better solution! > > > > > On Thu, 16 Jan 2020 at 13:42, Sam Pfeiffer <[12] > sammypfeiffer@gmail.com> > > > wrote: > > > > >> Hi Benda, > > > > >> Thanks for your reply. > > > > >> On Thu, 16 Jan 2020 at 13:01, Benda Xu <[13]heroxbd@gentoo.org> > wrote: > > > > >>> Hi Sam, > > > > >>> Sam Pfeiffer <[14]sammypfeiffer@gmail.com> writes: > > > > >>> > To ease automated building and debugging of the bootstrap of Gentoo > > >>> > Prefix I would like to propose a patch to add the option to stop > the > > >>> > non-interactive bootstrap on specific stages of the bootstrap. > > >>> > > > >>> > I'm currently doing this in my CI projects [1][2] and I'd love to > have > > >>> > this added upstream. > > >>> > > > >>> > I propose a patch like this: > > >>> >[15] > https://gist.github.com/awesomebytes/3468477c6c90fe3d985372d50aabba9f > > >>> > > > >>> > Which is what I'm currently using. It just adds another optional > > >>> >parameter to the commandline (a 3rd one) in non-interactive mode > which > > >>> >you tell it after which stage to stop (stage1, stage2, stage3). > > > > >>> > I'm not the most versed person in bash, so maybe there is a more > > >>> > conceptually beautiful way of doing this. My patch is just a > proposal > > >>> > (that has been tested and running). > > >>> > > > >>> > This patch is useful because it allows: > > >>> > 1) Easier debugging by stages. > > > > >>> > 2) Allows to split the bootstrap job in parts in CI environments > with > > >>> > time limits on jobs to run (e.g. Azure Pipelines time limit is 6h*, > > >>> > and the bootstrap takes in between 5h40-6h40 depending on random > > >>> > things like time to download things, and load on the machine... or > > >>> > packages just taking longer to build). > > >>> > > > >>> > And this patch should not change anything to anyone else (AFAIK). > > > > >>> Are you aware that in bootstrap-prefix.sh you can specify the stage > you > > >>> are after? e.g. > > > > >>> [16] > https://wiki.gentoo.org/wiki/Project:Prefix/Manual_Bootstrap#Stage_1 > > > > >>> The text above that could be outdated, but PATH exports and stage1~3 > are > > >>> correct. > > > > >> I'm aware, but the behaviour (relating to environment variables and > choosing > > >> CPU numbers) differs from the > > > > >> noninteractive argument. I'd like to keep the CI environment as close > as a > > >> user executing ./bootstrap-prefix.sh EPREFIX as possible. > > > > >> That's why I'm proposing this addition. Another approach could be > modifying > > >> the current argument to do the same than the noninteractive one. > > > > >> But that may break the workflow of someone, and I don't want that. > > > > >> > > > > >>> > Thanks for your time and feedback! > > >>> > > > >>> > P.S.: I'm used to the GitHub/GitLab workflow of having a graphical > > >>> > interface to do a Pull Request... I don't know how to do this with > > >>> > [17]https://gitweb.gentoo.org/repo/proj/prefix.git/ If there is > no way, > > >>> > aside of the goal of this email, I'd propose to have a mirror in > > >>> > GitHub. > > > > >>> Yes, that's a good to have for me, but not in high priority. > > > > >>> > *If you are curious, Stage 1 takes 8min~, Stage 2 takes 29min~, > Stage > > >>> > 3 takes 2h30min~, emerge system takes 2h35min~ on Azure Pipelines, > > >>> > approximately. > > > > >>> Thank you so much for all your work! That's extremely helpful to > keep > > >>> the quality of Prefix codebase. > > > > >>> > [1] [18]https://github.com/awesomebytes/gentoo_prefix_ci > > >>> > [2] [19]https://github.com/awesomebytes/gentoo_prefix_ci_32b > > > > >>> Yours, > > >>> Benda > > > > >> -- > > > > >> Sammy Pfeiffer > > >> PhD Candidate at The Magic Lab within UTS. > > > > > -- > > > > > Sammy Pfeiffer > > > PhD Candidate at The Magic Lab within UTS. > > > > > > > > References: > > 1. mailto:sammypfeiffer@gmail.com > > 2. > https://gist.github.com/awesomebytes/ba4df9c6ef3ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L3033 > > 3. > https://gist.github.com/awesomebytes/ba4df9c6ef3ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L3082 > > 4. > https://gist.github.com/awesomebytes/ba4df9c6ef3ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L2200-L2247 > > 5. > https://gist.github.com/awesomebytes/ba4df9c6ef3ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L2273-L2433 > > 6. > https://gist.github.com/awesomebytes/ba4df9c6ef3ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L2443-L2501 > > 7. > https://gist.github.com/awesomebytes/ba4df9c6ef3ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L2503-L2576 > > 8. > https://gist.github.com/awesomebytes/ba4df9c6ef3ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L2578-L2608 > > 9. > https://gist.github.com/awesomebytes/ba4df9c6ef3ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L2610-L2681 > > 10. > https://gist.github.com/awesomebytes/ba4df9c6ef3ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L2724-L2726 > > 11. > https://gist.github.com/awesomebytes/ba4df9c6ef3ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L1355 > > 12. mailto:sammypfeiffer@gmail.com > > 13. mailto:heroxbd@gentoo.org > > 14. mailto:sammypfeiffer@gmail.com > > 15. > https://gist.github.com/awesomebytes/3468477c6c90fe3d985372d50aabba9f > > 16. > https://wiki.gentoo.org/wiki/Project:Prefix/Manual_Bootstrap#Stage_1 > > 17. https://gitweb.gentoo.org/repo/proj/prefix.git/ > > 18. https://github.com/awesomebytes/gentoo_prefix_ci > > 19. https://github.com/awesomebytes/gentoo_prefix_ci_32b > > > > read_char: errno==EILSEQ; invalid byte sequence for UTF-8: > -- > Fabian Groffen > Gentoo on a different level > -- *Sammy Pfeiffer* PhD Candidate at The Magic Lab within UTS. --000000000000de6a06059da3e1c1 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hey Fabian,

Thanks for chipp= ing in. I do think it's useful and user-friendly to be able to control = the target of the interactive/noninteractive script.
I proposed i= n my first email in the thread something similar with the $STOP_AFTER_STAGE variable, even though I was capturing it from= the commandline, and as I understand
you propose, it would probably be better to capture it from an env= ironment variable (it would actually be more in-line with other existing va= riables).

Benda, I'm sorry for the noise I'm = causing. I just want to improve the experience for the next person that tri= es to use this amazing project by
taking advantage of already implemented stuff. And also just having th= is discussion is good to have more information archived to google about Gen= too
Prefix and its bootstrap = process. I think.

On Sun, 2 Feb 2020 at 23:08, Fabian Groff= en <grobian@gentoo.org> wro= te:
What I gathe= r from this discussion is that the bootstrap script is
complex, and grew more complexity over the years.

Back in the early days, there was just documentation, one needed to
follow.=C2=A0 Since this frequently needed an update, and people kept using=
older copies, at some point the bootstrap-prefix.sh script was
introduced.

This script followed the documentation to the letter.=C2=A0 It was an exact=
replica of the instructions in the bootstrap document.=C2=A0 This included<= br> the notions of 3 stages and a world recompile.

Today, the purpose of the stages have blurred, and seem more like
batches in which to do things.=C2=A0 Some of the initial intentions still stand out, e.g. that stage1 is there to provide a working portage
environment with some minimal efforts.=C2=A0 Minimal got redefined as soon = as
we had to install python and more and more tools seemed necesary (sed,
path, awk, etc.).

Now, it seems you need to cut time and sort of bootstrap up-to a stage.
Would it be an option to simply add a(nother) var that will control the
interactive target in terms of how far it will run along.=C2=A0 Normal resu= me
strategies it employs already would simply allow a next run to run along further.

I think above can be done easily with a few lines of code.

Let me know what you think.

Thanks,
Fabian

On 23-01-2020 13:31:05 +1100, Sam Pfeiffer wrote:
> Grobian, any input about this?
>
> On Thu, Jan 16, 2020, 17:39 Sam Pfeiffer <[1]sammypfeiffer@gmail.com> wrot= e:
>
> > Hello Benda,
>
> > To be more specific... To easily go thru it I made a gist with th= e latest
> > version of bootstrap-prefix.sh and I'll be pointing to specif= ic lines (this
> > also helps me recap why I did this, as I remember using stage{1,2= ,3} didn't
> > work for me when I first came into this):
>
> > When you execute bootstrap-prefix.sh without arguments (other tha= n the
> > EPREFIX, ./bootstrap-prefix.sh /MY/EPREFIX) it calls this line: >
> > [2]https://gist.github.com/awesomebytes/ba4df9c6ef3ab5742cb9aceed= 7998c3d#file-bootstrap-prefix-sh-L3033
>
> > Calling bootstrap_interactive
>
> > If you give an extra argument (./bootstrap-prefix.sh /MY/EPREFIX<= br> > > [noninteractive, stage{1,2,3}]) it calls this line:
> > [3]https://gist.github.com/awesomebytes/ba4df9c6ef3ab5742cb9aceed= 7998c3d#file-bootstrap-prefix-sh-L3082
>
> > bootstrap_${TODO#non} || exit 1
>
> > Which, on the noninteractive case, goes to the same place than no= arguments
> > (bootstrap_interactive). But in the stage{1,2,3} calls directly t= he
> > bootstrap_stageX function.
>
> > The thing is, noninteractive does some checks, finds resources an= d sets up
> > environment variables like:
>
> >=C2=A0 =C2=A0* Check for bad flags:
> >=C2=A0 =C2=A0[4]https://gist.github.com/awesomebytes/ba4df9c= 6ef3ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L2200-L2247
> >=C2=A0 =C2=A0* We find a gcc:
> >=C2=A0 =C2=A0[5]https://gist.github.com/awesomebytes/ba4df9c= 6ef3ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L2273-L2433
> >=C2=A0 =C2=A0* We get a cpu count and adjust -j for Make:
> >=C2=A0 =C2=A0[6]https://gist.github.com/awesomebytes/ba4df9c= 6ef3ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L2443-L2501
> >=C2=A0 =C2=A0* Check if we deal with multilib systems:
> >=C2=A0 =C2=A0[7]https://gist.github.com/awesomebytes/ba4df9c= 6ef3ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L2503-L2576
> >=C2=A0 =C2=A0* Check if we are boostrapping from a Gentoo system:<= br> > >=C2=A0 =C2=A0[8]https://gist.github.com/awesomebytes/ba4df9c= 6ef3ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L2578-L2608
> >=C2=A0 =C2=A0* Set our EPREFIX:
> > [9]https://gist.github.com/awesomebytes/ba4df9c6ef3ab5742cb= 9aceed7998c3d#file-bootstrap-prefix-sh-L2610-L2681
>
> > Which is summarised at the end by setting the environment variabl= es EPREFIX,
> > CHOST, PATH, MAKEOPTS.
>
> > And then finally calls bootstrap_stage1_log, which runs bootstrap= _stage1
> > itself.
>
> > [10]https://gist.github.com/awesomebytes/ba4df9c6ef3ab5742c= b9aceed7998c3d#file-bootstrap-prefix-sh-L2724-L2726
>
> > Then, we compare with calling bootstrap-prefix.sh EPREFIX stage1.= ..
>
> > Calls bootstrap_stage1():
> > [11]https://gist.github.com/awesomebytes/ba4df9c6ef3ab5742cb9acee= d7998c3d#file-bootstrap-prefix-sh-L1355
>
> > And as you can see it won't do any of the extra work done in<= br> > > bootstrap_interactive().
>
> > I'm not that versed in this script, so I may be missing somet= hing easy to
> > overcome this (or I don't know enough about what is going on)= , but there is no
> > separate function that does everything in bootstrap_interactive w= ithout
> > calling bootstrap_stage1. And bootstrap_stage1 does not do all th= at setup that
> > bootstrap_interactive does. So that's how I came to my propos= al.
>
> > If there is some patch to enable that setup from bootstrap_intera= ctive with
> > bootstrap_stage1, that would be a good alternative, I guess. I st= ill like the
> > fact that with my approach you are just saying "bootstrap ev= erything until
> > this step" instead of needing to manually do bootstrap_setup= (imaginary
> > function I just named) then bootstrap_stage1, then bootstrap_stag= e2, then
> > bootstrap_stage3, and then no argument for the last emerge -e sys= tem, to get
> > to a specific bootstrap stage, but that's just commodity.
>
> > Let me know if I'm wrong somewhere or you think of any better= solution!
>
> > On Thu, 16 Jan 2020 at 13:42, Sam Pfeiffer <[12]sammypfeiffer@gmail.com&= gt;
> > wrote:
>
> >> Hi Benda,
>
> >> Thanks for your reply.
>
> >> On Thu, 16 Jan 2020 at 13:01, Benda Xu <[13]heroxbd@gentoo.org> wrote:=
>
> >>> Hi Sam,
>
> >>> Sam Pfeiffer <[14]sammypfeiffer@gmail.com> writes:
>
> >>> > To ease automated building and debugging of the boot= strap of Gentoo
> >>> > Prefix I would like to propose a patch to add the op= tion to stop the
> >>> > non-interactive bootstrap on specific stages of the = bootstrap.
> >>> >
> >>> > I'm currently doing this in my CI projects [1][2= ] and I'd love to have
> >>> > this added upstream.
> >>> >
> >>> > I propose a patch like this:
> >>> >[15]https= ://gist.github.com/awesomebytes/3468477c6c90fe3d985372d50aabba9f
> >>> >
> >>> > Which is what I'm currently using. It just adds = another optional
> >>> >parameter to the commandline (a 3rd one) in non-inter= active mode which
> >>> >you tell it after which stage to stop (stage1, stage2= , stage3).
>
> >>> > I'm not the most versed person in bash, so maybe= there is a more
> >>> > conceptually beautiful way of doing this. My patch i= s just a proposal
> >>> > (that has been tested and running).
> >>> >
> >>> > This patch is useful because it allows:
> >>> > 1) Easier debugging by stages.
>
> >>> > 2) Allows to split the bootstrap job in parts in CI = environments with
> >>> > time limits on jobs to run (e.g. Azure Pipelines tim= e limit is 6h*,
> >>> > and the bootstrap takes in between 5h40-6h40 dependi= ng on random
> >>> > things like time to download things, and load on the= machine... or
> >>> > packages just taking longer to build).
> >>> >
> >>> > And this patch should not change anything to anyone = else (AFAIK).
>
> >>> Are you aware that in bootstrap-prefix.sh you can specify= the stage you
> >>> are after?=C2=A0 e.g.
>
> >>> =C2=A0 =C2=A0[16]https://wiki.gentoo.org/wiki/Project:Prefix/Manual_Bootstrap#Stage_1=
>
> >>> The text above that could be outdated, but PATH exports a= nd stage1~3 are
> >>> correct.
>
> >> I'm aware, but the behaviour (relating to environment var= iables and choosing
> >> CPU numbers) differs from the
>
> >> noninteractive argument. I'd like to keep the CI environm= ent as close as a
> >> user executing ./bootstrap-prefix.sh EPREFIX as possible.
>
> >> That's why I'm proposing this addition. Another appro= ach could be modifying
> >> the current argument to do the same than the noninteractive o= ne.
>
> >> But that may break the workflow of someone, and I don't w= ant that.
>
> >> =C2=A0
>
> >>> > Thanks for your time and feedback!
> >>> >
> >>> > P.S.: I'm used to the GitHub/GitLab workflow of = having a graphical
> >>> > interface to do a Pull Request... I don't know h= ow to do this with
> >>> > [17]https://gitweb.gentoo.org/= repo/proj/prefix.git/ If there is no way,
> >>> > aside of the goal of this email, I'd propose to = have a mirror in
> >>> > GitHub.
>
> >>> Yes, that's a good to have for me, but not in high pr= iority.
>
> >>> > *If you are curious, Stage 1 takes 8min~, Stage 2 ta= kes 29min~, Stage
> >>> > 3 takes 2h30min~, emerge system takes 2h35min~ on Az= ure Pipelines,
> >>> > approximately.
>
> >>> Thank you so much for all your work!=C2=A0 That's ext= remely helpful to keep
> >>> the quality of Prefix codebase.
>
> >>> > [1] [18]https://github.com/aw= esomebytes/gentoo_prefix_ci
> >>> > [2] [19]https://github.co= m/awesomebytes/gentoo_prefix_ci_32b
>
> >>> Yours,
> >>> Benda
>
> >> --
>
> >> Sammy Pfeiffer
> >> PhD Candidate at The Magic Lab within UTS.
>
> > --
>
> > Sammy Pfeiffer
> > PhD Candidate at The Magic Lab within UTS.
>
>
>
>=C2=A0 References:
>=C2=A0 =C2=A0 1. mailto:sammypfeiffer@gmail.com
>=C2=A0 =C2=A0 2. https://gist.github.com/awesomebytes/ba4df9c6ef3ab5742= cb9aceed7998c3d#file-bootstrap-prefix-sh-L3033
>=C2=A0 =C2=A0 3. https://gist.github.com/awesomebytes/ba4df9c6ef3ab5742= cb9aceed7998c3d#file-bootstrap-prefix-sh-L3082
>=C2=A0 =C2=A0 4. https://gist.github.com/awesomebytes/ba4df9c6ef3= ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L2200-L2247
>=C2=A0 =C2=A0 5. https://gist.github.com/awesomebytes/ba4df9c6ef3= ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L2273-L2433
>=C2=A0 =C2=A0 6. https://gist.github.com/awesomebytes/ba4df9c6ef3= ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L2443-L2501
>=C2=A0 =C2=A0 7. https://gist.github.com/awesomebytes/ba4df9c6ef3= ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L2503-L2576
>=C2=A0 =C2=A0 8. https://gist.github.com/awesomebytes/ba4df9c6ef3= ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L2578-L2608
>=C2=A0 =C2=A0 9. https://gist.github.com/awesomebytes/ba4df9c6ef3= ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L2610-L2681
>=C2=A0 =C2=A010. https://gist.github.com/awesomebytes/ba4df9c6ef3= ab5742cb9aceed7998c3d#file-bootstrap-prefix-sh-L2724-L2726
>=C2=A0 =C2=A011. https://gist.github.com/awesomebytes/ba4df9c6ef3ab5742= cb9aceed7998c3d#file-bootstrap-prefix-sh-L1355
>=C2=A0 =C2=A012. mailto:sammypfeiffer@gmail.com
>=C2=A0 =C2=A013. mailto:heroxbd@gentoo.org
>=C2=A0 =C2=A014. mailto:sammypfeiffer@gmail.com
>=C2=A0 =C2=A015. https://gi= st.github.com/awesomebytes/3468477c6c90fe3d985372d50aabba9f
>=C2=A0 =C2=A016. https://wik= i.gentoo.org/wiki/Project:Prefix/Manual_Bootstrap#Stage_1
>=C2=A0 =C2=A017. https://gitweb.gentoo.org/repo/p= roj/prefix.git/
>=C2=A0 =C2=A018. https://github.com/awesomebytes= /gentoo_prefix_ci
>=C2=A0 =C2=A019. https://github.com/awesomeb= ytes/gentoo_prefix_ci_32b
>
> read_char: errno=3D=3DEILSEQ; invalid byte sequence for UTF-8:
--
Fabian Groffen
Gentoo on a different level


--

Sammy Pfeiffer
PhD Candidate at The Magic Lab within UTS.
--000000000000de6a06059da3e1c1--