From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <gentoo-catalyst+bounces-3754-garchives=archives.gentoo.org@lists.gentoo.org> Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id AB21A138359 for <garchives@archives.gentoo.org>; Tue, 3 Nov 2020 13:04:34 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 005C6E0A95; Tue, 3 Nov 2020 13:04:34 +0000 (UTC) Received: from smtp.gentoo.org (smtp.gentoo.org [140.211.166.183]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id A16F4E0A95 for <gentoo-catalyst@lists.gentoo.org>; Tue, 3 Nov 2020 13:04:33 +0000 (UTC) Date: Tue, 3 Nov 2020 08:04:12 -0500 From: Brian Dolbec <dolsen@gentoo.org> To: gentoo-catalyst@lists.gentoo.org Subject: Re: [gentoo-catalyst] catalyst changes for improving automation Message-ID: <20201103080412.3fd4e5a0@rogue1> In-Reply-To: <CAEdQ38H27b86LQ+HBUAmk7njS7-KJvTiqdNczn6mqcB3n7bF7w@mail.gmail.com> References: <CAEdQ38H27b86LQ+HBUAmk7njS7-KJvTiqdNczn6mqcB3n7bF7w@mail.gmail.com> X-Mailer: Claws Mail 3.17.7 (GTK+ 2.24.32; x86_64-pc-linux-gnu) Precedence: bulk List-Post: <mailto:gentoo-catalyst@lists.gentoo.org> List-Help: <mailto:gentoo-catalyst+help@lists.gentoo.org> List-Unsubscribe: <mailto:gentoo-catalyst+unsubscribe@lists.gentoo.org> List-Subscribe: <mailto:gentoo-catalyst+subscribe@lists.gentoo.org> List-Id: Gentoo Linux mail <gentoo-catalyst.gentoo.org> X-BeenThere: gentoo-catalyst@lists.gentoo.org Reply-to: gentoo-catalyst@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Archives-Salt: fbebd1cd-aaeb-4ccf-8161-c67de27af5c6 X-Archives-Hash: 480aa0fcf04a4af0555114d8038a4a08 On Mon, 2 Nov 2020 22:44:07 -0500 Matt Turner <mattst88@gentoo.org> wrote: > The catalyst-auto automation scripts live in a repo separate from > catalyst. That increases the difficulty of changing catalyst's > interface, and it doesn't seem to offer any advantages otherwise. > (Keeping build specs in a separate repo allows them to be updated > independent of catalyst and that is valuable). Additionally, since the > primary way catalyst is used is via this automation, it makes sense to > support this workflow in catalyst directly. > > But to get there, there are some changes to catalyst that I think are > improvements on their own and simplify the path to integrating > automation capabilities directly into catalyst. That's what I'd like > to discuss here. > I have been thinking for the past few years that the automation could benefit from using a buildbot to run the stages. In that way it would set the required variables, run the stages in sequence. Upon failure, buildbot makes it easy to re-run failed steps from where they left off. Or initiate unscheduled builds. It also makes it easy to see detailed logs (by anyone with a browser if the buildbot is public viewable) of the various steps for debugging, etc.. Perhaps with your new spec file, you add a varialbe that lists the stages to run in sequence. In that way it would preserve the old capability of running single stages independantly or a full sequence. Or perhaps a cli option to override the setting on an unedited spec file. ie: [build.stages] stage1 stage2 stage3 livecd1 livecd2 sorry, not familiar with toml In that way a spec file could be edited simply to restart from any point with a single variable edit without removing unused [build.???] definitions for the full run. This would be particularly useful when troubleshooting hidden/delayed stage build fails. Overall, I do agree that the releng automation scripts capabilities should be part of the caltlyst repo in order for them to be up-to-date with catalyst code. I have limited time and resources lately, so can't help out much until I get back home (probably Xmas) largely due to covid... I only have this small laptop and my eyes are not that good to be doing lots of work on a tiny 14 inch screen and a stage run to take hours instead of minutes... (yes, I got spoiled with 2 28 inch 4K displays, 16-core 128k ram system at home...) Brian Dolbec <dolsen@gentoo.org>