From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id 5269613933E for ; Mon, 19 Jul 2021 10:29:10 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 88D45E0C19; Mon, 19 Jul 2021 10:29:09 +0000 (UTC) Received: from smtp.gentoo.org (dev.gentoo.org [IPv6:2001:470:ea4a:1:5054:ff:fec7:86e4]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id 94529E0C19 for ; Mon, 19 Jul 2021 10:29:08 +0000 (UTC) Received: from [2a0c:b641:69c:e7f1::2] (port=42930 helo=aurora) by muon with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94) (envelope-from ) id 1m5QWG-0003W2-Ox for gentoo-soc@lists.gentoo.org; Mon, 19 Jul 2021 10:29:04 +0000 From: Benda Xu To: gentoo-soc@lists.gentoo.org Subject: Re: [gentoo-soc] Week 6 Report for Big Data Infrastructure and H2O ebuilds Project References: Date: Mon, 19 Jul 2021 18:29:01 +0800 In-Reply-To: (Yuan Liao's message of "Sun, 18 Jul 2021 22:52:32 -0700") Message-ID: <87wnpmk36a.fsf@gentoo.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-soc@lists.gentoo.org Reply-to: gentoo-soc@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 Content-Type: text/plain X-Archives-Salt: 5ad68e27-854a-436a-9b37-55a66e0d3b47 X-Archives-Hash: 02043e71ae44011544aa132f4bb1ceb2 "Yuan Liao (Leo)" writes: > This week, I have been busy with an eclectic mix of tasks. First of > all, I continued to make improvements to ebuild-commander so it can > run well both in CI environments and on any developers' personal > workstations. Another related accomplishment was a new ebuild > installation test case to be run in GitHub Actions that would have > coverage over all ebuilds in the Spark overlay. Last but not least, I > further expanded the documentation for the Kotlin packages I had > created a few weeks ago with the addition of a maintainer-oriented > wiki page. > > The bulk of the improvements to ebuild-commander are new features > designed to optimize its user experience in an interactive > environment. While I was developing ebuild-commander and creating new > test cases for ebuilds in the Spark overlay, I would often run the > test cases from an interactive shell on my own computer before > creating a Git commit. Sometimes I realized something was wrong > immediately after the tests were launched, so I would interrupt > ebuild-commander at once, fix the problem, and re-run the tests. The > interruption would almost always be followed by a deletion of the > Docker container created by ebuild-commander, which was tedious when > it was done manually. So, I added SIGINT handling to ebuild-commander > to let it automatically clean up the container it created upon > interruption. By contrast, ebuild-commander might also perform an > automatic clean-up when the user might want the container to be > retained. For example, if the test fails, the user probably wishes to > open up a shell in the container to inspect it, in which case the > container should not be cleaned up. To deal with this situation, I > added a new command-line option to ebuild-commander for controlling > the automatic clean-up behavior. By default, the clean-up would be > skipped if the test failed, but the user can also choose to keep the > container even if the test succeeded or always remove it even upon > test failure. > > As per my original project proposal, I am also adding a test case for > the ebuild installation tests which will ensure every package in the > Spark overlay can be installed at least once. Adding every package to > the emerge command theoretically works, but the command would be too > long. Invoking emerge separately for each package would resolve this > problem, but the overhead of emerge's dependency calculation would > seriously impact the test runtime. I came up with a solution that > could address both issues: write a script to compute a list of leaf > packages in the Spark overlay and pass the packages in the list to > emerge, so every package in the overlay would be installed, and the > emerge command can be simplified to have a shorter length too. The > script can also act as a helpful tool for any ebuild repository's > maintainers to find out all leaf packages in the repository for > maintenance tasks like last-rite and package clean-up. After some > initial optimization and tuning, the script (written in Python) can > compute a list of leaf packages among about 500 packages in the Spark > overlay within only a few minutes. The optimization and tuning is > also the topic for this week's blog post of mine [1]. This post > covers some knowledge and topics from computer science, including > graph theory, graph algorithms, data structure, and time complexity. > If you are interested in any of those subjects, make sure you don't > miss it! > > Meanwhile, I created a new page on Gentoo Wiki for the Kotlin Package > Maintainer Guide [2] and added explanations for how a Kotlin package > is built by the upstream, instructions to find the commands that > should be used to compile the Kotlin packages, and a tutorial for > using the eclasses I authored to support the Kotlin ebuilds. I hope > it can be useful to both any future developers who would like to help > maintain those Kotlin packages and myself after a few months in case I > forget the details of my own work. > > This concludes my work during the past week and this report. Thank > you for reading it (and my blog post in case you are checking it out)! Thank you! Keeping going with good work. Yours, Benda