From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id AF2CA158020 for ; Sun, 6 Nov 2022 08:15:47 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 7A8F1E0858; Sun, 6 Nov 2022 08:15:44 +0000 (UTC) Received: from smtp.gentoo.org (woodpecker.gentoo.org [140.211.166.183]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id 3F403E0826 for ; Sun, 6 Nov 2022 08:15:44 +0000 (UTC) Message-ID: <2bd8402513dd47b06aa38a0d49e1c29880c1aad4.camel@gentoo.org> Subject: [gentoo-dev] Disturbing state of arch testing in Gentoo From: =?UTF-8?Q?Micha=C5=82_G=C3=B3rny?= To: gentoo-dev@lists.gentoo.org Date: Sun, 06 Nov 2022 09:15:40 +0100 Organization: Gentoo Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.44.4 Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-dev@lists.gentoo.org Reply-to: gentoo-dev@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 X-Archives-Salt: fd5f04f7-40fd-4c55-94ae-213001db944e X-Archives-Hash: b9a014a18091cac95368e15233fe02d2 Hi, everyone. Arch testing's relying on automation a lot these days. Not saying that's bad, if it improves the state of affairs. However, I have some concerns, based on what I've seen lately. On top of that, it seems that most of it still relies on proprietary software and we have no clue how *exactly* it works, and it's really, really hard to get a straight answer. So, my questions are: 1. Is "runtime testing required" field being respected? Obviously not every package can be (sufficiently) tested via FEATURES=3Dtest, so we've added that fields. However, if arch testers just ignore it and push things stable based on pure build testing... 2. How are kernels being tested? Given the speed with which new gentoo- sources stablereqs are handled, I really feel like "arch testing" there means "checking if sources install", and have little to do with working kernels. 3. How does the automation handle packages that aren't trivially installable? I recall that in the past stablereqs were stalled for months without a single comment because automation couldn't figure out how to proceed, and nobody bothered reporting a problem. --=20 Best regards, Micha=C5=82 G=C3=B3rny