From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) by finch.gentoo.org (Postfix) with ESMTP id 74333138262 for ; Tue, 17 May 2016 08:46:56 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id D8014141A8; Tue, 17 May 2016 08:46:46 +0000 (UTC) Received: from mail.schwarzvogel.de (skade.schwarzvogel.de [144.76.18.87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id E6F7A21C054 for ; Tue, 17 May 2016 08:46:45 +0000 (UTC) Received: from klausman by mail.schwarzvogel.de with local (Exim 4.87) (envelope-from ) id 1b2aeJ-0007fQ-Fi for gentoo-dev@lists.gentoo.org; Tue, 17 May 2016 10:46:43 +0200 Date: Tue, 17 May 2016 10:46:43 +0200 From: Tobias Klausmann To: gentoo-dev@lists.gentoo.org Subject: Re: [gentoo-dev] Proposal for changes for the next EAPI version Message-ID: <20160517084643.GA24972@skade.schwarzvogel.de> Mail-Followup-To: gentoo-dev@lists.gentoo.org References: <20160516183840.4b241463@gentp.lnet> Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-dev@lists.gentoo.org Reply-to: gentoo-dev@lists.gentoo.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.6.1 (2016-04-27) Sender: Tobias Klausmann X-Archives-Salt: 413a0c0a-3840-4f2e-bca1-317aa5c6ffbd X-Archives-Hash: feeced6344d04a48756131fbbbb98b6d Hi! On Tue, 17 May 2016, Kent Fredric wrote: > On 17 May 2016 at 19:37, Pallav Agarwal wrote: > > For normal users we wouldn't. But currently, arch-testers need to make a > > judgement call on what to test when a stable-req bug is filed. Tests run in > > src_test are provided by upstream, and does not guarantee that a package > > that has been merged will actually run on the system. > > If the maintainer could add a couple small scripts to check basic > > functionality > > of the merged package, it would make testing for arch testers much easier > > and reliable. > > Let me give an example. Let's say > > app-misc/screenfetch-2.7.7 is the current stable package for screenfetch in > > the portage tree. > > However, on running, it produces an error on the top, along with the proper > > output. > > If screenfetch-3.0.0 happens to fix that error, maintainer can add a simple > > script > > > > if [ "$(screenfetch 2>&1 1>/dev/null)" != "" ] then > > eerror "Still producing error" > > fi > > > > To make sure the build is properly updating the screenfetch version, and > > that > > the bug has in fact been fixed. This is the only way I can see to reliabily > > and automatically test packages that have been merged successfully > > > I don't think this needs to be an EAPI change. And if we can acheive > the goal without one, the better. Agreed. > [... lots of interesting stuff ...] Or maintainers and teams could do what the Emacs team does: provide test plans[0]. Naturally, I'd prefer something automated over the very hands-on tests that are a bit of a necessity for an application like Emacs, but they are still better than nothing. Either way, I think it is preferable to have non-upstream tests of whatever shape we pick to be separate from the ebuilds and the source files, for the simple reason that most users won't care about them. Those who do can retrieve them in the same manner as the ATs. And as for my pet peeve, tests that are known to fail, can we also annotate that somehow so I don't waste hours running a test suite that gives zero signal on whether I should add the stable keyword? Even a one-line hin in the stabilization request would be nice. As it is, I keep a list of known-to-fail packages and my testing machinery tells me to not bother with FEATURES=test in those case. Not ideal, but less time-wasty. Regards, Tobias [0] https://wiki.gentoo.org/wiki/Project:Emacs/Test_plans -- if (user_specified) /* Didn't work, but the user is convinced this is the * place. */ linux-2.4.0-test2/drivers/parport/parport_pc.c