From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id 0EAD4158043 for ; Tue, 16 Apr 2024 01:40:30 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 7B61AE29F3; Tue, 16 Apr 2024 01:40:28 +0000 (UTC) Received: from smtp.gentoo.org (mail.gentoo.org [IPv6:2001:470:ea4a:1:5054:ff:fec7:86e4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id 4E197E29F3 for ; Tue, 16 Apr 2024 01:40:28 +0000 (UTC) Received: from oystercatcher.gentoo.org (oystercatcher.gentoo.org [148.251.78.52]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp.gentoo.org (Postfix) with ESMTPS id 3CF3C343168 for ; Tue, 16 Apr 2024 01:40:27 +0000 (UTC) Received: from localhost.localdomain (localhost [IPv6:::1]) by oystercatcher.gentoo.org (Postfix) with ESMTP id A07BA16FA for ; Tue, 16 Apr 2024 01:40:25 +0000 (UTC) From: "Sam James" To: gentoo-commits@lists.gentoo.org Content-Transfer-Encoding: 8bit Content-type: text/plain; charset=UTF-8 Reply-To: gentoo-dev@lists.gentoo.org, "Sam James" Message-ID: <1713231528.ef9d26ba1e410d25b94cab81a6606e96e85d35d8.sam@gentoo> Subject: [gentoo-commits] repo/gentoo:master commit in: eclass/ X-VCS-Repository: repo/gentoo X-VCS-Files: eclass/toolchain.eclass X-VCS-Directories: eclass/ X-VCS-Committer: sam X-VCS-Committer-Name: Sam James X-VCS-Revision: ef9d26ba1e410d25b94cab81a6606e96e85d35d8 X-VCS-Branch: master Date: Tue, 16 Apr 2024 01:40:25 +0000 (UTC) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-commits@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply X-Archives-Salt: a14a6f0b-6867-4448-b389-2ef26fcf91f2 X-Archives-Hash: 3ed073070aa1072ae400f33d5ea85ee2 commit: ef9d26ba1e410d25b94cab81a6606e96e85d35d8 Author: Gabi Falk gmx com> AuthorDate: Mon Apr 15 08:00:00 2024 +0000 Commit: Sam James gentoo org> CommitDate: Tue Apr 16 01:38:48 2024 +0000 URL: https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=ef9d26ba toolchain.eclass: add a variable to force baseline regeneration Suggested-by: Sam James gentoo.org> Signed-off-by: Gabi Falk gmx.com> Closes: https://github.com/gentoo/gentoo/pull/36244 Signed-off-by: Sam James gentoo.org> eclass/toolchain.eclass | 33 +++++++++++++++++++++++++-------- 1 file changed, 25 insertions(+), 8 deletions(-) diff --git a/eclass/toolchain.eclass b/eclass/toolchain.eclass index 786d65da2d9a..2a74e6f165df 100644 --- a/eclass/toolchain.eclass +++ b/eclass/toolchain.eclass @@ -117,6 +117,13 @@ tc_version_is_between() { # Ignore missing baseline/reference data and create new baseline. : "${GCC_TESTS_IGNORE_NO_BASELINE:=}" +# @ECLASS_VARIABLE: GCC_TESTS_REGEN_BASELINE +# @DEFAULT_UNSET +# @USER_VARIABLE +# @DESCRIPTION: +# Ignore baseline/reference data and create new baseline. +: "${GCC_TESTS_REGEN_BASELINE:=}" + # @ECLASS_VARIABLE: GCC_TESTS_CHECK_TARGET # @USER_VARIABLE # @DESCRIPTION: @@ -1917,17 +1924,25 @@ toolchain_src_test() { --manifest="${T}"/${CHOST}.xfail \ --produce_manifest &> /dev/null - if [[ -f "${GCC_TESTS_COMPARISON_DIR}/${GCC_TESTS_COMPARISON_SLOT}/${CHOST}.xfail" ]] ; then + local manifest="${GCC_TESTS_COMPARISON_DIR}/${GCC_TESTS_COMPARISON_SLOT}/${CHOST}.xfail" + + if [[ -f "${manifest}" ]] ; then # TODO: Distribute some baseline results in e.g. gcc-patches.git? # validate_failures.py manifest files support include directives. - einfo "Comparing with previous cached results at GCC_TESTS_COMPARISON_DIR=${GCC_TESTS_COMPARISON_DIR}/${GCC_TESTS_COMPARISON_SLOT}/${CHOST}.xfail" + einfo "Comparing with previous cached results at ${manifest}" - edo "${T}"/validate_failures.py \ + nonfatal edo "${T}"/validate_failures.py \ --srcpath="${S}" \ --build_dir="${WORKDIR}"/build \ - --manifest="${GCC_TESTS_COMPARISON_DIR}/${GCC_TESTS_COMPARISON_SLOT}/${CHOST}.xfail" + --manifest="${manifest}" + ret=$? + + if [[ -n ${GCC_TESTS_REGEN_BASELINE} ]] ; then + eerror "GCC_TESTS_REGEN_BASELINE is set, ignoring test result and creating a new baseline..." + elif [[ ${ret} != 0 ]]; then + die "Tests failed (failures not listed in the baseline data)" + fi else - # nonfatal first because we want to run again with comparison data if available. nonfatal edo "${T}"/validate_failures.py \ --srcpath="${S}" \ --build_dir="${WORKDIR}"/build @@ -1935,13 +1950,15 @@ toolchain_src_test() { # We have no reference data saved from a previous run to know if # the failures are tolerable or not, so we bail out. - eerror "No reference test data at GCC_TESTS_COMPARISON_DIR=${GCC_TESTS_COMPARISON_DIR}/${GCC_TESTS_COMPARISON_SLOT}/${CHOST}.xfail!" + eerror "No reference test data at ${manifest}!" eerror "GCC's tests require a baseline to compare with for any reasonable interpretation of results." if [[ -n ${GCC_TESTS_IGNORE_NO_BASELINE} ]] ; then - eerror "GCC_TESTS_IGNORE_NO_BASELINE is set, creating new baseline manifest..." + eerror "GCC_TESTS_IGNORE_NO_BASELINE is set, ignoring test result and creating a new baseline..." + elif [[ -n ${GCC_TESTS_REGEN_BASELINE} ]] ; then + eerror "GCC_TESTS_REGEN_BASELINE is set, ignoring test result and creating using a new baseline..." elif [[ ${ret} != 0 ]]; then - eerror "(Set GCC_TESTS_IGNORE_NO_BASELINE=1 to make this non-fatal for initial run.)" + eerror "(Set GCC_TESTS_IGNORE_NO_BASELINE=1 to make this non-fatal and generate a baseline.)" die "Tests failed (failures occurred with no reference data)" fi fi