From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id 32EB9138334 for ; Wed, 23 Oct 2019 10:15:20 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 32F85E0A5C; Wed, 23 Oct 2019 10:15:15 +0000 (UTC) Received: from smtp.gentoo.org (woodpecker.gentoo.org [IPv6:2001:470:ea4a:1:5054:ff:fec7:86e4]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id 124C0E08D6 for ; Wed, 23 Oct 2019 10:15:14 +0000 (UTC) Received: from localhost.localdomain (c134-66.icpnet.pl [85.221.134.66]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: mgorny) by smtp.gentoo.org (Postfix) with ESMTPSA id 2D51B34BD1B; Wed, 23 Oct 2019 10:15:12 +0000 (UTC) From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= To: gentoo-dev@lists.gentoo.org Cc: python@gentoo.org, =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Subject: [gentoo-dev] [PATCH] install-qa-check.d: Add check for missing Python bytecode (.pyc) Date: Wed, 23 Oct 2019 12:14:47 +0200 Message-Id: <20191023101447.157198-1-mgorny@gentoo.org> X-Mailer: git-send-email 2.23.0 Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-dev@lists.gentoo.org Reply-to: gentoo-dev@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Archives-Salt: 33119b1c-aa0a-4793-b6ef-3c5db110308d X-Archives-Hash: b2deb6cba3dc6be08f697aafc937f65a Add a check that detects Python modules that were not compiled after installation. To limit false positives, this is only done on modules installed to site-packages. Early testing of this check made it possible to detect a bug in python_optimize. Signed-off-by: Michał Górny --- metadata/install-qa-check.d/60python-pyc | 83 ++++++++++++++++++++++++ 1 file changed, 83 insertions(+) create mode 100644 metadata/install-qa-check.d/60python-pyc diff --git a/metadata/install-qa-check.d/60python-pyc b/metadata/install-qa-check.d/60python-pyc new file mode 100644 index 000000000000..7fe3b0c963d1 --- /dev/null +++ b/metadata/install-qa-check.d/60python-pyc @@ -0,0 +1,83 @@ +# Copyright 2019 Gentoo Authors +# Distributed under the terms of the GNU General Public License v2 + +# QA check: ensure that Python modules are compiled after installing +# Maintainer: Python project + +inherit python-utils-r1 + +python_pyc_check() { + local impl missing=() outdated=() + for impl in "${_PYTHON_SUPPORTED_IMPLS[@]}"; do + python_export "${impl}" EPYTHON PYTHON + local sitedir=$(python_get_sitedir "${impl}") + + if [[ -d ${D}${sitedir} ]]; then + local suffixes=() subdir= + case ${EPYTHON} in + python2*) + suffixes=( .py{c,o} ) + ;; + pypy) + suffixes=( .pyc ) + ;; + python3*|pypy3*) + local tag=$("${PYTHON}" -c 'import sys; print(sys.implementation.cache_tag)') + suffixes=( ".${tag}"{,.opt-{1,2}}.pyc ) + subdir=__pycache__/ + ;; + *) + # skip testing unknown impl + continue + ;; + esac + + einfo "Verifying compiled files in ${sitedir}" + local f s + while read -d $'\0' -r f; do + local dir=${f%/*} + local basename=${f##*/} + basename=${basename%.py} + + for s in "${suffixes[@]}"; do + local cache=${dir}/${subdir}${basename}${s} + if [[ ! -f ${cache} ]]; then + missing+=( "${cache}" ) + elif [[ ${f} -nt ${cache} ]]; then + outdated+=( "${cache}" ) + fi + done + done < <(find "${D}${sitedir}" -name '*.py' -print0) + fi + done + + if [[ ${missing[@]} ]]; then + eqawarn + eqawarn "This package installs one or more Python modules that are not byte-compiled." + eqawarn "The following files are missing:" + eqawarn + eqatag -v python-pyc.missing "${missing[@]#${D}}" + fi + + if [[ ${outdated[@]} ]]; then + eqawarn + eqawarn "This package installs one or more compiled Python modules that have older" + eqawarn "timestamps than the corresponding source files:" + eqawarn + eqatag -v python-pyc.outdated "${outdated[@]#${D}}" + fi + + if [[ ${missing[@]} || ${outdated[@]} ]]; then + eqawarn + eqawarn "Please either fix the upstream build system to byte-compile Python modules" + eqawarn "correctly, or call python_optimize after installing them. For more" + eqawarn "information, see:" + eqawarn "https://wiki.gentoo.org/wiki/Project:Python/Byte_compiling" + eqawarn + fi +} + +python_pyc_check +: # guarantee successful exit + +# vim:ft=ebuild -- 2.23.0