[gentoo-dev] [PATCH 0/4] install-qa-check.d/60distutils-use-setuptools future-proof and fixes

[gentoo-dev] [PATCH 0/4] install-qa-check.d/60distutils-use-setuptools future-proof and fixes

[Michał Górny]

Hi,

This patchset introduces three changes to the DISTUTILS_USE_SETUPTOOLS
check:

1. Packages that do not install .egg-info (i.e. do not use distutils)
   will now be required to use DISTUTILS_USE_SETUPTOOLS=no to avoid
   unnecessary dep on setuptools.  Previously, they were ignored.

2. Another class of entry points, 'gui_scripts' are now supported
   besides 'console_scripts'.

3. The check has been relaxed to allow both 'bdepend' and 'rdepend'
   for packages with entry points, the former being the new
   recommendation.


Explanation:

Originally, runner scripts using entry points were implemented using
setuptools' pkg_resources package.  Therefore, the check originally
enforced RDEPEND on dev-python/setuptools (i.e. pkg_resources provider)
whenever appropriate entry points were used.

However, modern versions of setuptools use either of three options,
in order of preference: importlib.metadata (built-in since Python 3.8),
importlib_metadata (backport) or pkg_resources.  This means that for
Python 3.8+, no additional dependency is necessary, and for Python 3.7
it could be either dev-python/importlib_metadata
or dev-python/setuptools.

Since the removal of Python 3.7 was not that far away, we didn't update
the check before and assumed that the overdependence on setuptools
is acceptable.  Now that it is getting really close, it is time to start
updating packages for it.

However, I don't think it's a good idea to suddenly make the check start
complaining about all the DISTUTILS_USE_SETUPTOOLS values that were
correct before, or for ago to start filing thousands of bugs about it.
For this reason, the check will now run in 'relaxed' mode that allows
both old (rdepend) and new (bdepend) value for entry point use.

Developers who want to start updating their packages early will be able
to set DISTUTILS_STRICT_ENTRY_POINTS=1 in their make.conf, and start
updating the variable on version bumps.

The new value is not 100% strictly correct until we actually remove
Python 3.7 (i.e. until 2021-06-01) but it shouldn't cause any real
issues.

-- 
Best regards,
Michał Górny


Michał Górny (4):
  install-qa-check.d: Make DUS check distinguish entry points
  install-qa-check.d: Extend DUS check to handle no egg-info
  install-qa-check.d: Future-proof entry point values for DUS
  install-qa-check.d: Make DUS aware of gui_scripts

 .../60distutils-use-setuptools                | 37 ++++++++++++-------
 1 file changed, 24 insertions(+), 13 deletions(-)

-- 
2.31.1

[gentoo-dev] [PATCH 1/4] install-qa-check.d: Make DUS check distinguish entry points

[Michał Górny]

Make DISTUTILS_USE_SETUPTOOLS check explicitly distinguish rdepend due
to entry points from explicit rdepend.  This currently does not change
the result but it will change in the future.

Signed-off-by: Michał Górny <mgorny@gentoo.org>
---
 .../60distutils-use-setuptools                   | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/metadata/install-qa-check.d/60distutils-use-setuptools b/metadata/install-qa-check.d/60distutils-use-setuptools
index 3b9523fcdaba..8970a6ab9989 100644
--- a/metadata/install-qa-check.d/60distutils-use-setuptools
+++ b/metadata/install-qa-check.d/60distutils-use-setuptools
@@ -1,4 +1,4 @@
-# Copyright 2020 Gentoo Authors
+# Copyright 2020-2021 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 # QA check: verify correctness of DISTUTILS_USE_SETUPTOOLS
@@ -24,17 +24,16 @@ distutils_use_setuptools_check() {
 				if [[ -f ${egg} ]]; then
 					# if .egg-info is a file, it's plain distutils
 					new_expected=no
-				elif grep -q -s -F '[console_scripts]' \
-						"${egg}"/entry_points.txt
-				then
-					# entry_points == we need rdepend
-					new_expected=rdepend
 				elif [[ -f ${egg}/requires.txt ]] &&
 						grep -q -s '^setuptools' \
 						<(sed -e '/^\[/,$d' "${egg}"/requires.txt)
 				then
 					# explicit *unconditional* rdepend in package metadata
 					new_expected=rdepend
+				elif grep -q -s -F '[console_scripts]' \
+						"${egg}"/entry_points.txt
+				then
+					new_expected=entry-point
 				else
 					new_expected=bdepend
 				fi
@@ -46,7 +45,6 @@ distutils_use_setuptools_check() {
 		fi
 	done
 
-	# at this point, expected can contain: no bdepend rdepend
 	if [[ ${#expected[@]} -gt 1 ]] && has no "${expected[@]}"; then
 		# 'no' and '[rb]depend' are mutually exclusive
 		eerror "The package seems to have used distutils and setuptools simultaneously."
@@ -54,8 +52,10 @@ distutils_use_setuptools_check() {
 		eerror "https://dev.gentoo.org/~mgorny/python-guide/distutils.html#conditional-distutils-setuptools-use-in-packages"
 		eerror "Please report a bug about this and CC python@"
 	elif [[ ${#expected[@]} -gt 0 ]]; then
-		# bdepend+rdepend=rdepend
+		# *+rdepend=rdepend
 		has rdepend "${expected[@]}" && expected=( rdepend )
+		# for the time being, entry points imply rdepend
+		has entry-point "${expected[@]}" && expected=( rdepend )
 		# at this point, expected should have exactly one value
 		[[ ${#expected[@]} -eq 1 ]] || die "integrity error"
 
-- 
2.31.1

[gentoo-dev] [PATCH 2/4] install-qa-check.d: Extend DUS check to handle no egg-info

[Michał Górny]

If the package does not install any .egg-info files, i.e. distutils
is probably not used at all, we do not the eclass to create a setuptools
dependency.

Signed-off-by: Michał Górny <mgorny@gentoo.org>
---
 metadata/install-qa-check.d/60distutils-use-setuptools | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/metadata/install-qa-check.d/60distutils-use-setuptools b/metadata/install-qa-check.d/60distutils-use-setuptools
index 8970a6ab9989..61209302ef3e 100644
--- a/metadata/install-qa-check.d/60distutils-use-setuptools
+++ b/metadata/install-qa-check.d/60distutils-use-setuptools
@@ -51,7 +51,10 @@ distutils_use_setuptools_check() {
 		eerror "This could mean the package has bad conditions:"
 		eerror "https://dev.gentoo.org/~mgorny/python-guide/distutils.html#conditional-distutils-setuptools-use-in-packages"
 		eerror "Please report a bug about this and CC python@"
-	elif [[ ${#expected[@]} -gt 0 ]]; then
+	else
+		# if we did not find anything, also assume 'no' is desired,
+		# we do not want the setuptools dep
+		[[ ${#expected[@]} -eq 0 ]] && expected=( no )
 		# *+rdepend=rdepend
 		has rdepend "${expected[@]}" && expected=( rdepend )
 		# for the time being, entry points imply rdepend
-- 
2.31.1

[gentoo-dev] [PATCH 3/4] install-qa-check.d: Future-proof entry point values for DUS

[Michał Górny]

Accept both 'rdepend' and 'bdepend' as DISTUTILS_USE_SETUPTOOLS values
for packages installing entry points.  Starting with Python 3.8, these
packages do not require any runtime dependency.  However, we do not want
to proactively bother people with updating everything and instead
provide the option to start performing opt-in updates.

Note that technically PyPy3 (that implements Python 3.7 at the moment)
continues requiring additional dependencies for entry point support.
However, we do not support PyPy3 target in stable and generally do not
consider it important enough to maintain extra complexity for.

Signed-off-by: Michał Górny <mgorny@gentoo.org>
---
 .../60distutils-use-setuptools                | 20 +++++++++++++------
 1 file changed, 14 insertions(+), 6 deletions(-)

diff --git a/metadata/install-qa-check.d/60distutils-use-setuptools b/metadata/install-qa-check.d/60distutils-use-setuptools
index 61209302ef3e..69164608fcaa 100644
--- a/metadata/install-qa-check.d/60distutils-use-setuptools
+++ b/metadata/install-qa-check.d/60distutils-use-setuptools
@@ -57,18 +57,26 @@ distutils_use_setuptools_check() {
 		[[ ${#expected[@]} -eq 0 ]] && expected=( no )
 		# *+rdepend=rdepend
 		has rdepend "${expected[@]}" && expected=( rdepend )
-		# for the time being, entry points imply rdepend
-		has entry-point "${expected[@]}" && expected=( rdepend )
-		# at this point, expected should have exactly one value
-		[[ ${#expected[@]} -eq 1 ]] || die "integrity error"
 
-		if [[ ${DISTUTILS_USE_SETUPTOOLS} != ${expected} ]]; then
+		if has entry-point "${expected[@]}"; then
+			if [[ ${DISTUTILS_STRICT_ENTRY_POINTS} ]]; then
+				# option for devs to future-proof their packages
+				# please don't let ago enable it
+				expected=( bdepend )
+			else
+				# accept either rdepend or bdepend for the time being
+				# but suggest bdepend as the future-proof option
+				expected=( bdepend rdepend )
+			fi
+		fi
+
+		if ! has ${DISTUTILS_USE_SETUPTOOLS} "${expected[@]}"; then
 				local def=
 				[[ ${DISTUTILS_USE_SETUPTOOLS} == bdepend ]] && def=' (or unset)'
 
 				eqawarn "DISTUTILS_USE_SETUPTOOLS value is probably incorrect"
 				eqawarn "  have:     DISTUTILS_USE_SETUPTOOLS=${DISTUTILS_USE_SETUPTOOLS}${def}"
-				eqawarn "  expected: DISTUTILS_USE_SETUPTOOLS=${expected}"
+				eqawarn "  expected: DISTUTILS_USE_SETUPTOOLS=${expected[0]}"
 		fi
 	fi
 }
-- 
2.31.1

[gentoo-dev] [PATCH 4/4] install-qa-check.d: Make DUS aware of gui_scripts

[Michał Górny]

Make DISTUTILS_USE_SETUPTOOLS check aware of gui_scripts in addition
to console_scripts.  Thanks to Arfrever for reporting.

Signed-off-by: Michał Górny <mgorny@gentoo.org>
---
 metadata/install-qa-check.d/60distutils-use-setuptools | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/metadata/install-qa-check.d/60distutils-use-setuptools b/metadata/install-qa-check.d/60distutils-use-setuptools
index 69164608fcaa..d5b9f87a21a1 100644
--- a/metadata/install-qa-check.d/60distutils-use-setuptools
+++ b/metadata/install-qa-check.d/60distutils-use-setuptools
@@ -30,7 +30,7 @@ distutils_use_setuptools_check() {
 				then
 					# explicit *unconditional* rdepend in package metadata
 					new_expected=rdepend
-				elif grep -q -s -F '[console_scripts]' \
+				elif grep -q -s '\[\(console\|gui\)_scripts\]' \
 						"${egg}"/entry_points.txt
 				then
 					new_expected=entry-point
-- 
2.31.1