[gentoo-dev] [PATCH 1/2] distutils-r1.eclass: disable setuptools validation of pypi.org allowed strings

[gentoo-dev] [PATCH 1/2] distutils-r1.eclass: disable setuptools validation of pypi.org allowed strings

[Eli Schwartz]

Trove classifiers, and their officialness, have no effect on a wheel
other than determining whether they are allowed to be uploaded to a
non-Gentoo website, and enabling the search index of that other site.

We don't need this, and we don't need to validate it. Setuptools will
disable validation if both of:

- network downloads failed

- cannot successfully import the `trove_classifiers` module

occurs. If trove-classifiers is installed by coincidence, this breaks
builds when it doesn't get updated on an extremely rapid basis and some
random package in dev-python/* uses a classifier that was made official
just the other day.

We could solve this another way, by making dev-python/setuptools
PDEPEND on trove-classifiers, and constantly bump the >= dependency. But
this is a pointless hassle. In fact, we're actually doing it, and it's
been a pointless hassle. We need to maintain up-to-the-minute minimum
bounds on the very latest version, and bump setuptools to a new -rX just
to update the minimum version of a package it doesn't even depend on. We
need to package new versions of trove-classifiers before *other* Gentoo
Devs outside of the python project, can successfully revbump their own
packages. We need to coordinate stabilization of trove-classifiers in
combination with those other packages. We force people to install a
pointless package. We overuse PDEPEND.

Instead, prevent the module from being successfully imported if the
package being built, doesn't actually depend on it. Then we don't
actually need it to be installed, and all is well. This can be done at
the small cost of installing a fake "trove_classifiers" module and
mutating PYTHONPATH. It is inelegant and meh, but upstream has stated
that they will not implement a real fix.

Bug: https://github.com/pypa/setuptools/issues/4459
Signed-off-by: Eli Schwartz <eschwartz@gentoo.org>
---
 eclass/distutils-r1.eclass | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/eclass/distutils-r1.eclass b/eclass/distutils-r1.eclass
index 7ab8dcae3265..9b9ba4b803ac 100644
--- a/eclass/distutils-r1.eclass
+++ b/eclass/distutils-r1.eclass
@@ -1360,6 +1360,17 @@ distutils_pep517_install() {
 					EOF
 				)
 			fi
+			if [[ ${BDEPEND} != *dev-python/trove-classifiers* ]]; then
+				# setuptools will try to import this package, if it is installed,
+				# and run validations we don't need or want that depend on
+				# having the most recent trove_classifiers version installed.
+				#
+				# https://github.com/pypa/setuptools/issues/4459
+				cat >> "${T}/trove_classifiers.py" <<-EOF || die
+					raise ImportError("hide real trove_classifiers")
+				EOF
+				local -x PYTHONPATH="${T}${PYTHONPATH+:${PYTHONPATH}}"
+			fi
 			;;
 		sip)
 			if [[ -n ${DISTUTILS_ARGS[@]} ]]; then
-- 
2.45.2

[gentoo-dev] [PATCH 2/2] dev-python/setuptools: drop no longer necessary trove-classifiers workaround

[Eli Schwartz]

Signed-off-by: Eli Schwartz <eschwartz@gentoo.org>
---
 ...uptools-74.1.3-r1.ebuild => setuptools-74.1.3-r2.ebuild} | 6 ------
 ...setuptools-75.2.0.ebuild => setuptools-75.2.0-r1.ebuild} | 6 ------
 ...setuptools-75.3.0.ebuild => setuptools-75.3.0-r1.ebuild} | 6 ------
 3 files changed, 18 deletions(-)
 rename dev-python/setuptools/{setuptools-74.1.3-r1.ebuild => setuptools-74.1.3-r2.ebuild} (93%)
 rename dev-python/setuptools/{setuptools-75.2.0.ebuild => setuptools-75.2.0-r1.ebuild} (93%)
 rename dev-python/setuptools/{setuptools-75.3.0.ebuild => setuptools-75.3.0-r1.ebuild} (93%)

diff --git a/dev-python/setuptools/setuptools-74.1.3-r1.ebuild b/dev-python/setuptools/setuptools-74.1.3-r2.ebuild
similarity index 93%
rename from dev-python/setuptools/setuptools-74.1.3-r1.ebuild
rename to dev-python/setuptools/setuptools-74.1.3-r2.ebuild
index 9cc97e5921d2..b44344c86a82 100644
--- a/dev-python/setuptools/setuptools-74.1.3-r1.ebuild
+++ b/dev-python/setuptools/setuptools-74.1.3-r2.ebuild
@@ -64,14 +64,8 @@ BDEPEND="
 "
 # setuptools-scm is here because installing plugins apparently breaks stuff at
 # runtime, so let's pull it early. See bug #663324.
-#
-# trove-classifiers are optionally used in validation, if they are
-# installed.  Since we really oughtn't block them, let's always enforce
-# the newest version for the time being to avoid errors.
-# https://github.com/pypa/setuptools/issues/4459
 PDEPEND="
 	dev-python/setuptools-scm[${PYTHON_USEDEP}]
-	>=dev-python/trove-classifiers-2024.10.16[${PYTHON_USEDEP}]
 "
 
 src_prepare() {
diff --git a/dev-python/setuptools/setuptools-75.2.0.ebuild b/dev-python/setuptools/setuptools-75.2.0-r1.ebuild
similarity index 93%
rename from dev-python/setuptools/setuptools-75.2.0.ebuild
rename to dev-python/setuptools/setuptools-75.2.0-r1.ebuild
index c66232a1e7d2..bd3597cd7edd 100644
--- a/dev-python/setuptools/setuptools-75.2.0.ebuild
+++ b/dev-python/setuptools/setuptools-75.2.0-r1.ebuild
@@ -66,14 +66,8 @@ BDEPEND="
 "
 # setuptools-scm is here because installing plugins apparently breaks stuff at
 # runtime, so let's pull it early. See bug #663324.
-#
-# trove-classifiers are optionally used in validation, if they are
-# installed.  Since we really oughtn't block them, let's always enforce
-# the newest version for the time being to avoid errors.
-# https://github.com/pypa/setuptools/issues/4459
 PDEPEND="
 	dev-python/setuptools-scm[${PYTHON_USEDEP}]
-	>=dev-python/trove-classifiers-2024.10.16[${PYTHON_USEDEP}]
 "
 
 src_prepare() {
diff --git a/dev-python/setuptools/setuptools-75.3.0.ebuild b/dev-python/setuptools/setuptools-75.3.0-r1.ebuild
similarity index 93%
rename from dev-python/setuptools/setuptools-75.3.0.ebuild
rename to dev-python/setuptools/setuptools-75.3.0-r1.ebuild
index aa6b581cf0dd..2a2973bd6249 100644
--- a/dev-python/setuptools/setuptools-75.3.0.ebuild
+++ b/dev-python/setuptools/setuptools-75.3.0-r1.ebuild
@@ -66,14 +66,8 @@ BDEPEND="
 "
 # setuptools-scm is here because installing plugins apparently breaks stuff at
 # runtime, so let's pull it early. See bug #663324.
-#
-# trove-classifiers are optionally used in validation, if they are
-# installed.  Since we really oughtn't block them, let's always enforce
-# the newest version for the time being to avoid errors.
-# https://github.com/pypa/setuptools/issues/4459
 PDEPEND="
 	dev-python/setuptools-scm[${PYTHON_USEDEP}]
-	>=dev-python/trove-classifiers-2024.10.16[${PYTHON_USEDEP}]
 "
 
 src_prepare() {
-- 
2.45.2

Re: [gentoo-dev] [PATCH 1/2] distutils-r1.eclass: disable setuptools validation of pypi.org allowed strings

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2585 bytes --]

On Tue, 2024-11-12 at 12:21 -0500, Eli Schwartz wrote:
> Trove classifiers, and their officialness, have no effect on a wheel
> other than determining whether they are allowed to be uploaded to a
> non-Gentoo website, and enabling the search index of that other site.
> 
> We don't need this, and we don't need to validate it. Setuptools will
> disable validation if both of:
> 
> - network downloads failed
> 
> - cannot successfully import the `trove_classifiers` module
> 
> occurs. If trove-classifiers is installed by coincidence, this breaks
> builds when it doesn't get updated on an extremely rapid basis and some
> random package in dev-python/* uses a classifier that was made official
> just the other day.
> 
> We could solve this another way, by making dev-python/setuptools
> PDEPEND on trove-classifiers, and constantly bump the >= dependency. But
> this is a pointless hassle. In fact, we're actually doing it, and it's
> been a pointless hassle. We need to maintain up-to-the-minute minimum
> bounds on the very latest version, and bump setuptools to a new -rX just
> to update the minimum version of a package it doesn't even depend on. We
> need to package new versions of trove-classifiers before *other* Gentoo
> Devs outside of the python project, can successfully revbump their own
> packages. We need to coordinate stabilization of trove-classifiers in
> combination with those other packages. We force people to install a
> pointless package. We overuse PDEPEND.
> 
> Instead, prevent the module from being successfully imported if the
> package being built, doesn't actually depend on it. Then we don't
> actually need it to be installed, and all is well. This can be done at
> the small cost of installing a fake "trove_classifiers" module and
> mutating PYTHONPATH. It is inelegant and meh, but upstream has stated
> that they will not implement a real fix.
> 
> Bug: https://github.com/pypa/setuptools/issues/4459
> Signed-off-by: Eli Schwartz <eschwartz@gentoo.org>
> ---
>  eclass/distutils-r1.eclass | 11 +++++++++++
>  1 file changed, 11 insertions(+)
> 
> diff --git a/eclass/distutils-r1.eclass b/eclass/distutils-r1.eclass
> index 7ab8dcae3265..9b9ba4b803ac 100644
> --- a/eclass/distutils-r1.eclass
> +++ b/eclass/distutils-r1.eclass
> @@ -1360,6 +1360,17 @@ distutils_pep517_install() {
>  					EOF
>  				)
>  			fi
> +			if [[ ${BDEPEND} != *dev-python/trove-classifiers* ]]; then

It's not valid to access stacked variables like BDEPEND from ebuilds.

-- 
Best regards,
Michał Górny


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

Re: [gentoo-dev] [PATCH 1/2] distutils-r1.eclass: disable setuptools validation of pypi.org allowed strings

[Eli Schwartz]

[-- Attachment #1.1: Type: text/plain, Size: 861 bytes --]

On 11/12/24 1:37 PM, Michał Górny wrote:
>> diff --git a/eclass/distutils-r1.eclass b/eclass/distutils-r1.eclass
>> index 7ab8dcae3265..9b9ba4b803ac 100644
>> --- a/eclass/distutils-r1.eclass
>> +++ b/eclass/distutils-r1.eclass
>> @@ -1360,6 +1360,17 @@ distutils_pep517_install() {
>>  					EOF
>>  				)
>>  			fi
>> +			if [[ ${BDEPEND} != *dev-python/trove-classifiers* ]]; then
> 
> It's not valid to access stacked variables like BDEPEND from ebuilds.


For what reason is it invalid? Is it about whether this will work
correctly or is there a policy reason banning it or... ?


Note that this happens inside of a phase function, not at global scope.
So the value of the variable is determined by the contents of the
environment file, not from the incremental value when sourcing the
eclass at global scope.


-- 
Eli Schwartz

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

Re: [gentoo-dev] [PATCH 1/2] distutils-r1.eclass: disable setuptools validation of pypi.org allowed strings

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1156 bytes --]

On Tue, 2024-11-12 at 15:00 -0500, Eli Schwartz wrote:
> On 11/12/24 1:37 PM, Michał Górny wrote:
> > > diff --git a/eclass/distutils-r1.eclass b/eclass/distutils-r1.eclass
> > > index 7ab8dcae3265..9b9ba4b803ac 100644
> > > --- a/eclass/distutils-r1.eclass
> > > +++ b/eclass/distutils-r1.eclass
> > > @@ -1360,6 +1360,17 @@ distutils_pep517_install() {
> > >  					EOF
> > >  				)
> > >  			fi
> > > +			if [[ ${BDEPEND} != *dev-python/trove-classifiers* ]]; then
> > 
> > It's not valid to access stacked variables like BDEPEND from ebuilds.
> 
> 
> For what reason is it invalid? Is it about whether this will work
> correctly or is there a policy reason banning it or... ?

Both.

> e that this happens inside of a phase function, not at global scope.
> So the value of the variable is determined by the contents of the
> environment file, not from the incremental value when sourcing the
> eclass at global scope.
> 

Precisely.  The spec doesn't guarantee that PM will reexport
the merged BDEPEND back into ebuild env, rather than leaving whatever
the last eclass sourced set.

-- 
Best regards,
Michał Górny


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 512 bytes --]