From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id 18EE9138334 for ; Mon, 25 Nov 2019 17:38:58 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 4E45EE09C9; Mon, 25 Nov 2019 17:38:54 +0000 (UTC) Received: from smtp.gentoo.org (woodpecker.gentoo.org [IPv6:2001:470:ea4a:1:5054:ff:fec7:86e4]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id ECA6DE0855 for ; Mon, 25 Nov 2019 17:38:53 +0000 (UTC) Received: from pomiot (c134-66.icpnet.pl [85.221.134.66]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: mgorny) by smtp.gentoo.org (Postfix) with ESMTPSA id 5E48034D34A; Mon, 25 Nov 2019 17:38:52 +0000 (UTC) Message-ID: Subject: [gentoo-dev] [RFC] dev-python/setuptools deps in distutils-r1 packages From: =?UTF-8?Q?Micha=C5=82_G=C3=B3rny?= To: gentoo-dev Date: Mon, 25 Nov 2019 18:38:47 +0100 Organization: Gentoo Content-Type: multipart/signed; micalg="pgp-sha512"; protocol="application/pgp-signature"; boundary="=-fSX1t+iVkS3hhFVVPAxM" User-Agent: Evolution 3.32.4 Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-dev@lists.gentoo.org Reply-to: gentoo-dev@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 X-Archives-Salt: fd9d30ea-945f-42bf-b812-3ea0e20fd4b7 X-Archives-Hash: 0c4b80b78dce5fbc2db1bf7c6ef275c4 --=-fSX1t+iVkS3hhFVVPAxM Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi, TL;DR: should we depend on setuptools by default? Alternatively, should we add distutils_enable_setuptools API to provide at least partial validity checks. The problem =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D The vast majority of Python packages nowadays uses setuptools as their build system. According to a cheap grep, 1633 out of 2415 packages using distutils-r1 depends on it -- and I suspect the vast majority of those that do not are simply missing the dependency. Are there valid cases for not using setuptools? Notably, packages needed to bootstrap setuptools aren't using it. Plus some simple packages that really don't need its features. There are also packages that use setuptools but have distutils fallback. We don't like them because switching between build systems involves file <-> directory replacement that isn't handled cleanly by our package managers. Therefore, we want to always force one build system in them. The setuptools dependency is so common it's easy to miss. Notably, it's a prerequisite of specifying package dependencies, so it's normally not listed in dependencies. Finally, while most of the time setuptools is just BDEPEND, there are cases when it's RDEPEND as well. The most common is the use of entry_points -- and again, upstreams don't mention it explicitly. All that considered, I think we should work on providing a better API for depending on setuptools, to reduce the number of missing dependencies and make it possible to automatically test for them (reminder: PMS doesn't permit inspecting *DEPEND). Variant 1: automatic dependency on setuptools =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Basically, we add a new trinary pre-inherit variable: DISTUTILS_USE_SETUPTOOLS=3Dno -> no deps DISTUTILS_USE_SETUPTOOLS=3Dbdepend -> add to BDEPEND (the default) DISTUTILS_USE_SETUPTOOLS=3Drdepend -> add to BDEPEND+RDEPEND This is roughly 'erring on the safe side'. The default will work for the majority of packages. We will have to disable it for setuptools bootstrap deps, and devs will be able to adjust it to correct values as they update ebuilds. For the time being, existing *DEPEND on setuptools will avoid breaking stuff. This will also enable me to add extra QA checks to esetup.py. It should be able to reasonably detect incorrect value and report it. This will imply some 'false positives' for packages that use the old method of specifying setuptools in RDEPEND but that's a minor hassle. Pros: - works out of the box for the majority of packages - enables full-range QA checking Cons: - pre-inherit variable - some (harmless) false positives on existing packages Variant 2: distutils_enable_setuptools =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D The alternative method is to add another function to the distutils_enable* series: distutils_enable_setuptools [-r] The basic form adds it to BDEPEND, -r B+RDEPEND. Of course, no dep is present by default. The main difference from setting deps explicitly is that it permits us to do a minimal QA check between pure BDEPEND and B+RDEPEND for now. When all ebuilds are migrated from explicit dependencies to this method, we can also start detecting missing deps completely. However, that presumes we require using this function rather than explicit deps. Pros: - no pre-inherit variables Cons: - only partial QA check possible for the time being - requires migrating all ebuilds long-term Variant 3: leave as-is, add minimal install-qa-check.d =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D We can just continue adding deps manually, and add a minimal install-qa- check.d that greps installed scripts for entry_points usage. This will let us detect missing RDEPEND on setuptools but not BDEPEND. Pros: - no changes to ebuilds Cons: - only partial QA check possible WDYT? =3D=3D=3D=3D=3D Both options have their pros and cons. I think V1 is the best since it avoids a common mistake, and gives full range QA check. It also doesn't interfere with existing deps. V2 neatly fits into the recent series but still requires users to remember to call it, and we can't report missing calls until we clean up all ebuilds. V3 provides only minimal QA improvement without changing the eclass. WDYT? Do you have any other ideas? --=20 Best regards, Micha=C5=82 G=C3=B3rny --=-fSX1t+iVkS3hhFVVPAxM Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- iQGTBAABCgB9FiEEx2qEUJQJjSjMiybFY5ra4jKeJA4FAl3cEahfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEM3 NkE4NDUwOTQwOThEMjhDQzhCMjZDNTYzOUFEQUUyMzI5RTI0MEUACgkQY5ra4jKe JA7n9QgA0uSry3uKOQ4P8Lb7FOAmk1fRB95vTP/j7voMO4bN5sBBYDbp9cMM58qF vRfELUkOuaxbDJfJmX5VoLigXTjRNgIXx6P5/nvxGPFiHAkUKNMzXCiGfHSbS7hU r/BPCF7POzKF/ekR5ln7gCBuyXhlBo3O1BfcJviX71W3D8w4GdxYj5Enq5ml4vgd DiVKpbSLdZ/FYxqhuii8nffAh/gwjpkozDlfWpxHrCijqC7FrGDfIaX6z67ZoTxw rxISQJ916IlJrwMQh8KnkhjM00/CtZY51onP4JWh5nquz7swkfnBRaiZfIXdQsRX QcE9OOrNdi0GTI3lQVP87EGjR+lNjw== =j6La -----END PGP SIGNATURE----- --=-fSX1t+iVkS3hhFVVPAxM--