public inbox for gentoo-dev@lists.gentoo.org
 help / color / mirror / Atom feed
* [gentoo-dev] [PATCH v4 0/6] cargo.eclass: Handle LDFLAGS and RUSTFLAGS better
@ 2024-07-25 18:02 James Le Cuirot
  2024-07-25 18:02 ` [gentoo-dev] [PATCH 1/6] cargo.eclass: Use newer Cargo config file name James Le Cuirot
                   ` (6 more replies)
  0 siblings, 7 replies; 9+ messages in thread
From: James Le Cuirot @ 2024-07-25 18:02 UTC (permalink / raw
  To: gentoo-dev; +Cc: rust, python, James Le Cuirot

Sorry for yet another iteration of this. I noticed that the build host flags
were not being applied to rustc when cross-compiling. Upon investigation, I
found that this isn't actually possible right now. I have reworked it to be
close to what we need when it does become possible.

James Le Cuirot (6):
  cargo.eclass: Use newer Cargo config file name
  cargo.eclass: Add cargo_env helper and use it in compile, test,
    install
  cargo.eclass: Handle LDFLAGS and RUSTFLAGS better
  cargo.eclass: Explicitly tell rustc not to strip binaries
  cargo.eclass: Shadow flag variables so that LTO filtering remains
    local
  distutils-r1.eclass: Use cargo_env when appropriate for flag handling

 eclass/cargo.eclass        | 93 ++++++++++++++++++++++++++------------
 eclass/distutils-r1.eclass | 16 ++-----
 2 files changed, 69 insertions(+), 40 deletions(-)

--
2.45.2


^ permalink raw reply	[flat|nested] 9+ messages in thread

* [gentoo-dev] [PATCH 1/6] cargo.eclass: Use newer Cargo config file name
  2024-07-25 18:02 [gentoo-dev] [PATCH v4 0/6] cargo.eclass: Handle LDFLAGS and RUSTFLAGS better James Le Cuirot
@ 2024-07-25 18:02 ` James Le Cuirot
  2024-07-25 18:02 ` [gentoo-dev] [PATCH 2/6] cargo.eclass: Add cargo_env helper and use it in compile, test, install James Le Cuirot
                   ` (5 subsequent siblings)
  6 siblings, 0 replies; 9+ messages in thread
From: James Le Cuirot @ 2024-07-25 18:02 UTC (permalink / raw
  To: gentoo-dev; +Cc: rust, python, James Le Cuirot

"config" is deprecated and "config.toml" has been valid for ages.

Signed-off-by: James Le Cuirot <chewi@gentoo.org>
---
 eclass/cargo.eclass | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/eclass/cargo.eclass b/eclass/cargo.eclass
index 7db34efb4e174..aab28dbbac167 100644
--- a/eclass/cargo.eclass
+++ b/eclass/cargo.eclass
@@ -248,7 +248,7 @@ cargo_crate_uris() {
 
 # @FUNCTION: cargo_gen_config
 # @DESCRIPTION:
-# Generate the $CARGO_HOME/config necessary to use our local registry and settings.
+# Generate the $CARGO_HOME/config.toml necessary to use our local registry and settings.
 # Cargo can also be configured through environment variables in addition to the TOML syntax below.
 # For each configuration key below of the form foo.bar the environment variable CARGO_FOO_BAR
 # can also be used to define the value.
@@ -261,7 +261,7 @@ cargo_gen_config() {
 
 	mkdir -p "${ECARGO_HOME}" || die
 
-	cat > "${ECARGO_HOME}/config" <<- _EOF_ || die "Failed to create cargo config"
+	cat > "${ECARGO_HOME}/config.toml" <<- _EOF_ || die "Failed to create cargo config"
 	[source.gentoo]
 	directory = "${ECARGO_VENDOR}"
 
-- 
2.45.2



^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [gentoo-dev] [PATCH 2/6] cargo.eclass: Add cargo_env helper and use it in compile, test, install
  2024-07-25 18:02 [gentoo-dev] [PATCH v4 0/6] cargo.eclass: Handle LDFLAGS and RUSTFLAGS better James Le Cuirot
  2024-07-25 18:02 ` [gentoo-dev] [PATCH 1/6] cargo.eclass: Use newer Cargo config file name James Le Cuirot
@ 2024-07-25 18:02 ` James Le Cuirot
  2024-07-25 18:02 ` [gentoo-dev] [PATCH 3/6] cargo.eclass: Handle LDFLAGS and RUSTFLAGS better James Le Cuirot
                   ` (4 subsequent siblings)
  6 siblings, 0 replies; 9+ messages in thread
From: James Le Cuirot @ 2024-07-25 18:02 UTC (permalink / raw
  To: gentoo-dev; +Cc: rust, python, James Le Cuirot

Rust packages have a tendency to rebuild parts during test and install.
It is not clear whether this can be addressed. We were therefore relying
on some environment variables set during the compile phase for
cross-compiling to work in the later phases. This is not ideal,
especially if you need to build for multiple targets.

These environment variables can also be useful in other contexts, such
as the build runner in app-misc/anki.

This change moves the setting of these variables into a separate helper
that is now used in all these phases and can be used by ebuilds too. The
variables are now kept local to each invocation of this helper,
preventing leakage.

Signed-off-by: James Le Cuirot <chewi@gentoo.org>
---
 eclass/cargo.eclass | 37 +++++++++++++++++++++++--------------
 1 file changed, 23 insertions(+), 14 deletions(-)

diff --git a/eclass/cargo.eclass b/eclass/cargo.eclass
index aab28dbbac167..d598af8825adf 100644
--- a/eclass/cargo.eclass
+++ b/eclass/cargo.eclass
@@ -523,26 +523,23 @@ cargo_src_configure() {
 	[[ ${ECARGO_ARGS[@]} ]] && einfo "Configured with: ${ECARGO_ARGS[@]}"
 }
 
-# @FUNCTION: cargo_src_compile
+# @FUNCTION: cargo_env
+# @USAGE: Command with its arguments
 # @DESCRIPTION:
-# Build the package using cargo build.
-cargo_src_compile() {
-	debug-print-function ${FUNCNAME} "$@"
-
-	[[ ${_CARGO_GEN_CONFIG_HAS_RUN} ]] || \
-		die "FATAL: please call cargo_gen_config before using ${FUNCNAME}"
-
+# Run the given command under an environment needed for performing tasks with
+# Cargo such as building.
+cargo_env() {
 	filter-lto
 	tc-export AR CC CXX PKG_CONFIG
 
 	if tc-is-cross-compiler; then
-		export CARGO_BUILD_TARGET=$(rust_abi)
+		declare -x CARGO_BUILD_TARGET=$(rust_abi)
 		local TRIPLE=${CARGO_BUILD_TARGET//-/_}
-		export CARGO_TARGET_"${TRIPLE^^}"_LINKER=$(tc-getCC)
+		declare -x CARGO_TARGET_"${TRIPLE^^}"_LINKER=$(tc-getCC)
 
 		# Set vars for cc-rs crate.
 		tc-export_build_env
-		export \
+		declare -x \
 			HOST_AR=$(tc-getBUILD_AR)
 			HOST_CC=$(tc-getBUILD_CC)
 			HOST_CXX=$(tc-getBUILD_CXX)
@@ -550,9 +547,21 @@ cargo_src_compile() {
 			HOST_CXXFLAGS=${BUILD_CXXFLAGS}
 	fi
 
+	"${@}"
+}
+
+# @FUNCTION: cargo_src_compile
+# @DESCRIPTION:
+# Build the package using cargo build.
+cargo_src_compile() {
+	debug-print-function ${FUNCNAME} "$@"
+
+	[[ ${_CARGO_GEN_CONFIG_HAS_RUN} ]] || \
+		die "FATAL: please call cargo_gen_config before using ${FUNCNAME}"
+
 	set -- cargo build $(usex debug "" --release) ${ECARGO_ARGS[@]} "$@"
 	einfo "${@}"
-	"${@}" || die "cargo build failed"
+	cargo_env "${@}" || die "cargo build failed"
 }
 
 # @FUNCTION: cargo_src_install
@@ -573,7 +582,7 @@ cargo_src_install() {
 		$(usex debug --debug "") \
 		${ECARGO_ARGS[@]} "$@"
 	einfo "${@}"
-	"${@}" || die "cargo install failed"
+	cargo_env "${@}" || die "cargo install failed"
 
 	rm -f "${ED}/usr/.crates.toml" || die
 	rm -f "${ED}/usr/.crates2.json" || die
@@ -590,7 +599,7 @@ cargo_src_test() {
 
 	set -- cargo test $(usex debug "" --release) ${ECARGO_ARGS[@]} "$@"
 	einfo "${@}"
-	"${@}" || die "cargo test failed"
+	cargo_env "${@}" || die "cargo test failed"
 }
 
 fi
-- 
2.45.2



^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [gentoo-dev] [PATCH 3/6] cargo.eclass: Handle LDFLAGS and RUSTFLAGS better
  2024-07-25 18:02 [gentoo-dev] [PATCH v4 0/6] cargo.eclass: Handle LDFLAGS and RUSTFLAGS better James Le Cuirot
  2024-07-25 18:02 ` [gentoo-dev] [PATCH 1/6] cargo.eclass: Use newer Cargo config file name James Le Cuirot
  2024-07-25 18:02 ` [gentoo-dev] [PATCH 2/6] cargo.eclass: Add cargo_env helper and use it in compile, test, install James Le Cuirot
@ 2024-07-25 18:02 ` James Le Cuirot
  2024-07-25 18:02 ` [gentoo-dev] [PATCH 4/6] cargo.eclass: Explicitly tell rustc not to strip binaries James Le Cuirot
                   ` (3 subsequent siblings)
  6 siblings, 0 replies; 9+ messages in thread
From: James Le Cuirot @ 2024-07-25 18:02 UTC (permalink / raw
  To: gentoo-dev; +Cc: rust, python, James Le Cuirot

LDFLAGS are not currently honoured by Cargo builds at all. It would be
particularly advantageous to honour -fuse-ld because alternative linkers
like mold are known to be significantly faster at handling Rust.

As things stand, the eclass sets the linker to CC when cross-compiling,
but it does so erroneously due to a shell quoting issue. If CC includes
arguments, an error occurs when setting the CARGO_TARGET_*_LINKER
variable. Even with the right quoting, Cargo still fails because this
variable is not allowed to include arguments. They have to be specified
via RUSTFLAGS instead.

The build host linker is frequently invoked due to the use of build
scripts by some crates. We would therefore also like to configure the
build host linker properly when cross-compiling, but unfortunately there
is currently no way to do this without using a Rust nightly feature.

The build host linker is now configured in the cargo_gen_config function
because this configuration is static. Conversely, the target host linker
is configured with environment variables in the cargo_env function
because the target may change.

Some ebuilds already set RUSTFLAGS, so some consideration was given to
how to handle these. When set, Cargo prioritises RUSTFLAGS and similar
over specific configuration for the build and target hosts, so these
must be unset. We can still include their contents in that configuration
for convenience though. CARGO_BUILD_RUSTFLAGS can be used by ebuilds to
set flags only for the build host.

It should not be necessary for ebuilds to figure out which Rust ABI is
applicable in order to set flags only for the target host, so the helper
reads from a simple CARGO_TARGET_RUSTFLAGS variable without the triple
for convenience.

Signed-off-by: James Le Cuirot <chewi@gentoo.org>
---
 eclass/cargo.eclass | 65 +++++++++++++++++++++++++++++----------------
 1 file changed, 42 insertions(+), 23 deletions(-)

diff --git a/eclass/cargo.eclass b/eclass/cargo.eclass
index d598af8825adf..6ea37ec5e4e4b 100644
--- a/eclass/cargo.eclass
+++ b/eclass/cargo.eclass
@@ -109,7 +109,7 @@ ECARGO_VENDOR="${ECARGO_HOME}/gentoo"
 #
 # If you enable CARGO_OPTIONAL, you have to set BDEPEND on virtual/rust
 # for your package and call at least cargo_gen_config manually before using
-# other src_functions of this eclass.
+# other src_functions or cargo_env of this eclass.
 # Note that cargo_gen_config is automatically called by cargo_src_unpack.
 
 # @ECLASS_VARIABLE: myfeatures
@@ -259,6 +259,19 @@ cargo_crate_uris() {
 cargo_gen_config() {
 	debug-print-function ${FUNCNAME} "$@"
 
+	# The default linker is "cc" so override by setting linker to CC in the
+	# RUSTFLAGS. The given linker cannot include any arguments, so split these
+	# into link-args along with LDFLAGS. Also include external RUSTFLAGS.
+	# Note that as of Rust 1.80, the build host RUSTFLAGS are ignored when
+	# cross-compiling unless you use the unstable host-config feature available
+	# with USE=nightly. There is no simple way around this.
+	tc-export_build_env
+	local LD_A=( $(tc-getBUILD_CC) ${BUILD_LDFLAGS} )
+	local BUILD_RUSTFLAGS="-C linker=${LD_A[0]}"
+	[[ ${#LD_A[@]} -gt 1 ]] && BUILD_RUSTFLAGS+="$(printf -- ' -C link-arg=%s' "${LD_A[@]:1}")"
+	BUILD_RUSTFLAGS+=" ${RUSTFLAGS} ${CARGO_BUILD_RUSTFLAGS}"
+	tc-is-cross-compiler || BUILD_RUSTFLAGS+=" ${CARGO_TARGET_RUSTFLAGS}"
+
 	mkdir -p "${ECARGO_HOME}" || die
 
 	cat > "${ECARGO_HOME}/config.toml" <<- _EOF_ || die "Failed to create cargo config"
@@ -273,6 +286,7 @@ cargo_gen_config() {
 	offline = true
 
 	[build]
+	rustflags = "${BUILD_RUSTFLAGS}"
 	jobs = $(makeopts_jobs)
 	incremental = false
 
@@ -527,27 +541,41 @@ cargo_src_configure() {
 # @USAGE: Command with its arguments
 # @DESCRIPTION:
 # Run the given command under an environment needed for performing tasks with
-# Cargo such as building.
+# Cargo such as building. RUSTFLAGS is used for both the build and target host.
+# CARGO_BUILD_RUSTFLAGS and CARGO_TARGET_RUSTFLAGS are used for just the build
+# host and target host respectively. Ensure these are set consistently between
+# Cargo invocations, otherwise rebuilds will occur.
 cargo_env() {
+	[[ ${_CARGO_GEN_CONFIG_HAS_RUN} ]] || \
+		die "FATAL: please call cargo_gen_config before using ${FUNCNAME}"
+
 	filter-lto
 	tc-export AR CC CXX PKG_CONFIG
 
+	# Set vars for cc-rs crate.
+	local -x \
+		HOST_AR=$(tc-getBUILD_AR)
+		HOST_CC=$(tc-getBUILD_CC)
+		HOST_CXX=$(tc-getBUILD_CXX)
+		HOST_CFLAGS=${BUILD_CFLAGS}
+		HOST_CXXFLAGS=${BUILD_CXXFLAGS}
+
 	if tc-is-cross-compiler; then
-		declare -x CARGO_BUILD_TARGET=$(rust_abi)
+		local -x CARGO_BUILD_TARGET=$(rust_abi)
 		local TRIPLE=${CARGO_BUILD_TARGET//-/_}
-		declare -x CARGO_TARGET_"${TRIPLE^^}"_LINKER=$(tc-getCC)
-
-		# Set vars for cc-rs crate.
-		tc-export_build_env
-		declare -x \
-			HOST_AR=$(tc-getBUILD_AR)
-			HOST_CC=$(tc-getBUILD_CC)
-			HOST_CXX=$(tc-getBUILD_CXX)
-			HOST_CFLAGS=${BUILD_CFLAGS}
-			HOST_CXXFLAGS=${BUILD_CXXFLAGS}
+		local TRIPLE=${TRIPLE^^} LD_A=( $(tc-getCC) ${LDFLAGS} )
+		local -x CARGO_TARGET_"${TRIPLE}"_RUSTFLAGS="-C linker=${LD_A[0]}"
+		[[ ${#LD_A[@]} -gt 1 ]] && local CARGO_TARGET_"${TRIPLE}"_RUSTFLAGS+="$(printf -- ' -C link-arg=%s' "${LD_A[@]:1}")"
+		local CARGO_TARGET_"${TRIPLE}"_RUSTFLAGS+=" ${RUSTFLAGS} ${CARGO_TARGET_RUSTFLAGS}"
 	fi
 
-	"${@}"
+	(
+		# These variables will override the above, even if empty, so unset them
+		# locally. Do this in a subshell so that they remain set afterwards.
+		unset CARGO_BUILD_RUSTFLAGS CARGO_ENCODED_RUSTFLAGS RUSTFLAGS
+
+		"${@}"
+	)
 }
 
 # @FUNCTION: cargo_src_compile
@@ -556,9 +584,6 @@ cargo_env() {
 cargo_src_compile() {
 	debug-print-function ${FUNCNAME} "$@"
 
-	[[ ${_CARGO_GEN_CONFIG_HAS_RUN} ]] || \
-		die "FATAL: please call cargo_gen_config before using ${FUNCNAME}"
-
 	set -- cargo build $(usex debug "" --release) ${ECARGO_ARGS[@]} "$@"
 	einfo "${@}"
 	cargo_env "${@}" || die "cargo build failed"
@@ -573,9 +598,6 @@ cargo_src_compile() {
 cargo_src_install() {
 	debug-print-function ${FUNCNAME} "$@"
 
-	[[ ${_CARGO_GEN_CONFIG_HAS_RUN} ]] || \
-		die "FATAL: please call cargo_gen_config before using ${FUNCNAME}"
-
 	set -- cargo install $(has --path ${@} || echo --path ./) \
 		--root "${ED}/usr" \
 		${GIT_CRATES[@]:+--frozen} \
@@ -594,9 +616,6 @@ cargo_src_install() {
 cargo_src_test() {
 	debug-print-function ${FUNCNAME} "$@"
 
-	[[ ${_CARGO_GEN_CONFIG_HAS_RUN} ]] || \
-		die "FATAL: please call cargo_gen_config before using ${FUNCNAME}"
-
 	set -- cargo test $(usex debug "" --release) ${ECARGO_ARGS[@]} "$@"
 	einfo "${@}"
 	cargo_env "${@}" || die "cargo test failed"
-- 
2.45.2



^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [gentoo-dev] [PATCH 4/6] cargo.eclass: Explicitly tell rustc not to strip binaries
  2024-07-25 18:02 [gentoo-dev] [PATCH v4 0/6] cargo.eclass: Handle LDFLAGS and RUSTFLAGS better James Le Cuirot
                   ` (2 preceding siblings ...)
  2024-07-25 18:02 ` [gentoo-dev] [PATCH 3/6] cargo.eclass: Handle LDFLAGS and RUSTFLAGS better James Le Cuirot
@ 2024-07-25 18:02 ` James Le Cuirot
  2024-07-25 18:02 ` [gentoo-dev] [PATCH 5/6] cargo.eclass: Shadow flag variables so that LTO filtering remains local James Le Cuirot
                   ` (2 subsequent siblings)
  6 siblings, 0 replies; 9+ messages in thread
From: James Le Cuirot @ 2024-07-25 18:02 UTC (permalink / raw
  To: gentoo-dev; +Cc: rust, python, James Le Cuirot

Most projects don't strip binaries in release mode by default, but there
are exceptions like app-misc/broot.

Signed-off-by: James Le Cuirot <chewi@gentoo.org>
---
 eclass/cargo.eclass | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/eclass/cargo.eclass b/eclass/cargo.eclass
index 6ea37ec5e4e4b..84f72168fbb3c 100644
--- a/eclass/cargo.eclass
+++ b/eclass/cargo.eclass
@@ -267,7 +267,7 @@ cargo_gen_config() {
 	# with USE=nightly. There is no simple way around this.
 	tc-export_build_env
 	local LD_A=( $(tc-getBUILD_CC) ${BUILD_LDFLAGS} )
-	local BUILD_RUSTFLAGS="-C linker=${LD_A[0]}"
+	local BUILD_RUSTFLAGS="-C strip=none -C linker=${LD_A[0]}"
 	[[ ${#LD_A[@]} -gt 1 ]] && BUILD_RUSTFLAGS+="$(printf -- ' -C link-arg=%s' "${LD_A[@]:1}")"
 	BUILD_RUSTFLAGS+=" ${RUSTFLAGS} ${CARGO_BUILD_RUSTFLAGS}"
 	tc-is-cross-compiler || BUILD_RUSTFLAGS+=" ${CARGO_TARGET_RUSTFLAGS}"
@@ -564,7 +564,7 @@ cargo_env() {
 		local -x CARGO_BUILD_TARGET=$(rust_abi)
 		local TRIPLE=${CARGO_BUILD_TARGET//-/_}
 		local TRIPLE=${TRIPLE^^} LD_A=( $(tc-getCC) ${LDFLAGS} )
-		local -x CARGO_TARGET_"${TRIPLE}"_RUSTFLAGS="-C linker=${LD_A[0]}"
+		local -x CARGO_TARGET_"${TRIPLE}"_RUSTFLAGS="-C strip=none -C linker=${LD_A[0]}"
 		[[ ${#LD_A[@]} -gt 1 ]] && local CARGO_TARGET_"${TRIPLE}"_RUSTFLAGS+="$(printf -- ' -C link-arg=%s' "${LD_A[@]:1}")"
 		local CARGO_TARGET_"${TRIPLE}"_RUSTFLAGS+=" ${RUSTFLAGS} ${CARGO_TARGET_RUSTFLAGS}"
 	fi
-- 
2.45.2



^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [gentoo-dev] [PATCH 5/6] cargo.eclass: Shadow flag variables so that LTO filtering remains local
  2024-07-25 18:02 [gentoo-dev] [PATCH v4 0/6] cargo.eclass: Handle LDFLAGS and RUSTFLAGS better James Le Cuirot
                   ` (3 preceding siblings ...)
  2024-07-25 18:02 ` [gentoo-dev] [PATCH 4/6] cargo.eclass: Explicitly tell rustc not to strip binaries James Le Cuirot
@ 2024-07-25 18:02 ` James Le Cuirot
  2024-07-25 18:02 ` [gentoo-dev] [PATCH 6/6] distutils-r1.eclass: Use cargo_env when appropriate for flag handling James Le Cuirot
  2024-07-25 21:22 ` [gentoo-dev] [PATCH 7/8] distutils-r1.eclass: Add python_get_stdlib helper function James Le Cuirot
  6 siblings, 0 replies; 9+ messages in thread
From: James Le Cuirot @ 2024-07-25 18:02 UTC (permalink / raw
  To: gentoo-dev; +Cc: rust, python, James Le Cuirot

This is currently done for Cargo by distutils-r1.eclass. The next commit
will remove that code, leaving cargo_env responsible for it.

Signed-off-by: James Le Cuirot <chewi@gentoo.org>
---
 eclass/cargo.eclass | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/eclass/cargo.eclass b/eclass/cargo.eclass
index 84f72168fbb3c..25acf010287d8 100644
--- a/eclass/cargo.eclass
+++ b/eclass/cargo.eclass
@@ -549,7 +549,16 @@ cargo_env() {
 	[[ ${_CARGO_GEN_CONFIG_HAS_RUN} ]] || \
 		die "FATAL: please call cargo_gen_config before using ${FUNCNAME}"
 
+	# Shadow flag variables so that filtering below remains local.
+	local flag
+	for flag in $(all-flag-vars); do
+		local -x "${flag}=${!flag}"
+	done
+
+	# Rust extensions are incompatible with C/C++ LTO compiler see e.g.
+	# https://bugs.gentoo.org/910220
 	filter-lto
+
 	tc-export AR CC CXX PKG_CONFIG
 
 	# Set vars for cc-rs crate.
-- 
2.45.2



^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [gentoo-dev] [PATCH 6/6] distutils-r1.eclass: Use cargo_env when appropriate for flag handling
  2024-07-25 18:02 [gentoo-dev] [PATCH v4 0/6] cargo.eclass: Handle LDFLAGS and RUSTFLAGS better James Le Cuirot
                   ` (4 preceding siblings ...)
  2024-07-25 18:02 ` [gentoo-dev] [PATCH 5/6] cargo.eclass: Shadow flag variables so that LTO filtering remains local James Le Cuirot
@ 2024-07-25 18:02 ` James Le Cuirot
  2024-07-25 21:22 ` [gentoo-dev] [PATCH 7/8] distutils-r1.eclass: Add python_get_stdlib helper function James Le Cuirot
  6 siblings, 0 replies; 9+ messages in thread
From: James Le Cuirot @ 2024-07-25 18:02 UTC (permalink / raw
  To: gentoo-dev; +Cc: rust, python, James Le Cuirot

cargo_env handles linker flags and enables cross-compiling. It also
handles LTO filtering, so we can remove that from this eclass.

Signed-off-by: James Le Cuirot <chewi@gentoo.org>
---
 eclass/distutils-r1.eclass | 16 ++++------------
 1 file changed, 4 insertions(+), 12 deletions(-)

diff --git a/eclass/distutils-r1.eclass b/eclass/distutils-r1.eclass
index fa8edb5cdfb77..743242b2acee8 100644
--- a/eclass/distutils-r1.eclass
+++ b/eclass/distutils-r1.eclass
@@ -1251,7 +1251,9 @@ distutils_pep517_install() {
 		die "mydistutilsargs are banned in PEP517 mode (use DISTUTILS_ARGS)"
 	fi
 
-	local config_settings=
+	local cmd=() config_settings=
+	has cargo ${INHERITED} && cmd+=( cargo_env )
+
 	case ${DISTUTILS_USE_PEP517} in
 		maturin)
 			# `maturin pep517 build-wheel --help` for options
@@ -1390,7 +1392,7 @@ distutils_pep517_install() {
 
 	local build_backend=$(_distutils-r1_get_backend)
 	einfo "  Building the wheel for ${PWD#${WORKDIR}/} via ${build_backend}"
-	local cmd=(
+	cmd+=(
 		"${EPYTHON}" -m gpep517 build-wheel
 			--prefix="${EPREFIX}/usr"
 			--backend "${build_backend}"
@@ -1792,16 +1794,6 @@ distutils-r1_run_phase() {
 		# bug fixes from Cython (this works only when setup.py is using
 		# cythonize() but it's better than nothing)
 		local -x CYTHON_FORCE_REGEN=1
-
-		# Rust extensions are incompatible with C/C++ LTO compiler
-		# see e.g. https://bugs.gentoo.org/910220
-		if has cargo ${INHERITED}; then
-			local x
-			for x in $(all-flag-vars); do
-				local -x "${x}=${!x}"
-			done
-			filter-lto
-		fi
 	fi
 
 	# silence warnings when pydevd is loaded on Python 3.11+
-- 
2.45.2



^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [gentoo-dev] [PATCH 7/8] distutils-r1.eclass: Add python_get_stdlib helper function
  2024-07-25 18:02 [gentoo-dev] [PATCH v4 0/6] cargo.eclass: Handle LDFLAGS and RUSTFLAGS better James Le Cuirot
                   ` (5 preceding siblings ...)
  2024-07-25 18:02 ` [gentoo-dev] [PATCH 6/6] distutils-r1.eclass: Use cargo_env when appropriate for flag handling James Le Cuirot
@ 2024-07-25 21:22 ` James Le Cuirot
  2024-07-25 21:22   ` [gentoo-dev] [PATCH 8/8] distutils-r1.eclass: Support cross-compiling with PyO3 James Le Cuirot
  6 siblings, 1 reply; 9+ messages in thread
From: James Le Cuirot @ 2024-07-25 21:22 UTC (permalink / raw
  To: gentoo-dev; +Cc: python, James Le Cuirot

This is just like python_get_sitedir, but it returns the stdlib
directory such as /usr/lib/python3.12. This is useful for locating the
sysconfigdata file.

Signed-off-by: James Le Cuirot <chewi@gentoo.org>
---
 eclass/python-utils-r1.eclass | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/eclass/python-utils-r1.eclass b/eclass/python-utils-r1.eclass
index c47565fa1db2a..b544f52312880 100644
--- a/eclass/python-utils-r1.eclass
+++ b/eclass/python-utils-r1.eclass
@@ -332,6 +332,17 @@ _python_export() {
 				export PYTHON=${BROOT-${EPREFIX}}/usr/bin/${impl}
 				debug-print "${FUNCNAME}: PYTHON = ${PYTHON}"
 				;;
+			PYTHON_STDLIB)
+				[[ -n ${PYTHON} ]] || die "PYTHON needs to be set for ${var} to be exported, or requested before it"
+				PYTHON_STDLIB=$(
+					"${PYTHON}" - "${EPREFIX}/usr" <<-EOF || die
+						import sys, sysconfig
+						print(sysconfig.get_path("stdlib", vars={"installed_base": sys.argv[1]}))
+					EOF
+				)
+				export PYTHON_STDLIB
+				debug-print "${FUNCNAME}: PYTHON_STDLIB = ${PYTHON_STDLIB}"
+				;;
 			PYTHON_SITEDIR)
 				[[ -n ${PYTHON} ]] || die "PYTHON needs to be set for ${var} to be exported, or requested before it"
 				PYTHON_SITEDIR=$(
@@ -466,6 +477,18 @@ _python_export() {
 	done
 }

+# @FUNCTION: python_get_stdlib
+# @USAGE: [<impl>]
+# @DESCRIPTION:
+# Obtain and print the 'stdlib' path for the given implementation. If no
+# implementation is provided, ${EPYTHON} will be used.
+python_get_stdlib() {
+	debug-print-function ${FUNCNAME} "${@}"
+
+	_python_export "${@}" PYTHON_STDLIB
+	echo "${PYTHON_STDLIB}"
+}
+
 # @FUNCTION: python_get_sitedir
 # @USAGE: [<impl>]
 # @DESCRIPTION:
--
2.45.2


^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [gentoo-dev] [PATCH 8/8] distutils-r1.eclass: Support cross-compiling with PyO3
  2024-07-25 21:22 ` [gentoo-dev] [PATCH 7/8] distutils-r1.eclass: Add python_get_stdlib helper function James Le Cuirot
@ 2024-07-25 21:22   ` James Le Cuirot
  0 siblings, 0 replies; 9+ messages in thread
From: James Le Cuirot @ 2024-07-25 21:22 UTC (permalink / raw
  To: gentoo-dev; +Cc: python, James Le Cuirot

Only one variable needs to be set. For details, see
https://pyo3.rs/latest/building-and-distribution.html#cross-compiling.

Signed-off-by: James Le Cuirot <chewi@gentoo.org>
---
 eclass/distutils-r1.eclass | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/eclass/distutils-r1.eclass b/eclass/distutils-r1.eclass
index 743242b2acee8..9289392f84b38 100644
--- a/eclass/distutils-r1.eclass
+++ b/eclass/distutils-r1.eclass
@@ -1390,6 +1390,9 @@ distutils_pep517_install() {
 			;;
 	esac

+	# https://pyo3.rs/latest/building-and-distribution.html#cross-compiling
+	tc-is-cross-compiler && local -x PYO3_CROSS_LIB_DIR=${SYSROOT}/$(python_get_stdlib)
+
 	local build_backend=$(_distutils-r1_get_backend)
 	einfo "  Building the wheel for ${PWD#${WORKDIR}/} via ${build_backend}"
 	cmd+=(
--
2.45.2


^ permalink raw reply related	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2024-07-25 21:26 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-07-25 18:02 [gentoo-dev] [PATCH v4 0/6] cargo.eclass: Handle LDFLAGS and RUSTFLAGS better James Le Cuirot
2024-07-25 18:02 ` [gentoo-dev] [PATCH 1/6] cargo.eclass: Use newer Cargo config file name James Le Cuirot
2024-07-25 18:02 ` [gentoo-dev] [PATCH 2/6] cargo.eclass: Add cargo_env helper and use it in compile, test, install James Le Cuirot
2024-07-25 18:02 ` [gentoo-dev] [PATCH 3/6] cargo.eclass: Handle LDFLAGS and RUSTFLAGS better James Le Cuirot
2024-07-25 18:02 ` [gentoo-dev] [PATCH 4/6] cargo.eclass: Explicitly tell rustc not to strip binaries James Le Cuirot
2024-07-25 18:02 ` [gentoo-dev] [PATCH 5/6] cargo.eclass: Shadow flag variables so that LTO filtering remains local James Le Cuirot
2024-07-25 18:02 ` [gentoo-dev] [PATCH 6/6] distutils-r1.eclass: Use cargo_env when appropriate for flag handling James Le Cuirot
2024-07-25 21:22 ` [gentoo-dev] [PATCH 7/8] distutils-r1.eclass: Add python_get_stdlib helper function James Le Cuirot
2024-07-25 21:22   ` [gentoo-dev] [PATCH 8/8] distutils-r1.eclass: Support cross-compiling with PyO3 James Le Cuirot

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox