public inbox for gentoo-dev@lists.gentoo.org
 help / color / mirror / Atom feed
* [gentoo-dev] [PATCH 0/5] Handle LTO with static libraries
@ 2025-05-02 20:13 Sam James
  2025-05-02 20:13 ` [gentoo-dev] [PATCH 1/5] dot-a.eclass: new eclass for handling LTO in static archives Sam James
                   ` (4 more replies)
  0 siblings, 5 replies; 10+ messages in thread
From: Sam James @ 2025-05-02 20:13 UTC (permalink / raw
  To: gentoo-dev; +Cc: Sam James

See the commit message for dot-a.eclass for the details. More packages
will need to be ported (libpcre2, Qt, ...) later.

Sam James (5):
  dot-a.eclass: new eclass for handling LTO in static archives
  sys-libs/zlib: use dot-a.eclass for LTO
  dev-libs/glib: use dot-a.eclass for LTO
  dev-util/sysprof-capture: use dot-a.eclass for LTO
  sys-devel/flex: use dot-a.eclass for LTO

 dev-libs/glib/glib-2.78.6.ebuild              |   7 +
 dev-libs/glib/glib-2.80.5-r1.ebuild           |   7 +
 dev-libs/glib/glib-2.82.5.ebuild              |   7 +
 dev-libs/glib/glib-2.84.0.ebuild              |   7 +
 dev-libs/glib/glib-2.84.1.ebuild              |   9 +-
 .../sysprof-capture-3.48.0.ebuild             |  14 +-
 .../sysprof-capture-46.0.ebuild               |  12 +-
 .../sysprof-capture-47.2.ebuild               |  12 +-
 eclass/dot-a.eclass                           | 124 +++++++
 eclass/tests/dot-a.sh                         | 314 ++++++++++++++++++
 sys-devel/flex/flex-2.6.4-r6.ebuild           |   7 +-
 sys-libs/zlib/zlib-1.3.1-r1.ebuild            |  11 +-
 12 files changed, 521 insertions(+), 10 deletions(-)
 create mode 100644 eclass/dot-a.eclass
 create mode 100755 eclass/tests/dot-a.sh

-- 
2.49.0



^ permalink raw reply	[flat|nested] 10+ messages in thread

* [gentoo-dev] [PATCH 1/5] dot-a.eclass: new eclass for handling LTO in static archives
  2025-05-02 20:13 [gentoo-dev] [PATCH 0/5] Handle LTO with static libraries Sam James
@ 2025-05-02 20:13 ` Sam James
  2025-05-03  4:27   ` Michał Górny
  2025-05-02 20:13 ` [gentoo-dev] [PATCH 2/5] sys-libs/zlib: use dot-a.eclass for LTO Sam James
                   ` (3 subsequent siblings)
  4 siblings, 1 reply; 10+ messages in thread
From: Sam James @ 2025-05-02 20:13 UTC (permalink / raw
  To: gentoo-dev; +Cc: Sam James, Eli Schwartz

Introduce a new eclass with utility functions for handling LTO bytecode
(or internal representation, IR) inside static archives (.a files).

Static libraries when built with LTO will contain LTO bytecode which is
not portable across compiler versions or compiler vendors. To avoid pessimising
the library and always filtering LTO, we can build it with -ffat-lto-objects
instead, which builds some components twice. The installed part will then
have the LTO contents stripped out, leaving the regular objects in the
static archive.

It's not feasible to make these work otherwise, as we'd need tracking
for whether a library was built by a specific compiler and its version,
and that compatibility can vary based on other factors (e.g. with gcc,
sys-devel/gcc[zstd] controls if it supports zstd compression for LTO). We
also discourage static libraries anyway.

Provide two functions:

* lto-guarantee-fat

  If LTO is currently enabled (as determined by `tc-is-lto`, added in
  2aea6c3ff2181ad96187e456a3307609fd288d4c), add `-ffat-lto-objects`
  to CFLAGS and CXXFLAGS if supported.

  This guarantees that produced archives are "fat" (contain both IR
  and regular object files) for later pruning.

* strip-lto-bytecode

  Process a given static archive (.a file) and remove its IR component,
  leaving a regular object.

This approach is also taken by Fedora, openSUSE, and Debian/Ubuntu. An
honourable mention to `lto-rebuild` which fulfilled the same task for many
in the LTO overlay too.

We did consider an alternative approach where we'd relink objects using
the driver in src_install (or some hook afterwards), but this would be
more brittle, as we'd need to extract the right arguments to use (see
e.g. the recent Wireshark issues in fad8ff8a45afc83559f8df695cf96dfec51d3e8a
for how this can be subtle) and not PM-agnostic given we don't have portable
hooks right now (and even if we did, suspect they wouldn't work in a way
that facilitated this). It's also not clear if such an approach would've
worked for Clang.

All of this wasn't worth pursuing until H. J. Lu's patches for binutils
landed, which they have now in binutils-2.44 [0], which made bfd's handling
of mixed objects much more robust.

[0] https://inbox.sourceware.org/binutils/20250112220244.597636-1-hjl.tools@gmail.com/

Bug: https://bugs.gentoo.org/926120
Thanks-to: Arsen Arsenović <arsen@gentoo.org>
Co-authored-by: Eli Schwartz <eschwartz@gentoo.org>
Signed-off-by: Sam James <sam@gentoo.org>
---
 eclass/dot-a.eclass   | 124 +++++++++++++++++
 eclass/tests/dot-a.sh | 314 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 438 insertions(+)
 create mode 100644 eclass/dot-a.eclass
 create mode 100755 eclass/tests/dot-a.sh

diff --git a/eclass/dot-a.eclass b/eclass/dot-a.eclass
new file mode 100644
index 0000000000000..20a0fa1dfc206
--- /dev/null
+++ b/eclass/dot-a.eclass
@@ -0,0 +1,124 @@
+# Copyright 2025 Gentoo Authors
+# Distributed under the terms of the GNU General Public License v2
+
+# @ECLASS: dot-a.eclass
+# @MAINTAINER:
+# Toolchain
+# Toolchain Ninjas <toolchain@gentoo.org>
+# @AUTHOR:
+# Sam James <sam@gentoo.org>
+# Eli Schwartz <eschwartz@gentoo.org>
+# @SUPPORTED_EAPIS: 8
+# @BLURB: Functions to handle stripping LTO bytecode out of static archives.
+# @DESCRIPTION:
+# This eclass provides functions to strip LTO bytecode out of static archives
+# (.a files).
+#
+# Static libraries when built with LTO will contain LTO bytecode which is
+# not portable across compiler versions or compiler vendors. To avoid pessimising
+# the library and always filtering LTO, we can build it with -ffat-lto-objects
+# instead, which builds some components twice. The installed part will then
+# have the LTO contents stripped out, leaving the regular objects in the
+# static archive.
+#
+# Use should be passing calling lto-guarantee-fat before configure-time
+# and calling strip-lto-bytecode after installation.
+#
+# @EXAMPLE:
+# @CODE
+#
+# inherit dot-a
+#
+# src_configure() {
+#     lto-guarantee-fat
+#     econf
+# }
+#
+# src_install() {
+#     default
+#     strip-lto-bytecode
+# }
+case ${EAPI} in
+	8) ;;
+	*) die "${ECLASS}: EAPI ${EAPI:-0} not supported" ;;
+esac
+
+if [[ -z ${_DOT_A_ECLASS} ]] ; then
+_DOT_A_ECLASS=1
+
+inherit flag-o-matic toolchain-funcs
+
+# TODO: QA check
+
+# @FUNCTION: lto-guarantee-fat
+# @DESCRIPTION:
+# If LTO is enabled, appends -ffat-lto-objects or any other flags needed
+# to provide fat LTO objects.
+lto-guarantee-fat() {
+	tc-is-lto || return
+
+	# We add this for all languages as LTO obviously can't be done
+	# if different compilers are used for e.g. C vs C++ anyway.
+	append-flags $(test-flags-CC -ffat-lto-objects)
+}
+
+# @FUNCTION: strip-lto-bytecode
+# @USAGE: [library|directory] [...]
+# @DESCRIPTION:
+# Strips LTO bytecode from libraries (static archives) passed as arguments.
+# Defaults to operating on ${ED} as a whole if no arguments are passed.
+#
+# As an optimisation, if USE=static-libs exists for a package and is disabled,
+# the default-searching behaviour with no arguments is suppressed.
+strip-lto-bytecode() {
+	tc-is-lto || return
+
+	local files=()
+
+	if [[ ${#} -eq 0 ]]; then
+		if ! in_iuse static-libs || use static-libs ; then
+			# maybe we are USE=static-libs. Alternatively, maybe the ebuild doesn't
+			# offer such a choice. In both cases, the user specified the function,
+			# so we expect to be called on *something*, but nothing was explicitly
+			# passed. Try scanning ${ED} automatically.
+			set -- "${ED}"
+		fi
+	fi
+
+	# Check if any of our arguments are directories to be recursed
+	# into.
+	local arg
+	for arg in "$@" ; do
+		if [[ -d ${arg} ]] ; then
+			mapfile -t -d '' -O "${#files[@]}" files < <(find "${arg}" -type f -iname '*.a' -print0)
+		else
+			files+=( "${arg}" )
+		fi
+	done
+
+	toolchain_type=
+	tc-is-gcc && toolchain_type=gnu
+	tc-is-clang && toolchain_type=llvm
+
+	local file
+	for file in "${files[@]}" ; do
+		case ${toolchain_type} in
+			gnu)
+				$(tc-getSTRIP) \
+					-R .gnu.lto_* \
+					-R .gnu.debuglto_* \
+					-N __gnu_lto_v1 \
+					"${file}" || die "Stripping bytecode in ${file} failed"
+				;;
+			llvm)
+				llvm-bitcode-strip \
+					-r "${file}" \
+					-o "${file}" || die "Stripping bytecode in ${file} failed"
+				;;
+			*)
+				;;
+		esac
+	done
+}
+
+fi
diff --git a/eclass/tests/dot-a.sh b/eclass/tests/dot-a.sh
new file mode 100755
index 0000000000000..5c153b3b85fba
--- /dev/null
+++ b/eclass/tests/dot-a.sh
@@ -0,0 +1,314 @@
+#!/bin/bash
+# Copyright 2025 Gentoo Authors
+# Distributed under the terms of the GNU General Public License v2
+
+EAPI=8
+
+source tests-common.sh || exit
+source version-funcs.sh || exit
+
+inherit dot-a
+
+_create_test_progs() {
+	cat <<-EOF > a.c
+	int foo();
+
+	int foo() {
+		return 42;
+	}
+	EOF
+
+	cat <<-EOF > main.c
+	#include <stdio.h>
+	int foo();
+
+	int main() {
+		printf("Got magic number: %d\n", foo());
+		return 0;
+	}
+	EOF
+}
+
+test_lto_guarantee_fat() {
+	# Check whether lto-guarantee-fat adds -ffat-lto-objects and it
+	# results in a successful link (and a failed link without it).
+	LDFLAGS="-fuse-ld=${linker}"
+
+	$(tc-getCC) ${CFLAGS} -flto a.c -o a.o -c || die
+	$(tc-getCC) ${CFLAGS} ${LDFLAGS} -flto main.c a.o -o main || die
+	if ./main | grep -q "Got magic number: 42" ; then
+		:;
+	else
+		die "Pure LTO check failed"
+	fi
+
+	tbegin "lto-guarantee-fat (CC=$(tc-getCC), linker=${linker}): check linking w/ fat LTO object w LTO"
+	ret=0
+	(
+		export CFLAGS="-O2 -flto"
+		lto-guarantee-fat
+
+		$(tc-getCC) ${CFLAGS} a.c -o a.o -c 2>/dev/null || return 1
+		$(tc-getCC) ${CFLAGS} ${LDFLAGS} main.c a.o 2>/dev/null || return 1
+	) || ret=1
+	tend ${ret} "Linking LTO executable w/ fat archive failed"
+
+	tbegin "lto-guarantee-fat (CC=$(tc-getCC), linker=${linker}): check linking w/ fat LTO object w/o LTO"
+	ret=0
+	(
+		export CFLAGS="-O2 -flto"
+		lto-guarantee-fat
+
+		# Linking here will fail if a.o isn't a fat object, as there's nothing
+		# to fall back on with -fno-lto.
+		$(tc-getCC) ${CFLAGS} a.c -o a.o -c 2>/dev/null || return 1
+		$(tc-getCC) ${CFLAGS} ${LDFLAGS} -fno-lto main.c a.o 2>/dev/null || return 1
+	) || ret=1
+	tend ${ret} "Linking non-LTO executable w/ fat archive failed"
+}
+
+test_strip_lto_bytecode() {
+        # Check whether strip-lto-bytecode does its job on a single argument, but
+	# focus of this test is more basic, not checking all possible option
+	# handling.
+	#
+	# i.e. If we use strip-lto-bytecode, does it remove the LTO bytecode
+	# and allow linking? If we use it w/o -ffat-lto-objects, do we get
+	# a failed link as we expect?
+	LDFLAGS="-fuse-ld=${linker}"
+
+	tbegin "strip-lto-bytecode (CC=$(tc-getCC), linker=${linker}): check that linking w/ stripped non-fat archive breaks"
+	ret=0
+	(
+		export CFLAGS="-O2 -flto"
+
+		# strip-lto-bytecode will error out early with LLVM,
+		# so stop the test here.
+		tc-is-clang && return 0
+
+		$(tc-getCC) ${CFLAGS} a.c -o a.o -c 2>/dev/null || return 1
+
+		# This should corrupt a.o and make linking below fail.
+		strip-lto-bytecode a.o
+
+		$(tc-getCC) ${CFLAGS} ${LDFLAGS} main.c a.o -o main 2>/dev/null && return 1
+
+		return 0
+	) || ret=1
+	tend ${ret} "Linking corrupted non-fat archive unexpectedly worked"
+
+	tbegin "strip-lto-bytecode (CC=$(tc-getCC), linker=${linker}): check that linking w/ stripped fat archive works"
+	ret=0
+	(
+		export CFLAGS="-O2 -flto"
+
+		lto-guarantee-fat
+
+		$(tc-getCC) ${CFLAGS} a.c -o a.o -c 2>/dev/null || return 1
+
+		# This should NOT corrupt a.o, so linking below should succeed.
+		strip-lto-bytecode a.o
+
+		$(tc-getCC) ${CFLAGS} ${LDFLAGS} main.c a.o -o main 2>/dev/null || return 1
+	) || ret=1
+	tend ${ret} "Linking stripped fat archive failed"
+}
+
+test_mixed_objects_after_stripping() {
+	# Check whether mixing objects from two compilers (${CC_1} and ${CC_2})
+	# fails without lto-guarantee-fat and strip-lto-bytecode and works
+	# once they're used.
+	LDFLAGS="-fuse-ld=${linker}"
+
+	tbegin "strip-lto-bytecode (CC_1=${CC_1}, CC_2=${CC_2}, linker=${linker}): check that unstripped LTO objects from ${CC_1} fail w/ ${CC_2}"
+	ret=0
+	(
+		export CFLAGS="-O2 -flto"
+
+		${CC_1} ${CFLAGS} a.c -o a.o -c 2>/dev/null || return 1
+		# Using CC_1 IR with CC_2 should fail.
+		${CC_2} ${CFLAGS} ${LDFLAGS} main.c a.o -o main 2>/dev/null && return 1
+
+		return 0
+	) || ret=1
+	tend ${ret} "Mixing unstripped objects unexpectedly worked"
+
+	tbegin "strip-lto-bytecode (CC_1=${CC_1}, CC_2=${CC_2}, linker=${linker}): check that unstripped LTO objects from ${CC_2} fail w/ ${CC_1}"
+	ret=0
+	(
+		export CFLAGS="-O2 -flto"
+
+		${CC_2} ${CFLAGS} a.c -o a.o -c 2>/dev/null || return 1
+		# Using CC_2 IR with CC_1 should fail.
+		${CC_1} ${CFLAGS} ${LDFLAGS} main.c a.o -o main 2>/dev/null && return 1
+
+		return 0
+	) || ret=1
+	tend ${ret} "Mixing unstripped objects unexpectedly worked"
+
+	tbegin "strip-lto-bytecode (CC_1=${CC_1}, CC_2=${CC_2}, linker=${linker}): check that stripped LTO objects from ${CC_1} work w/ ${CC_2}"
+	ret=0
+	(
+		export CFLAGS="-O2 -flto"
+
+		lto-guarantee-fat
+		${CC_1} ${CFLAGS} a.c -o a.o -c 2>/dev/null || return 1
+		# The object should now be "vendor-neutral" and work.
+		CC=${CC_1} strip-lto-bytecode a.o
+		${CC_2} ${CFLAGS} ${LDFLAGS} main.c a.o -o main 2>/dev/null || return 1
+	) || ret=1
+	tend ${ret} "Mixing stripped objects failed"
+
+	tbegin "strip-lto-bytecode (CC_1=${CC_1}, CC_2=${CC_2}, linker=${linker}): check that stripped LTO objects from ${CC_2} work w/ ${CC_1}"
+	ret=0
+	(
+		export CFLAGS="-O2 -flto"
+
+		lto-guarantee-fat
+		${CC_2} ${CFLAGS} a.c -o a.o -c 2>/dev/null || return 1
+		# The object should now be "vendor-neutral" and work.
+		CC=${CC_2} strip-lto-bytecode a.o
+		${CC_1} ${CFLAGS} ${LDFLAGS} main.c a.o -o main 2>/dev/null || return 1
+	) || ret=1
+	tend ${ret} "Mixing stripped objects failed"
+}
+
+_check_if_lto_object() {
+	# Adapted from tc-is-lto
+	local ret=1
+	case $(tc-get-compiler-type) in
+		clang)
+			# If LTO is used, clang will output bytecode and llvm-bcanalyzer
+			# will run successfully.  Otherwise, it will output plain object
+			# file and llvm-bcanalyzer will exit with error.
+			llvm-bcanalyzer "$1" &>/dev/null && ret=0
+			;;
+		gcc)
+			[[ $($(tc-getREADELF) -S "$1") == *.gnu.lto* ]] && ret=0
+			;;
+	esac
+	return "${ret}"
+}
+
+test_search_recursion() {
+	# Test whether the argument handling and logic of strip-lto-bytecode
+	# works as expected.
+	tbegin "whether default search behaviour of \${ED} works"
+	ret=0
+	(
+		CC=gcc
+		CFLAGS="-O2 -flto"
+
+		_create_test_progs
+		lto-guarantee-fat
+		$(tc-getCC) ${CFLAGS} a.c -o a.o -c 2>/dev/null || return 1
+		ar q foo.a a.o 2>/dev/null || return 1
+
+		_check_if_lto_object "${tmpdir}/lto/foo.a" || return 1
+		# It should search ${ED} if no arguments are passed, find
+		# the LTO'd foo.o, and strip it.
+		ED="${tmpdir}/lto" strip-lto-bytecode
+		# foo.a should be a regular object here.
+		_check_if_lto_object "${tmpdir}/lto/foo.a" && return 1
+
+		return 0
+	) || ret=1
+	tend ${ret} "Unexpected LTO object found"
+
+	tbegin "whether a single file argument works"
+	ret=0
+	(
+		CC=gcc
+		CFLAGS="-O2 -flto"
+
+		_create_test_progs
+		lto-guarantee-fat
+		$(tc-getCC) ${CFLAGS} a.c -o a.o -c 2>/dev/null || return 1
+		ar q foo.a a.o 2>/dev/null || return 1
+
+		_check_if_lto_object "${tmpdir}/lto/foo.a" || return 1
+		# It should search ${ED} if no arguments are passed, find
+		# the LTO'd foo.o, and strip it.
+		ED="${tmpdir}/lto" strip-lto-bytecode "${tmpdir}/lto/foo.a"
+		# foo.a should be a regular object here.
+		_check_if_lto_object "${tmpdir}/lto/foo.a" && return 1
+
+		return 0
+	) || ret=1
+	tend ${ret} "Unexpected LTO object found"
+
+	tbegin "whether a directory and file argument works"
+	ret=0
+	(
+		mkdir "${tmpdir}"/lto2 || die
+
+		CC=gcc
+		CFLAGS="-O2 -flto"
+
+		_create_test_progs
+		lto-guarantee-fat
+		$(tc-getCC) ${CFLAGS} "${tmpdir}"/lto/a.c -o "${tmpdir}"/lto/a.o -c 2>/dev/null || return 1
+		ar q foo.a a.o 2>/dev/null || return 1
+		ar q "${tmpdir}"/lto2/foo.a a.o 2>/dev/null || return 1
+
+		_check_if_lto_object "${tmpdir}/lto/foo.a" || return 1
+		_check_if_lto_object "${tmpdir}/lto2/foo.a" || return 1
+		# It should search ${ED} if no arguments are passed, find
+		# the LTO'd foo.o, and strip it.
+		ED="${tmpdir}/lto" strip-lto-bytecode "${tmpdir}/lto/foo.a" "${tmpdir}/lto2/foo.a"
+		# foo.a should be a regular object here.
+		_check_if_lto_object "${tmpdir}/lto/foo.a" && return 1
+		_check_if_lto_object "${tmpdir}/lto2/foo.a" && return 1
+
+		return 0
+	) || ret=1
+	tend ${ret} "Unexpected LTO object found"
+}
+
+_repeat_tests_with_compilers() {
+	# Call test_lto_guarantee_fat and test_strip_lto_bytecode with
+	# various compilers and linkers.
+	for CC in gcc clang ; do
+		type -P ${CC} &>/dev/null || continue
+
+		for linker in gold bfd lld mold gold ; do
+			# lld doesn't support GCC LTO: https://github.com/llvm/llvm-project/issues/41791
+			[[ ${CC} == gcc && ${linker} == lld ]] && continue
+			# Make sure the relevant linker is actually installed and usable.
+			LDFLAGS="-fuse-ld=${linker}" tc-ld-is-${linker} || continue
+			LDFLAGS="-fuse-ld=${linker}" test-compile 'c+ld' 'int main() { return 0; }' || continue
+
+			test_lto_guarantee_fat
+			test_strip_lto_bytecode
+		done
+	done
+}
+
+_repeat_mixed_tests_with_linkers() {
+	# Call test_mixed_objects_after_stripping with various linkers.
+	#
+	# Needs both GCC and Clang to test mixing their outputs.
+	if type -P gcc &>/dev/null && type -P clang &>/dev/null ; then
+		for linker in bfd lld mold gold ; do
+			# lld doesn't support GCC LTO: https://github.com/llvm/llvm-project/issues/41791
+			[[ ${CC} == gcc && ${linker} == lld ]] && continue
+			# Make sure the relevant linker is actually installed and usable.
+			LDFLAGS="-fuse-ld=${linker}" tc-ld-is-${linker} || continue
+			LDFLAGS="-fuse-ld=${linker}" test-compile 'c+ld' 'int main() { return 0; }' || continue
+
+			CC_1=gcc
+			CC_2=clang
+			test_mixed_objects_after_stripping
+		done
+	fi
+}
+
+mkdir -p "${tmpdir}/lto" || die
+pushd "${tmpdir}/lto" >/dev/null || die
+_create_test_progs
+_repeat_tests_with_compilers
+_repeat_mixed_tests_with_linkers
+test_search_recursion
+texit
+
+# TODO: test multiple files
-- 
2.49.0



^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [gentoo-dev] [PATCH 2/5] sys-libs/zlib: use dot-a.eclass for LTO
  2025-05-02 20:13 [gentoo-dev] [PATCH 0/5] Handle LTO with static libraries Sam James
  2025-05-02 20:13 ` [gentoo-dev] [PATCH 1/5] dot-a.eclass: new eclass for handling LTO in static archives Sam James
@ 2025-05-02 20:13 ` Sam James
  2025-05-02 20:13 ` [gentoo-dev] [PATCH 3/5] dev-libs/glib: " Sam James
                   ` (2 subsequent siblings)
  4 siblings, 0 replies; 10+ messages in thread
From: Sam James @ 2025-05-02 20:13 UTC (permalink / raw
  To: gentoo-dev; +Cc: Sam James

This means that USE=static-libs doesn't produce a "broken" library
when built w/ LTO.

Signed-off-by: Sam James <sam@gentoo.org>
---
 sys-libs/zlib/zlib-1.3.1-r1.ebuild | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/sys-libs/zlib/zlib-1.3.1-r1.ebuild b/sys-libs/zlib/zlib-1.3.1-r1.ebuild
index f81cf5fac8d68..837b8564eec1b 100644
--- a/sys-libs/zlib/zlib-1.3.1-r1.ebuild
+++ b/sys-libs/zlib/zlib-1.3.1-r1.ebuild
@@ -1,4 +1,4 @@
-# Copyright 1999-2024 Gentoo Authors
+# Copyright 1999-2025 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 EAPI=8
@@ -6,7 +6,7 @@ EAPI=8
 # Worth keeping an eye on 'develop' branch upstream for possible backports.
 AUTOTOOLS_AUTO_DEPEND="no"
 VERIFY_SIG_OPENPGP_KEY_PATH=/usr/share/openpgp-keys/madler.asc
-inherit autotools edo multilib-minimal flag-o-matic verify-sig
+inherit autotools dot-a edo multilib-minimal flag-o-matic verify-sig
 
 DESCRIPTION="Standard (de)compression library"
 HOMEPAGE="https://zlib.net/"
@@ -68,6 +68,11 @@ src_prepare() {
 	esac
 }
 
+src_configure() {
+	lto-guarantee-fat
+	multilib-minimal_src_configure
+}
+
 multilib_src_configure() {
 	# We pass manually instead of relying on the configure script/makefile
 	# because it would pass it even for older binutils.
@@ -164,6 +169,8 @@ multilib_src_install() {
 }
 
 multilib_src_install_all() {
+	strip-lto-bytecode
+
 	dodoc FAQ README ChangeLog doc/*.txt
 
 	if use minizip ; then
-- 
2.49.0



^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [gentoo-dev] [PATCH 3/5] dev-libs/glib: use dot-a.eclass for LTO
  2025-05-02 20:13 [gentoo-dev] [PATCH 0/5] Handle LTO with static libraries Sam James
  2025-05-02 20:13 ` [gentoo-dev] [PATCH 1/5] dot-a.eclass: new eclass for handling LTO in static archives Sam James
  2025-05-02 20:13 ` [gentoo-dev] [PATCH 2/5] sys-libs/zlib: use dot-a.eclass for LTO Sam James
@ 2025-05-02 20:13 ` Sam James
  2025-05-02 20:13 ` [gentoo-dev] [PATCH 4/5] dev-util/sysprof-capture: " Sam James
  2025-05-02 20:13 ` [gentoo-dev] [PATCH 5/5] sys-devel/flex: " Sam James
  4 siblings, 0 replies; 10+ messages in thread
From: Sam James @ 2025-05-02 20:13 UTC (permalink / raw
  To: gentoo-dev; +Cc: Sam James

This means that USE=static-libs doesn't produce a "broken" library
when built w/ LTO.

Bug: https://bugs.gentoo.org/927994
Signed-off-by: Sam James <sam@gentoo.org>
---
 dev-libs/glib/glib-2.78.6.ebuild    | 7 +++++++
 dev-libs/glib/glib-2.80.5-r1.ebuild | 7 +++++++
 dev-libs/glib/glib-2.82.5.ebuild    | 7 +++++++
 dev-libs/glib/glib-2.84.0.ebuild    | 7 +++++++
 dev-libs/glib/glib-2.84.1.ebuild    | 9 ++++++++-
 5 files changed, 36 insertions(+), 1 deletion(-)

diff --git a/dev-libs/glib/glib-2.78.6.ebuild b/dev-libs/glib/glib-2.78.6.ebuild
index e5481fe4d0052..b9609c7c059f2 100644
--- a/dev-libs/glib/glib-2.78.6.ebuild
+++ b/dev-libs/glib/glib-2.78.6.ebuild
@@ -173,6 +173,11 @@ src_prepare() {
 	# TODO: python_name sedding for correct python shebang? Might be relevant mainly for glib-utils only
 }
 
+src_configure() {
+	lto-guarantee-fat
+	meson-multilib_src_configure
+}
+
 multilib_src_configure() {
 	# TODO: figure a way to pass appropriate values for all cross properties
 	# that glib uses (search for get_cross_property)
@@ -245,6 +250,8 @@ multilib_src_install() {
 }
 
 multilib_src_install_all() {
+	strip-lto-bytecode
+
 	# These are installed by dev-util/glib-utils
 	# TODO: With patching we might be able to get rid of the python-any deps
 	# and removals, and test depend on glib-utils instead; revisit now with
diff --git a/dev-libs/glib/glib-2.80.5-r1.ebuild b/dev-libs/glib/glib-2.80.5-r1.ebuild
index fb26093bb4256..d1794a03ff0c3 100644
--- a/dev-libs/glib/glib-2.80.5-r1.ebuild
+++ b/dev-libs/glib/glib-2.80.5-r1.ebuild
@@ -210,6 +210,11 @@ src_prepare() {
 	# TODO: python_name sedding for correct python shebang? Might be relevant mainly for glib-utils only
 }
 
+src_configure() {
+	lto-guarantee-fat
+	meson-multilib_src_configure
+}
+
 multilib_src_configure() {
 	# TODO: figure a way to pass appropriate values for all cross properties
 	# that glib uses (search for get_cross_property)
@@ -371,6 +376,8 @@ multilib_src_install() {
 }
 
 multilib_src_install_all() {
+	strip-lto-bytecode
+
 	# These are installed by dev-util/glib-utils
 	# TODO: With patching we might be able to get rid of the python-any deps
 	# and removals, and test depend on glib-utils instead; revisit now with
diff --git a/dev-libs/glib/glib-2.82.5.ebuild b/dev-libs/glib/glib-2.82.5.ebuild
index b16e62189f6ac..859f0f25c4897 100644
--- a/dev-libs/glib/glib-2.82.5.ebuild
+++ b/dev-libs/glib/glib-2.82.5.ebuild
@@ -198,6 +198,11 @@ src_prepare() {
 	# TODO: python_name sedding for correct python shebang? Might be relevant mainly for glib-utils only
 }
 
+src_configure() {
+	lto-guarantee-fat
+	meson-multilib_src_configure
+}
+
 multilib_src_configure() {
 	# TODO: figure a way to pass appropriate values for all cross properties
 	# that glib uses (search for get_cross_property)
@@ -359,6 +364,8 @@ multilib_src_install() {
 }
 
 multilib_src_install_all() {
+	strip-lto-bytecode
+
 	# These are installed by dev-util/glib-utils
 	# TODO: With patching we might be able to get rid of the python-any deps
 	# and removals, and test depend on glib-utils instead; revisit now with
diff --git a/dev-libs/glib/glib-2.84.0.ebuild b/dev-libs/glib/glib-2.84.0.ebuild
index 8ec0a9d7d48d3..50dfe9b47f1c1 100644
--- a/dev-libs/glib/glib-2.84.0.ebuild
+++ b/dev-libs/glib/glib-2.84.0.ebuild
@@ -202,6 +202,11 @@ src_prepare() {
 	# TODO: python_name sedding for correct python shebang? Might be relevant mainly for glib-utils only
 }
 
+src_configure() {
+	lto-guarantee-fat
+	meson-multilib_src_configure
+}
+
 multilib_src_configure() {
 	# TODO: figure a way to pass appropriate values for all cross properties
 	# that glib uses (search for get_cross_property)
@@ -363,6 +368,8 @@ multilib_src_install() {
 }
 
 multilib_src_install_all() {
+	strip-lto-bytecode
+
 	# These are installed by dev-util/glib-utils
 	# TODO: With patching we might be able to get rid of the python-any deps
 	# and removals, and test depend on glib-utils instead; revisit now with
diff --git a/dev-libs/glib/glib-2.84.1.ebuild b/dev-libs/glib/glib-2.84.1.ebuild
index e5e5ff280c0f2..342d919ebecd7 100644
--- a/dev-libs/glib/glib-2.84.1.ebuild
+++ b/dev-libs/glib/glib-2.84.1.ebuild
@@ -5,7 +5,7 @@ EAPI=8
 PYTHON_REQ_USE="xml(+)"
 PYTHON_COMPAT=( python3_{10..13} )
 
-inherit eapi9-ver gnome.org gnome2-utils linux-info meson-multilib multilib python-any-r1 toolchain-funcs xdg
+inherit dot-a eapi9-ver gnome.org gnome2-utils linux-info meson-multilib multilib python-any-r1 toolchain-funcs xdg
 
 DESCRIPTION="The GLib library of C routines"
 HOMEPAGE="https://www.gtk.org/"
@@ -203,6 +203,11 @@ src_prepare() {
 	# TODO: python_name sedding for correct python shebang? Might be relevant mainly for glib-utils only
 }
 
+src_configure() {
+	lto-guarantee-fat
+	meson-multilib_src_configure
+}
+
 multilib_src_configure() {
 	# TODO: figure a way to pass appropriate values for all cross properties
 	# that glib uses (search for get_cross_property)
@@ -364,6 +369,8 @@ multilib_src_install() {
 }
 
 multilib_src_install_all() {
+	strip-lto-bytecode
+
 	# These are installed by dev-util/glib-utils
 	# TODO: With patching we might be able to get rid of the python-any deps
 	# and removals, and test depend on glib-utils instead; revisit now with
-- 
2.49.0



^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [gentoo-dev] [PATCH 4/5] dev-util/sysprof-capture: use dot-a.eclass for LTO
  2025-05-02 20:13 [gentoo-dev] [PATCH 0/5] Handle LTO with static libraries Sam James
                   ` (2 preceding siblings ...)
  2025-05-02 20:13 ` [gentoo-dev] [PATCH 3/5] dev-libs/glib: " Sam James
@ 2025-05-02 20:13 ` Sam James
  2025-05-02 20:13 ` [gentoo-dev] [PATCH 5/5] sys-devel/flex: " Sam James
  4 siblings, 0 replies; 10+ messages in thread
From: Sam James @ 2025-05-02 20:13 UTC (permalink / raw
  To: gentoo-dev; +Cc: Sam James

This means that USE=static-libs doesn't produce a "broken" library
when built w/ LTO.

Bug: https://bugs.gentoo.org/818097
Bug: https://bugs.gentoo.org/889004
Bug: https://bugs.gentoo.org/927994
Bug: https://bugs.gentoo.org/939515
Signed-off-by: Sam James <sam@gentoo.org>
---
 .../sysprof-capture/sysprof-capture-3.48.0.ebuild  | 14 ++++++++++++--
 .../sysprof-capture/sysprof-capture-46.0.ebuild    | 12 +++++++++++-
 .../sysprof-capture/sysprof-capture-47.2.ebuild    | 12 +++++++++++-
 3 files changed, 34 insertions(+), 4 deletions(-)

diff --git a/dev-util/sysprof-capture/sysprof-capture-3.48.0.ebuild b/dev-util/sysprof-capture/sysprof-capture-3.48.0.ebuild
index 0502a2eca20da..095cff44673d1 100644
--- a/dev-util/sysprof-capture/sysprof-capture-3.48.0.ebuild
+++ b/dev-util/sysprof-capture/sysprof-capture-3.48.0.ebuild
@@ -1,10 +1,10 @@
-# Copyright 1999-2023 Gentoo Authors
+# Copyright 1999-2025 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 EAPI=8
 GNOME_ORG_MODULE="sysprof"
 
-inherit gnome.org meson-multilib systemd
+inherit dot-a gnome.org meson-multilib systemd
 
 DESCRIPTION="Static library for sysprof capture data generation"
 HOMEPAGE="http://sysprof.com/"
@@ -21,6 +21,11 @@ BDEPEND="
 	virtual/pkgconfig
 "
 
+src_configure() {
+	lto-guarantee-fat
+	meson-multilib_src_configure
+}
+
 multilib_src_configure() {
 	local emesonargs=(
 		-Dgtk=false
@@ -38,3 +43,8 @@ multilib_src_configure() {
 	)
 	meson_src_configure
 }
+
+multilib_src_install_all() {
+	einstalldocs
+	strip-lto-bytecode
+}
diff --git a/dev-util/sysprof-capture/sysprof-capture-46.0.ebuild b/dev-util/sysprof-capture/sysprof-capture-46.0.ebuild
index 2934a83c17343..c8e9fc6d93a81 100644
--- a/dev-util/sysprof-capture/sysprof-capture-46.0.ebuild
+++ b/dev-util/sysprof-capture/sysprof-capture-46.0.ebuild
@@ -4,7 +4,7 @@
 EAPI=8
 GNOME_ORG_MODULE="sysprof"
 
-inherit gnome.org meson-multilib systemd
+inherit dot-a gnome.org meson-multilib systemd
 
 DESCRIPTION="Static library for sysprof capture data generation"
 HOMEPAGE="http://sysprof.com/"
@@ -21,6 +21,11 @@ BDEPEND="
 	virtual/pkgconfig
 "
 
+src_configure() {
+	lto-guarantee-fat
+	meson-multilib_src_configure
+}
+
 multilib_src_configure() {
 	local emesonargs=(
 		-Dgtk=false
@@ -36,3 +41,8 @@ multilib_src_configure() {
 	)
 	meson_src_configure
 }
+
+multilib_src_install_all() {
+	einstalldocs
+	strip-lto-bytecode
+}
diff --git a/dev-util/sysprof-capture/sysprof-capture-47.2.ebuild b/dev-util/sysprof-capture/sysprof-capture-47.2.ebuild
index 24b308d6b635e..6618277e10104 100644
--- a/dev-util/sysprof-capture/sysprof-capture-47.2.ebuild
+++ b/dev-util/sysprof-capture/sysprof-capture-47.2.ebuild
@@ -4,7 +4,7 @@
 EAPI=8
 GNOME_ORG_MODULE="sysprof"
 
-inherit gnome.org meson-multilib systemd
+inherit dot-a gnome.org meson-multilib systemd
 
 DESCRIPTION="Static library for sysprof capture data generation"
 HOMEPAGE="http://sysprof.com/"
@@ -21,6 +21,11 @@ BDEPEND="
 	virtual/pkgconfig
 "
 
+src_configure() {
+	lto-guarantee-fat
+	meson-multilib_src_configure
+}
+
 multilib_src_configure() {
 	local emesonargs=(
 		-Dgtk=false
@@ -36,3 +41,8 @@ multilib_src_configure() {
 	)
 	meson_src_configure
 }
+
+multilib_src_install_all() {
+	einstalldocs
+	strip-lto-bytecode
+}
-- 
2.49.0



^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [gentoo-dev] [PATCH 5/5] sys-devel/flex: use dot-a.eclass for LTO
  2025-05-02 20:13 [gentoo-dev] [PATCH 0/5] Handle LTO with static libraries Sam James
                   ` (3 preceding siblings ...)
  2025-05-02 20:13 ` [gentoo-dev] [PATCH 4/5] dev-util/sysprof-capture: " Sam James
@ 2025-05-02 20:13 ` Sam James
  4 siblings, 0 replies; 10+ messages in thread
From: Sam James @ 2025-05-02 20:13 UTC (permalink / raw
  To: gentoo-dev; +Cc: Sam James

This means that USE=static-libs doesn't produce a "broken" library
when built w/ LTO.

Closes: https://bugs.gentoo.org/592868
Closes: https://bugs.gentoo.org/616106
Closes: https://bugs.gentoo.org/876430
Closes: https://bugs.gentoo.org/924183
Closes: https://bugs.gentoo.org/936172
Closes: https://bugs.gentoo.org/938858
Closes: https://bugs.gentoo.org/940541
Closes: https://bugs.gentoo.org/944291
Closes: https://bugs.gentoo.org/945923
Signed-off-by: Sam James <sam@gentoo.org>
---
 sys-devel/flex/flex-2.6.4-r6.ebuild | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/sys-devel/flex/flex-2.6.4-r6.ebuild b/sys-devel/flex/flex-2.6.4-r6.ebuild
index 41845f105a3d3..c024ea00fe94c 100644
--- a/sys-devel/flex/flex-2.6.4-r6.ebuild
+++ b/sys-devel/flex/flex-2.6.4-r6.ebuild
@@ -1,9 +1,9 @@
-# Copyright 1999-2023 Gentoo Authors
+# Copyright 1999-2025 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 EAPI=8
 
-inherit flag-o-matic libtool multilib-minimal toolchain-funcs
+inherit dot-a flag-o-matic libtool multilib-minimal toolchain-funcs
 
 DESCRIPTION="The Fast Lexical Analyzer"
 HOMEPAGE="https://github.com/westes/flex"
@@ -53,7 +53,7 @@ src_prepare() {
 
 src_configure() {
 	use static && append-ldflags -static
-
+	lto-guarantee-fat
 	multilib-minimal_src_configure
 }
 
@@ -86,6 +86,7 @@ multilib_src_install() {
 }
 
 multilib_src_install_all() {
+	strip-lto-bytecode
 	einstalldocs
 	dodoc ONEWS
 	find "${ED}" -name '*.la' -type f -delete || die
-- 
2.49.0



^ permalink raw reply related	[flat|nested] 10+ messages in thread

* Re: [gentoo-dev] [PATCH 1/5] dot-a.eclass: new eclass for handling LTO in static archives
  2025-05-02 20:13 ` [gentoo-dev] [PATCH 1/5] dot-a.eclass: new eclass for handling LTO in static archives Sam James
@ 2025-05-03  4:27   ` Michał Górny
  2025-05-03  6:22     ` Sam James
  0 siblings, 1 reply; 10+ messages in thread
From: Michał Górny @ 2025-05-03  4:27 UTC (permalink / raw
  To: gentoo-dev; +Cc: Sam James, Eli Schwartz

[-- Attachment #1: Type: text/plain, Size: 4723 bytes --]

Hi,

Sorry for not looking earlier.

On Fri, 2025-05-02 at 21:13 +0100, Sam James wrote:
> diff --git a/eclass/dot-a.eclass b/eclass/dot-a.eclass
> new file mode 100644
> index 0000000000000..20a0fa1dfc206
> --- /dev/null
> +++ b/eclass/dot-a.eclass
> @@ -0,0 +1,124 @@
> +# Copyright 2025 Gentoo Authors
> +# Distributed under the terms of the GNU General Public License v2
> +
> +# @ECLASS: dot-a.eclass
> +# @MAINTAINER:
> +# Toolchain
> +# Toolchain Ninjas <toolchain@gentoo.org>

You've got "toolchain" twice.

> +# @AUTHOR:
> +# Sam James <sam@gentoo.org>
> +# Eli Schwartz <eschwartz@gentoo.org>
> +# @SUPPORTED_EAPIS: 8
> +# @BLURB: Functions to handle stripping LTO bytecode out of static archives.
> +# @DESCRIPTION:
> +# This eclass provides functions to strip LTO bytecode out of static archives
> +# (.a files).
> +#
> +# Static libraries when built with LTO will contain LTO bytecode which is
> +# not portable across compiler versions or compiler vendors. To avoid pessimising
> +# the library and always filtering LTO, we can build it with -ffat-lto-objects
> +# instead, which builds some components twice. The installed part will then
> +# have the LTO contents stripped out, leaving the regular objects in the
> +# static archive.
> +#
> +# Use should be passing calling lto-guarantee-fat before configure-time
> +# and calling strip-lto-bytecode after installation.

Do I understand correctly that this is to be always used when you're
installing static libraries?  Or are there more specific criteria? 
Perhaps that'd be worth including in the doc.

> +#
> +# @EXAMPLE:
> +# @CODE
> +#
> +# inherit dot-a
> +#
> +# src_configure() {
> +#     lto-guarantee-fat
> +#     econf
> +# }
> +#
> +# src_install() {
> +#     default
> +#     strip-lto-bytecode
> +# }
> +case ${EAPI} in
> +	8) ;;
> +	*) die "${ECLASS}: EAPI ${EAPI:-0} not supported" ;;
> +esac
> +
> +if [[ -z ${_DOT_A_ECLASS} ]] ; then
> +_DOT_A_ECLASS=1
> +
> +inherit flag-o-matic toolchain-funcs
> +
> +# TODO: QA check
> +
> +# @FUNCTION: lto-guarantee-fat
> +# @DESCRIPTION:
> +# If LTO is enabled, appends -ffat-lto-objects or any other flags needed
> +# to provide fat LTO objects.
> +lto-guarantee-fat() {
> +	tc-is-lto || return
> +
> +	# We add this for all languages as LTO obviously can't be done
> +	# if different compilers are used for e.g. C vs C++ anyway.
> +	append-flags $(test-flags-CC -ffat-lto-objects)
> +}
> +
> +# @FUNCTION: strip-lto-bytecode
> +# @USAGE: [library|directory] [...]
> +# @DESCRIPTION:
> +# Strips LTO bytecode from libraries (static archives) passed as arguments.
> +# Defaults to operating on ${ED} as a whole if no arguments are passed.
> +#
> +# As an optimisation, if USE=static-libs exists for a package and is disabled,
> +# the default-searching behaviour with no arguments is suppressed.
> +strip-lto-bytecode() {
> +	tc-is-lto || return
> +
> +	local files=()
> +
> +	if [[ ${#} -eq 0 ]]; then
> +		if ! in_iuse static-libs || use static-libs ; then
> +			# maybe we are USE=static-libs. Alternatively, maybe the ebuild doesn't
> +			# offer such a choice. In both cases, the user specified the function,
> +			# so we expect to be called on *something*, but nothing was explicitly
> +			# passed. Try scanning ${ED} automatically.
> +			set -- "${ED}"
> +		fi
> +	fi
> +
> +	# Check if any of our arguments are directories to be recursed
> +	# into.
> +	local arg
> +	for arg in "$@" ; do
> +		if [[ -d ${arg} ]] ; then
> +			mapfile -t -d '' -O "${#files[@]}" files < <(find "${arg}" -type f -iname '*.a' -print0)
> +		else
> +			files+=( "${arg}" )
> +		fi
> +	done

Why not just pass all arguments to find(1)?  If you pass a file path to
it, it will just return that file, i.e.:

mapfile -t -d '' files < <(find -H "${@}" -type f -iname '*.a' -print0)

('-H' to follow symbolic links if passed directly in "$@")

> +
> +	toolchain_type=
> +	tc-is-gcc && toolchain_type=gnu
> +	tc-is-clang && toolchain_type=llvm
> +
> +	local file
> +	for file in "${files[@]}" ; do
> +		case ${toolchain_type} in
> +			gnu)
> +				$(tc-getSTRIP) \
> +					-R .gnu.lto_* \
> +					-R .gnu.debuglto_* \
> +					-N __gnu_lto_v1 \
> +					"${file}" || die "Stripping bytecode in ${file} failed"
> +				;;

Technically, strip accepts multiple files, but I can guess there's
no point in optimizing the GNU branch here.

> +			llvm)
> +				llvm-bitcode-strip \
> +					-r "${file}" \
> +					-o "${file}" || die "Stripping bytecode in ${file} failed"
> +				;;
> +			*)
> +				;;
> +		esac
> +	done
> +}
> +
> +fi

-- 
Best regards,
Michał Górny


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [gentoo-dev] [PATCH 1/5] dot-a.eclass: new eclass for handling LTO in static archives
  2025-05-03  4:27   ` Michał Górny
@ 2025-05-03  6:22     ` Sam James
  2025-05-03  6:44       ` Sam James
  0 siblings, 1 reply; 10+ messages in thread
From: Sam James @ 2025-05-03  6:22 UTC (permalink / raw
  To: Michał Górny; +Cc: gentoo-dev, Eli Schwartz

Michał Górny <mgorny@gentoo.org> writes:

> Hi,
>
> Sorry for not looking earlier.
>

Thanks for reviewing (and no worries, this has been loooooong overdue).


> On Fri, 2025-05-02 at 21:13 +0100, Sam James wrote:
>> diff --git a/eclass/dot-a.eclass b/eclass/dot-a.eclass
>> new file mode 100644
>> index 0000000000000..20a0fa1dfc206
>> --- /dev/null
>> +++ b/eclass/dot-a.eclass
>> @@ -0,0 +1,124 @@
>> +# Copyright 2025 Gentoo Authors
>> +# Distributed under the terms of the GNU General Public License v2
>> +
>> +# @ECLASS: dot-a.eclass
>> +# @MAINTAINER:
>> +# Toolchain
>> +# Toolchain Ninjas <toolchain@gentoo.org>
>
> You've got "toolchain" twice.

ACK.

>
>> +# @AUTHOR:
>> +# Sam James <sam@gentoo.org>
>> +# Eli Schwartz <eschwartz@gentoo.org>
>> +# @SUPPORTED_EAPIS: 8
>> +# @BLURB: Functions to handle stripping LTO bytecode out of static archives.
>> +# @DESCRIPTION:
>> +# This eclass provides functions to strip LTO bytecode out of static archives
>> +# (.a files).
>> +#
>> +# Static libraries when built with LTO will contain LTO bytecode which is
>> +# not portable across compiler versions or compiler vendors. To avoid pessimising
>> +# the library and always filtering LTO, we can build it with -ffat-lto-objects
>> +# instead, which builds some components twice. The installed part will then
>> +# have the LTO contents stripped out, leaving the regular objects in the
>> +# static archive.
>> +#
>> +# Use should be passing calling lto-guarantee-fat before configure-time
>> +# and calling strip-lto-bytecode after installation.
>
> Do I understand correctly that this is to be always used when you're
> installing static libraries?  Or are there more specific criteria?
> Perhaps that'd be worth including in the doc.

Correct, with the exception of packages where LTO is currently filtered
(but people should add it to future-proof unless it's something like
glibc).

I'll document that.

>
>> +#
>> +# @EXAMPLE:
>> +# @CODE
>> +#
>> +# inherit dot-a
>> +#
>> +# src_configure() {
>> +#     lto-guarantee-fat
>> +#     econf
>> +# }
>> +#
>> +# src_install() {
>> +#     default
>> +#     strip-lto-bytecode
>> +# }
>> +case ${EAPI} in
>> +	8) ;;
>> +	*) die "${ECLASS}: EAPI ${EAPI:-0} not supported" ;;
>> +esac
>> +
>> +if [[ -z ${_DOT_A_ECLASS} ]] ; then
>> +_DOT_A_ECLASS=1
>> +
>> +inherit flag-o-matic toolchain-funcs
>> +
>> +# TODO: QA check
>> +
>> +# @FUNCTION: lto-guarantee-fat
>> +# @DESCRIPTION:
>> +# If LTO is enabled, appends -ffat-lto-objects or any other flags needed
>> +# to provide fat LTO objects.
>> +lto-guarantee-fat() {
>> +	tc-is-lto || return
>> +
>> +	# We add this for all languages as LTO obviously can't be done
>> +	# if different compilers are used for e.g. C vs C++ anyway.
>> +	append-flags $(test-flags-CC -ffat-lto-objects)
>> +}
>> +
>> +# @FUNCTION: strip-lto-bytecode
>> +# @USAGE: [library|directory] [...]
>> +# @DESCRIPTION:
>> +# Strips LTO bytecode from libraries (static archives) passed as arguments.
>> +# Defaults to operating on ${ED} as a whole if no arguments are passed.
>> +#
>> +# As an optimisation, if USE=static-libs exists for a package and is disabled,
>> +# the default-searching behaviour with no arguments is suppressed.
>> +strip-lto-bytecode() {
>> +	tc-is-lto || return
>> +
>> +	local files=()
>> +
>> +	if [[ ${#} -eq 0 ]]; then
>> +		if ! in_iuse static-libs || use static-libs ; then
>> +			# maybe we are USE=static-libs. Alternatively, maybe the ebuild doesn't
>> +			# offer such a choice. In both cases, the user specified the function,
>> +			# so we expect to be called on *something*, but nothing was explicitly
>> +			# passed. Try scanning ${ED} automatically.
>> +			set -- "${ED}"
>> +		fi
>> +	fi
>> +
>> +	# Check if any of our arguments are directories to be recursed
>> +	# into.
>> +	local arg
>> +	for arg in "$@" ; do
>> +		if [[ -d ${arg} ]] ; then
>> +			mapfile -t -d '' -O "${#files[@]}" files < <(find "${arg}" -type f -iname '*.a' -print0)
>> +		else
>> +			files+=( "${arg}" )
>> +		fi
>> +	done
>
> Why not just pass all arguments to find(1)?  If you pass a file path to
> it, it will just return that file, i.e.:
>
> mapfile -t -d '' files < <(find -H "${@}" -type f -iname '*.a' -print0)
>
> ('-H' to follow symbolic links if passed directly in "$@")

Oh! Good point. Let me try that.

>
>> +
>> +	toolchain_type=
>> +	tc-is-gcc && toolchain_type=gnu
>> +	tc-is-clang && toolchain_type=llvm
>> +
>> +	local file
>> +	for file in "${files[@]}" ; do
>> +		case ${toolchain_type} in
>> +			gnu)
>> +				$(tc-getSTRIP) \
>> +					-R .gnu.lto_* \
>> +					-R .gnu.debuglto_* \
>> +					-N __gnu_lto_v1 \
>> +					"${file}" || die "Stripping bytecode in ${file} failed"
>> +				;;
>
> Technically, strip accepts multiple files, but I can guess there's
> no point in optimizing the GNU branch here.

I thought about this, and it's so tempting because the GNU case is the
common one, but it'd uglify the function, and the eclass already feels
(unavoidably) complex for something which is simple.

>
>> +			llvm)
>> +				llvm-bitcode-strip \
>> +					-r "${file}" \
>> +					-o "${file}" || die "Stripping bytecode in ${file} failed"
>> +				;;
>> +			*)
>> +				;;
>> +		esac
>> +	done
>> +}
>> +
>> +fi

thanks,
sam


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [gentoo-dev] [PATCH 1/5] dot-a.eclass: new eclass for handling LTO in static archives
  2025-05-03  6:22     ` Sam James
@ 2025-05-03  6:44       ` Sam James
  2025-05-03  7:18         ` Sam James
  0 siblings, 1 reply; 10+ messages in thread
From: Sam James @ 2025-05-03  6:44 UTC (permalink / raw
  To: Michał Górny; +Cc: gentoo-dev, Eli Schwartz

Sam James <sam@gentoo.org> writes:

> Michał Górny <mgorny@gentoo.org> writes:
> [...]
>>> +	local arg
>>> +	for arg in "$@" ; do
>>> +		if [[ -d ${arg} ]] ; then
>>> +			mapfile -t -d '' -O "${#files[@]}" files < <(find "${arg}" -type f -iname '*.a' -print0)
>>> +		else
>>> +			files+=( "${arg}" )
>>> +		fi
>>> +	done
>>
>> Why not just pass all arguments to find(1)?  If you pass a file path to
>> it, it will just return that file, i.e.:
>>
>> mapfile -t -d '' files < <(find -H "${@}" -type f -iname '*.a' -print0)
>>
>> ('-H' to follow symbolic links if passed directly in "$@")
>
> Oh! Good point. Let me try that.

This breaks some of the tests because we do some direct testing on .o
files. But I can't think of why a package would do that besides
toolchain packages where special care will be needed anyway, so I'll
adjust the tests.


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [gentoo-dev] [PATCH 1/5] dot-a.eclass: new eclass for handling LTO in static archives
  2025-05-03  6:44       ` Sam James
@ 2025-05-03  7:18         ` Sam James
  0 siblings, 0 replies; 10+ messages in thread
From: Sam James @ 2025-05-03  7:18 UTC (permalink / raw
  To: Michał Górny; +Cc: gentoo-dev, Eli Schwartz

Sam James <sam@gentoo.org> writes:

> Sam James <sam@gentoo.org> writes:
>
>> Michał Górny <mgorny@gentoo.org> writes:
>> [...]
>>>> +	local arg
>>>> +	for arg in "$@" ; do
>>>> +		if [[ -d ${arg} ]] ; then
>>>> +			mapfile -t -d '' -O "${#files[@]}" files < <(find "${arg}" -type f -iname '*.a' -print0)
>>>> +		else
>>>> +			files+=( "${arg}" )
>>>> +		fi
>>>> +	done
>>>
>>> Why not just pass all arguments to find(1)?  If you pass a file path to
>>> it, it will just return that file, i.e.:
>>>
>>> mapfile -t -d '' files < <(find -H "${@}" -type f -iname '*.a' -print0)
>>>
>>> ('-H' to follow symbolic links if passed directly in "$@")
>>
>> Oh! Good point. Let me try that.
>
> This breaks some of the tests because we do some direct testing on .o
> files. But I can't think of why a package would do that besides
> toolchain packages where special care will be needed anyway, so I'll
> adjust the tests.

It needs some more thought as .a requires special handling (bug #603594
and bug #866422). I'll think aobut it more.


^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2025-05-03  7:19 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-05-02 20:13 [gentoo-dev] [PATCH 0/5] Handle LTO with static libraries Sam James
2025-05-02 20:13 ` [gentoo-dev] [PATCH 1/5] dot-a.eclass: new eclass for handling LTO in static archives Sam James
2025-05-03  4:27   ` Michał Górny
2025-05-03  6:22     ` Sam James
2025-05-03  6:44       ` Sam James
2025-05-03  7:18         ` Sam James
2025-05-02 20:13 ` [gentoo-dev] [PATCH 2/5] sys-libs/zlib: use dot-a.eclass for LTO Sam James
2025-05-02 20:13 ` [gentoo-dev] [PATCH 3/5] dev-libs/glib: " Sam James
2025-05-02 20:13 ` [gentoo-dev] [PATCH 4/5] dev-util/sysprof-capture: " Sam James
2025-05-02 20:13 ` [gentoo-dev] [PATCH 5/5] sys-devel/flex: " Sam James

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox