Re: [gentoo-dev] [PATCH 1/5] dot-a.eclass: new eclass for handling LTO in static archives

[Sam James <sam@gentoo.org>]

Michał Górny <mgorny@gentoo.org> writes:

> Hi,
>
> Sorry for not looking earlier.
>

Thanks for reviewing (and no worries, this has been loooooong overdue).


> On Fri, 2025-05-02 at 21:13 +0100, Sam James wrote:
>> diff --git a/eclass/dot-a.eclass b/eclass/dot-a.eclass
>> new file mode 100644
>> index 0000000000000..20a0fa1dfc206
>> --- /dev/null
>> +++ b/eclass/dot-a.eclass
>> @@ -0,0 +1,124 @@
>> +# Copyright 2025 Gentoo Authors
>> +# Distributed under the terms of the GNU General Public License v2
>> +
>> +# @ECLASS: dot-a.eclass
>> +# @MAINTAINER:
>> +# Toolchain
>> +# Toolchain Ninjas <toolchain@gentoo.org>
>
> You've got "toolchain" twice.

ACK.

>
>> +# @AUTHOR:
>> +# Sam James <sam@gentoo.org>
>> +# Eli Schwartz <eschwartz@gentoo.org>
>> +# @SUPPORTED_EAPIS: 8
>> +# @BLURB: Functions to handle stripping LTO bytecode out of static archives.
>> +# @DESCRIPTION:
>> +# This eclass provides functions to strip LTO bytecode out of static archives
>> +# (.a files).
>> +#
>> +# Static libraries when built with LTO will contain LTO bytecode which is
>> +# not portable across compiler versions or compiler vendors. To avoid pessimising
>> +# the library and always filtering LTO, we can build it with -ffat-lto-objects
>> +# instead, which builds some components twice. The installed part will then
>> +# have the LTO contents stripped out, leaving the regular objects in the
>> +# static archive.
>> +#
>> +# Use should be passing calling lto-guarantee-fat before configure-time
>> +# and calling strip-lto-bytecode after installation.
>
> Do I understand correctly that this is to be always used when you're
> installing static libraries?  Or are there more specific criteria?
> Perhaps that'd be worth including in the doc.

Correct, with the exception of packages where LTO is currently filtered
(but people should add it to future-proof unless it's something like
glibc).

I'll document that.

>
>> +#
>> +# @EXAMPLE:
>> +# @CODE
>> +#
>> +# inherit dot-a
>> +#
>> +# src_configure() {
>> +#     lto-guarantee-fat
>> +#     econf
>> +# }
>> +#
>> +# src_install() {
>> +#     default
>> +#     strip-lto-bytecode
>> +# }
>> +case ${EAPI} in
>> +	8) ;;
>> +	*) die "${ECLASS}: EAPI ${EAPI:-0} not supported" ;;
>> +esac
>> +
>> +if [[ -z ${_DOT_A_ECLASS} ]] ; then
>> +_DOT_A_ECLASS=1
>> +
>> +inherit flag-o-matic toolchain-funcs
>> +
>> +# TODO: QA check
>> +
>> +# @FUNCTION: lto-guarantee-fat
>> +# @DESCRIPTION:
>> +# If LTO is enabled, appends -ffat-lto-objects or any other flags needed
>> +# to provide fat LTO objects.
>> +lto-guarantee-fat() {
>> +	tc-is-lto || return
>> +
>> +	# We add this for all languages as LTO obviously can't be done
>> +	# if different compilers are used for e.g. C vs C++ anyway.
>> +	append-flags $(test-flags-CC -ffat-lto-objects)
>> +}
>> +
>> +# @FUNCTION: strip-lto-bytecode
>> +# @USAGE: [library|directory] [...]
>> +# @DESCRIPTION:
>> +# Strips LTO bytecode from libraries (static archives) passed as arguments.
>> +# Defaults to operating on ${ED} as a whole if no arguments are passed.
>> +#
>> +# As an optimisation, if USE=static-libs exists for a package and is disabled,
>> +# the default-searching behaviour with no arguments is suppressed.
>> +strip-lto-bytecode() {
>> +	tc-is-lto || return
>> +
>> +	local files=()
>> +
>> +	if [[ ${#} -eq 0 ]]; then
>> +		if ! in_iuse static-libs || use static-libs ; then
>> +			# maybe we are USE=static-libs. Alternatively, maybe the ebuild doesn't
>> +			# offer such a choice. In both cases, the user specified the function,
>> +			# so we expect to be called on *something*, but nothing was explicitly
>> +			# passed. Try scanning ${ED} automatically.
>> +			set -- "${ED}"
>> +		fi
>> +	fi
>> +
>> +	# Check if any of our arguments are directories to be recursed
>> +	# into.
>> +	local arg
>> +	for arg in "$@" ; do
>> +		if [[ -d ${arg} ]] ; then
>> +			mapfile -t -d '' -O "${#files[@]}" files < <(find "${arg}" -type f -iname '*.a' -print0)
>> +		else
>> +			files+=( "${arg}" )
>> +		fi
>> +	done
>
> Why not just pass all arguments to find(1)?  If you pass a file path to
> it, it will just return that file, i.e.:
>
> mapfile -t -d '' files < <(find -H "${@}" -type f -iname '*.a' -print0)
>
> ('-H' to follow symbolic links if passed directly in "$@")

Oh! Good point. Let me try that.

>
>> +
>> +	toolchain_type=
>> +	tc-is-gcc && toolchain_type=gnu
>> +	tc-is-clang && toolchain_type=llvm
>> +
>> +	local file
>> +	for file in "${files[@]}" ; do
>> +		case ${toolchain_type} in
>> +			gnu)
>> +				$(tc-getSTRIP) \
>> +					-R .gnu.lto_* \
>> +					-R .gnu.debuglto_* \
>> +					-N __gnu_lto_v1 \
>> +					"${file}" || die "Stripping bytecode in ${file} failed"
>> +				;;
>
> Technically, strip accepts multiple files, but I can guess there's
> no point in optimizing the GNU branch here.

I thought about this, and it's so tempting because the GNU case is the
common one, but it'd uglify the function, and the eclass already feels
(unavoidably) complex for something which is simple.

>
>> +			llvm)
>> +				llvm-bitcode-strip \
>> +					-r "${file}" \
>> +					-o "${file}" || die "Stripping bytecode in ${file} failed"
>> +				;;
>> +			*)
>> +				;;
>> +		esac
>> +	done
>> +}
>> +
>> +fi

thanks,
sam