[gentoo-dev] [RFC] Initial proof-of-concept for explicit x86 multilib flags

[gentoo-dev] [RFC] Initial proof-of-concept for explicit x86 multilib flags

[Michał Górny]

Hello,

Following my earlier mail, I'm sending two patches which describe how
I see the potential of introducing explicit multilib flags.

The idea is that each arch has its own ABI_<arch> USE_EXPAND, specifying
the multilib ABIs for choice. For example, x86 has ABI_X86="32 64".

All of those USE_EXPANDs are hidden (using USE_EXPAND_HIDDEN)
in the base profile, and all of their flags are masked.

In the proper multilib profiles, e.g. the amd64 multilib profile,
the relevant USE_EXPAND is removed from USE_EXPAND_HIDDEN, the flags
are unmasked and the default ABI flag is use.forced.

The eclass exports *all* possible ABIs for all arches in IUSE (due
to the necessity of constant metadata). However, it checks only
the flags relevant to the arch (avoids wasting time) and when no flags
are set (e.g. non-multilib system) does a non-multilib build.

The use.force default ABI means that for a typical user the native build
is forced and therefore regular package dependencies are correct.
The user is allowed to disable it in his own profile but he does that
on his own responsibility.

For multilib or non-native builds, a proper USE dependencies need
be used. Multilib builds take [${MULTILIB_USEDEP}] for them; the Skype
example mentioned by aballier would use [abi_x86_32].

I think those are all the important ideas. The patches shall be
considered mostly proof-of-concept, and I'm awaiting further discussion
on the topic.

If anyone is interested in testing the multilib work of mine (which
doesn't use the new flags yet, just the profile-forced 'multilib' flag),
I have converted the live ebuilds corresponding to packages from
emul-linux-x86-xlibs. The packages can be found in the x11 overlay,
'multilib' branch.

  $ layman -a x11
  $ ( cd /var/lib/layman/x11; git checkout multilib )
  $ diffmask -a libX11 # yep, X11 live ebuilds are package.masked
  $ emerge -v emul-linux-x86-xlibs

For easier testing, there's media-libs/libtxc_dxtn in the gx86 tree.
But it's nothing really special to see, it doesn't even have
dependencies to prove the major points.

[gentoo-dev] [PATCH 1/2] Add multilib flags for x86.

[Michał Górny]

64- and 32-bit libs involved. No x32 yet since I have no idea about it.
---
 gx86/profiles/arch/amd64/make.defaults | 4 ++++
 gx86/profiles/arch/amd64/use.force     | 4 ++++
 gx86/profiles/arch/amd64/use.mask      | 5 +++++
 gx86/profiles/base/make.defaults       | 4 ++--
 gx86/profiles/base/use.mask            | 5 +++++
 gx86/profiles/desc/abi_x86.desc        | 9 +++++++++
 6 files changed, 29 insertions(+), 2 deletions(-)
 create mode 100644 gx86/profiles/desc/abi_x86.desc

diff --git a/gx86/profiles/arch/amd64/make.defaults b/gx86/profiles/arch/amd64/make.defaults
index bd020bb..27c480a 100644
--- a/gx86/profiles/arch/amd64/make.defaults
+++ b/gx86/profiles/arch/amd64/make.defaults
@@ -45,3 +45,7 @@ VIDEO_CARDS="fbdev glint intel mach64 mga nouveau nv r128 radeon savage sis tdfx
 # 2006/12/22 - Danny van Dyk <kugelfang@gentoo.org>
 # Default for ALSA_CARDS USE_EXPAND variable.
 ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci"
+
+# Michał Górny <mgorny@gentoo.org> (23 Jan 2013)
+# Make the ABI_X86 multilib USE_EXPAND visible for the profile.
+USE_EXPAND_HIDDEN="-ABI_X86"
diff --git a/gx86/profiles/arch/amd64/use.force b/gx86/profiles/arch/amd64/use.force
index b54bac8..51d7a75 100644
--- a/gx86/profiles/arch/amd64/use.force
+++ b/gx86/profiles/arch/amd64/use.force
@@ -1,2 +1,6 @@
 # Force the flag which corresponds to ARCH.
 amd64
+
+# Michał Górny <mgorny@gentoo.org> (23 Jan 2013)
+# Force building native libraries for the platform.
+abi_x86_64
diff --git a/gx86/profiles/arch/amd64/use.mask b/gx86/profiles/arch/amd64/use.mask
index 123bdfc..4fc14c3 100644
--- a/gx86/profiles/arch/amd64/use.mask
+++ b/gx86/profiles/arch/amd64/use.mask
@@ -177,4 +177,9 @@ capslib
 # fdk-aac is already keyworded here
 -fdk
 
+# Michał Górny <mgorny@gentoo.org> (23 Jan 2013)
+# Unmask multilib flags for the platform.
+-abi_x86_32
+-abi_x86_64
+
 # NOT NECESSARY - SECTION
diff --git a/gx86/profiles/base/make.defaults b/gx86/profiles/base/make.defaults
index 00761b6..07e19cf 100644
--- a/gx86/profiles/base/make.defaults
+++ b/gx86/profiles/base/make.defaults
@@ -16,11 +16,11 @@ USE_EXPAND_VALUES_USERLAND="BSD GNU"
 
 # Env vars to expand into USE vars.  Modifying this requires prior
 # discussion on gentoo-dev@gentoo.org.
-USE_EXPAND="APACHE2_MODULES APACHE2_MPMS CALLIGRA_FEATURES ENLIGHTENMENT_MODULES FOO2ZJS_DEVICES MISDN_CARDS FRITZCAPI_CARDS FCDSL_CARDS VIDEO_CARDS DVB_CARDS LIRC_DEVICES INPUT_DEVICES LINGUAS USERLAND KERNEL ELIBC CROSSCOMPILE_OPTS ALSA_CARDS ALSA_PCM_PLUGINS LCD_DEVICES CAMERAS NETBEANS_MODULES QEMU_SOFTMMU_TARGETS QEMU_USER_TARGETS SANE_BACKENDS RUBY_TARGETS PHP_TARGETS NGINX_MODULES_HTTP NGINX_MODULES_MAIL XFCE_PLUGINS XTABLES_ADDONS GPSD_PROTOCOLS COLLECTD_PLUGINS DRACUT_MODULES OFED_DRIVERS GRUB_PLATFORMS FFTOOLS PYTHON_TARGETS CURL_SSL OPENMPI_FABRICS OPENMPI_RM OPENMPI_OFED_FEATURES LIBREOFFICE_EXTENSIONS VOICEMAIL_STORAGE PYTHON_SINGLE_TARGET"
+USE_EXPAND="APACHE2_MODULES APACHE2_MPMS CALLIGRA_FEATURES ENLIGHTENMENT_MODULES FOO2ZJS_DEVICES MISDN_CARDS FRITZCAPI_CARDS FCDSL_CARDS VIDEO_CARDS DVB_CARDS LIRC_DEVICES INPUT_DEVICES LINGUAS USERLAND KERNEL ELIBC CROSSCOMPILE_OPTS ALSA_CARDS ALSA_PCM_PLUGINS LCD_DEVICES CAMERAS NETBEANS_MODULES QEMU_SOFTMMU_TARGETS QEMU_USER_TARGETS SANE_BACKENDS RUBY_TARGETS PHP_TARGETS NGINX_MODULES_HTTP NGINX_MODULES_MAIL XFCE_PLUGINS XTABLES_ADDONS GPSD_PROTOCOLS COLLECTD_PLUGINS DRACUT_MODULES OFED_DRIVERS GRUB_PLATFORMS FFTOOLS PYTHON_TARGETS CURL_SSL OPENMPI_FABRICS OPENMPI_RM OPENMPI_OFED_FEATURES LIBREOFFICE_EXTENSIONS VOICEMAIL_STORAGE PYTHON_SINGLE_TARGET ABI_X86"
 
 # USE_EXPAND variables whose contents are not shown in package manager
 # output. Changes need discussion on gentoo-dev.
-USE_EXPAND_HIDDEN="USERLAND KERNEL ELIBC CROSSCOMPILE_OPTS"
+USE_EXPAND_HIDDEN="USERLAND KERNEL ELIBC CROSSCOMPILE_OPTS ABI_X86"
 
 CONFIG_PROTECT="/etc"
 CONFIG_PROTECT_MASK="/etc/env.d /etc/gconf"
diff --git a/gx86/profiles/base/use.mask b/gx86/profiles/base/use.mask
index 811fa3b..3dc0c36 100644
--- a/gx86/profiles/base/use.mask
+++ b/gx86/profiles/base/use.mask
@@ -323,3 +323,8 @@ python_targets_pypy2_0
 python_single_target_pypy1_8
 python_single_target_pypy1_9
 python_single_target_pypy2_0
+
+# Michał Górny <mgorny@gentoo.org> (23 Jan 2013)
+# Mask all of the multilib flags for non-multilib profiles.
+abi_x86_32
+abi_x86_64
diff --git a/gx86/profiles/desc/abi_x86.desc b/gx86/profiles/desc/abi_x86.desc
new file mode 100644
index 0000000..5a11f2a
--- /dev/null
+++ b/gx86/profiles/desc/abi_x86.desc
@@ -0,0 +1,9 @@
+# Copyright 2013 Gentoo Foundation.
+# Distributed under the terms of the GNU General Public License v2
+# $Header: $
+
+# This file contains descriptions of ABI_X86 USE_EXPAND flags.
+
+# Keep it sorted.
+64 - 64-bit (amd64) libraries
+32 - 32-bit (x86) libraries
-- 
1.8.1.1

[gentoo-dev] [PATCH 2/2] Use new multilib flags in autotools-multilib.

[Michał Górny]

This is mostly a proof-of-concept. If approved, I will work on moving
the code into a separate eclass, possibly named 'multilib-build' ;).
---
 gx86/eclass/autotools-multilib.eclass | 24 +++++++++++++++++++++---
 1 file changed, 21 insertions(+), 3 deletions(-)

diff --git a/gx86/eclass/autotools-multilib.eclass b/gx86/eclass/autotools-multilib.eclass
index 7c8697a..eef7bcc 100644
--- a/gx86/eclass/autotools-multilib.eclass
+++ b/gx86/eclass/autotools-multilib.eclass
@@ -32,7 +32,23 @@ inherit autotools-utils multilib
 
 EXPORT_FUNCTIONS src_configure src_compile src_test src_install
 
-IUSE=multilib
+# Declare all of them, profiles will control their visibility.
+IUSE='abi_x86_32 abi_x86_64'
+
+# @FUNCTION: _autotools-multilib_get_enabled_abis
+# @DESCRIPTION:
+# Get the list of enabled ABIs. The returned names are suitable for use
+# with multilib.eclass.
+#
+# If multilib is not enabled or not supported, returns an empty list.
+_autotools-multilib_get_enabled_abis() {
+	debug-print-function ${FUNCNAME} "${@}"
+
+	if use amd64; then
+		use abi_x86_64 && echo amd64
+		use abi_x86_32 && echo x86
+	fi
+}
 
 # @FUNCTION: autotools-multilib_foreach_abi
 # @USAGE: argv...
@@ -46,9 +62,11 @@ IUSE=multilib
 autotools-multilib_foreach_abi() {
 	local initial_dir=${BUILD_DIR:-${S}}
 
-	if use multilib; then
+	local multilib_abis=$(_autotools-multilib_get_enabled_abis)
+
+	if [[ ${multilib_abis} ]]; then
 		local ABI
-		for ABI in $(get_all_abis); do
+		for ABI in ${multilib_abis}; do
 			multilib_toolchain_setup "${ABI}"
 			BUILD_DIR=${initial_dir%%/}-${ABI} "${@}"
 		done
-- 
1.8.1.1

Re: [gentoo-dev] [PATCH 2/2] Use new multilib flags in autotools-multilib.

[Alexis Ballier]

On Thu, 24 Jan 2013 00:23:57 +0100
Michał Górny <mgorny@gentoo.org> wrote:

> This is mostly a proof-of-concept. If approved, I will work on moving
> the code into a separate eclass, possibly named 'multilib-build' ;).
> ---
>  gx86/eclass/autotools-multilib.eclass | 24 +++++++++++++++++++++---
>  1 file changed, 21 insertions(+), 3 deletions(-)
> 
> diff --git a/gx86/eclass/autotools-multilib.eclass
> b/gx86/eclass/autotools-multilib.eclass index 7c8697a..eef7bcc 100644
> --- a/gx86/eclass/autotools-multilib.eclass
> +++ b/gx86/eclass/autotools-multilib.eclass
> @@ -32,7 +32,23 @@ inherit autotools-utils multilib
>  
>  EXPORT_FUNCTIONS src_configure src_compile src_test src_install
>  
> -IUSE=multilib
> +# Declare all of them, profiles will control their visibility.
> +IUSE='abi_x86_32 abi_x86_64'
> +
> +# @FUNCTION: _autotools-multilib_get_enabled_abis
> +# @DESCRIPTION:
> +# Get the list of enabled ABIs. The returned names are suitable for
> use +# with multilib.eclass.
> +#
> +# If multilib is not enabled or not supported, returns an empty list.
> +
> +	debug-print-function ${FUNCNAME} "${@}"
> +
> +	if use amd64; then
> +		use abi_x86_64 && echo amd64
> +		use abi_x86_32 && echo x86
> +	fi
> +}

I would rather iterate over a variable than hardcoding and duplicating
it here:

MULTILIB_ABIS='abi_x86_32:x86 abi_x86_64:amd64'
IUSE=""
for i in $MULTILIB_ABIS ; do
   IUSE+=" ${i%:*}"
done

_autotools-multilib_get_enabled_abis() {
   for i in $MULTILIB_ABIS ; do
      use ${i%:*} && echo ${i#*:}
   done
}

(maybe protect it with has_multilib_profile if you wish)    

[...]

Alexis.

Re: [gentoo-dev] [PATCH 2/2] Use new multilib flags in autotools-multilib.

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2061 bytes --]

On Wed, 23 Jan 2013 21:40:13 -0300
Alexis Ballier <aballier@gentoo.org> wrote:

> On Thu, 24 Jan 2013 00:23:57 +0100
> Michał Górny <mgorny@gentoo.org> wrote:
> 
> > This is mostly a proof-of-concept. If approved, I will work on moving
> > the code into a separate eclass, possibly named 'multilib-build' ;).
> > ---
> >  gx86/eclass/autotools-multilib.eclass | 24 +++++++++++++++++++++---
> >  1 file changed, 21 insertions(+), 3 deletions(-)
> > 
> > diff --git a/gx86/eclass/autotools-multilib.eclass
> > b/gx86/eclass/autotools-multilib.eclass index 7c8697a..eef7bcc 100644
> > --- a/gx86/eclass/autotools-multilib.eclass
> > +++ b/gx86/eclass/autotools-multilib.eclass
> > @@ -32,7 +32,23 @@ inherit autotools-utils multilib
> >  
> >  EXPORT_FUNCTIONS src_configure src_compile src_test src_install
> >  
> > -IUSE=multilib
> > +# Declare all of them, profiles will control their visibility.
> > +IUSE='abi_x86_32 abi_x86_64'
> > +
> > +# @FUNCTION: _autotools-multilib_get_enabled_abis
> > +# @DESCRIPTION:
> > +# Get the list of enabled ABIs. The returned names are suitable for
> > use +# with multilib.eclass.
> > +#
> > +# If multilib is not enabled or not supported, returns an empty list.
> > +
> > +	debug-print-function ${FUNCNAME} "${@}"
> > +
> > +	if use amd64; then
> > +		use abi_x86_64 && echo amd64
> > +		use abi_x86_32 && echo x86
> > +	fi
> > +}
> 
> I would rather iterate over a variable than hardcoding and duplicating
> it here:
> 
> MULTILIB_ABIS='abi_x86_32:x86 abi_x86_64:amd64'
> IUSE=""
> for i in $MULTILIB_ABIS ; do
>    IUSE+=" ${i%:*}"
> done
> 
> _autotools-multilib_get_enabled_abis() {
>    for i in $MULTILIB_ABIS ; do
>       use ${i%:*} && echo ${i#*:}
>    done
> }

What are the advantages? I feel like the explicit solution is much more
readable and obvious at the first glance.

> (maybe protect it with has_multilib_profile if you wish)    

Well, the current code assumes that no flags == non-multilib profile.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] [PATCH 2/2] Use new multilib flags in autotools-multilib.

[Alexis Ballier]

On Sat, 26 Jan 2013 13:11:41 +0100
Michał Górny <mgorny@gentoo.org> wrote:

> On Wed, 23 Jan 2013 21:40:13 -0300
> Alexis Ballier <aballier@gentoo.org> wrote:
> 
> > On Thu, 24 Jan 2013 00:23:57 +0100
> > Michał Górny <mgorny@gentoo.org> wrote:
> > 
> > > This is mostly a proof-of-concept. If approved, I will work on
> > > moving the code into a separate eclass, possibly named
> > > 'multilib-build' ;). ---
> > >  gx86/eclass/autotools-multilib.eclass | 24
> > > +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3
> > > deletions(-)
> > > 
> > > diff --git a/gx86/eclass/autotools-multilib.eclass
> > > b/gx86/eclass/autotools-multilib.eclass index 7c8697a..eef7bcc
> > > 100644 --- a/gx86/eclass/autotools-multilib.eclass
> > > +++ b/gx86/eclass/autotools-multilib.eclass
> > > @@ -32,7 +32,23 @@ inherit autotools-utils multilib
> > >  
> > >  EXPORT_FUNCTIONS src_configure src_compile src_test src_install
> > >  
> > > -IUSE=multilib
> > > +# Declare all of them, profiles will control their visibility.
> > > +IUSE='abi_x86_32 abi_x86_64'
> > > +
> > > +# @FUNCTION: _autotools-multilib_get_enabled_abis
> > > +# @DESCRIPTION:
> > > +# Get the list of enabled ABIs. The returned names are suitable
> > > for use +# with multilib.eclass.
> > > +#
> > > +# If multilib is not enabled or not supported, returns an empty
> > > list. +
> > > +	debug-print-function ${FUNCNAME} "${@}"
> > > +
> > > +	if use amd64; then
> > > +		use abi_x86_64 && echo amd64
> > > +		use abi_x86_32 && echo x86
> > > +	fi
> > > +}
> > 
> > I would rather iterate over a variable than hardcoding and
> > duplicating it here:
> > 
> > MULTILIB_ABIS='abi_x86_32:x86 abi_x86_64:amd64'
> > IUSE=""
> > for i in $MULTILIB_ABIS ; do
> >    IUSE+=" ${i%:*}"
> > done
> > 
> > _autotools-multilib_get_enabled_abis() {
> >    for i in $MULTILIB_ABIS ; do
> >       use ${i%:*} && echo ${i#*:}
> >    done
> > }
> 
> What are the advantages? I feel like the explicit solution is much
> more readable and obvious at the first glance.

yes it is more readable but IMHO it's better to avoid to have to touch
multiple places when adding a new ABI: you only have to document
that adding a new ABI consists simply in adding it to this list (and
document the useflag) instead of 'add the useflag, add support for it to
function foo and bar, etc.'
your call in the end, but I fear not trying to separate code from data
could make the eclass harder to maintain.

also, it'll make code much shorter when all the exotic ABIs will be
added :)


> 
> > (maybe protect it with has_multilib_profile if you wish)    
> 
> Well, the current code assumes that no flags == non-multilib profile.

in the code you posted you do not seem to take into account amd64
non-multilib then :)

also, IMHO you shouldn't use arch to guess what useflag to check or
not: they'll all be in IUSE in the end and the profiles should be the
one deciding what to mask or not (which you'll have to do anyway), not
the eclass.

Alexis.

Re: [gentoo-dev] [PATCH 2/2] Use new multilib flags in autotools-multilib.

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 3648 bytes --]

On Sat, 26 Jan 2013 11:51:22 -0300
Alexis Ballier <aballier@gentoo.org> wrote:

> On Sat, 26 Jan 2013 13:11:41 +0100
> Michał Górny <mgorny@gentoo.org> wrote:
> 
> > On Wed, 23 Jan 2013 21:40:13 -0300
> > Alexis Ballier <aballier@gentoo.org> wrote:
> > 
> > > On Thu, 24 Jan 2013 00:23:57 +0100
> > > Michał Górny <mgorny@gentoo.org> wrote:
> > > 
> > > > This is mostly a proof-of-concept. If approved, I will work on
> > > > moving the code into a separate eclass, possibly named
> > > > 'multilib-build' ;). ---
> > > >  gx86/eclass/autotools-multilib.eclass | 24
> > > > +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3
> > > > deletions(-)
> > > > 
> > > > diff --git a/gx86/eclass/autotools-multilib.eclass
> > > > b/gx86/eclass/autotools-multilib.eclass index 7c8697a..eef7bcc
> > > > 100644 --- a/gx86/eclass/autotools-multilib.eclass
> > > > +++ b/gx86/eclass/autotools-multilib.eclass
> > > > @@ -32,7 +32,23 @@ inherit autotools-utils multilib
> > > >  
> > > >  EXPORT_FUNCTIONS src_configure src_compile src_test src_install
> > > >  
> > > > -IUSE=multilib
> > > > +# Declare all of them, profiles will control their visibility.
> > > > +IUSE='abi_x86_32 abi_x86_64'
> > > > +
> > > > +# @FUNCTION: _autotools-multilib_get_enabled_abis
> > > > +# @DESCRIPTION:
> > > > +# Get the list of enabled ABIs. The returned names are suitable
> > > > for use +# with multilib.eclass.
> > > > +#
> > > > +# If multilib is not enabled or not supported, returns an empty
> > > > list. +
> > > > +	debug-print-function ${FUNCNAME} "${@}"
> > > > +
> > > > +	if use amd64; then
> > > > +		use abi_x86_64 && echo amd64
> > > > +		use abi_x86_32 && echo x86
> > > > +	fi
> > > > +}
> > > 
> > > I would rather iterate over a variable than hardcoding and
> > > duplicating it here:
> > > 
> > > MULTILIB_ABIS='abi_x86_32:x86 abi_x86_64:amd64'
> > > IUSE=""
> > > for i in $MULTILIB_ABIS ; do
> > >    IUSE+=" ${i%:*}"
> > > done
> > > 
> > > _autotools-multilib_get_enabled_abis() {
> > >    for i in $MULTILIB_ABIS ; do
> > >       use ${i%:*} && echo ${i#*:}
> > >    done
> > > }
> > 
> > What are the advantages? I feel like the explicit solution is much
> > more readable and obvious at the first glance.
> 
> yes it is more readable but IMHO it's better to avoid to have to touch
> multiple places when adding a new ABI: you only have to document
> that adding a new ABI consists simply in adding it to this list (and
> document the useflag) instead of 'add the useflag, add support for it to
> function foo and bar, etc.'
> your call in the end, but I fear not trying to separate code from data
> could make the eclass harder to maintain.

Ok, separating data from code seems a valid concern. However, I feel
like this is not really clear concept in bash, and the intent is that
the ABI list will be used only in that function.

> > > (maybe protect it with has_multilib_profile if you wish)    
> > 
> > Well, the current code assumes that no flags == non-multilib profile.
> 
> in the code you posted you do not seem to take into account amd64
> non-multilib then :)

Well, I didn't touch the non-multilib profile indeed.

> also, IMHO you shouldn't use arch to guess what useflag to check or
> not: they'll all be in IUSE in the end and the profiles should be the
> one deciding what to mask or not (which you'll have to do anyway), not
> the eclass.

Well, I think I've stated that already. That 'use foo &&' is just
an optimization hack -- to not check all the irrelevant flags all
the time.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] [PATCH 2/2] Use new multilib flags in autotools-multilib.

[Alexis Ballier]

On Sat, 26 Jan 2013 16:06:35 +0100
Michał Górny <mgorny@gentoo.org> wrote:

> On Sat, 26 Jan 2013 11:51:22 -0300
> Alexis Ballier <aballier@gentoo.org> wrote:
> 
> > On Sat, 26 Jan 2013 13:11:41 +0100
> > Michał Górny <mgorny@gentoo.org> wrote:
> > 
> > > On Wed, 23 Jan 2013 21:40:13 -0300
> > > Alexis Ballier <aballier@gentoo.org> wrote:
> > > 
> > > > On Thu, 24 Jan 2013 00:23:57 +0100
> > > > Michał Górny <mgorny@gentoo.org> wrote:
> > > > 
> > > > > This is mostly a proof-of-concept. If approved, I will work on
> > > > > moving the code into a separate eclass, possibly named
> > > > > 'multilib-build' ;). ---
> > > > >  gx86/eclass/autotools-multilib.eclass | 24
> > > > > +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3
> > > > > deletions(-)
> > > > > 
> > > > > diff --git a/gx86/eclass/autotools-multilib.eclass
> > > > > b/gx86/eclass/autotools-multilib.eclass index 7c8697a..eef7bcc
> > > > > 100644 --- a/gx86/eclass/autotools-multilib.eclass
> > > > > +++ b/gx86/eclass/autotools-multilib.eclass
> > > > > @@ -32,7 +32,23 @@ inherit autotools-utils multilib
> > > > >  
> > > > >  EXPORT_FUNCTIONS src_configure src_compile src_test
> > > > > src_install 
> > > > > -IUSE=multilib
> > > > > +# Declare all of them, profiles will control their
> > > > > visibility. +IUSE='abi_x86_32 abi_x86_64'
> > > > > +
> > > > > +# @FUNCTION: _autotools-multilib_get_enabled_abis
> > > > > +# @DESCRIPTION:
> > > > > +# Get the list of enabled ABIs. The returned names are
> > > > > suitable for use +# with multilib.eclass.
> > > > > +#
> > > > > +# If multilib is not enabled or not supported, returns an
> > > > > empty list. +
> > > > > +	debug-print-function ${FUNCNAME} "${@}"
> > > > > +
> > > > > +	if use amd64; then
> > > > > +		use abi_x86_64 && echo amd64
> > > > > +		use abi_x86_32 && echo x86
> > > > > +	fi
> > > > > +}
> > > > 
> > > > I would rather iterate over a variable than hardcoding and
> > > > duplicating it here:
> > > > 
> > > > MULTILIB_ABIS='abi_x86_32:x86 abi_x86_64:amd64'
> > > > IUSE=""
> > > > for i in $MULTILIB_ABIS ; do
> > > >    IUSE+=" ${i%:*}"
> > > > done
> > > > 
> > > > _autotools-multilib_get_enabled_abis() {
> > > >    for i in $MULTILIB_ABIS ; do
> > > >       use ${i%:*} && echo ${i#*:}
> > > >    done
> > > > }
> > > 
> > > What are the advantages? I feel like the explicit solution is much
> > > more readable and obvious at the first glance.
> > 
> > yes it is more readable but IMHO it's better to avoid to have to
> > touch multiple places when adding a new ABI: you only have to
> > document that adding a new ABI consists simply in adding it to this
> > list (and document the useflag) instead of 'add the useflag, add
> > support for it to function foo and bar, etc.'
> > your call in the end, but I fear not trying to separate code from
> > data could make the eclass harder to maintain.
> 
> Ok, separating data from code seems a valid concern. However, I feel
> like this is not really clear concept in bash, and the intent is that
> the ABI list will be used only in that function.

makes sense; the foo:bar trick is what I've came up in trying to emulate
python dictionaries :)


> > also, IMHO you shouldn't use arch to guess what useflag to check or
> > not: they'll all be in IUSE in the end and the profiles should be
> > the one deciding what to mask or not (which you'll have to do
> > anyway), not the eclass.
> 
> Well, I think I've stated that already. That 'use foo &&' is just
> an optimization hack -- to not check all the irrelevant flags all
> the time.

I'm not sure 'optimizations' are relevant when we're going to build
twice the package here :P
Also, # of arches is about the same order of magnitude than # of abis,
thus checking # of arches time use $arch or # of abis times use $abi is,
in the end, comparable while checking arch adds the constraint that
the eclass and the profiles have to be in sync.

Alexis.

Re: [gentoo-dev] [PATCH 2/2] Use new multilib flags in autotools-multilib.

[Alexis Ballier]

On Sat, 26 Jan 2013 13:11:41 +0100
Michał Górny <mgorny@gentoo.org> wrote:

> > (maybe protect it with has_multilib_profile if you wish)    
> 
> Well, the current code assumes that no flags == non-multilib profile.

.. and I hit send to quickly:
coming back to the skype example, with this assumption, on x86
libitneeds will have no abi useflag enabled while on amd64 it'll have
abi_x86_32, so you'll end with deps like: x86? ( libitneeds ) !x86?
( libitneeds[abi_x86_32] )

Re: [gentoo-dev] [PATCH 2/2] Use new multilib flags in autotools-multilib.

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 813 bytes --]

On Sat, 26 Jan 2013 11:54:44 -0300
Alexis Ballier <aballier@gentoo.org> wrote:

> On Sat, 26 Jan 2013 13:11:41 +0100
> Michał Górny <mgorny@gentoo.org> wrote:
> 
> > > (maybe protect it with has_multilib_profile if you wish)    
> > 
> > Well, the current code assumes that no flags == non-multilib profile.
> 
> .. and I hit send to quickly:
> coming back to the skype example, with this assumption, on x86
> libitneeds will have no abi useflag enabled while on amd64 it'll have
> abi_x86_32, so you'll end with deps like: x86? ( libitneeds ) !x86?
> ( libitneeds[abi_x86_32] )

Hmm, maybe we should make 'plain x86' use.force the abi_x86_32 flag
(though keeping it hidden). Then the following dep will work for both
arches:

  libitneeds[abi_x86_32]

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] [PATCH 2/2] Use new multilib flags in autotools-multilib.

[Alexis Ballier]

On Sat, 26 Jan 2013 16:08:45 +0100
Michał Górny <mgorny@gentoo.org> wrote:

> On Sat, 26 Jan 2013 11:54:44 -0300
> Alexis Ballier <aballier@gentoo.org> wrote:
> 
> > On Sat, 26 Jan 2013 13:11:41 +0100
> > Michał Górny <mgorny@gentoo.org> wrote:
> > 
> > > > (maybe protect it with has_multilib_profile if you wish)    
> > > 
> > > Well, the current code assumes that no flags == non-multilib
> > > profile.
> > 
> > .. and I hit send to quickly:
> > coming back to the skype example, with this assumption, on x86
> > libitneeds will have no abi useflag enabled while on amd64 it'll
> > have abi_x86_32, so you'll end with deps like: x86? ( libitneeds
> > ) !x86? ( libitneeds[abi_x86_32] )
> 
> Hmm, maybe we should make 'plain x86' use.force the abi_x86_32 flag
> (though keeping it hidden). Then the following dep will work for both
> arches:
> 
>   libitneeds[abi_x86_32]

yep, that's the best thing to do IMHO, but then if you want no multilib
-> returns empty list then you'll have to check it with
has_multilib_profile :)

Alexis.

Re: [gentoo-dev] [PATCH 2/2] Use new multilib flags in autotools-multilib.

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1471 bytes --]

On Sat, 26 Jan 2013 12:30:16 -0300
Alexis Ballier <aballier@gentoo.org> wrote:

> On Sat, 26 Jan 2013 16:08:45 +0100
> Michał Górny <mgorny@gentoo.org> wrote:
> 
> > On Sat, 26 Jan 2013 11:54:44 -0300
> > Alexis Ballier <aballier@gentoo.org> wrote:
> > 
> > > On Sat, 26 Jan 2013 13:11:41 +0100
> > > Michał Górny <mgorny@gentoo.org> wrote:
> > > 
> > > > > (maybe protect it with has_multilib_profile if you wish)    
> > > > 
> > > > Well, the current code assumes that no flags == non-multilib
> > > > profile.
> > > 
> > > .. and I hit send to quickly:
> > > coming back to the skype example, with this assumption, on x86
> > > libitneeds will have no abi useflag enabled while on amd64 it'll
> > > have abi_x86_32, so you'll end with deps like: x86? ( libitneeds
> > > ) !x86? ( libitneeds[abi_x86_32] )
> > 
> > Hmm, maybe we should make 'plain x86' use.force the abi_x86_32 flag
> > (though keeping it hidden). Then the following dep will work for both
> > arches:
> > 
> >   libitneeds[abi_x86_32]
> 
> yep, that's the best thing to do IMHO, but then if you want no multilib
> -> returns empty list then you'll have to check it with
> has_multilib_profile :)

Is this really necessary? AFAIU from the endclass standpoint there's
no real difference between non-multilib build and build for one ABI.
I could even make the multilib code paths used only when at least 2 ABIs
are enabled.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] [PATCH 2/2] Use new multilib flags in autotools-multilib.

[Alexis Ballier]

On Sat, 26 Jan 2013 18:06:32 +0100
Michał Górny <mgorny@gentoo.org> wrote:

> On Sat, 26 Jan 2013 12:30:16 -0300
> Alexis Ballier <aballier@gentoo.org> wrote:
> 
> > On Sat, 26 Jan 2013 16:08:45 +0100
> > Michał Górny <mgorny@gentoo.org> wrote:
> > 
> > > On Sat, 26 Jan 2013 11:54:44 -0300
> > > Alexis Ballier <aballier@gentoo.org> wrote:
> > > 
> > > > On Sat, 26 Jan 2013 13:11:41 +0100
> > > > Michał Górny <mgorny@gentoo.org> wrote:
> > > > 
> > > > > > (maybe protect it with has_multilib_profile if you wish)    
> > > > > 
> > > > > Well, the current code assumes that no flags == non-multilib
> > > > > profile.
> > > > 
> > > > .. and I hit send to quickly:
> > > > coming back to the skype example, with this assumption, on x86
> > > > libitneeds will have no abi useflag enabled while on amd64 it'll
> > > > have abi_x86_32, so you'll end with deps like: x86? ( libitneeds
> > > > ) !x86? ( libitneeds[abi_x86_32] )
> > > 
> > > Hmm, maybe we should make 'plain x86' use.force the abi_x86_32
> > > flag (though keeping it hidden). Then the following dep will work
> > > for both arches:
> > > 
> > >   libitneeds[abi_x86_32]
> > 
> > yep, that's the best thing to do IMHO, but then if you want no
> > multilib -> returns empty list then you'll have to check it with
> > has_multilib_profile :)
> 
> Is this really necessary? AFAIU from the endclass standpoint there's
> no real difference between non-multilib build and build for one ABI.
> I could even make the multilib code paths used only when at least 2
> ABIs are enabled.

I don't think it's necessary, but that's what your function had as a
spec :)
You could probably just use one code path, since, like you said,
non-multilib is multilib with only one abi :)

(not sure if it's safe to do non-multilib when there's only one abi, it
may not be the default abi and need multilib_toolchain_setup)

by the way, you'd probably want _autotools-multilib_get_enabled_abis to
return the default ABI last with the same trick as get_all_abis so that
in make install the default ABI gets 'preference' as it overwrites the
others upon collision (it should, in theory, not matter but think
about installed headers that get processed by autotools)

Re: [gentoo-dev] [PATCH 2/2] Use new multilib flags in autotools-multilib.

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2552 bytes --]

On Sat, 26 Jan 2013 14:43:38 -0300
Alexis Ballier <aballier@gentoo.org> wrote:

> On Sat, 26 Jan 2013 18:06:32 +0100
> Michał Górny <mgorny@gentoo.org> wrote:
> 
> > On Sat, 26 Jan 2013 12:30:16 -0300
> > Alexis Ballier <aballier@gentoo.org> wrote:
> > 
> > > On Sat, 26 Jan 2013 16:08:45 +0100
> > > Michał Górny <mgorny@gentoo.org> wrote:
> > > 
> > > > On Sat, 26 Jan 2013 11:54:44 -0300
> > > > Alexis Ballier <aballier@gentoo.org> wrote:
> > > > 
> > > > > On Sat, 26 Jan 2013 13:11:41 +0100
> > > > > Michał Górny <mgorny@gentoo.org> wrote:
> > > > > 
> > > > > > > (maybe protect it with has_multilib_profile if you wish)    
> > > > > > 
> > > > > > Well, the current code assumes that no flags == non-multilib
> > > > > > profile.
> > > > > 
> > > > > .. and I hit send to quickly:
> > > > > coming back to the skype example, with this assumption, on x86
> > > > > libitneeds will have no abi useflag enabled while on amd64 it'll
> > > > > have abi_x86_32, so you'll end with deps like: x86? ( libitneeds
> > > > > ) !x86? ( libitneeds[abi_x86_32] )
> > > > 
> > > > Hmm, maybe we should make 'plain x86' use.force the abi_x86_32
> > > > flag (though keeping it hidden). Then the following dep will work
> > > > for both arches:
> > > > 
> > > >   libitneeds[abi_x86_32]
> > > 
> > > yep, that's the best thing to do IMHO, but then if you want no
> > > multilib -> returns empty list then you'll have to check it with
> > > has_multilib_profile :)
> > 
> > Is this really necessary? AFAIU from the endclass standpoint there's
> > no real difference between non-multilib build and build for one ABI.
> > I could even make the multilib code paths used only when at least 2
> > ABIs are enabled.
> 
> I don't think it's necessary, but that's what your function had as a
> spec :)
> You could probably just use one code path, since, like you said,
> non-multilib is multilib with only one abi :)
> 
> (not sure if it's safe to do non-multilib when there's only one abi, it
> may not be the default abi and need multilib_toolchain_setup)

Correct.

> by the way, you'd probably want _autotools-multilib_get_enabled_abis to
> return the default ABI last with the same trick as get_all_abis so that
> in make install the default ABI gets 'preference' as it overwrites the
> others upon collision (it should, in theory, not matter but think
> about installed headers that get processed by autotools)

Good catch. I'll attach new patches soon ;).

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

[gentoo-dev] Re: [PATCH 2/2] Use new multilib flags in autotools-multilib.

[Mike Doty]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 01/23/13 15:23, Michał Górny wrote:
> This is mostly a proof-of-concept. If approved, I will work on moving
> the code into a separate eclass, possibly named 'multilib-build' ;).
> ---
>  gx86/eclass/autotools-multilib.eclass | 24 +++++++++++++++++++++---
>  1 file changed, 21 insertions(+), 3 deletions(-)
> 
> diff --git a/gx86/eclass/autotools-multilib.eclass b/gx86/eclass/autotools-multilib.eclass
> index 7c8697a..eef7bcc 100644
> --- a/gx86/eclass/autotools-multilib.eclass
> +++ b/gx86/eclass/autotools-multilib.eclass
> @@ -32,7 +32,23 @@ inherit autotools-utils multilib
>  
>  EXPORT_FUNCTIONS src_configure src_compile src_test src_install
>  
> -IUSE=multilib
> +# Declare all of them, profiles will control their visibility.
> +IUSE='abi_x86_32 abi_x86_64'
> +
> +# @FUNCTION: _autotools-multilib_get_enabled_abis
> +# @DESCRIPTION:
> +# Get the list of enabled ABIs. The returned names are suitable for use
> +# with multilib.eclass.
> +#
> +# If multilib is not enabled or not supported, returns an empty list.
> +_autotools-multilib_get_enabled_abis() {
> +	debug-print-function ${FUNCNAME} "${@}"
> +
> +	if use amd64; then
> +		use abi_x86_64 && echo amd64
> +		use abi_x86_32 && echo x86
> +	fi
> +}
>  
>  # @FUNCTION: autotools-multilib_foreach_abi
>  # @USAGE: argv...
> @@ -46,9 +62,11 @@ IUSE=multilib
>  autotools-multilib_foreach_abi() {
>  	local initial_dir=${BUILD_DIR:-${S}}
>  
> -	if use multilib; then
> +	local multilib_abis=$(_autotools-multilib_get_enabled_abis)
> +
> +	if [[ ${multilib_abis} ]]; then
>  		local ABI
> -		for ABI in $(get_all_abis); do
> +		for ABI in ${multilib_abis}; do
>  			multilib_toolchain_setup "${ABI}"
>  			BUILD_DIR=${initial_dir%%/}-${ABI} "${@}"
>  		done
> 
Why CC devrel on this?


- -- 
=======================================================
Mike Doty                      kingtaco -at- gentoo.org
Gentoo Infrastructure Operations Manager
Gentoo/AMD64 Strategic Lead
GPG: 0094 7F06 913E 78D6 F1BB  06BA D0AD D125 A797 C7A7
=======================================================
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iEYEARECAAYFAlEBbpwACgkQ0K3RJaeXx6cwsgCg1tVjaMe3iEJEdAaUpRQhJy6v
q0kAnjjil5zCt19Ep08O5hLVQ2BvRwTc
=HgIF
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Re: [PATCH 2/2] Use new multilib flags in autotools-multilib.

[Michał Górny]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Thu, 24 Jan 2013 09:25:48 -0800
Mike Doty <kingtaco@gentoo.org> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On 01/23/13 15:23, Michał Górny wrote:
> > This is mostly a proof-of-concept. If approved, I will work on moving
> > the code into a separate eclass, possibly named 'multilib-build' ;).
> > ---
> >  gx86/eclass/autotools-multilib.eclass | 24 +++++++++++++++++++++---
> >  1 file changed, 21 insertions(+), 3 deletions(-)
> > 
> > diff --git a/gx86/eclass/autotools-multilib.eclass b/gx86/eclass/autotools-multilib.eclass
> > index 7c8697a..eef7bcc 100644
> > --- a/gx86/eclass/autotools-multilib.eclass
> > +++ b/gx86/eclass/autotools-multilib.eclass
> > @@ -32,7 +32,23 @@ inherit autotools-utils multilib
> >  
> >  EXPORT_FUNCTIONS src_configure src_compile src_test src_install
> >  
> > -IUSE=multilib
> > +# Declare all of them, profiles will control their visibility.
> > +IUSE='abi_x86_32 abi_x86_64'
> > +
> > +# @FUNCTION: _autotools-multilib_get_enabled_abis
> > +# @DESCRIPTION:
> > +# Get the list of enabled ABIs. The returned names are suitable for use
> > +# with multilib.eclass.
> > +#
> > +# If multilib is not enabled or not supported, returns an empty list.
> > +_autotools-multilib_get_enabled_abis() {
> > +	debug-print-function ${FUNCNAME} "${@}"
> > +
> > +	if use amd64; then
> > +		use abi_x86_64 && echo amd64
> > +		use abi_x86_32 && echo x86
> > +	fi
> > +}
> >  
> >  # @FUNCTION: autotools-multilib_foreach_abi
> >  # @USAGE: argv...
> > @@ -46,9 +62,11 @@ IUSE=multilib
> >  autotools-multilib_foreach_abi() {
> >  	local initial_dir=${BUILD_DIR:-${S}}
> >  
> > -	if use multilib; then
> > +	local multilib_abis=$(_autotools-multilib_get_enabled_abis)
> > +
> > +	if [[ ${multilib_abis} ]]; then
> >  		local ABI
> > -		for ABI in $(get_all_abis); do
> > +		for ABI in ${multilib_abis}; do
> >  			multilib_toolchain_setup "${ABI}"
> >  			BUILD_DIR=${initial_dir%%/}-${ABI} "${@}"
> >  		done
> > 
> Why CC devrel on this?

Oops, sorry. I meant to CC releng...

- -- 
Best regards,
Michał Górny
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iJwEAQEIAAYFAlEBpHoACgkQfXuS5UK5QB2znQP/YTR6zos9ATcaXSjoo1+bhiVl
tSpvNXGDRmYEIVGWdW7831oTd1/5bazGaoALnC5Y8iURblI8MUENv7xx4u9RQnQb
okIKHLXWFsirMrx3XAs0nTbRotZxFZY00N93mhYHYGjg4l0bnsIVCQC6dBpHLKRX
wMUEF2WjATT0ACxgyos=
=qTSP
-----END PGP SIGNATURE-----