[gentoo-dev] RFC: cartesian product extension to keyword system

[gentoo-dev] RFC: cartesian product extension to keyword system

[heroxbd]

Dear all,

In GLEP22[1], reasonable defaults has been introduced to prevent the
explosion of keywords. With the growth of Gentoo Prefix, however, a
substantial amount of keywords are introduced. Among them, duplex
information exists. For example, an ebuild keyworded x86-linux(Gentoo
Prefix on x86 linux) will certainly build on x86 (Gentoo vanilla on x86
linux). Furthermore if the ebuild has "amd64" keyword, it will certainly
build on amd64-linux too.

This duplication becomes more manifested during the development of
Gentoo Prefix on arm. Introducing an arm-linux keyword is objected by
developers because we are having too many keywords[2]. But it is far
from consensus yet, as people are already adding arm-linux to
ebuilds[2].

There were also a proposal to treat arm the same as arm-linux, but
received more objections for its vagueness.

Therefore we propose the following cartesian product extension to
keyword system.

a. introducing four more variables, in order, KEYWORDS_ARCH,
   KEYWORDS_KERNEL, KEYWORDS_USERLAND, KEYWORDS_LIBC. An order relation
   is defined as arch > kernel > userland > libc

b. KEYWORDS_var1 must be defined if KEYWORDS_var2 is defined and var2 <
   var1. Otherwise it can be omitted.

c. KEYWORDS_PROD is defined as cartesian product of KEYWORDS_ARCH,
   KEYWORDS_KERNEL, KEYWORDS_USERLAND, KEYWORDS_LIBC. Omitted variable
   is interpreted as empty and not counted in the product. e.g., if
   KEYWORDS_USERLAND and KEYWORDS_LIBC are not defined, KEYWORDS_PROD is
   the product of KEYWORDS_ARCH and KEYWORDS_KERNEL.

c1. "~" is attached to KEYWORDS_PROD elements if any of its oprand has "~".

d. KEYWORDS is redefined as union of KEYWORDS_PROD and KEYWORDS (for
   backward compatibility)

e. reasonable defaults in GLEP22 applys.

Therefore, for sys-devel/bison, the keywords proposed will be:

KEYWORDS="~hppa-hpux ~m68k-mint ~ppc-aix ~x64-solaris ~x86-interix

KEYWORDS_ARCH="~alpha ~amd64 ~arm ~hppa ~ia64 ~m68k ~mips ~ppc ~ppc64 ~s390 ~sh ~sparc ~x86"

KEYWORDS_KERNEL="~linux ~solaris ~freebsd ~fbsd ~macos"

Problems remaining:

a. how to do stablization in this case?

b. how to handle things that does not exist? e.g. sparc-macosx?

c. how to clean up confusions? e.g. x64-solaris vs amd64-solaris,
   freebsd (Gentoo Prefix on FreeBSD) vs fbsd (Gentoo/FreeBSD)

Thanks,
Benda

1. http://www.gentoo.org/proj/en/glep/glep-0022.html

2. see sys-devel/bison, ~alpha ~amd64 ~amd64-fbsd ~amd64-linux ~arm ~arm-linux
   ~hppa ~hppa-hpux ~ia64 ~ia64-hpux ~ia64-linux ~m68k ~m68k-mint ~mips
   ~ppc ~ppc-aix ~ppc-macos ~ppc64 ~s390 ~sh ~sparc ~sparc-fbsd
   ~sparc-solaris ~sparc64-solaris ~x64-freebsd ~x64-macos ~x64-solaris
   ~x86 ~x86-fbsd ~x86-freebsd ~x86-interix ~x86-linux ~x86-macos
   ~x86-solaris

Re: [gentoo-dev] RFC: cartesian product extension to keyword system

[Fabian Groffen]

[-- Attachment #1: Type: text/plain, Size: 948 bytes --]

Hi,

On 29-04-2013 16:14:51 +0900, heroxbd wrote:
> In GLEP22[1], reasonable defaults has been introduced to prevent the
> explosion of keywords. With the growth of Gentoo Prefix, however, a
> substantial amount of keywords are introduced. Among them, duplex
> information exists. For example, an ebuild keyworded x86-linux(Gentoo
> Prefix on x86 linux) will certainly build on x86 (Gentoo vanilla on x86
> linux).

Likely, but not necessary true.

> Furthermore if the ebuild has "amd64" keyword, it will certainly
> build on amd64-linux too.

Somewhat likely, but absolutely not true.

> Therefore we propose the following cartesian product extension to
> keyword system.

We added the keywords as is, to uniquely define which systems it works
on.  Assuming something will work because it should be similar is (and
has been proven in the past to be) wrong.

My €0.02

-- 
Fabian Groffen
Gentoo on a different level

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [gentoo-dev] RFC: cartesian product extension to keyword system

[heroxbd]

[-- Attachment #1: Type: text/plain, Size: 914 bytes --]

Dearr Fabian,

Fabian Groffen <grobian@gentoo.org> writes:

>> Furthermore if the ebuild has "amd64" keyword, it will certainly
>> build on amd64-linux too.
>
> Somewhat likely, but absolutely not true.

Sorry, the original phrase was vague. I meant, if an ebuild is keyworded
"amd64" and "x86-linux", it will certainly work on "amd64-linux"

>> Therefore we propose the following cartesian product extension to
>> keyword system.
>
> We added the keywords as is, to uniquely define which systems it works
> on. Assuming something will work because it should be similar is (and
> has been proven in the past to be) wrong.

Every system is different. We are grouping similar systems into
corresponding keywords, and already assuming ability for
extrapolation. And now we observe that ARCH and KERNEL are quite
independent. They can be decomposed into two dimensions. Why don't me
exploit this fact?

Cheers,
Benda


[-- Attachment #2: Type: application/pgp-signature, Size: 489 bytes --]

Re: [gentoo-dev] RFC: cartesian product extension to keyword system

[Ben de Groot]

On 29 April 2013 15:14, heroxbd <heroxbd@gentoo.org> wrote:
>
> Dear all,
>
> In GLEP22[1], reasonable defaults has been introduced to prevent the
> explosion of keywords. With the growth of Gentoo Prefix, however, a
> substantial amount of keywords are introduced. Among them, duplex
> information exists. For example, an ebuild keyworded x86-linux(Gentoo
> Prefix on x86 linux) will certainly build on x86 (Gentoo vanilla on x86
> linux). Furthermore if the ebuild has "amd64" keyword, it will certainly
> build on amd64-linux too.
>
> This duplication becomes more manifested during the development of
> Gentoo Prefix on arm. Introducing an arm-linux keyword is objected by
> developers because we are having too many keywords[2]. But it is far
> from consensus yet, as people are already adding arm-linux to
> ebuilds[2].
>
> There were also a proposal to treat arm the same as arm-linux, but
> received more objections for its vagueness.
>
> Therefore we propose the following cartesian product extension to
> keyword system.
>
> a. introducing four more variables, in order, KEYWORDS_ARCH,
>    KEYWORDS_KERNEL, KEYWORDS_USERLAND, KEYWORDS_LIBC. An order relation
>    is defined as arch > kernel > userland > libc
>
> b. KEYWORDS_var1 must be defined if KEYWORDS_var2 is defined and var2 <
>    var1. Otherwise it can be omitted.
>
> c. KEYWORDS_PROD is defined as cartesian product of KEYWORDS_ARCH,
>    KEYWORDS_KERNEL, KEYWORDS_USERLAND, KEYWORDS_LIBC. Omitted variable
>    is interpreted as empty and not counted in the product. e.g., if
>    KEYWORDS_USERLAND and KEYWORDS_LIBC are not defined, KEYWORDS_PROD is
>    the product of KEYWORDS_ARCH and KEYWORDS_KERNEL.
>
> c1. "~" is attached to KEYWORDS_PROD elements if any of its oprand has "~".
>
> d. KEYWORDS is redefined as union of KEYWORDS_PROD and KEYWORDS (for
>    backward compatibility)
>
> e. reasonable defaults in GLEP22 applys.
>
> Therefore, for sys-devel/bison, the keywords proposed will be:
>
> KEYWORDS="~hppa-hpux ~m68k-mint ~ppc-aix ~x64-solaris ~x86-interix
>
> KEYWORDS_ARCH="~alpha ~amd64 ~arm ~hppa ~ia64 ~m68k ~mips ~ppc ~ppc64 ~s390 ~sh ~sparc ~x86"
>
> KEYWORDS_KERNEL="~linux ~solaris ~freebsd ~fbsd ~macos"
>
> Problems remaining:
>
> a. how to do stablization in this case?
>
> b. how to handle things that does not exist? e.g. sparc-macosx?
>
> c. how to clean up confusions? e.g. x64-solaris vs amd64-solaris,
>    freebsd (Gentoo Prefix on FreeBSD) vs fbsd (Gentoo/FreeBSD)


To me this looks to be needlessly complex.

Cheers,

Ben | yngwin
Gentoo developer
Gentoo Qt project lead, Gentoo Wiki admin

Re: [gentoo-dev] RFC: cartesian product extension to keyword system

[heroxbd]

Hi Ben,

Ben de Groot <yngwin@gentoo.org> writes:

> To me this looks to be needlessly complex.

There is an alternative way to do this:

      @-linux

to mean "the ebuild works on every keyworded arch on Prefix with Linux
kernel".

Cheers,
Benda

Re: [gentoo-dev] RFC: cartesian product extension to keyword system

[Jeroen Roovers]

On Mon, 29 Apr 2013 16:14:51 +0900
heroxbd <heroxbd@gentoo.org> wrote:

> KEYWORDS="~hppa-hpux ~m68k-mint ~ppc-aix ~x64-solaris ~x86-interix["]
> 
> KEYWORDS_ARCH="~alpha ~amd64 ~arm ~hppa ~ia64 ~m68k ~mips ~ppc ~ppc64
> ~s390 ~sh ~sparc ~x86"

Regardless of your chance of success in making the extra complexity
manageable, I think moving the tradition KEYWORDS value to
KEYWORDS_ARCH, and reusing KEYWORDS for something else , would
needlessly increase the work required to "migrate" the portage tree.

Why not leave KEYWORDS what it is right now, and expand/change its
meaning using alternate variables so that you can (indefinitely)
maintain backward compatibility?


     jer

Re: [gentoo-dev] RFC: cartesian product extension to keyword system

[Alexey Shvetsov]

[-- Attachment #1: Type: text/plain, Size: 1312 bytes --]

В письме от 29 апреля 2013 15:43:03 пользователь Jeroen Roovers написал:
> On Mon, 29 Apr 2013 16:14:51 +0900
> 
> heroxbd <heroxbd@gentoo.org> wrote:
> > KEYWORDS="~hppa-hpux ~m68k-mint ~ppc-aix ~x64-solaris ~x86-interix["]
> > 
> > KEYWORDS_ARCH="~alpha ~amd64 ~arm ~hppa ~ia64 ~m68k ~mips ~ppc ~ppc64
> > ~s390 ~sh ~sparc ~x86"
> 
> Regardless of your chance of success in making the extra complexity
> manageable, I think moving the tradition KEYWORDS value to
> KEYWORDS_ARCH, and reusing KEYWORDS for something else , would
> needlessly increase the work required to "migrate" the portage tree.
> 
> Why not leave KEYWORDS what it is right now, and expand/change its
> meaning using alternate variables so that you can (indefinitely)
> maintain backward compatibility?

Its not necessary. We simply may not define KEYWORDS but only define KEYWORDS_* 
vars. So old package managers versions will treat that as we have empty 
keywords

> 
> 
>      jer
-- 
Best Regards,
Alexey 'Alexxy' Shvetsov
Petersburg Nuclear Physics Institute, NRC Kurchatov Institute, 
Gatchina, Russia
Department of Molecular and Radiation Biophysics
Gentoo Team Ru
Gentoo Linux Dev
mailto:alexxyum@gmail.com
mailto:alexxy@gentoo.org
mailto:alexxy@omrb.pnpi.spb.ru

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]