[gentoo-dev] [RFC] Adding features to Portage that work on top of any EAPI

[gentoo-dev] [RFC] Adding features to Portage that work on top of any EAPI

[Fabian Groffen]

Hi all,

The Prefix team has a separate Portage branch which implements the
"prefix" extensions.  In short, this encompasses the addition of the
variables EPREFIX, ED and EROOT, and the function eprefixify to the
ebuild/eclass environment, which may be used to make an ebuild work for
a given prefix offset.

I would like to get some input on how best to deal with these additions
in the light of EAPI and the main (gentoo-x86) tree and Portage.  Since
the Prefix extensions can currently be applied on top of any existing
EAPI, they are not requiring any special EAPI value as baseline.

Currently in Prefix we implemented EAPI as being a set of tokens that
are orthogonal to each other.  In other words, while 0, 1 and 2 are
mutual exclusive, "prefix" can be applied to 0, 1, or 2.  The result is
something like EAPI="prefix 1".  Of course with this approach the recent
introduction of EAPI=2, resulted in a problem with eclasses that do a
blind check on the EAPI variable to be e.g. 2.
An alternative is to create multiple new EAPIs, such as prefix-1 or
1-prefix, containing the Prefix extensions on top of EAPI=1.  Same
problem when checks on EAPI are done of course, but EAPI then always
consists of a single value.
Something that was raised in previous discussions was that maybe the
Prefix extensions don't need an EAPI in itself, but that an ebuild has
to be marked as Prefix-ready through e.g. the recently proposed
PROPERTIES variable, (a horrible hack) in KEYWORDS, or a newly to be
added variable.

Please discuss.


-- 
Fabian Groffen
Gentoo on a different level

Re: [gentoo-dev] [RFC] Adding features to Portage that work on top of any EAPI

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 2509 bytes --]

On Thu, 9 Oct 2008 19:46:55 +0200
Fabian Groffen <grobian@gentoo.org> wrote:
> Currently in Prefix we implemented EAPI as being a set of tokens that
> are orthogonal to each other.  In other words, while 0, 1 and 2 are
> mutual exclusive, "prefix" can be applied to 0, 1, or 2.  The result
> is something like EAPI="prefix 1".  Of course with this approach the
> recent introduction of EAPI=2, resulted in a problem with eclasses
> that do a blind check on the EAPI variable to be e.g. 2.

EAPI's defined as being a single value because mixing between EAPIs is
in general impossible. For example, I'm guessing prefix might need to
do something to econf / the default src_compile/configure functions at
some point, so it's not really truly independent.

> An alternative is to create multiple new EAPIs, such as prefix-1 or
> 1-prefix, containing the Prefix extensions on top of EAPI=1.  Same
> problem when checks on EAPI are done of course, but EAPI then always
> consists of a single value.

That's the sensible way of doing it...

The way things are with EAPIs... The only way you'll get things
supported in main tree eclasses is if you get the Council to approve a
formal specification of what you want. Unfortunately, they seem
reluctant to do that even if you're an official Gentoo project (see
kdebuild-1).

Is there anything stopping you from formalising your specification of
what you need? (The PMS team can probably help with the 'writing formal'
bit if necessary, given an informal description.) Could it be done in
such a way that non-prefix package managers can do minimal support to
get the current /usr behaviour, whilst optionally implementing
everything else? If this is the case, the best route's probably to get
the whole thing into the next EAPI, start using that EAPI for all your
overlay packages and persuade people to include the necessary prefixy
things in main-tree eclasses (which they should, once said EAPI is
Council approved).

> Something that was raised in previous discussions was that maybe the
> Prefix extensions don't need an EAPI in itself, but that an ebuild has
> to be marked as Prefix-ready through e.g. the recently proposed
> PROPERTIES variable, (a horrible hack) in KEYWORDS, or a newly to be
> added variable.

What's the scope of the changes? I think it'd be easiest to discuss
this if you posted an informal summary describing the differences in
terms of which bits of PMS are affected.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: [gentoo-dev] [RFC] Adding features to Portage that work on top of any EAPI

[Fabian Groffen]

On 09-10-2008 19:15:26 +0100, Ciaran McCreesh wrote:
> On Thu, 9 Oct 2008 19:46:55 +0200
> Fabian Groffen <grobian@gentoo.org> wrote:
> > Currently in Prefix we implemented EAPI as being a set of tokens that
> > are orthogonal to each other.  In other words, while 0, 1 and 2 are
> > mutual exclusive, "prefix" can be applied to 0, 1, or 2.  The result
> > is something like EAPI="prefix 1".  Of course with this approach the
> > recent introduction of EAPI=2, resulted in a problem with eclasses
> > that do a blind check on the EAPI variable to be e.g. 2.
> 
> EAPI's defined as being a single value because mixing between EAPIs is
> in general impossible. For example, I'm guessing prefix might need to
> do something to econf / the default src_compile/configure functions at
> some point, so it's not really truly independent.

While that is true, currently Prefix is designed in such a way that
an empty offset results in a fully "backwards" compatible Portage.

> Is there anything stopping you from formalising your specification of
> what you need? (The PMS team can probably help with the 'writing formal'
> bit if necessary, given an informal description.) Could it be done in
> such a way that non-prefix package managers can do minimal support to
> get the current /usr behaviour, whilst optionally implementing
> everything else? If this is the case, the best route's probably to get
> the whole thing into the next EAPI, start using that EAPI for all your
> overlay packages and persuade people to include the necessary prefixy
> things in main-tree eclasses (which they should, once said EAPI is
> Council approved).

A problem I have with requiring a "next" EAPI for each ebuild, is that
Prefix requires all base-system ebuilds to be "Prefix compatible".
However, this category of ebuilds requires being conservative with EAPI
requirements.

I once started on an attempt to document the stuff[1], but it's pretty
verbose, and it misses the necessary informal definitions of in
particular chapter [2].

> > Something that was raised in previous discussions was that maybe the
> > Prefix extensions don't need an EAPI in itself, but that an ebuild has
> > to be marked as Prefix-ready through e.g. the recently proposed
> > PROPERTIES variable, (a horrible hack) in KEYWORDS, or a newly to be
> > added variable.
> 
> What's the scope of the changes? I think it'd be easiest to discuss
> this if you posted an informal summary describing the differences in
> terms of which bits of PMS are affected.

[2] sums it up for as much as I can recall for the moment, the
non-privileged stuff is supposed to be separate, but its use is
inherently related to offset installations.  Our overlay[3] contains
enough material to get an idea of what it looks like in practise.  If
you want specific pointers, I can give you them.


[1] http://dev.gentoo.org/~grobian/gleps/glep-prefix.html
[2] http://dev.gentoo.org/~grobian/gleps/glep-prefix.html#ebuild-changes-in-prefix
[3] http://overlays.gentoo.org/proj/alt/browser/trunk/prefix-overlay

-- 
Fabian Groffen
Gentoo on a different level

Re: [gentoo-dev] [RFC] Adding features to Portage that work on top of any EAPI

[Jeremy Olexa]

On Thu, Oct 9, 2008 at 1:15 PM, Ciaran McCreesh
<ciaran.mccreesh@googlemail.com> wrote:
> On Thu, 9 Oct 2008 19:46:55 +0200
<large snip>
> What's the scope of the changes? I think it'd be easiest to discuss
> this if you posted an informal summary describing the differences in
> terms of which bits of PMS are affected.

Ciaran, others:

In a way I feel like we (the Prefix project) are mis-using the EAPI
value. If we have something that is designed to work with *any* EAPI,
is it really a special EAPI? haubi said something on the gentoo-alt
list that made me think about this more:
"When an usecase of something is orthogonal to what that thing is
designed for, one should consider using a different thing for this
usecase." -source unknown

Is this PROPERTIES-like information? Is that easily supportable in the PM?

Re: [gentoo-dev] [RFC] Adding features to Portage that work on top of any EAPI

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1693 bytes --]

On Fri, 10 Oct 2008 10:40:53 -0500
"Jeremy Olexa" <darkside@gentoo.org> wrote:
> In a way I feel like we (the Prefix project) are mis-using the EAPI
> value.

You're misusing it in the way you treat it as a set of strings rather
than a single value. But this being an EAPI thing seems right.

> If we have something that is designed to work with *any* EAPI,
> is it really a special EAPI? haubi said something on the gentoo-alt
> list that made me think about this more:
> "When an usecase of something is orthogonal to what that thing is
> designed for, one should consider using a different thing for this
> usecase." -source unknown
> 
> Is this PROPERTIES-like information? Is that easily supportable in
> the PM?

I don't see it as orthogonal.

Here's the thing: you can't use prefix ebuilds in a non-prefix-aware
package manager because things like ED will be unset. If prefix ebuilds
could work (as in, install unprefixed) in a purely vanilla package
manager with no prefix awareness, it could be done using PROPERTIES or
some similar variable. But prefix won't work at all unless its
extensions are present, and it also appears to require changes to
things that are defined differently in different EAPIs.

I suspect most of the problem is down to timescale. The prefix
development time is spread out over three EAPIs so far, so you need
three sets of (mostly similar) extensions. Had prefix taken less time
to be worked out, it'd fairly clearly be something that could just go
straight in to the next EAPI, with duplicated base system packages in
an overlay to avoid having to use new EAPIs for core things in the
main tree.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 197 bytes --]