[gentoo-pms] EAPI specification in ebuilds

[gentoo-pms] EAPI specification in ebuilds

[Ulrich Mueller]

Hi,

The council has decided in its April meeting [1] that we should take
the route of parsing the EAPI assignment in ebuilds. Please find below
a draft wording for PMS. Thanks to dleverton, ferringb, and zmedico
for their input.

The plan would be to allow optional parsing now, and make it mandatory
in future, when EAPI 5 is introduced (i.e. EAPI 5 aware package
managers would have to implement it).

Ulrich

[1] <http://www.gentoo.org/proj/en/council/meeting-logs/20120403.txt>
    <http://www.gentoo.org/proj/en/council/meeting-logs/20120403-summary.txt>


--- a/ebuild-vars.tex
+++ b/ebuild-vars.tex
@@ -75,7 +75,7 @@ Ebuilds may define any of the following variables:
 
 \begin{description}
 \item[DEPEND] See section~\ref{sec:dependencies}.
-\item[EAPI] The EAPI. See below for defaults.
+\item[EAPI] The EAPI. See below.
 \item[PDEPEND] See section~\ref{sec:dependencies}.
 \item[RDEPEND] See section~\ref{sec:dependencies}. For some EAPIs, \t{RDEPEND} has special behaviour
     for its value if unset and when used with an eclass. See section~\ref{sec:rdepend-depend} for
@@ -124,6 +124,23 @@ ensure that it is unset.
 If any of these variables are set to invalid values, the package manager's behaviour is undefined;
 ideally, an error in one ebuild should not prevent operations upon other ebuilds or packages.
 
+To enable parsing of ebuilds for the \t{EAPI} value, the \t{EAPI} assignment statement must adhere
+to the following syntax: It must consist of the string \t{EAPI}, followed by an equals sign,
+followed by the \t{EAPI} value, optionally enclosed in single or double quotes. The statement must
+be on its own line, optionally preceded or followed by whitespace (spaces or tabs), optionally
+followed by a comment. Thus, the line must match the following regular expression:
+\begin{verbatim}
+^[ \t]*EAPI=(['"]?)([A-Za-z0-9+_.-]*)\1[ \t]*(#.*)?$
+\end{verbatim}
+If an ebuild contains an \t{EAPI} assignment, the statement must occur within the first 20 lines.
+It is an error for an ebuild to contain more than one \t{EAPI} assignment.
+
+The package manager may optionally determine the \t{EAPI} of an ebuild by parsing it, using above
+regular expression and line number limit. If a recognised \t{EAPI} has been determined such, then
+the package manager must make sure that the \t{EAPI} value obtained by sourcing the ebuild with
+bash is identical to the one obtained by parsing. The ebuild must be treated as invalid if these
+values are different.
+
 \subsection{\t{RDEPEND} value}
 \label{sec:rdepend-depend}
 

Re: [gentoo-pms] EAPI specification in ebuilds

[Robin H. Johnson]

On Fri, Apr 06, 2012 at 09:40:16AM +0200, Ulrich Mueller wrote:
> +To enable parsing of ebuilds for the \t{EAPI} value, the \t{EAPI} assignment statement must adhere
> +to the following syntax: It must consist of the string \t{EAPI}, followed by an equals sign,
> +followed by the \t{EAPI} value, optionally enclosed in single or double quotes. The statement must
> +be on its own line, optionally preceded or followed by whitespace (spaces or tabs), optionally
> +followed by a comment. Thus, the line must match the following regular expression:
> +\begin{verbatim}
> +^[ \t]*EAPI=(['"]?)([A-Za-z0-9+_.-]*)\1[ \t]*(#.*)?$
EAPI=
is valid in this. Was that intended?

Will there be any issue overriding EAPI as an env var to call a
function/command?
EAPI=... foo

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee & Infrastructure Lead
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85

Re: [gentoo-pms] EAPI specification in ebuilds

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 930 bytes --]

On Fri, Apr 6, 2012 at 10:29 AM, Robin H. Johnson <robbat2@gentoo.org>wrote:

> On Fri, Apr 06, 2012 at 09:40:16AM +0200, Ulrich Mueller wrote:
> > +To enable parsing of ebuilds for the \t{EAPI} value, the \t{EAPI}
> assignment statement must adhere
> > +to the following syntax: It must consist of the string \t{EAPI},
> followed by an equals sign,
> > +followed by the \t{EAPI} value, optionally enclosed in single or double
> quotes. The statement must
> > +be on its own line, optionally preceded or followed by whitespace
> (spaces or tabs), optionally
> > +followed by a comment. Thus, the line must match the following regular
> expression:
> > +\begin{verbatim}
> > +^[ \t]*EAPI=(['"]?)([A-Za-z0-9+_.-]*)\1[ \t]*(#.*)?$
> EAPI=
> is valid in this. Was that intended?
>
> Will there be any issue overriding EAPI as an env var to call a
> function/command?
> EAPI=... foo
>

That's kind of a no no; specific usage?
~harring

[-- Attachment #2: Type: text/html, Size: 1279 bytes --]

Re: [gentoo-pms] EAPI specification in ebuilds

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 239 bytes --]

On Fri, 6 Apr 2012 17:29:17 +0000
"Robin H. Johnson" <robbat2@gentoo.org> wrote:
> Will there be any issue overriding EAPI as an env var to call a
> function/command?
> EAPI=... foo

That's never been legal.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-pms] EAPI specification in ebuilds

[Ulrich Mueller]

>>>>> On Fri, 6 Apr 2012, Robin H Johnson wrote:

>> +^[ \t]*EAPI=(['"]?)([A-Za-z0-9+_.-]*)\1[ \t]*(#.*)?$
> EAPI=
> is valid in this. Was that intended?

An empty EAPI is equivalent to EAPI 0 (says PMS), therefore "EAPI=" is
valid in ebuilds.

But this is sort of academic, because no ebuild in the tree contains
an empty EAPI assignment.

> Will there be any issue overriding EAPI as an env var to call a
> function/command?
> EAPI=... foo

This won't match the regexp, and I believe that such usage is illegal
anyhow.

Ulrich

Re: [gentoo-pms] EAPI specification in ebuilds

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1423 bytes --]

On Fri, 6 Apr 2012 09:40:16 +0200
Ulrich Mueller <ulm@gentoo.org> wrote:
> The plan would be to allow optional parsing now, and make it mandatory
> in future, when EAPI 5 is introduced (i.e. EAPI 5 aware package
> managers would have to implement it).

What does this do, if EAPI 5 includes per-package eclasses?

    inherit foo
    EAPI=5

Where 'foo' would be found per-package for EAPI 5, and globally
otherwise. (This is a general problem, and inherit is just an example,
so it needs to be addressed even if you say "well we won't allow that".
The same kind of thing happens if we stick versionator into the package
mangler, or anything else global scopeish.)

What about this, if the package manager does not support the new
behaviour yet?

    EAPI=5
    # 20 lines
    EAPI=4

The proposal as worded doesn't seem to outlaw this, and moving to
"mandatory for EAPI 5" doesn't help either, since under current rules
that ebuild isn't EAPI 5.

What about this?

    cat <<END
    EAPI=5
    END >/dev/null
    EAPI=4

That's also not EAPI 5, so it's not affected by the rules as written.

What about this?

    if [[ $EAPI = 5 ]] ; then
        blah
    fi

    EAPI=5

Also relevant: https://bugs.gentoo.org/show_bug.cgi?id=411069

The solution the Council decided upon doesn't do away with the whole
"pre-source" vs "post-source" EAPI problem.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-pms] EAPI specification in ebuilds

[David Leverton]

On 6 April 2012 08:40, Ulrich Mueller <ulm@gentoo.org> wrote:
> +It is an error for an ebuild to contain more than one \t{EAPI} assignment.

Are package managers expected to enforce this in the parsing stage (at
least within the first 20 lines) or is it OK to stop reading as soon
as you see a valid EAPI= line (so any conflicting later assignments
would get caught by making sure the post-source EAPI matches the
parsed one, but redundant assignments with the same EAPI would get
through)?

Re: [gentoo-pms] EAPI specification in ebuilds

[Ulrich Mueller]

>>>>> On Fri, 6 Apr 2012, David Leverton wrote:

>> +It is an error for an ebuild to contain more than one \t{EAPI} assignment.

> Are package managers expected to enforce this in the parsing stage
> (at least within the first 20 lines) or is it OK to stop reading as
> soon as you see a valid EAPI= line (so any conflicting later
> assignments would get caught by making sure the post-source EAPI
> matches the parsed one,

I guess the pragmatic approach is that the package manager would stop
reading when it encounters the first valid EAPI assignment (or after
line 20, whatever occurs first). The sanity check is that the two EAPI
values obtained by parsing and sourcing must agree.

In theory this may leave some loopholes, i.e. it is possible to
construct an ebuild that would be invalid by the wording of the spec
but would be accepted as valid ebuild by the package manager. I don't
think that this has any practical relevance though.

> but redundant assignments with the same EAPI would get through)?

Right, assigning the same EAPI twice would be an example for an ebuild
that is invalid but accepted by the package manager. But would this
cause any problems?

Ulrich

Re: [gentoo-pms] EAPI specification in ebuilds

[David Leverton]

On 6 April 2012 22:25, Ulrich Mueller <ulm@gentoo.org> wrote:
> Right, assigning the same EAPI twice would be an example for an ebuild
> that is invalid but accepted by the package manager. But would this
> cause any problems?
>
> Ulrich
>

It's not that it's a problem in itself, it just struck me as an edge
case with the potential for implementations to differ.  As long as
everyone's clear what the rules are it's fine.

Re: [gentoo-pms] EAPI specification in ebuilds

[Ulrich Mueller]

>>>>> On Fri, 6 Apr 2012, David Leverton wrote:

>> Right, assigning the same EAPI twice would be an example for an
>> ebuild that is invalid but accepted by the package manager. But
>> would this cause any problems?

> It's not that it's a problem in itself, it just struck me as an edge
> case with the potential for implementations to differ.  As long as
> everyone's clear what the rules are it's fine.

Maybe the spec should say that the first match must be used:

"The package manager may optionally determine the EAPI of an ebuild by
parsing it, using the first match for the above regular expression
within the above line number limit."

Ulrich

Re: [gentoo-pms] EAPI specification in ebuilds

[Ulrich Mueller]

>>>>> On Sat, 7 Apr 2012, Ulrich Mueller wrote:

>>>>> On Fri, 6 Apr 2012, David Leverton wrote:
>>> Right, assigning the same EAPI twice would be an example for an
>>> ebuild that is invalid but accepted by the package manager. But
>>> would this cause any problems?

>> It's not that it's a problem in itself, it just struck me as an edge
>> case with the potential for implementations to differ.  As long as
>> everyone's clear what the rules are it's fine.

> Maybe the spec should say that the first match must be used:

> "The package manager may optionally determine the EAPI of an ebuild by
> parsing it, using the first match for the above regular expression
> within the above line number limit."

See updated patch below.

Ulrich


--- a/ebuild-vars.tex
+++ b/ebuild-vars.tex
@@ -75,7 +75,7 @@
 
 \begin{description}
 \item[DEPEND] See section~\ref{sec:dependencies}.
-\item[EAPI] The EAPI. See below for defaults.
+\item[EAPI] The EAPI. See below.
 \item[PDEPEND] See section~\ref{sec:dependencies}.
 \item[RDEPEND] See section~\ref{sec:dependencies}. For some EAPIs, \t{RDEPEND} has special behaviour
     for its value if unset and when used with an eclass. See section~\ref{sec:rdepend-depend} for
@@ -124,6 +124,23 @@
 If any of these variables are set to invalid values, the package manager's behaviour is undefined;
 ideally, an error in one ebuild should not prevent operations upon other ebuilds or packages.
 
+To enable parsing of ebuilds for the \t{EAPI} value, the \t{EAPI} assignment statement must adhere
+to the following syntax: It must consist of the string \t{EAPI}, followed by an equals sign,
+followed by the \t{EAPI} value, optionally enclosed in single or double quotes. The statement must
+be on its own line, optionally preceded or followed by whitespace (spaces or tabs), optionally
+followed by a comment. Thus, the line must match the following regular expression:
+\begin{verbatim}
+^[ \t]*EAPI=(['"]?)([A-Za-z0-9+_.-]*)\1[ \t]*(#.*)?$
+\end{verbatim}
+If an ebuild contains an \t{EAPI} assignment, the statement must occur within the first 20 lines.
+An ebuild must not contain more than one \t{EAPI} assignment.
+
+The package manager may optionally determine the \t{EAPI} of an ebuild by parsing it, using the
+first match for the above regular expression within the above line number limit. If a recognised
+\t{EAPI} has been determined such, then the package manager must make sure that the \t{EAPI} value
+obtained by sourcing the ebuild with bash is identical to the one obtained by parsing. The ebuild
+must be treated as invalid if these values are different.
+
 \subsection{\t{RDEPEND} value}
 \label{sec:rdepend-depend}
 

Re: [gentoo-pms] EAPI specification in ebuilds

[Zac Medico]

On 04/10/2012 03:53 AM, Ulrich Mueller wrote:
> +^[ \t]*EAPI=(['"]?)([A-Za-z0-9+_.-]*)\1[ \t]*(#.*)?$

We could make the ([A-Za-z0-9+_.-]*) group "optional", but treat it as 
an error if the group is missing. That way, the package manager can 
detect invalid assignments earlier, and can skip sourcing the ebuild in 
this case.
--
Thanks,
Zac

Re: [gentoo-pms] EAPI specification in ebuilds

[Brian Harring]

On Tue, Apr 10, 2012 at 08:43:55AM -0700, Zac Medico wrote:
> On 04/10/2012 03:53 AM, Ulrich Mueller wrote:
> > +^[ \t]*EAPI=(['"]?)([A-Za-z0-9+_.-]*)\1[ \t]*(#.*)?$
> 
> We could make the ([A-Za-z0-9+_.-]*) group "optional", but treat it as 
> an error if the group is missing. That way, the package manager can 
> detect invalid assignments earlier, and can skip sourcing the ebuild in 
> this case.

Or disallow EAPI= .

There isn't any reason to reset it to an unknown EAPI manually like 
that; if code does that, the code is wrong imo, thus skip trying to 
parse that.
~harring

Re: [gentoo-pms] EAPI specification in ebuilds

[Zac Medico]

On 04/10/2012 09:30 AM, Brian Harring wrote:
> On Tue, Apr 10, 2012 at 08:43:55AM -0700, Zac Medico wrote:
>> On 04/10/2012 03:53 AM, Ulrich Mueller wrote:
>>> +^[ \t]*EAPI=(['"]?)([A-Za-z0-9+_.-]*)\1[ \t]*(#.*)?$
>>
>> We could make the ([A-Za-z0-9+_.-]*) group "optional", but treat it as
>> an error if the group is missing. That way, the package manager can
>> detect invalid assignments earlier, and can skip sourcing the ebuild in
>> this case.
>
> Or disallow EAPI= .

My intention was to catch anything that starts with the ^[ \t]*EAPI= 
part. For my optional group thing to work, I guess you'd also have to 
add a (.*)? group to match whatever invalid characters may be there.
-- 
Thanks,
Zac

Re: [gentoo-pms] EAPI specification in ebuilds

[Ulrich Mueller]

>>>>> On Tue, 10 Apr 2012, Brian Harring wrote:

> On Tue, Apr 10, 2012 at 08:43:55AM -0700, Zac Medico wrote:
>> We could make the ([A-Za-z0-9+_.-]*) group "optional", but treat it
>> as an error if the group is missing. That way, the package manager
>> can detect invalid assignments earlier, and can skip sourcing the
>> ebuild in this case.

Currently there isn't any ebuild in the tree that assigns an empty
EAPI, so I think we can neglect any performance issues.

> Or disallow EAPI= .

Probably it's simpler to keep the regexp as it is, but error out if
the second group is empty.

> There isn't any reason to reset it to an unknown EAPI manually like
> that; if code does that, the code is wrong imo, thus skip trying to
> parse that.

I see no valid usage case why an ebuild should assign the empty string
to the EAPI variable. So we could add a sentence like
"It is an error for an ebuild to assign an empty EAPI."
to the first paragraph of this subsection:
<http://dev.gentoo.org/~ulm/pms/4/pms.html#x1-710008.3.1>

Ulrich

Re: [gentoo-pms] EAPI specification in ebuilds

[Zac Medico]

On 04/10/2012 09:58 AM, Ulrich Mueller wrote:
>>>>>> On Tue, 10 Apr 2012, Brian Harring wrote:
>
>> On Tue, Apr 10, 2012 at 08:43:55AM -0700, Zac Medico wrote:
>>> We could make the ([A-Za-z0-9+_.-]*) group "optional", but treat it
>>> as an error if the group is missing. That way, the package manager
>>> can detect invalid assignments earlier, and can skip sourcing the
>>> ebuild in this case.
>
> Currently there isn't any ebuild in the tree that assigns an empty
> EAPI, so I think we can neglect any performance issues.

My intention was to detect EAPI assignments containing invalid 
characters such as spaces or unicode characters. Either way, I agree 
it's not really a performance issue. If it makes the specification too 
complex for your liking, then I agree it's not worth doing.
-- 
Thanks,
Zac

Re: [gentoo-pms] EAPI specification in ebuilds

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 549 bytes --]

On Tue, 10 Apr 2012 12:53:27 +0200
Ulrich Mueller <ulm@gentoo.org> wrote:
> +If an ebuild contains an \t{EAPI} assignment, the statement must
> occur within the first 20 lines. An ebuild must not contain more
> than one \t{EAPI} assignment.

This still doesn't explain what should happen here:

    inherit foo
    EAPI=5

There are two issues: which EAPI's 'inherit' behaviour is used, and
what is the value of the $EAPI variable when sourcing foo.eclass?
Eclasses seem to like doing $EAPI-dependent things...

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-pms] EAPI specification in ebuilds

[Ulrich Mueller]

>>>>> On Tue, 10 Apr 2012, Ciaran McCreesh wrote:

>> +If an ebuild contains an \t{EAPI} assignment, the statement must
>> occur within the first 20 lines. An ebuild must not contain more
>> than one \t{EAPI} assignment.

> This still doesn't explain what should happen here:

>     inherit foo
>     EAPI=5

> There are two issues: which EAPI's 'inherit' behaviour is used, and
> what is the value of the $EAPI variable when sourcing foo.eclass?
> Eclasses seem to like doing $EAPI-dependent things...

Hm, the EAPI cannot be set to the probed value when sourcing the
ebuild. Otherwise, the sanity check could succeed in cases where it
should really fail. So I guess the current PMS wording still applies
here:

| The package manager must either pre-set the EAPI variable to 0 or
| ensure that it is unset before sourcing the ebuild for metadata
| generation. When using the ebuild for other purposes, the package
| manager must either pre-set EAPI to the value specified by the
| ebuild's metadata or ensure that it is unset.

Anyway, what's the usage case for having the EAPI assignment after the
inherit command? Currently this is forbidden.

Ulrich

Re: [gentoo-pms] EAPI specification in ebuilds

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 475 bytes --]

On Tue, 10 Apr 2012 22:13:09 +0200
Ulrich Mueller <ulm@gentoo.org> wrote:
> Anyway, what's the usage case for having the EAPI assignment after the
> inherit command? Currently this is forbidden.

It is? By the spec, or by QA policy? If it's just the latter, we can't
rely upon people following it.

I think this whole thing goes away if you also require in the spec
(i.e. PMS, not QA) that the EAPI assignment be done before anything
else.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-pms] EAPI specification in ebuilds

[Ulrich Mueller]

>>>>> On Tue, 10 Apr 2012, Ciaran McCreesh wrote:

> On Tue, 10 Apr 2012 22:13:09 +0200
> Ulrich Mueller <ulm@gentoo.org> wrote:
>> Anyway, what's the usage case for having the EAPI assignment after
>> the inherit command? Currently this is forbidden.

> It is? By the spec, or by QA policy? If it's just the latter, we can't
> rely upon people following it.

Policy. The devmanual says "if you want to override the EAPI variable,
you have to specify it at the top of the ebuild" and "eclasses may
have EAPI-conditional code".

> I think this whole thing goes away if you also require in the spec
> (i.e. PMS, not QA) that the EAPI assignment be done before anything
> else.

Agreed, that would be best. Now we only need a wording for "before
anything else" or "at the top of the ebuild" that is suitable for the
spec. ;-)

And if possible, it should be specified in a way that doesn't
invalidate ebuilds with harmless permutations like this:

   WANT_AUTOCONF="2.1"
   EAPI="3"

Ulrich

Re: [gentoo-pms] EAPI specification in ebuilds

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 996 bytes --]

On Tue, 10 Apr 2012 23:19:18 +0200
Ulrich Mueller <ulm@gentoo.org> wrote:
> Agreed, that would be best. Now we only need a wording for "before
> anything else" or "at the top of the ebuild" that is suitable for the
> spec. ;-)

If the EAPI is to be specified, it must be assigned to precisely once.
The assignment must occur within the first 20 lines of the ebuild, must
not be preceded by any lines other than blank lines or those that start
with optional whitespace followed by a # character, and the assignment
statement must match the following regular expression:

If EAPI is not specified in this way, the EAPI environment variable may
be either empty or set to the value "0".

> And if possible, it should be specified in a way that doesn't
> invalidate ebuilds with harmless permutations like this:
> 
>    WANT_AUTOCONF="2.1"
>    EAPI="3"

That would make it an utter pain in the ass to specify and verify...
How much does this right now?

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-pms] EAPI specification in ebuilds

[Ulrich Mueller]

>>>>> On Tue, 10 Apr 2012, Ciaran McCreesh wrote:

>> And if possible, it should be specified in a way that doesn't
>> invalidate ebuilds with harmless permutations like this:
>> 
>> WANT_AUTOCONF="2.1"
>> EAPI="3"

> That would make it an utter pain in the ass to specify and verify...
> How much does this right now?

If my script is working correctly, 130 ebuilds in the Portage tree
(which is less than I had expected). I haven't checked any overlays.

Ulrich

Re: [gentoo-pms] EAPI specification in ebuilds

[David Leverton]

On 10 April 2012 22:41, Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
> If the EAPI is to be specified, it must be assigned to precisely once.
> The assignment must occur within the first 20 lines of the ebuild, must
> not be preceded by any lines other than blank lines or those that start
> with optional whitespace followed by a # character, and the assignment
> statement must match the following regular expression:

Do we still need the 20 line limit if we're doing it this way?

Re: [gentoo-pms] EAPI specification in ebuilds

[Ulrich Mueller]

>>>>> On Wed, 11 Apr 2012, David Leverton wrote:

> On 10 April 2012 22:41, Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
>> If the EAPI is to be specified, it must be assigned to precisely once.
>> The assignment must occur within the first 20 lines of the ebuild, must
>> not be preceded by any lines other than blank lines or those that start
>> with optional whitespace followed by a # character, and the assignment
>> statement must match the following regular expression:

> Do we still need the 20 line limit if we're doing it this way?

I see no advantage in keeping the 20 line limit.

Ulrich

Re: [gentoo-pms] EAPI specification in ebuilds

[Zac Medico]

On 04/11/2012 01:04 AM, Ulrich Mueller wrote:
>>>>>> On Wed, 11 Apr 2012, David Leverton wrote:
> 
>> On 10 April 2012 22:41, Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
>>> If the EAPI is to be specified, it must be assigned to precisely once.
>>> The assignment must occur within the first 20 lines of the ebuild, must
>>> not be preceded by any lines other than blank lines or those that start
>>> with optional whitespace followed by a # character, and the assignment
>>> statement must match the following regular expression:
> 
>> Do we still need the 20 line limit if we're doing it this way?
> 
> I see no advantage in keeping the 20 line limit.

Yeah, it makes the 20 line limit redundant, and the only purpose of the
20 line limit was to support existing ebuilds in the wild.

Since the EAPI parsing behavior is optional for older EAPIs, package
managers can simply trigger a warning when they detect non-conformant
ebuilds in the wild, and eventually make it fatal after the community
has had sufficient warning.
-- 
Thanks,
Zac

Re: [gentoo-pms] EAPI specification in ebuilds

[David Leverton]

Zac Medico wrote:
> Since the EAPI parsing behavior is optional for older EAPIs, package
> managers can simply trigger a warning when they detect non-conformant
> ebuilds in the wild, and eventually make it fatal after the community
> has had sufficient warning.

Just to be clear, do you mean "if the PM doesn't detect an EAPI 
assignment during parsing but the EAPI is non-0 after generating the 
metadata, warn"? ¹  Or something else?

[1] As opposed to the already-decided "if the PM detects an EAPI line 
during parsing but the EAPI is something else after generating the 
metadata, fail" rule.

Re: [gentoo-pms] EAPI specification in ebuilds

[Zac Medico]

On 04/11/2012 11:02 AM, David Leverton wrote:
> Zac Medico wrote:
>> Since the EAPI parsing behavior is optional for older EAPIs, package
>> managers can simply trigger a warning when they detect non-conformant
>> ebuilds in the wild, and eventually make it fatal after the community
>> has had sufficient warning.
>
> Just to be clear, do you mean "if the PM doesn't detect an EAPI
> assignment during parsing but the EAPI is non-0 after generating the
> metadata, warn"? ¹ Or something else?
>
> [1] As opposed to the already-decided "if the PM detects an EAPI line
> during parsing but the EAPI is something else after generating the
> metadata, fail" rule.

The spec says "The package manager may optionally determine the \t{EAPI} 
of an ebuild by parsing it", which implies that you can implement a 
"pretend" mode that doesn't really determine the EAPI by parsing it, but 
parses it anyway and uses the result to generate deprecation warnings. A 
"pretend" mode like this is useful for initial deployment, in order to 
warn people with non-conformant ebuilds that are outside of the main tree.
--
Thanks,
Zac

Re: [gentoo-pms] EAPI specification in ebuilds

[Ulrich Mueller]

There seems to be consensus that the EAPI assignment should be in the
first non-blank non-comment line, and that the 20 lines limit should
be dropped. Please find a new iteration of the patch below.

Changes to previous version:
- Incorporated Ciaran's wording about position of EAPI assignment.
- Dropped 20 lines limit.
- Changed description of package manager's action accordingly. (Is it
  OK to simply say "first non-blank non-comment line" here?)
- Distinguish typographically between EAPI as a concept (roman type)
  and the EAPI environment variable (typewriter type).

Ulrich


--- a/ebuild-vars.tex
+++ b/ebuild-vars.tex
@@ -75,7 +75,7 @@ Ebuilds may define any of the following variables:
 
 \begin{description}
 \item[DEPEND] See section~\ref{sec:dependencies}.
-\item[EAPI] The EAPI. See below for defaults.
+\item[EAPI] The EAPI. See below.
 \item[PDEPEND] See section~\ref{sec:dependencies}.
 \item[RDEPEND] See section~\ref{sec:dependencies}. For some EAPIs, \t{RDEPEND} has special behaviour
     for its value if unset and when used with an eclass. See section~\ref{sec:rdepend-depend} for
@@ -124,6 +124,20 @@ ensure that it is unset.
 If any of these variables are set to invalid values, the package manager's behaviour is undefined;
 ideally, an error in one ebuild should not prevent operations upon other ebuilds or packages.
 
+If the EAPI is to be specified in an ebuild, the \t{EAPI} variable must be assigned to precisely
+once. The assignment must not be preceded by any lines other than blank lines or those that start
+with optional whitespace (spaces or tabs) followed by a \t{\#} character, and the line containing
+the assignment statement must match the following regular expression:
+\begin{verbatim}
+^[ \t]*EAPI=(['"]?)([A-Za-z0-9+_.-]*)\1[ \t]*(#.*)?$
+\end{verbatim}
+
+The package manager may optionally determine the EAPI of an ebuild by parsing its first non-blank
+and non-comment line, using above regular expression. If a recognised EAPI has been determined
+such, then the package manager must make sure that the \t{EAPI} value obtained by sourcing the
+ebuild with bash is identical to the EAPI obtained by parsing. The ebuild must be treated as
+invalid if these values are different.
+
 \subsection{\t{RDEPEND} value}
 \label{sec:rdepend-depend}
 

Re: [gentoo-pms] EAPI specification in ebuilds

[Zac Medico]

On 04/12/2012 04:55 AM, Ulrich Mueller wrote:
> There seems to be consensus that the EAPI assignment should be in the
> first non-blank non-comment line, and that the 20 lines limit should
> be dropped. Please find a new iteration of the patch below.

Here's a repoman patch:

http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=db49b97f8d9d855482ecdc9a5760d3d620948d1d
-- 
Thanks,
Zac

Re: [gentoo-pms] EAPI specification in ebuilds

[Ulrich Mueller]

>>>>> On Sun, 15 Apr 2012, Zac Medico wrote:

> Here's a repoman patch:

> http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=db49b97f8d9d855482ecdc9a5760d3d620948d1d

Thank you.

It might be better not to reference specific section numbers in PMS,
as these tend to change. Section 8.3.1 in eapi-4-approved-2011-01-17
has become section 7.3.1 in the current head.

Ulrich

Re: [gentoo-pms] EAPI specification in ebuilds

[David Leverton]

Ulrich Mueller wrote:
 > Please find a new iteration of the patch below.

Apologies for not noticing these sooner, but now that I come to 
implement it...

> +\begin{verbatim}
> +^[ \t]*EAPI=(['"]?)([A-Za-z0-9+_.-]*)\1[ \t]*(#.*)?$
> +\end{verbatim}

1) Should an empty value be allowed here?
2) If there's a trailing comment it needs to have at least one 
whitespace between it and the end of the value:

     [dleverton@shiny-one pms] $ EAPI=123#foo
     [dleverton@shiny-one pms] $ echo ${EAPI}
     123#foo

> +The package manager may optionally determine the EAPI of an ebuild by parsing its first non-blank
> +and non-comment line, using above regular expression. If a recognised EAPI has been determined
> +such, then the package manager must make sure that the \t{EAPI} value obtained by sourcing the
> +ebuild with bash is identical to the EAPI obtained by parsing. The ebuild must be treated as
> +invalid if these values are different.

3) If the first non-blank non-comment /doesn't/ match the magic regexp, 
should the PM insist that the post-source EAPI is 0 (or unset), or 
should it accept whatever it gets?

Re: [gentoo-pms] EAPI specification in ebuilds

[Ulrich Mueller]

>>>>> On Sun, 13 May 2012, David Leverton wrote:

> Apologies for not noticing these sooner, but now that I come to 
> implement it...

>> +\begin{verbatim}
>> +^[ \t]*EAPI=(['"]?)([A-Za-z0-9+_.-]*)\1[ \t]*(#.*)?$
>> +\end{verbatim}

> 1) Should an empty value be allowed here?

That was the intention, because the spec says that an empty value is
equivalent to 0.

> 2) If there's a trailing comment it needs to have at least one 
> whitespace between it and the end of the value:

>      [dleverton@shiny-one pms] $ EAPI=123#foo
>      [dleverton@shiny-one pms] $ echo ${EAPI}
>      123#foo

Darn. So we need another [ \t] in the last subexpression:

-^[ \t]*EAPI=(['"]?)([A-Za-z0-9+_.-]*)\1[ \t]*(#.*)?$
+^[ \t]*EAPI=(['"]?)([A-Za-z0-9+_.-]*)\1[ \t]*([ \t]#.*)?$

>> +The package manager may optionally determine the EAPI of an ebuild by parsing its first non-blank
>> +and non-comment line, using above regular expression. If a recognised EAPI has been determined
>> +such, then the package manager must make sure that the \t{EAPI} value obtained by sourcing the
>> +ebuild with bash is identical to the EAPI obtained by parsing. The ebuild must be treated as
>> +invalid if these values are different.

> 3) If the first non-blank non-comment /doesn't/ match the magic regexp, 
> should the PM insist that the post-source EAPI is 0 (or unset), or 
> should it accept whatever it gets?

The former. If the first non-blank non-comment doesn't match the
regexp, then the EAPI determined by parsing is 0 (or empty, which is
equivalent).

Do we need to clarify the wording here? (Patches are welcome.)

Ulrich

[gentoo-pms] [PATCH 1/2] Clarify wording for EAPI parsing

[David Leverton]

---
 ebuild-vars.tex |    5 +++--
 1 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/ebuild-vars.tex b/ebuild-vars.tex
index f3b72bf..58ca42e 100644
--- a/ebuild-vars.tex
+++ b/ebuild-vars.tex
@@ -133,8 +133,9 @@ the assignment statement must match the following regular expression:
 \end{verbatim}
 
 The package manager may optionally determine the EAPI of an ebuild by parsing its first non-blank
-and non-comment line, using above regular expression. If a recognised EAPI has been determined
-such, then the package manager must make sure that the \t{EAPI} value obtained by sourcing the
+and non-comment line, using the above regular expression. If it matches, the \t{EAPI} is the
+substring matched by the capturing parentheses (\t{0} if empty), otherwise it is \t{0}.
+The package manager must make sure that the \t{EAPI} value obtained by sourcing the
 ebuild with bash is identical to the EAPI obtained by parsing. The ebuild must be treated as
 invalid if these values are different.
 
-- 
1.7.3.4

Re: [gentoo-pms] [PATCH 1/2] Clarify wording for EAPI parsing

[Ulrich Mueller]

>>>>> On Sun, 13 May 2012, David Leverton wrote:
 
>  The package manager may optionally determine the EAPI of an ebuild by parsing its first non-blank
> -and non-comment line, using above regular expression. If a recognised EAPI has been determined
> -such, then the package manager must make sure that the \t{EAPI} value obtained by sourcing the
> +and non-comment line, using the above regular expression. If it matches, the \t{EAPI} is the
> +substring matched by the capturing parentheses (\t{0} if empty), otherwise it is \t{0}.
> +The package manager must make sure that the \t{EAPI} value obtained by sourcing the
>  ebuild with bash is identical to the EAPI obtained by parsing. The ebuild must be treated as
>  invalid if these values are different.

Hm, the condition "If a recognised EAPI has been determined such,
then ..." is important and shouldn't be removed.

If the result of parsing is an EAPI not known to the package manager,
then the ebuild must not be sourced (but be rejected immediately).

Ulrich

Re: [gentoo-pms] [PATCH 1/2] Clarify wording for EAPI parsing

[David Leverton]

Ulrich Mueller wrote:
 > Hm, the condition "If a recognised EAPI has been determined such,
 > then ..." is important and shouldn't be removed.

I took that out because the PM should treat the parsed EAPI as 
authoritative whether or not it's recognised...

 > If the result of parsing is an EAPI not known to the package manager,
 > then the ebuild must not be sourced (but be rejected immediately).

... which is basically what you're saying here, but maybe you want 
explicit wording saying that the sourcing should be skipped in such a case?

Re: [gentoo-pms] [PATCH 1/2] Clarify wording for EAPI parsing

[Ulrich Mueller]

>>>>> On Sun, 13 May 2012, David Leverton wrote:

>> Hm, the condition "If a recognised EAPI has been determined such,
>> then ..." is important and shouldn't be removed.

> I took that out because the PM should treat the parsed EAPI as 
> authoritative whether or not it's recognised...

>> If the result of parsing is an EAPI not known to the package manager,
>> then the ebuild must not be sourced (but be rejected immediately).

> ... which is basically what you're saying here, but maybe you want 
> explicit wording saying that the sourcing should be skipped in such a case?

Updated patch is included below, as discussed on IRC.

Ulrich


From 9d8525e70b808f200cde7ea39f028ef1af883e38 Mon Sep 17 00:00:00 2001
From: David Leverton <levertond@googlemail.com>
Date: Sun, 13 May 2012 20:11:39 +0100
Subject: [PATCH] Clarify wording for EAPI parsing.

Clarify wording. Require a space before the trailing comment.
---
 ebuild-vars.tex |    7 ++++---
 1 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/ebuild-vars.tex b/ebuild-vars.tex
index f3b72bf..d39c491 100644
--- a/ebuild-vars.tex
+++ b/ebuild-vars.tex
@@ -129,12 +129,13 @@ once. The assignment must not be preceded by any lines other than blank lines or
 with optional whitespace (spaces or tabs) followed by a \t{\#} character, and the line containing
 the assignment statement must match the following regular expression:
 \begin{verbatim}
-^[ \t]*EAPI=(['"]?)([A-Za-z0-9+_.-]*)\1[ \t]*(#.*)?$
+^[ \t]*EAPI=(['"]?)([A-Za-z0-9+_.-]*)\1[ \t]*([ \t]#.*)?$
 \end{verbatim}
 
 The package manager may optionally determine the EAPI of an ebuild by parsing its first non-blank
-and non-comment line, using above regular expression. If a recognised EAPI has been determined
-such, then the package manager must make sure that the \t{EAPI} value obtained by sourcing the
+and non-comment line, using the above regular expression. If it matches, the EAPI is the substring
+matched by the capturing parentheses (\t{0} if empty), otherwise it is \t{0}. For a recognised 
+EAPI, the package manager must make sure that the \t{EAPI} value obtained by sourcing the
 ebuild with bash is identical to the EAPI obtained by parsing. The ebuild must be treated as
 invalid if these values are different.
 
-- 
1.7.8.6

Re: [gentoo-pms] [PATCH 1/2] Clarify wording for EAPI parsing

[Ulrich Mueller]

>>>>> On Sun, 13 May 2012, Ulrich Mueller wrote:

>>>>> On Sun, 13 May 2012, David Leverton wrote:
>>> Hm, the condition "If a recognised EAPI has been determined such,
>>> then ..." is important and shouldn't be removed.

>> I took that out because the PM should treat the parsed EAPI as 
>> authoritative whether or not it's recognised...

>>> If the result of parsing is an EAPI not known to the package manager,
>>> then the ebuild must not be sourced (but be rejected immediately).

>> ... which is basically what you're saying here, but maybe you want 
>> explicit wording saying that the sourcing should be skipped in such a case?

> Updated patch is included below, as discussed on IRC.

Committed.

Ulrich

[gentoo-pms] [PATCH 2/2] Require a space before the trailing comment

[David Leverton]

---
 ebuild-vars.tex |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/ebuild-vars.tex b/ebuild-vars.tex
index 58ca42e..60c7b04 100644
--- a/ebuild-vars.tex
+++ b/ebuild-vars.tex
@@ -129,7 +129,7 @@ once. The assignment must not be preceded by any lines other than blank lines or
 with optional whitespace (spaces or tabs) followed by a \t{\#} character, and the line containing
 the assignment statement must match the following regular expression:
 \begin{verbatim}
-^[ \t]*EAPI=(['"]?)([A-Za-z0-9+_.-]*)\1[ \t]*(#.*)?$
+^[ \t]*EAPI=(['"]?)([A-Za-z0-9+_.-]*)\1[ \t]*([ \t]#.*)?$
 \end{verbatim}
 
 The package manager may optionally determine the EAPI of an ebuild by parsing its first non-blank
-- 
1.7.3.4