[gentoo-dev] RFC: an eclass for github snapshots?

[gentoo-dev] RFC: an eclass for github snapshots?

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1470 bytes --]

Hello,

Right now, a quick 'grep -l github.*tarball' shows that there are about
147 ebuilds in portage using github snapshots. This evaluates to 83
different packages.

The problem with github is that it suffixes the tarballs with
a complete git commit id. This means that the `S' variable
in the ebuild needs to refer to a long hash changing randomly. Right
now, the problem is handled in a number of ways:

1) (from app-admin/rudy)

S="${WORKDIR}/solutious-${PN}-*"

I'm surprised if that actually works. I'd say that seems not
PMS-compliant but in fact PMS seems to almost not mention S.

2) (app-emacs/calfw and suggested solution for Sunrise)

src_unpack() {
	unpack ${A}
	mv kiwanami-emacs-calfw-* ${P} || die
}

3) (app-misc/bgrep)

GITHUB_USER="tmbinc"
GITHUB_HASH="49b098be9548d174023ad05c10f6af9d02b8e18e"
MY_P="${GITHUB_USER}-${PN}-${GITHUB_HASH:0:7}"
S="${WORKDIR}/${MY_P}"

4) (app-misc/tmux-mem-cpu-load)

src_prepare() {
	cd "${WORKDIR}"/thewtex-${PN}-*
	S=$(pwd)
}


What I'd like to do is creating a small github.eclass, encapsulating
a common, nice way of handling the S issue. I guess the best solution
would be to git with something like 2) above, with the eclass providing
github_src_unpack() for EAPIs 2+.

Maybe the eclass could be extended to support other kinds of snapshots.
I'm not sure if there aren't other services providing snapshots similar
to github.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

[gentoo-dev] Re: RFC: an eclass for github snapshots?

[Diego Elio Pettenò]

Il giorno lun, 30/05/2011 alle 08.27 +0200, Michał Górny ha scritto:
> 
> S="${WORKDIR}/solutious-${PN}-*"
> 
> I'm surprised if that actually works. I'd say that seems not
> PMS-compliant but in fact PMS seems to almost not mention S. 

Because you didn't follow the whole eclass tree ;)

ruby-ng handles the star as a special case, given how many of those we
had to use over time, so it is that, expanding SRC_URI, not the package
manager.

-- 
Diego Elio Pettenò — Flameeyes
http://blog.flameeyes.eu/

[gentoo-dev] Re: RFC: an eclass for github snapshots?

[Ulrich Mueller]

>>>>> On Mon, 30 May 2011, Diego Elio Pettenò wrote:
> Il giorno lun, 30/05/2011 alle 08.27 +0200, Michał Górny ha scritto:
>> S="${WORKDIR}/solutious-${PN}-*"
>> 
>> I'm surprised if that actually works. I'd say that seems not
>> PMS-compliant but in fact PMS seems to almost not mention S. 

> Because you didn't follow the whole eclass tree ;)

> ruby-ng handles the star as a special case, given how many of those
> we had to use over time, [...]

But it is not compliant with PMS:
"If S is assigned in the global scope of an ebuild, then the
restrictions of section 12.2 for global variables apply." (section 12.1)
"Global variables must only contain invariant values." (section 12.2)

Ulrich

[gentoo-dev] Re: RFC: an eclass for github snapshots?

[Diego Elio Pettenò]

Il giorno lun, 30/05/2011 alle 13.21 +0200, Ulrich Mueller ha scritto:
> 
> But it is not compliant with PMS:
> "If S is assigned in the global scope of an ebuild, then the
> restrictions of section 12.2 for global variables apply." (section
> 12.1)
> "Global variables must only contain invariant values." (section 12.2)

It is fixed for EAPI=4, but we didn't want to break the previous API
that worked quite fine for over an year.

-- 
Diego Elio Pettenò — Flameeyes
http://blog.flameeyes.eu/

Re: [gentoo-dev] Re: RFC: an eclass for github snapshots?

[Donnie Berkholz]

[-- Attachment #1: Type: text/plain, Size: 1251 bytes --]

On 13:21 Mon 30 May     , Ulrich Mueller wrote:
> >>>>> On Mon, 30 May 2011, Diego Elio Pettenò wrote:
> > Il giorno lun, 30/05/2011 alle 08.27 +0200, Michał Górny ha scritto:
> >> S="${WORKDIR}/solutious-${PN}-*"
> >> 
> >> I'm surprised if that actually works. I'd say that seems not
> >> PMS-compliant but in fact PMS seems to almost not mention S. 
> 
> > Because you didn't follow the whole eclass tree ;)
> 
> > ruby-ng handles the star as a special case, given how many of those
> > we had to use over time, [...]
> 
> But it is not compliant with PMS:
> "If S is assigned in the global scope of an ebuild, then the
> restrictions of section 12.2 for global variables apply." (section 12.1)
> "Global variables must only contain invariant values." (section 12.2)

It seems compliant to me, as S is assigned an invariant value that 
happens to contain the character '*', which is overwritten with a new 
value as a local variable in ebuild functions. Sample code in listing 
12.1 in my copy of the PMS seems to suggest this is perfectly fine 
behavior as long as the global invariant is restored after each 
function.

-- 
Thanks,
Donnie

Donnie Berkholz
Sr. Developer, Gentoo Linux
Blog: http://dberkholz.com

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Re: RFC: an eclass for github snapshots?

[Ulrich Mueller]

>>>>> On Tue, 7 Jun 2011, Donnie Berkholz wrote:

>> But it is not compliant with PMS:
>> "If S is assigned in the global scope of an ebuild, then the
>> restrictions of section 12.2 for global variables apply." (section 12.1)
>> "Global variables must only contain invariant values." (section 12.2)

> It seems compliant to me, as S is assigned an invariant value that 
> happens to contain the character '*', which is overwritten with a new 
> value as a local variable in ebuild functions. Sample code in listing 
> 12.1 in my copy of the PMS seems to suggest this is perfectly fine 
> behavior as long as the global invariant is restored after each 
> function.

Even if it fulfills the restrictions for global variables, it is still
an abuse of the spec, because PMS defines S as follows:
"The full path to the temporary build directory, used by src_compile,
src_install etc."

And for EAPI 4 it will fail, because S is required to exist as initial
working directory in most src_* phase functions.

Ulrich

Re: [gentoo-dev] Re: RFC: an eclass for github snapshots?

[Hans de Graaff]

[-- Attachment #1: Type: text/plain, Size: 854 bytes --]

On Tue, 2011-06-07 at 20:16 +0200, Ulrich Mueller wrote:

> Even if it fulfills the restrictions for global variables, it is still
> an abuse of the spec, because PMS defines S as follows:
> "The full path to the temporary build directory, used by src_compile,
> src_install etc."

I don't see how setting S violates this specification. For each ruby
implementation that we build for the definition of S holds. It just has
a different value for each implementation.

> And for EAPI 4 it will fail, because S is required to exist as initial
> working directory in most src_* phase functions.

Correct, so in EAPI 4 we now set the RUBY_S variable to handle the
initial setup, and then we set S as part of the environment setup when
we are iterating through each ruby implementation within each of the PMS
phases.

Kind regards,

Hans

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 230 bytes --]

Re: [gentoo-dev] Re: RFC: an eclass for github snapshots?

[Ulrich Mueller]

>>>>> On Wed, 08 Jun 2011, Hans de Graaff wrote:

>> Even if it fulfills the restrictions for global variables, it is
>> still an abuse of the spec, because PMS defines S as follows:
>> "The full path to the temporary build directory, used by
>> src_compile, src_install etc."

> I don't see how setting S violates this specification. For each ruby
> implementation that we build for the definition of S holds. It just
> has a different value for each implementation.

The value of S that is assigned in global scope (i.e. the one
containing the wildcard) violates it.

Ulrich

Re: [gentoo-dev] Re: RFC: an eclass for github snapshots?

[Hans de Graaff]

[-- Attachment #1: Type: text/plain, Size: 582 bytes --]

On Wed, 2011-06-08 at 09:17 +0200, Ulrich Mueller wrote:

> The value of S that is assigned in global scope (i.e. the one
> containing the wildcard) violates it.

Ah, right, I initially read Donnie's quotation from PMS as an
endorsement for our approach, but that is only true for our EAPI=4
solution where we only change S within a function.

That leaves the question what to do with the approach for EAPI=2,3. I'd
rather not risk breaking ebuilds by removing this support just for a
violation of PMS if there is no real problem exposed by it.

Kind regards,

Hans

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 230 bytes --]

Re: [gentoo-dev] Re: RFC: an eclass for github snapshots?

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 987 bytes --]

On Wed, 08 Jun 2011 17:43:38 +0200
Hans de Graaff <graaff@gentoo.org> wrote:
> That leaves the question what to do with the approach for EAPI=2,3.
> I'd rather not risk breaking ebuilds by removing this support just
> for a violation of PMS if there is no real problem exposed by it.

The 'invariant' restriction on S in PMS is, strictly speaking, stronger
than it has to be. However, working out exactly what set of weaker
rules would be ok proved to be too difficult -- historically, Portage
has had various different behaviours for global scope variables that
are assigned variable values. Thus, PMS is the way it is there because
we know for sure that if you follow those rules you're safe; if you
don't, you'll definitely cause problems for EAPI 4, and you may or may
not get away with it for earlier EAPIs.

It's a bit like assuming that it's ok to dereference a null pointer
and get a zero, since that's what one particular system does...

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Re: RFC: an eclass for github snapshots?

[Hans de Graaff]

[-- Attachment #1: Type: text/plain, Size: 1356 bytes --]

On Wed, 2011-06-08 at 16:50 +0100, Ciaran McCreesh wrote:
> On Wed, 08 Jun 2011 17:43:38 +0200
> Hans de Graaff <graaff@gentoo.org> wrote:
> > That leaves the question what to do with the approach for EAPI=2,3.
> > I'd rather not risk breaking ebuilds by removing this support just
> > for a violation of PMS if there is no real problem exposed by it.
> 
> The 'invariant' restriction on S in PMS is, strictly speaking, stronger
> than it has to be. However, working out exactly what set of weaker
> rules would be ok proved to be too difficult -- historically, Portage
> has had various different behaviours for global scope variables that
> are assigned variable values. Thus, PMS is the way it is there because
> we know for sure that if you follow those rules you're safe; if you
> don't, you'll definitely cause problems for EAPI 4, and you may or may
> not get away with it for earlier EAPIs.
> 
> It's a bit like assuming that it's ok to dereference a null pointer
> and get a zero, since that's what one particular system does...

Thanks for the background on this particular part of the specification.

I think I'll add an eqawarn to the eclass for EAPI=2,3 and migrate
ebuilds over naturally. I'll bump the remaining ones in 6 months or so.
That also gives overlays some time to move to EAPI=4.

Kind regards,

Hans


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 230 bytes --]