[gentoo-council] Comparison of GLEP 54 and 'live ebuild' proposal

[gentoo-council] Comparison of GLEP 54 and 'live ebuild' proposal

[Thomas Anderson]

[-- Attachment #1: Type: text/plain, Size: 328 bytes --]

Hi,

    Attached is my comparison of the two proposals for live sources.
    Sorry about getting it out late, I had to get ahold of a number of
    people to finish writing it up.

Cheers,
Thomas
-- 
---------
Thomas Anderson
Gentoo Developer
/////////
Areas of responsibility:
AMD64, Secretary to the Gentoo Council
---------

[-- Attachment #2: glep54comp.txt --]
[-- Type: text/plain, Size: 2847 bytes --]

GLEP54:
    This proposal attempts to have proper ordering with all version 
    parts(0.2-scm should be higher than 0-2.1) while providing information to
    the package manager and user that this package should be treated like an
    ebuild with live sources. This proposal has the added advantage that it's
    already supported in at least one package manager so it's behaviour is
    defined and predictable.

    The only real objection to this proposal is that some don't like the
    version component name('scm'). No technical objections have been made,
    other than that it "doesn't go far enough".

    One thing immediately found lacking in this proposal is the lack of time
    of installation or the revision used. As stated in the GLEP however, this
    information can be exposed in other ways by the underlying scm tool. The
    specific revision however should not be in the package version but
    otherwise available.

Live Ebuild:
    The liveebuild proposal attempts to give both proper ordering and an idea
    of point of time when the ebuild was installed, in other words,
    'traceability' of live package. A keyword 'live' in the
    package version part of the filename is expanded to the ISO
    date(YYYYMMDD). One difference with GLEP54 is that while package managers
    supporting GLEP54 can on that information alone determine if a package is
    a live-sources package, with the live ebuild method one needs
    PROPERTIES=live as well(once the version is expanded one cannot know if it
    was a normal ebuild or a .live ebuild).

    A problem with liveebuild is that there has been no real specification
    other than the outline provided. A consequence is that there has not been
    consideration of what happens when you have two ebuilds:
    	mythtv-0.20_prelive
	mythtv-0.20_pre20090309(today's date, which live would expand to)
    I've not seen any discussion or explanation of what would happen. This
    probably needs to be sorted out before this proposal can be accepted.
    This cannot be easily solved by saying "No two ebuilds in the same package
    can have the same version components because we never know what _live will
    expand to.

    One important issue is what happens in the following
    scenario:
    	1) world update starts at 20090301@2200hrs.
	2) this particular update involves 100 packages so it takes quite
	along time
	3) The _live package is not reached until 20090302 at 1AM.

     Is the package installed as 20090301 or 20090302?

     Similar to the above problem is what occurs when a user understandably
     puts =media-tv/mythtv-0.20_20090301 in package.{use,keywords} and the
     date changes. Also, what happens if the user =media-tv/mythtv-0.20.live
     in package.{use,keywords}? Is live expanded that early so it is invalid
     or is it still valid?

Re: [gentoo-council] Comparison of GLEP 54 and 'live ebuild' proposal

[Roy Bamford]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 2009.03.09 22:47, Thomas Anderson wrote:
> Hi,
> 
>     Attached is my comparison of the two proposals for live sources.
>     Sorry about getting it out late, I had to get ahold of a number 
> of
>     people to finish writing it up.
> 
> Cheers,
> Thomas
> -- 
> ---------
> Thomas Anderson
> Gentoo Developer
> /////////
> Areas of responsibility:
> AMD64, Secretary to the Gentoo Council
> ---------
> 
> 

- ------quoted attachment "glep54comp.txt"------
[snip]
> 
>     One important issue is what happens in the following
>     scenario:
>     	1) world update starts at 20090301@2200hrs.
> 	2) this particular update involves 100 packages so it takes
> quite
> 	along time
> 	3) The _live package is not reached until 20090302 at 1AM.
> 
>      Is the package installed as 20090301 or 20090302?
> 
[snip]

> 
Thomas,

Live has to expand to the date when the sources were fetched, otherwise 
its not 'live' by definition.
As an illustration, I install KDE 4.2 on my 25MHz 486DX with 64Mb RAM.
During the time it takes to build, 'live' is likely to have changed 
several times.

How do you handle prefetching of sources, or do you forbid 
prefetching ?
Live infers you fetch the sources at the time you need to build them 
and do the live expansion at that time. Without that you don't know how 
old your live version is.

live can change several times a day. With only one day resolution, how 
do you handle that?

- -- 
Regards,

Roy Bamford
(NeddySeagoon) a member of
gentoo-ops
forum-mods
treecleaners
trustees
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.10 (GNU/Linux)

iEYEARECAAYFAkm1pl4ACgkQTE4/y7nJvatrEACg6JF/DcOjL65KJG7XL6L1AzIx
WToAoL+JgmILR6rsxNn4G/jhnk+thcGv
=g/qP
-----END PGP SIGNATURE-----

Re: [gentoo-council] Comparison of GLEP 54 and 'live ebuild' proposal

[Thomas Anderson]

[-- Attachment #1: Type: text/plain, Size: 795 bytes --]

On Mon, Mar 09, 2009 at 11:29:29PM +0000, Roy Bamford wrote:
> live can change several times a day. With only one day resolution, how 
> do you handle that?


Hi Roy,

   I'm not entirely sure you can handle that with liveebuild. In my opinion the correct
   thing to do is not to attempt to put the revision in the package
   version but to have it exposed in some other way(say a metadata
   variable holding the installed revision). Otherwise you're just going
   to end up trying to get more precise and more precise times.

   Really, you shouldnt be putting dates in place of revisions in the
   package version. It's just a bad idea.
-- 
---------
Thomas Anderson
Gentoo Developer
/////////
Areas of responsibility:
AMD64, Secretary to the Gentoo Council
---------

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: [gentoo-council] Comparison of GLEP 54 and 'live ebuild' proposal

[Luca Barbato]

Roy Bamford wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On 2009.03.09 22:47, Thomas Anderson wrote:
>> Hi,
>>
>>     Attached is my comparison of the two proposals for live sources.
>>     Sorry about getting it out late, I had to get ahold of a number 
>> of
>>     people to finish writing it up.
>>
>> Cheers,
>> Thomas
>> -- 
>> ---------
>> Thomas Anderson
>> Gentoo Developer
>> /////////
>> Areas of responsibility:
>> AMD64, Secretary to the Gentoo Council
>> ---------
>>
>>
> 
> - ------quoted attachment "glep54comp.txt"------
> [snip]
>>     One important issue is what happens in the following
>>     scenario:
>>     	1) world update starts at 20090301@2200hrs.
>> 	2) this particular update involves 100 packages so it takes
>> quite
>> 	along time
>> 	3) The _live package is not reached until 20090302 at 1AM.
>>
>>      Is the package installed as 20090301 or 20090302?
>>
> [snip]
> 
> Thomas,
> 
> Live has to expand to the date when the sources were fetched, otherwise 
> its not 'live' by definition.
> As an illustration, I install KDE 4.2 on my 25MHz 486DX with 64Mb RAM.
> During the time it takes to build, 'live' is likely to have changed 
> several times.

It isn't exactly a problem (more will follow)

> 
> How do you handle prefetching of sources, or do you forbid 
> prefetching ?

live template ebuild require supporting src_fetch among the other stuff. 
Keep in mind that once you get  an ebuild from the template you can 
use&reuse it as a normal ebuild (so it works like the mythtv "not so 
live" ebuilds using svn on fixed revision)

> Live infers you fetch the sources at the time you need to build them 
> and do the live expansion at that time. Without that you don't know how 
> old your live version is.
> 
> live can change several times a day. With only one day resolution, how 
> do you handle that?

What is in the draft you can find on 
http://dev.gentoo.org/~lu_zero/glep/liveebuild.rst

"
Resolution and Version Comparison
---------------------------------

At resolution the live keyword is substituted with a timestamp in the 
form of
iso date (``YYYYMMDDhhmm``) and the version comparison follows the normal
version comparison rules.
"

Once you trigger the template -> ebuild generation you are working with 
a snapshot for all what concerns portage.

So once you start you are set.

lu

-- 

Luca Barbato
Gentoo Council Member
Gentoo/linux Gentoo/PPC
http://dev.gentoo.org/~lu_zero

Re: [gentoo-council] Comparison of GLEP 54 and 'live ebuild' proposal

[Luca Barbato]

Thomas Anderson wrote:
> Hi,
> 
>     Attached is my comparison of the two proposals for live sources.
>     Sorry about getting it out late, I had to get ahold of a number of
>     people to finish writing it up.

I'd be happier if you actually provided it with a better description 
and/or updated drafts along.

The glep54 doesn't state anything about how/where the specific revision 
is stored nor what the live property is and it implicitly 
provides/triggers in the package manager.

The main technical objection could be stated as "does nothing beside 
giving a token to describe infinity for a version component as version 
suffix".

The actual draft for the live templates is present at

http://dev.gentoo.org/~lu_zero/glep/liveebuild.rst

And should cover the concern you raised here beside the following

 > Similar to the above problem is what occurs when a user understandably
 > puts =media-tv/mythtv-0.20_20090301 in package.{use,keywords} and the
 > date changes. Also, what happens if the user
 > =media-tv/mythtv-0.20.live in package.{use,keywords}? Is live expanded
 > that early so it is invalid or is it still valid?

Having =cat/pkg-ver.live in package.{use, keywords} would translate to a 
sort of =cat/pkg-ver* but would be nicer putting directly an isodate to 
restrict better what you want in and what you want out.

lu

-- 

Luca Barbato
Gentoo Council Member
Gentoo/linux Gentoo/PPC
http://dev.gentoo.org/~lu_zero

Re: [gentoo-council] Comparison of GLEP 54 and 'live ebuild' proposal

[Thomas Anderson]

[-- Attachment #1: Type: text/plain, Size: 2422 bytes --]

On Tue, Mar 10, 2009 at 03:26:39AM +0100, Luca Barbato wrote:
> Thomas Anderson wrote:
>> Hi,
>>     Attached is my comparison of the two proposals for live sources.
>>     Sorry about getting it out late, I had to get ahold of a number of
>>     people to finish writing it up.
>
> I'd be happier if you actually provided it with a better description and/or 
> updated drafts along.

As per the Council summary we were suppose to write up a comparison of
the advantages/disadvantages of both. It was not in the summary that I
had to update the Glep as well as write a comparison.
>
> The glep54 doesn't state anything about how/where the specific revision is 
> stored nor what the live property is and it implicitly provides/triggers in 
> the package manager.

For one, the live property is rendered useless with glep54. Secondly,
the glep does state that those are outside the scope of this particular
glep, but can later be implemented once this goes through. Doug and I
had a conversation about this yesterday, and glep54 is the first
incremental step.

> The main technical objection could be stated as "does nothing beside giving 
> a token to describe infinity for a version component as version suffix".

That's not a technical objection in my opinion. That's an objection that
the doc doesn't go far enough, to which the answer is that it's the
first step. Just because the first step doesn't go as far as some would
like isn't a reason to take the first step.

> > Similar to the above problem is what occurs when a user understandably
> > puts =media-tv/mythtv-0.20_20090301 in package.{use,keywords} and the
> > date changes. Also, what happens if the user
> > =media-tv/mythtv-0.20.live in package.{use,keywords}? Is live expanded
> > that early so it is invalid or is it still valid?
>
> Having =cat/pkg-ver.live in package.{use, keywords} would translate to a 
> sort of =cat/pkg-ver* but would be nicer putting directly an isodate to 
> restrict better what you want in and what you want out.

Hm, so according the the wildcard way:

Keywording media-tv/mythtv-live in package.keywords keywords every
single version of mythtv!? I'm guessing that's not a very intuitive
method. Also, that applies to package.use too.

-- 
---------
Thomas Anderson
Gentoo Developer
/////////
Areas of responsibility:
AMD64, Secretary to the Gentoo Council
---------

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: [gentoo-council] Comparison of GLEP 54 and 'live ebuild' proposal

[Thomas Anderson]

On Tue, Mar 10, 2009 at 10:00:03AM -0400, Thomas Anderson wrote:
> On Tue, Mar 10, 2009 at 03:26:39AM +0100, Luca Barbato wrote:
> > Thomas Anderson wrote:
> That's not a technical objection in my opinion. That's an objection that
> the doc doesn't go far enough, to which the answer is that it's the
> first step. Just because the first step doesn't go as far as some would
> like isn't a reason to take the first step.

Make that "isn't a reason not to take the first step". *sigh*

-- 
---------
Thomas Anderson
Gentoo Developer
/////////
Areas of responsibility:
AMD64, Secretary to the Gentoo Council
---------

Re: [gentoo-council] Comparison of GLEP 54 and 'live ebuild' proposal

[Luca Barbato]

Thomas Anderson wrote:
> On Tue, Mar 10, 2009 at 03:26:39AM +0100, Luca Barbato wrote:
>> Thomas Anderson wrote:
>>> Hi,
>>>     Attached is my comparison of the two proposals for live sources.
>>>     Sorry about getting it out late, I had to get ahold of a number of
>>>     people to finish writing it up.
>> I'd be happier if you actually provided it with a better description and/or 
>> updated drafts along.
> 
> As per the Council summary we were suppose to write up a comparison of
> the advantages/disadvantages of both. It was not in the summary that I
> had to update the Glep as well as write a comparison.

having the updated drafts would be useful to highlight better and state 
what probably is lost in the countless mail threads.

>> The glep54 doesn't state anything about how/where the specific revision is 
>> stored nor what the live property is and it implicitly provides/triggers in 
>> the package manager.
> 
> For one, the live property is rendered useless with glep54.

Try to explain why without defining it or at least tell what's supposed 
to provide. Glep54 doesn't state anything about it.

> Secondly,
> the glep does state that those are outside the scope of this particular
> glep, but can later be implemented once this goes through. Doug and I
> had a conversation about this yesterday, and glep54 is the first
> incremental step.

That should be stated in the glep and should help knowing the other 
steps (see below why)

>> The main technical objection could be stated as "does nothing beside giving 
>> a token to describe infinity for a version component as version suffix".
> 
> That's not a technical objection in my opinion. That's an objection that
> the doc doesn't go far enough, to which the answer is that it's the
> first step. Just because the first step doesn't go as far as some would
> like isn't a reason to take the first step.

first step -> "does nothing, but you need to change the eapi in a pretty 
radical way"

I cannot disagree with people that are against it either because they 
don't use even -9999 or they consider worthless doing anything since 
they are about 0.003 of portage and shouldn't be used at all by common 
users. The technical objection is about the effort/usefulness ratio.

If the usefulness is next to 0 the ratio goes next to too big quite easily.

>>> Similar to the above problem is what occurs when a user understandably
>>> puts =media-tv/mythtv-0.20_20090301 in package.{use,keywords} and the
>>> date changes. Also, what happens if the user
>>> =media-tv/mythtv-0.20.live in package.{use,keywords}? Is live expanded
>>> that early so it is invalid or is it still valid?
>> Having =cat/pkg-ver.live in package.{use, keywords} would translate to a 
>> sort of =cat/pkg-ver* but would be nicer putting directly an isodate to 
>> restrict better what you want in and what you want out.
> 
> Hm, so according the the wildcard way:
> 
> Keywording media-tv/mythtv-live in package.keywords keywords every
> single version of mythtv!?

Let me explain better:

Having =media-tv/mythtv-1.2.3.live would always let you unmask or define 
useflags for whatever is the ebuild that is resolved. So it works as 
you'd expect.
That also means that having =media-tv/mythtv-1.2.3.live will apply to 
the whole set of =media-tv/mythtv-1.2.3.{ebuilds you installed using 
that template}.

In that regard is a sort of =cat/pkg-ver* since it covers a list of 
ebuild and not just one (consider that you can issue re-emerge of 
ebuilds and you want to keep the same useflag set you'd expect)

I'll update the draft with those two specific usecases (package.* files 
and behaviour with a long list of applications to be emerged).

lu

BTW: what happens when you take much time to emerge a set of -scm 
ebuilds? Assuming that in a determined timespan the sources are more or 
less stable, once you add up hours between merges the chance you have 
glitches raises a lot.
With live templates you can use emerge -f to reduce the risk with -9999 
and -scm you should be quite careful if you have large sets and upstream 
is quite active.



-- 

Luca Barbato
Gentoo Council Member
Gentoo/linux Gentoo/PPC
http://dev.gentoo.org/~lu_zero