[gentoo-council] Re: Comparison of GLEP 54 and 'live ebuild' proposal

[Ryan Hill <dirtyepic@gentoo.org>]

[-- Attachment #1: Type: text/plain, Size: 8515 bytes --]

On Wed, 11 Mar 2009 10:55:52 +0100
Luca Barbato <lu_zero@gentoo.org> wrote:

> Ryan Hill wrote:
> > I have some questions about the -live proposal.  I'm sorry if some
> > of this has been answered already; I haven't had the opportunity to
> > follow it more closely.
> > 
> > The draft (http://dev.gentoo.org/~lu_zero/glep/liveebuild.rst) says
> > that "At resolution the live keyword is substituted with a
> > timestamp in the form of iso date".  What is meant by "resolution"
> > here?  Does this mean that, having a gcc-4.4.0_prelive ebuild,
> > 'emerge -p gcc' would show something like:
> > 
> > [ebuild     U ] sys-devel/gcc-4.4.0_pre20090310
> > 
> > If so, is there any way to identify that this is a live ebuild?
> 
> The ebuild itself has some embedded information so portage
> could/should provide something like.
> 
> [ebuild     U ] sys-devel/gcc-4.4.0_pre20090310 [from svn master
> r12345]

As you said in another mail, this could only be shown for a rebuild,
but not an update, true?

> 
> > If I have an eclass that needs to do stuff to only live ebuilds
> > (like kde4-base.eclass setting SLOT=live when PV is 9999), how can I
> > differentiate between live ebuilds and snapshots?  Do eclasses see
> > -live or the expanded datestamp in PV?
> 
> it see the expanded datestamp but sees also LIVE_URI LIVE_REVISION 
> LIVE_BRANCH (the variables that keep track of the exact revision of
> the sources you are going to use), so you can also slot by branch
> 
> SLOT=${LIVE_BRANCH}

Yeah, immediately after posting I remembered there would be certain
variables present.  Even if they weren't you could just stick a
LIVE="yes" in the live ebuilds or check PROPERTIES or whatever.

> > How do I know if a user has a live ebuild installed when they file a
> > bug without having to check if there's a snapshot with that date in
> > the tree every single time the PV has a datestamp in it?  (minor
> > gripe admittedly)
> 
> He will provide the revision and branch so you have more information
> not less.
> 
> > If I build a live package today, will I see it as an update when
> > running emerge -pu @world tomorrow?
> 
> the template will be evaluated again so you get another snapshot 
> proposed for update.

Ooh, i really don't like that.  This would force me to either do daily
updates manually package by package or mask any live ebuilds before i
do.  That would _really_ suck if I were still running the kde-crazy
overlay and had 50+ live ebuilds installed.

> > If I have 20090309 installed what does 'emerge
> > =gcc-4.4.0_pre20090309' do tomorrow?  (It might be a neat trick to
> > disable fetch and just rebuild the current checkout in this case.)
> 
> it will try to build the exact revision it used by the time you
> issued the previous emerge.

Cool.

> > Does 'emerge =gcc-4.4.0_pre<date>' even work, or just `..._prelive`?
> 
> =gcc-4.4.0_pre<date> will try to reinstall what you installed by
> <date>

Will a list of date -> revision mappings be kept?  Where is this info
stored?

> =gcc-4.4.0_prelive will get resolved out of the template and then 
> installed as =gcc-4.4.0_pre<now>
> 
> > Does the user at any point ever see "live" in the ebuild version or
> > is it always replaced by the date?  If the latter, how do users
> > know they have to put '=sys-devel/gcc-4.4.0_prelive' in package.*
> > and not pre<date>?
> 
> the user will get the version render but also from where it is
> 
> > Are there any facilities to allow a user to checkout a specific
> > revision from the repo, or is that beyond the scope of this GLEP?
> > If we ever do implement such a thing, it seems like the datestamp
> > approach wouldn't mesh well; 20090310 doesn't make much sense when
> > the revision is from a month ago.
> 
> if you want to checkout a specific revision from the repo you aren't 
> creating a live ebuild but a snapshot with a specific src uri.
> 
> so you don't use a template but just the specific eclasses and mark
> the ebuild so you can show up on emerge -p the informations as stated
> above.

I was thinking about dynamic revisions, where the user could do
something like 'emerge -av gcc --rev 123456'.  I just checked a couple
eclasses and it seems we already have this feature, for example
ESVN_REVISION in subversion.eclass, so let's abandon this train of
thought. ;)

> > I'll be honest, I much prefer the -scm proposal.  But I want to
> > make sure I'm not completely out-to-lunch about -live before
> > making judgements.
> 
> At least now we have some more scenarios we could consider as
> use-cases.
> 
> Thank you for the input

Thanks for your answers.  I'd like to think about it a bit, but off the
top of my head what would you say to a hybrid approach where the ebuild
doesn't have the datestamp expansion on resolution, but rather this
information were available after building?  In all cases, the ebuild
would be referred to by its proper name (ie. gcc-4.4.0_pre.live), but
enough information would be recorded at build time so the package
manager could do groovy things like

[ebuild     U ] sys-devel/gcc-4.4.0_pre.live (last built 20090310 - rev.
14991) USE="use -flags"

and 

# emerge --info gcc

sys-devel/gcc-4.4.0_pre9999 was built with the following:
CFLAGS="-O2 -march=core2 -pipe"
CXXFLAGS="-O2 -march=core2 -pipe"

>>> Attempting to run pkg_info() for 'sys-devel/gcc-4.4.0_pre9999'

svn: 20081206 23:31:22 - gcc-4.3.3_pre9999:4.3-svn was merged at revision 142535
svn: 20090123 17:09:12 - gcc-4.4.0_pre9999:4.4-svn was merged at revision 143592
svn: 20090131 16:35:23 - gcc-4.4.0_pre9999:4.4-svn was merged at revision 143833
svn: 20090131 17:05:29 - gcc-4.4.0_pre9999:4.4-svn was merged at revision 143834
svn: 20090223 00:33:50 - gcc-4.4.0_pre9999:4.4-svn was merged at revision 144379
svn: 20090226 22:10:09 - gcc-4.4.0_pre9999:4.4-svn was merged at revision 144460
svn: 20090226 23:00:57 - gcc-4.3.3_pre9999:4.3-svn was merged at revision 144460
svn: 20090304 20:16:48 - gcc-4.4.0_pre9999:4.4-svn was merged at revision 144634
svn: 20090307 15:12:27 - gcc-4.4.0_pre9999:4.4-svn was merged at revision 144702

(note the latter is already possible with zero modifications to anything)

It seems to me that the datestamp expansion is a lot of complication
for very little gain.  

Looking at the list of shortcomings in the draft:

> There are many obvious and non obvious shortcomings in this
> situations:
> - you have to hand produce "high enough" version values and be sure
> they do not clash (e.g. 2.3_pre9999 live ebuild being shadowed by
> 2.3_pre20050201 snapshot).

Solved by both this and the -scm GLEP.

> - you cannot track what did you install since you don't have a precise
>   information about it, emerge logs will just provide you the build
> date.

Most SCM eclasses have facilities for storing this data, eg. ESVN_LOG_DIR.
Standardizing it could be helpful.

> - you cannot do exact reemerges and that may break revdep-rebuild

You can use eclass features that skip the updating phase and just build the
current sources (eg. ESVN_OFFLINE).  And as I mentioned above, some can
build a specific revision.

> - the package manager isn't aware of the "liveness" condition.

Solved by both this and the -scm GLEP.

> - in order to refresh/update the installed package automatically you
> need either to rely on script or on sets hand produced or
> heuristically defined (e.g "all ebuilds that inherit eclass svn go in
> svn set").

Do people really want live ebuilds updating automatically?  It sounds
like a recipe for breakage.  I don't want live sources updating until
i explicitly tell them to, but maybe that's just me.  If someone does
want this behaviour, I'd suggest using cron. Also, doesn't portage have
a @live-rebuild set that makes this trivial?

> - since you fetch on unpack phase you cannot use emerge --fetch
>   consistently.

How does either proposal address this?

In the end, I don't see anything that datename expansion does that
can't be done more easily and less confusingly with other techniques.
Is there a specific problem it solves that can't be addressed through
other means?


-- 
gcc-porting,                                      by design, by neglect
treecleaner,                              for a fact or just for effect
wxwidgets @ gentoo     EFFD 380E 047A 4B51 D2BD C64F 8AA8 8346 F9A4 0662

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]