[gentoo-catalyst] Catalyst 2.0 weirdness

[gentoo-catalyst] Catalyst 2.0 weirdness

[Bardur Arantsson]

Hello all,

I was trying to build a livecd-stage2 target with Catalyst 2.0 and had 
some 'interesting' problems. When doing a

$ catalyst -f stage2.spec

on my stage2.spec I got the following error:

[Some of the standard Catalyst output removed]
Emptying directory 
/var/tmp/catalyst/tmp/default/.autoresume-livecd-stage2-x86-CURRENT/
Emptying directory /var/tmp/catalyst/tmp/default/livecd-stage2-x86-CURRENT/
Catalyst aborting....
Traceback (most recent call last):
   File "/usr/bin/catalyst", line 380, in ?
     build_target(addlargs, targetmap)
   File "/usr/bin/catalyst", line 198, in build_target
     mytarget.run()
   File "/usr/lib/catalyst/modules/generic_stage_target.py", line 1063, 
in run
     apply(getattr(self,x))
   File "/usr/lib/catalyst/modules/livecd_stage2_target.py", line 116, 
in unpack
     print display_msg
UnboundLocalError: local variable 'display_msg' referenced before assignment


Looking at the code in livecd_stage2_target.py it seems that the code 
simply assumes that the display_msg variable gets set even though it 
doesn't when os.path.isdir(self.settings["source_path"]) == False.

I added a print statement for self.settings["source_path"] and that prints

	/var/tmp/catalyst/builds/stage3-x86-2006.0.tar.bz2

which seems rather strange to me. The (hopefully) relevant portion of my 
stage2.spec file is as follows:

   subarch: x86
   version_stamp: CURRENT
   target: livecd-stage2
   rel_type: default
   profile: default-linux/x86/2006.0
   snapshot: 20060821
   source_subpath: stage3-x86-2006.0

The only thing resembling the file name above is the source_subpath 
setting, but I'm not sure why that would get turned into a .tar.bz2 file 
name by catalyst when catalyst itself clearly expects a directory name.

Anyone have any clues to impart?

Cheers,

-- 
Bardur Arantsson
<bardurREMOVE@THISscientician.net>

Every program has at least one bug and can be shortened by at
least one instruction -- from which, by induction, one can deduce
that every program can be reduced to one instruction which
doesn't work.

-- 
gentoo-catalyst@gentoo.org mailing list

[gentoo-catalyst] Re: Catalyst 2.0 weirdness

[Bardur Arantsson]

Bardur Arantsson wrote:
> Hello all,
> 
[--snip--]
> I added a print statement for self.settings["source_path"] and that prints
> 
>     /var/tmp/catalyst/builds/stage3-x86-2006.0.tar.bz2
> 
> which seems rather strange to me. The (hopefully) relevant portion of my 
> stage2.spec file is as follows:
> 
>   subarch: x86
>   version_stamp: CURRENT
>   target: livecd-stage2
>   rel_type: default
>   profile: default-linux/x86/2006.0
>   snapshot: 20060821
>   source_subpath: stage3-x86-2006.0

... and of course the source_subpath was the problem; for some reason 
I'd completely forgotten that it's supposed to be a stage1. Changing this to

   source_subpath: default/livecd-stage1-x86-CURRENT

(My version stamp is CURRENT) worked.

I think the livecd_stage2_target.py should probably give the user a 
slightly more informative error message ;).

Maybe something like "Could not find directory $source_path. Your may be 
source_subpath incorrectly set." would be in order?

> 
> The only thing resembling the file name above is the source_subpath 
> setting, but I'm not sure why that would get turned into a .tar.bz2 file 
> name by catalyst when catalyst itself clearly expects a directory name.
> 

The "change to the name of a .tar.bz2" bit still puzzles me and I'm not 
sure it's desirable behavior.

Cheers,

-- 
Bardur Arantsson
<bardurREMOVE@THISscientician.net>

I haven't slept for ten days... because that would be too long.
                                                     Mitch Hedberg

-- 
gentoo-catalyst@gentoo.org mailing list

Re: [gentoo-catalyst] Re: Catalyst 2.0 weirdness

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 1303 bytes --]

On Wed, 2006-08-23 at 06:58 +0200, Bardur Arantsson wrote:
> I think the livecd_stage2_target.py should probably give the user a 
> slightly more informative error message ;).

Feel free to file a bug with a patch.  Remember that catalyst isn't
really targeted for end-users (how many times do you hear that about a
project) but instead Gentoo developers.  We release it mostly for
transparency reasons and because of our social contract.  That being
said, we do work bug reports, especially ones with patches.

> > The only thing resembling the file name above is the source_subpath 
> > setting, but I'm not sure why that would get turned into a .tar.bz2 file 
> > name by catalyst when catalyst itself clearly expects a directory name.
> > 
> 
> The "change to the name of a .tar.bz2" bit still puzzles me and I'm not 
> sure it's desirable behavior.

It is.  It is all due to seedcache and how it works.  It first searches
for a directory, then for a tarball.  That's not saying that there isn't
a bug in there somewhere, but the design is to use a directory first
when options=seedcache and a tarball only otherwise for all targets
*except* livecd-stage2.

-- 
Chris Gianelloni
Release Engineering - Strategic Lead
x86 Architecture Team
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 192 bytes --]

[gentoo-catalyst] Re: Catalyst 2.0 weirdness

[Bardur Arantsson]

[-- Attachment #1: Type: text/plain, Size: 2173 bytes --]

Chris Gianelloni wrote:
> On Wed, 2006-08-23 at 06:58 +0200, Bardur Arantsson wrote:
>> I think the livecd_stage2_target.py should probably give the user a 
>> slightly more informative error message ;).
> 
> Feel free to file a bug with a patch.  Remember that catalyst isn't
> really targeted for end-users (how many times do you hear that about a
> project) but instead Gentoo developers.  We release it mostly for
> transparency reasons and because of our social contract.  That being
> said, we do work bug reports, especially ones with patches.
> 

Fair enough, but bugs are meant to be squashed aren't they? ;) All your 
hard work _is_ appreciated even if it wasn't meant for general 
consumption :).

I can't be bothered to open up a bug report, but I've attached a trivial 
patch.

I think the logic in there still looks suspect though: AFAICT there may 
be other scenarios than the one I accidentally tried where 'unpack' may 
remain True while unpack_cmd does not get set to anything meaningful. 
With my patch this will result in the new error message as well -- I'm 
not sure how appropriate that is.

Anyway, the patch certainly certainly shouldn't cause any regressions.
The new message only triggers when an undefined variable would be 
accessed anyway.

>>> The only thing resembling the file name above is the source_subpath 
>>> setting, but I'm not sure why that would get turned into a .tar.bz2 file 
>>> name by catalyst when catalyst itself clearly expects a directory name.
>>>
>> The "change to the name of a .tar.bz2" bit still puzzles me and I'm not 
>> sure it's desirable behavior.
> 
> It is.  It is all due to seedcache and how it works.  It first searches
> for a directory, then for a tarball.  That's not saying that there isn't
> a bug in there somewhere, but the design is to use a directory first
> when options=seedcache and a tarball only otherwise for all targets
> *except* livecd-stage2.
> 

Ok, I'll take your word for it :).

-- 
Bardur Arantsson
<bardurREMOVE@THISscientician.net>

- I've got a plan so cunning, you could put a tail on it and call
it a weasel.
                                   Edmund Blackadder, 'Blackadder'

[-- Attachment #2: blah.patch --]
[-- Type: text/plain, Size: 776 bytes --]

--- livecd_stage2_target.py.orig	2006-08-23 19:33:53.000000000 +0200
+++ livecd_stage2_target.py	2006-08-23 20:20:13.000000000 +0200
@@ -75,6 +75,7 @@
 	
 	def unpack(self):
                 unpack=True
+		display_msg=None
 
                 clst_unpack_hash=read_from_clst(self.settings["autoresume_path"]+"unpack")
 
@@ -113,6 +114,9 @@
                                 if not os.path.exists(self.settings["pkgcache_path"]):
                                         os.makedirs(self.settings["pkgcache_path"],0755)
 
+			if not display_msg:
+				raise CatalystError,"Could not find appropriate source. Please check the 'source_subpath' setting in the spec file."
+
                         print display_msg
                         cmd(unpack_cmd,error_msg,env=self.env)
 

Re: [gentoo-catalyst] Re: Catalyst 2.0 weirdness

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 1034 bytes --]

On Wed, 2006-08-23 at 21:04 +0200, Bardur Arantsson wrote:
> I can't be bothered to open up a bug report, but I've attached a trivial 
> patch.

...

> I think the logic in there still looks suspect though: AFAICT there may 
> be other scenarios than the one I accidentally tried where 'unpack' may 
> remain True while unpack_cmd does not get set to anything meaningful. 
> With my patch this will result in the new error message as well -- I'm 
> not sure how appropriate that is.
> 
> Anyway, the patch certainly certainly shouldn't cause any regressions.
> The new message only triggers when an undefined variable would be 
> accessed anyway.

I'll have to test it.  It sure would be nice if there were a bug report
for this so that I didn't lose this email in all of my pre-release
hustle.  Maybe I'll happen to remember this email when I get to doing
catalyst testing again.  Who knows...

-- 
Chris Gianelloni
Release Engineering - Strategic Lead
x86 Architecture Team
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

[gentoo-catalyst] Re: Catalyst 2.0 weirdness

[Bardur Arantsson]

Chris Gianelloni wrote:
> On Wed, 2006-08-23 at 21:04 +0200, Bardur Arantsson wrote:
>> I can't be bothered to open up a bug report, but I've attached a trivial 
>> patch.
> 
> ...
> 
>> I think the logic in there still looks suspect though: AFAICT there may 
>> be other scenarios than the one I accidentally tried where 'unpack' may 
>> remain True while unpack_cmd does not get set to anything meaningful. 
>> With my patch this will result in the new error message as well -- I'm 
>> not sure how appropriate that is.
>>
>> Anyway, the patch certainly certainly shouldn't cause any regressions.
>> The new message only triggers when an undefined variable would be 
>> accessed anyway.
> 
> I'll have to test it.  It sure would be nice if there were a bug report
> for this so that I didn't lose this email in all of my pre-release
> hustle.  Maybe I'll happen to remember this email when I get to doing
> catalyst testing again.  Who knows...
> 

Paste
	http://www.mail-archive.com/gentoo-catalyst%40lists.gentoo.org/msg00395.html

into your TODO file.

-- 
Bardur Arantsson
<bardurREMOVE@THISscientician.net>

- You'll get more with a kind word and a 2x4" than with just a
kind word.
                                                       Marcus Cole

-- 
gentoo-catalyst@gentoo.org mailing list

Re: [gentoo-catalyst] Re: Catalyst 2.0 weirdness

[Andrew Gaffney]

Bardur Arantsson wrote:
> Paste
>     http://www.mail-archive.com/gentoo-catalyst%40lists.gentoo.org/msg00395.html 
> 
> 
> into your TODO file.

It probably took you more time to find that URL than it would have to file a 
bug. Why do users have such an aversion to filing bugs?

-- 
Andrew Gaffney                            http://dev.gentoo.org/~agaffney/
Gentoo Linux Developer                                   Installer Project
-- 
gentoo-catalyst@gentoo.org mailing list

Re: [gentoo-catalyst] Re: Catalyst 2.0 weirdness

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 1078 bytes --]

On Thu, 2006-08-24 at 07:13 -0500, Andrew Gaffney wrote:
> Bardur Arantsson wrote:
> > Paste
> >     http://www.mail-archive.com/gentoo-catalyst%40lists.gentoo.org/msg00395.html 
> > 
> > 
> > into your TODO file.
> 
> It probably took you more time to find that URL than it would have to file a 
> bug. Why do users have such an aversion to filing bugs?

Probably for the same reason developers don't keep stupid TODO lists and
use a bug tracker to track issues like this.  Personal preference.

As you are well aware, if it isn't in a bug, I won't remember it.  I
don't keep a TODO of any kind, as bugzilla does this for me.  Why should
I have to duplicate functionality that's already there?  It takes only a
minute or two to sign up for an account.  No more than it takes to sign
up for a mailing list.

Anyway, it's pretty simple.  If it's not a "bug" and it isn't in our bug
tracker, I'm not likely to ever do it.

C'est la vie.

-- 
Chris Gianelloni
Release Engineering - Strategic Lead
x86 Architecture Team
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

[gentoo-catalyst] Re: Catalyst 2.0 weirdness

[Bardur Arantsson]

Chris Gianelloni wrote:
> On Thu, 2006-08-24 at 07:13 -0500, Andrew Gaffney wrote:
>> Bardur Arantsson wrote:
>>> Paste
>>>     http://www.mail-archive.com/gentoo-catalyst%40lists.gentoo.org/msg00395.html 
>>>
>>>
>>> into your TODO file.
>> It probably took you more time to find that URL than it would have to file a 
>> bug. Why do users have such an aversion to filing bugs?
> 
> Probably for the same reason developers don't keep stupid TODO lists and
> use a bug tracker to track issues like this.  Personal preference.
> 
> As you are well aware, if it isn't in a bug, I won't remember it.

I know how you feel. I'm afflicted with the same problem.

> I don't keep a TODO of any kind, as bugzilla does this for me.  Why should
> I have to duplicate functionality that's already there?  It takes only a
> minute or two to sign up for an account.  No more than it takes to sign
> up for a mailing list.

Personally, I usually do like to keep a TODO list for my projects so I 
can paste random stuff like URLs or snippets of e-mails into merely as a 
"service to my users" if you will. However, like you say Catalyst isn't 
really a project which exists for the user's sakes as such, so I can 
certainly understand that you wouldn't want to make things more 
complicated for yourself than absolutely necessary.

I'll shut up in a minute, but just a minor point: In my development 
projects (both commercial and noncommercial) I've found that the best 
way to avoid getting bug reports is to just consistently say "file a bug 
report" whenever someone wants to report a bug. I'm not really sure why 
that is; maybe it has something to do with a perception -- however 
imagined or real it may be -- that all bug reporting systems are clunky 
and annoying. Certainly bugzilla is _reasonable_ when compared to some 
other systems, but it really doesn't compare to email when looking 
solely at the "ease of reporting a bug" metric. From personal experience 
I know that I'm about 1000 times more likely to file a bug if all I have 
to do is send an email somewhere. Otherwise it just *feels* like I'm 
doing work that a computer should be doing.

Cheers,

-- 
Bardur Arantsson
<bardurREMOVE@THISscientician.net>

- Rakes... my old arch-enemy.
                                      Sideshow Bob, 'The Simpsons'

-- 
gentoo-catalyst@gentoo.org mailing list

[gentoo-catalyst] Re: Catalyst 2.0 weirdness

[Bardur Arantsson]

Andrew Gaffney wrote:
> Bardur Arantsson wrote:
>> Paste
>>     
>> http://www.mail-archive.com/gentoo-catalyst%40lists.gentoo.org/msg00395.html 
>>
>>
>> into your TODO file.
> 
> It probably took you more time to find that URL than it would have to 
> file a bug.

Not really. I think I had to click about 3-5 times and type about 3 or 4 
words.

> Why do users have such an aversion to filing bugs?

Personally, it's usually because it often adds overhead relative to an 
email when the fix is trivial. If I could email bugzilla...

-- 
Bardur Arantsson
<bardurREMOVE@THISscientician.net>

- The best way to predict the future is to invent it.
                                                          Alan Kay

-- 
gentoo-catalyst@gentoo.org mailing list