[gentoo-catalyst] Couple of questions

[gentoo-catalyst] Couple of questions

[Mikey]

[-- Attachment #1: Type: text/plain, Size: 278 bytes --]

Are there any hooks to allow housecleaning tasks after stage processing 
completes?  Some way to provide for script in the spec file.  

For example, I would like to be able to do things like prune python 2.3 
after it is upgraded to 2.4 during a stage2 or stage3 build...

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-catalyst] Couple of questions

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 1000 bytes --]

On Tue, 2005-12-13 at 20:49 -0600, Mikey wrote:
> Are there any hooks to allow housecleaning tasks after stage processing 
> completes?  Some way to provide for script in the spec file.

Nope, nor will there ever be.

At stage in Gentoo is a sacred thing.  It is created a certain way and
that should not be changed.  If you're wanting to make changes, then you
are looking for a stage4.

> For example, I would like to be able to do things like prune python 2.3 
> after it is upgraded to 2.4 during a stage2 or stage3 build...

Using a seed with only python 2.4 is the proper solution.  If you do not
have such a seed, build a stage4 tarball with only python in it with
your new snapshot.  You can use a 2005.1-r1 stage3 as your seed.  Have
the stage4 run python-updater in your fsscript.  You can then use this
updated stage4 as your seed for building stage1.

-- 
Chris Gianelloni
Release Engineering - Strategic Lead
x86 Architecture Team
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-catalyst] Couple of questions

[Mikey]

[-- Attachment #1: Type: text/plain, Size: 2430 bytes --]

On Wednesday 14 December 2005 09:03, Chris Gianelloni spammed:

> Using a seed with only python 2.4 is the proper solution.  If you do not
> have such a seed, build a stage4 tarball with only python in it with
> your new snapshot.  You can use a 2005.1-r1 stage3 as your seed.  Have
> the stage4 run python-updater in your fsscript.  You can then use this
> updated stage4 as your seed for building stage1.

Python will be automatically upgraded at stage3, which means that catalyst 
is creating a stage3 with both versions of python.  Actually both versions 
of anything that might be slotted during that build of the stage3 target.  
Call me anal but that seems to be a rather poor state to leave it in.  
Regardless, the functionality provided by the stage4 fsscript is a 
workaround...

Perhaps I am wanting  catalyst to do something that it is not intended to 
do, maybe you can clarify for me.

I want to be able to create clean, pristine, generic x86 stage1s with the 
current toolchain to use as my seed for subsequent builds.  From mistakes I 
have been making and what you have been telling me, I am going to assume 
the following is the correct way to do this?

Create a stage1 seed to use for future stage2s

1) Use an unaltered stage1-x86-2005.0-r1.tar.bz2 from gentoo with a generic 
spec file, no changes in anything, only specify the source and snapshot, to 
create a stage2.  This will create a stage2 with the current gcc, binutils, 
glibc, etc...

2) Use the resulting stage2, with no customizations, to create a stage3.

3) Use the resulting stage3 to create a new stage1, which would now contain 
the updated toolchain.

Use my new stage1 to create my customized stage2/3s

1) Use the stage1 seed created above, create spec file for stage2 with my 
cflags, chost, cxxflags, and (where do use flags fit in)?

2) Use the new stage2 to create a stage3, I am assuming the cflags, chost, 
cxxflags, and use flags will migrate from the stage2 via the 
catalyst-generated make.conf and need no alteration?

I am a little confused about where my USE preferences need to be applied and 
how they migrate from stage2 -> stage3 -> stage4.  Would they go in the 
stage2 spec file with stage2/use, or a env script, or would I need to hand 
edit the stage2 make.conf before using it?  Afterwards, will I ever need to 
change the USE flags again for that particular release?

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-catalyst] Couple of questions

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 3946 bytes --]

On Wed, 2005-12-14 at 09:51 -0600, Mikey wrote:
> On Wednesday 14 December 2005 09:03, Chris Gianelloni spammed:
> 
> > Using a seed with only python 2.4 is the proper solution.  If you do not
> > have such a seed, build a stage4 tarball with only python in it with
> > your new snapshot.  You can use a 2005.1-r1 stage3 as your seed.  Have
> > the stage4 run python-updater in your fsscript.  You can then use this
> > updated stage4 as your seed for building stage1.
> 
> Python will be automatically upgraded at stage3, which means that catalyst 
> is creating a stage3 with both versions of python.  Actually both versions 
> of anything that might be slotted during that build of the stage3 target.  
> Call me anal but that seems to be a rather poor state to leave it in.  
> Regardless, the functionality provided by the stage4 fsscript is a 
> workaround...

OK.  I'm going to list the proper process for creating "clean" stages.

> Perhaps I am wanting  catalyst to do something that it is not intended to 
> do, maybe you can clarify for me.
> 
> I want to be able to create clean, pristine, generic x86 stage1s with the 
> current toolchain to use as my seed for subsequent builds.  From mistakes I 
> have been making and what you have been telling me, I am going to assume 
> the following is the correct way to do this?

It isn't.  I'll answer in-line.

> Create a stage1 seed to use for future stage2s

Yes and no.  You will want to create a stage1, but not in any way that
you have described.

> 1) Use an unaltered stage1-x86-2005.0-r1.tar.bz2 from gentoo with a generic 
> spec file, no changes in anything, only specify the source and snapshot, to 
> create a stage2.  This will create a stage2 with the current gcc, binutils, 
> glibc, etc...

This is unnecessary.

> 2) Use the resulting stage2, with no customizations, to create a stage3.

Again, unnecessary.

> 3) Use the resulting stage3 to create a new stage1, which would now contain 
> the updated toolchain.

Instead of wasting all the time of building your own stage3 just to
build a stage1, use stage3-x86-2005.1-r1.tar.bz2 and build your stage1
from this.  Building a stage1 will *always* be up-to-date to what is in
your snapshot.  No files from the stage3 are actually put into the
stage1 tarball, so it can be older than your snapshot.  This is how we
build newer Gentoo releases.

> Use my new stage1 to create my customized stage2/3s

Correct.

> 1) Use the stage1 seed created above, create spec file for stage2 with my 
> cflags, chost, cxxflags, and (where do use flags fit in)?

You cannot set USE flags for your lower stages.  You will need to create
a profile.  The USE flags *must* match what is in the profile.

> 2) Use the new stage2 to create a stage3, I am assuming the cflags, chost, 
> cxxflags, and use flags will migrate from the stage2 via the 
> catalyst-generated make.conf and need no alteration?

Correct.  These come from the seed stage for stages 1 and 3 (under _rc9
and up).

> I am a little confused about where my USE preferences need to be applied and 
> how they migrate from stage2 -> stage3 -> stage4.  Would they go in the 
> stage2 spec file with stage2/use, or a env script, or would I need to hand 
> edit the stage2 make.conf before using it?  Afterwards, will I ever need to 
> change the USE flags again for that particular release?

Well, stage2/use is invalid and ignored.  Also, stage3/use is invalid
and ignored.  You do not change the USE in stages 1 through 3.  Period.
To make changes to these stages, you need to make a new profile with
your changes.  This could either be done via editing your portage tree
before making the snapshot, or as portdir_overlay in each spec file.  No
hand-editing is ever required.

-- 
Chris Gianelloni
Release Engineering - Strategic Lead
x86 Architecture Team
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-catalyst] Couple of questions

[Mikey]

[-- Attachment #1: Type: text/plain, Size: 1743 bytes --]

On Wednesday 14 December 2005 12:42, Chris Gianelloni spammed:

> Instead of wasting all the time of building your own stage3 just to
> build a stage1, use stage3-x86-2005.1-r1.tar.bz2 and build your stage1
> from this.  Building a stage1 will *always* be up-to-date to what is in
> your snapshot.  No files from the stage3 are actually put into the

Well, I will have to argue with you on this point.  It is important to me 
that the toolchain is compiled against itself.  Catching weird toolchain 
errors on down the road in a production environment is not my idea of fun.  
I suppose I will repeat the use of the word anal again...


> You cannot set USE flags for your lower stages.  You will need to create
> a profile.  The USE flags *must* match what is in the profile.

Lower stages being 1-3?  During which stage are USE flags intended to be 
used with catalyst?

> Well, stage2/use is invalid and ignored.  Also, stage3/use is invalid
> and ignored.  You do not change the USE in stages 1 through 3.  Period.
> To make changes to these stages, you need to make a new profile with
> your changes.  This could either be done via editing your portage tree
> before making the snapshot, or as portdir_overlay in each spec file.  No
> hand-editing is ever required.

What about stage4?  If I want to build snapshots with nptl, userlocales, and 
-nls, for example, I have to use a stage4?

On the bug I submitted yesterday.  I just completed a stage1 -> stage2 -> 
stage3 -> stage1 cycle, using no environment script, no USE flags, no 
optimizations, the same thing happens.  It attempts to emerge glibc before 
linux-headers, glibc looks for the headers in stage1root, they don't exist, 
so it bombs....

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-catalyst] Couple of questions

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 2894 bytes --]

On Wed, 2005-12-14 at 14:35 -0600, Mikey wrote:
> Well, I will have to argue with you on this point.  It is important to me 
> that the toolchain is compiled against itself.  Catching weird toolchain 
> errors on down the road in a production environment is not my idea of fun.  
> I suppose I will repeat the use of the word anal again...

That is what the later stages are there for, to recompile the toolchain.
Argue all you want, this is fact.

> > You cannot set USE flags for your lower stages.  You will need to create
> > a profile.  The USE flags *must* match what is in the profile.
> 
> Lower stages being 1-3?  During which stage are USE flags intended to be 
> used with catalyst?

Correct.  Only stage4 is safe for changing USE flags.

> > Well, stage2/use is invalid and ignored.  Also, stage3/use is invalid
> > and ignored.  You do not change the USE in stages 1 through 3.  Period.
> > To make changes to these stages, you need to make a new profile with
> > your changes.  This could either be done via editing your portage tree
> > before making the snapshot, or as portdir_overlay in each spec file.  No
> > hand-editing is ever required.
> 
> What about stage4?  If I want to build snapshots with nptl, userlocales, and 
> -nls, for example, I have to use a stage4?

No, you do what I said and make a profile.  I am honestly getting a
little frustrated with having to repeat this.  With a proper profile,
you can have a stage2/stage3 with nptl.  It isn't possible to have a
stage1 with nptl on x86, since nptl isn't ported to i386 CHOST.  Look at
default-linux/x86/dev/2006.0 as a good example, as we're using nptl by
default there.

> On the bug I submitted yesterday.  I just completed a stage1 -> stage2 -> 
> stage3 -> stage1 cycle, using no environment script, no USE flags, no 
> optimizations, the same thing happens.  It attempts to emerge glibc before 
> linux-headers, glibc looks for the headers in stage1root, they don't exist, 
> so it bombs....

I have no way of guaranteeing that the stages you made are good.  Please
follow the instructions I sent in my last email for your builds, as
there is no way for me to verify anything other than the recommended
procedures.  Use a known good stage3 tarball as a stage1 seed.  At that
point, you don't need to rebuild stage1.  Basically, to duplicate what
you have done you would need: stage3 -> stage1 -> stage2 -> stage3
(remember that the stage3 *must* be generic to build a stage1) ->
stage1.

Unfortunately, in many cases, catalyst is a "fix it yourself"
application.  If it doesn't break our release-building, it is reduced in
priority, as catalyst is first and foremost our release-building tool.

Patches are always welcome, however.

-- 
Chris Gianelloni
Release Engineering - Strategic Lead
x86 Architecture Team
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-catalyst] Couple of questions

[Mikey]

[-- Attachment #1: Type: text/plain, Size: 3168 bytes --]

On Wednesday 14 December 2005 15:27, Chris Gianelloni spammed:

> > > You cannot set USE flags for your lower stages.  You will need to
> > > create a profile.  The USE flags *must* match what is in the profile.

Understood, finally.  

> Correct.  Only stage4 is safe for changing USE flags.
>
> No, you do what I said and make a profile.  I am honestly getting a
> little frustrated with having to repeat this.  With a proper profile,
> you can have a stage2/stage3 with nptl.  It isn't possible to have a
> stage1 with nptl on x86, since nptl isn't ported to i386 CHOST.  Look at
> default-linux/x86/dev/2006.0 as a good example, as we're using nptl by
> default there.

Is stage4 ok with nptl in the USE flags?  Or am I going to HAVE to go the 
profile route to get nptl?  You can't stick nptl in the profile and expect 
generic x86 arch builds to work, unless the chost was changed...

> I have no way of guaranteeing that the stages you made are good.  Please
> follow the instructions I sent in my last email for your builds, as
> there is no way for me to verify anything other than the recommended
> procedures.  Use a known good stage3 tarball as a stage1 seed.  At that
> point, you don't need to rebuild stage1.  Basically, to duplicate what
> you have done you would need: stage3 -> stage1 -> stage2 -> stage3
> (remember that the stage3 *must* be generic to build a stage1) ->
> stage1.

Just did, with a stage3-x86-2005.1-r1, this time with a correct subarch :)  
Hah hah, can't slip anything by you, can I!

It is still broken.  I won't bother to file any more bug reports about it.  
You can try building a stage1-x86 from the stage3-x86 yourself if you want 
to verify, or you can wait for a release to break, or whatever.  It keeps 
trying to install glibc in /tmp/stage1root, before linux-headers, and the 
glibc build keeps looking for those headers to be present 
in /tmp/stage1root.  The weird thing about it is that before the actual 
emerge start it indicates the headers will be installed first, then 
proceeds to do glibc before.  Maybe it is a circular dependency, I don't 
know.  I am sick of screwing with it, I just mounted everything and 
installed the headers into /tmp/stage1root and resumed.  Maybe it is just a 
bug somewhere in the snapshot from 20051205, I don't know.  I give up.

> Unfortunately, in many cases, catalyst is a "fix it yourself"
> application.  If it doesn't break our release-building, it is reduced in
> priority, as catalyst is first and foremost our release-building tool.

Just out of curiosity, WHY is it designed to be so inflexible as far as use 
flags in stage2?  The bootstrap.sh in stage2 strips all use flags except 
nls, userlocales, nptl, nptlonly, multilib I think, and runs with USE="-* 
build bootstrap allowed_flags", so why the restriction from using nptl and 
such in the stage2 via catalyst?  And stage3 for that matter.  I am not 
trying to be facetious here, I am just wondering what the reasoning behind 
the design is...

> Patches are always welcome, however.

Believe me, you wouldn't like what patches I would have in mind :)

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-catalyst] Couple of questions

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 4271 bytes --]

On Wed, 2005-12-14 at 16:20 -0600, Mikey wrote:
> > No, you do what I said and make a profile.  I am honestly getting a
> > little frustrated with having to repeat this.  With a proper profile,
> > you can have a stage2/stage3 with nptl.  It isn't possible to have a
> > stage1 with nptl on x86, since nptl isn't ported to i386 CHOST.  Look at
> > default-linux/x86/dev/2006.0 as a good example, as we're using nptl by
> > default there.
> 
> Is stage4 ok with nptl in the USE flags?  Or am I going to HAVE to go the 
> profile route to get nptl?  You can't stick nptl in the profile and expect 
> generic x86 arch builds to work, unless the chost was changed...

*Only* if stage4 is your final target.  As for nptl, it is 100% safe
even with CHOST=i386 because glibc ignores it.  Also, don't set
STAGE1_USE="nptl"... If you would look at the example that I told youto,
you would realize this.

> It is still broken.  I won't bother to file any more bug reports about it.  
> You can try building a stage1-x86 from the stage3-x86 yourself if you want 
> to verify, or you can wait for a release to break, or whatever.  It keeps 
> trying to install glibc in /tmp/stage1root, before linux-headers, and the 
> glibc build keeps looking for those headers to be present 
> in /tmp/stage1root.  The weird thing about it is that before the actual 
> emerge start it indicates the headers will be installed first, then 
> proceeds to do glibc before.  Maybe it is a circular dependency, I don't 
> know.  I am sick of screwing with it, I just mounted everything and 
> installed the headers into /tmp/stage1root and resumed.  Maybe it is just a 
> bug somewhere in the snapshot from 20051205, I don't know.  I give up.

It is a bug in the snapshot/tree, as I stated in bug #115445.  I'm
almost getting tired of explaining things seeing as how I have to do it
multiple times.  Rather than me repeating myself, please read each
message a few times instead? ;P

The issue is something with the tree that is being investigated.  I was
unable to reproduce it because I am using an older snapshot for my
builds.  I also mentioned this in the aforementioned bug.  I guess you
missed it.

> > Unfortunately, in many cases, catalyst is a "fix it yourself"
> > application.  If it doesn't break our release-building, it is reduced in
> > priority, as catalyst is first and foremost our release-building tool.
> 
> Just out of curiosity, WHY is it designed to be so inflexible as far as use 
> flags in stage2?  The bootstrap.sh in stage2 strips all use flags except

Because making changes outside of the scope of the profile is broken.
It produces an inconsistent stage, since the USE flags passed via an
envscript *never* see the inside of make.conf, giving you a stage that
*will not* behave as you expect it to.  Why leave this option available
when it is *obviously* misunderstood and causes errors?  You have proven
here exactly *why* the option is not there.

> nls, userlocales, nptl, nptlonly, multilib I think, and runs with USE="-* 
> build bootstrap allowed_flags", so why the restriction from using nptl and 

There is no such restriction, as stated over and over and over and over
again.  Use a profile to make these changes.

> such in the stage2 via catalyst?  And stage3 for that matter.  I am not 
> trying to be facetious here, I am just wondering what the reasoning behind 
> the design is...

No, you're being dense, rather.  I have explained it.  You just seem to
dislike my answer.  Stages 2 and 3 must match the profile or they will
not install properly in all cases.  Period.

> > Patches are always welcome, however.
> 
> Believe me, you wouldn't like what patches I would have in mind :)

Then I wouldn't accept them.  Catalyst is modular.  You're more than
welcome to replace any of the scripts to do things the way you want
them.  Just don't come asking us for help when you do things that we
know will not work as expected.  We are not stupid.  We have been
working with portage and Gentoo for a long time and know what is and is
not possible quite well by now.

-- 
Chris Gianelloni
Release Engineering - Strategic Lead
x86 Architecture Team
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-catalyst] Couple of questions

[Mikey]

[-- Attachment #1: Type: text/plain, Size: 5878 bytes --]

On Thursday 15 December 2005 07:34, Chris Gianelloni spammed:

> It is a bug in the snapshot/tree, as I stated in bug #115445.  I'm
> almost getting tired of explaining things seeing as how I have to do it
> multiple times.  Rather than me repeating myself, please read each
> message a few times instead? ;P

> know will not work as expected.  We are not stupid.  We have been
> working with portage and Gentoo for a long time and know what is and is
> not possible quite well by now.

As I am tired of repeating over and over that something is wrong with the 
stage1 target and all you seem to be able to do is blame me.  It only took 
me 5 freaking times to explain something was wrong.  So hey, you are not 
the only one who gets tired of repeating shit to thick headed people.  My 
snapshot version was right there in the spec files I posted, a simple 20 
minute test on your part would have confirmed the bug.  Apparently you 
portage people are not as smart as you would like me to believe, portage 
has its share of bugs and rarely behaves consistently.  I have been 
compiling linux from source for over 12 years, I happen to know a little 
bit about what I am talking about as well.  Probably not as much as you 
know about gentoo, but hey, I can recognize an obvious glaring problem when 
I see it.  

If you would have looked at what I posted in my bug report you would find a 
myriad of bugs.  Catalyst let me do things it is not supposed to do, like 
use a subarch during stage1 when it shouldn't.  Emerge told you it was 
going to install packages in a certain order, yet it didn't (not your bug).  
For some reason, baselayout was missing from the packages that were going 
to be installed, even though catalyst asked for it to be installed.  You 
indicate that usage of an envscript in catalyst.conf is normal usage, but 
fail to inform the user NOT to use USE flags in there or it will break 
something (that should be handled by catalyst).  You tell me catalyst is 
designed to not allow customization in the lower stages, then tell me the 
bug is invalid because I tried customization in lower stages.

But hey, I am sure it is a winning strategy to just treat all of your 
debuggers like they are idiots.

> Because making changes outside of the scope of the profile is broken.
> It produces an inconsistent stage, since the USE flags passed via an
> envscript *never* see the inside of make.conf, giving you a stage that
> *will not* behave as you expect it to.  Why leave this option available
> when it is *obviously* misunderstood and causes errors?  You have proven
> here exactly *why* the option is not there.

Gee, listen to what you are saying.  Using USE flags in stage2 and beyond is 
broken and produces inconsistent stages.  I would say that is a major 
problem that needs to be fixed before anything else, considering it is the 
most basic functionality that sets gentoo apart from all other linux 
distributions.  Stage2 ignores improper use flags via bootstrap.sh, stage3 
should be able to handle whatever use flag you throw at it, portage itself 
should be able to handle that.  So what is the problem with letting 
catalyst generate the use flags and put them into the make.conf right along 
with the cflags, mirrors, etc?  Don't worry, it is a rhetorical question.

> > nls, userlocales, nptl, nptlonly, multilib I think, and runs with
> > USE="-* build bootstrap allowed_flags", so why the restriction from
> > using nptl and
>
> There is no such restriction, as stated over and over and over and over
> again.  Use a profile to make these changes.

I see, using USE flags in the profile is not broken, letting the user 
specify USE flags anywhere else is.  Encouraging users to dork around with 
the profile is a good idea, set USE flags bad idea.  Brilliant.  
Particularly when you consider the fact that you can build "safe" USE flags 
into catalyst just like you build in "safe" cflags.  Which, by the way, are 
deprecated for gcc 3.4.4...

> No, you're being dense, rather.  I have explained it.  You just seem to
> dislike my answer.  Stages 2 and 3 must match the profile or they will
> not install properly in all cases.  Period.

Then something needs to be fixed, because the underlying process is broken.  
It needs to be the USE flags that match, it shouldn't matter where they 
come from.  Catalyst is not producing consistent results right now in spite 
of locking it down to the profile.  Locking catalyst down to extremely 
narrow functionality because portage might break at any moment is a shame, 
because it is a great tool.

But I just don't buy your explanation, USE flags are calculated the same way 
regardless of where you put them, in the profile or the environment.  The 
precedence might change, but they are still calculated the same way.  If I 
create a profile and take out nls, catalyst should handle it the exact same 
way as if I stuck it in the environment....

> Then I wouldn't accept them.  Catalyst is modular.  You're more than
> welcome to replace any of the scripts to do things the way you want

I don't think it is catalyst that is broken, but catalyst can be used to 
insure consistent results.  Simple things like install the toolchain in the 
proper order instead of using multi-package emerge statements that might or 
might not work correctly depending on what day of the week it is...

> them.  Just don't come asking us for help when you do things that we
> know will not work as expected.  We are not stupid.  We have been
> working with portage and Gentoo for a long time and know what is and is
> not possible quite well by now.

Did I say you were stupid?  And after this little experience, _I_ would be 
the stupid one for asking for help, as long as this is the response I get.

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-catalyst] Couple of questions

[Eric Edgar]

[-- Attachment #1: Type: text/plain, Size: 11201 bytes --]

On 08:47 Thu 15 Dec     , Mikey wrote:
> On Thursday 15 December 2005 07:34, Chris Gianelloni spammed:
> 
> > It is a bug in the snapshot/tree, as I stated in bug #115445.  I'm
> > almost getting tired of explaining things seeing as how I have to do it
> > multiple times.  Rather than me repeating myself, please read each
> > message a few times instead? ;P
> 
> > know will not work as expected.  We are not stupid.  We have been
> > working with portage and Gentoo for a long time and know what is and is
> > not possible quite well by now.
> 
> As I am tired of repeating over and over that something is wrong with the 
> stage1 target and all you seem to be able to do is blame me.  It only took 
> me 5 freaking times to explain something was wrong.
You also have to concider that you were doing things incorrectly.  We
only have so much time in the day and have to go after the low hanging
fruit first per say.  While you are correct that something is wrong in
the tree the initial results did not indicate that.
> So hey, you are not 
> the only one who gets tired of repeating shit to thick headed people.  My 
> snapshot version was right there in the spec files I posted, a simple 20 
> minute test on your part would have confirmed the bug.  
For you it might be a simple 20 minute test.  For us it takes much
longer.  We are working on getting catalyst ready to deploy a release.
Catalyst is the tool that allows us to do this .. that is its primary
purpose from our perspective.  While we do accept patches and generally
take the view point that catalyst should be as generic of a tool as
possible we have to shift gears into release mode when it comes time to
put out a new gentoo cd.  There are only 2 main developers working on
catalyst at the moment, Myself and wolf31o2.  

We are fully aware that there are bugs and inconsistencies and warts and
who knows what else.  Thats what happens when you have numerous
developers working in the tree.  I dont want to get into a political
flamewar on how it should be different.  Frankly we dont have the time.
We are developers who are donating our time to try and do things that
interest us.  

I am a catalyst developer now because the concept interested me.  I
started hacking on it to make it better.  I know its not perfect but its
good enough right now for its stated purpose of releasing gentoo install
media.  I am working on generalizing things and if you look at the code
between catalyst1 and catalyst2 you will see significant changes that
include additional error checks and user warnings for the most common
issues we see.
> Apparently you 
> portage people are not as smart as you would like me to believe, portage 
> has its share of bugs and rarely behaves consistently.  I have been 
> compiling linux from source for over 12 years, I happen to know a little 
> bit about what I am talking about as well.  Probably not as much as you 
> know about gentoo, but hey, I can recognize an obvious glaring problem when 
> I see it.  
from my standpoint I have seen you as quite firm in your convictions
that you were right.  I agree that the stage1 bug you posted is valid,
     but you could have also performed the few requested steps to follow
     wolf31o2's suggestions.  

Catalyst is designed to be as repeatable as possible.  We havent found
every issue yet and are always looking at smoothing those bumps over.  
Making a new profile is really very easy to do and with catalysts
portage_overlay features quite painless to do.  It works just like an
overlay on a host gentoo system.  I can understand that it might have
sounded difficult or time consuming to follow the recommendation but a
better course of action from you would have been how can I do this .. or
how do you guys do this.  If it sounds like it sucks for you maybe it
sucks for us too, but we just havent had the time to code a simpler way
yet.  We are not out to make anyones life difficult, remember we are
volunteers here with limited time.
> If you would have looked at what I posted in my bug report you would find a 
> myriad of bugs.  Catalyst let me do things it is not supposed to do, like 
> use a subarch during stage1 when it shouldn't.  Emerge told you it was 
> going to install packages in a certain order, yet it didn't (not your bug).  
> For some reason, baselayout was missing from the packages that were going 
> to be installed, even though catalyst asked for it to be installed.  You 
> indicate that usage of an envscript in catalyst.conf is normal usage, but 
> fail to inform the user NOT to use USE flags in there or it will break 
> something (that should be handled by catalyst).  You tell me catalyst is 
> designed to not allow customization in the lower stages, then tell me the 
> bug is invalid because I tried customization in lower stages.
> 
> But hey, I am sure it is a winning strategy to just treat all of your 
> debuggers like they are idiots.
You are not an idiot, you were just very firm in thinking the tool
should be used in X way and basically need to learn more of how the tool
works.  Suggestions were made to get you in the right direction yet it
seemed like you were not interested in following those initially.

> > Because making changes outside of the scope of the profile is broken.
> > It produces an inconsistent stage, since the USE flags passed via an
> > envscript *never* see the inside of make.conf, giving you a stage that
> > *will not* behave as you expect it to.  Why leave this option available
> > when it is *obviously* misunderstood and causes errors?  You have proven
> > here exactly *why* the option is not there.
> 
> Gee, listen to what you are saying.  Using USE flags in stage2 and beyond is 
> broken and produces inconsistent stages.  
He did NOT say using use flags was broken.  He said using use flags via
an environment variable was broken.  The envscript is a tool to help
build the initial hardened stages for example.  It is for developers who
need additional functionality that only a small number of the developers need.
We get our use flags out of the profiles because then every one can
build a stage1 the same way.  We grab the use flags and the packages to
be built right out of portage.  So this is not something that is broken.  
If you want different use flags for example setup a new profile and use
that ..  That is how catalyst is designed to work.  It works with the
other tools such as portage instead of trying to work around them.
Obviously the tree changes and we have to deal with that .. but that has
already been discussed.
> I would say that is a major 
> problem that needs to be fixed before anything else, considering it is the 
> most basic functionality that sets gentoo apart from all other linux 
> distributions.  Stage2 ignores improper use flags via bootstrap.sh, stage3 
> should be able to handle whatever use flag you throw at it, portage itself 
> should be able to handle that.  So what is the problem with letting 
> catalyst generate the use flags and put them into the make.conf right along 
> with the cflags, mirrors, etc?  Don't worry, it is a rhetorical question.
The problem is we already have a way to do that via profiles as I have
discussed.  Why code 2 methods to do something when we already have one
in place that works as expected for us.  Doing it this way allows us to
have very simple stage1 stage2 and stage3 spec files rather than
cluttering them up with ever changing use flags that may or may not be
missed because some developer changed something in the tree and didnt
inform us.  As it stands now a developer can change it in the tree and
update the profiles and most of the time it just works.  


> > > nls, userlocales, nptl, nptlonly, multilib I think, and runs with
> > > USE="-* build bootstrap allowed_flags", so why the restriction from
> > > using nptl and
> >
> > There is no such restriction, as stated over and over and over and over
> > again.  Use a profile to make these changes.
> 
> I see, using USE flags in the profile is not broken, letting the user 
> specify USE flags anywhere else is.  Encouraging users to dork around with 
> the profile is a good idea, set USE flags bad idea.  Brilliant.  
> Particularly when you consider the fact that you can build "safe" USE flags 
> into catalyst just like you build in "safe" cflags.  Which, by the way, are 
> deprecated for gcc 3.4.4...
> 
> > No, you're being dense, rather.  I have explained it.  You just seem to
> > dislike my answer.  Stages 2 and 3 must match the profile or they will
> > not install properly in all cases.  Period.

you are also missing the fact that the profile stays consistent from one
stage to the next.  custom use flags in a spec file can and will very
easily be missed or dropped or typoed etc.  It really is a better design
choice to do it with profiles.


> Then something needs to be fixed, because the underlying process is broken.  
> It needs to be the USE flags that match, it shouldn't matter where they 
> come from.  Catalyst is not producing consistent results right now in spite 
> of locking it down to the profile.  Locking catalyst down to extremely 
> narrow functionality because portage might break at any moment is a shame, 
> because it is a great tool.
> 
> But I just don't buy your explanation, USE flags are calculated the same way 
> regardless of where you put them, in the profile or the environment.  The 
> precedence might change, but they are still calculated the same way.  If I 
> create a profile and take out nls, catalyst should handle it the exact same 
> way as if I stuck it in the environment....
> 
> > Then I wouldn't accept them.  Catalyst is modular.  You're more than
> > welcome to replace any of the scripts to do things the way you want
> 
> I don't think it is catalyst that is broken, but catalyst can be used to 
> insure consistent results.  Simple things like install the toolchain in the 
> proper order instead of using multi-package emerge statements that might or 
> might not work correctly depending on what day of the week it is...
>
You assume that its a simple thing to compile a tool chain in order.
But what you miss is the fact that upstream developers consistently
change how things are done.  We have coded catalyst the best way for us.
Using profiles allows us to leverage other developers time in the tree
to catch a majority of the issues for us.  They can help reorganize the
way things are built if necessary via a profile.  Otherwise we would be
constantly updating our spec files and never know exactly what was going
on.


> > them.  Just don't come asking us for help when you do things that we
> > know will not work as expected.  We are not stupid.  We have been
> > working with portage and Gentoo for a long time and know what is and is
> > not possible quite well by now.
> 
> Did I say you were stupid?  And after this little experience, _I_ would be 
> the stupid one for asking for help, as long as this is the response I get.

Rocket

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-catalyst] Couple of questions

[Mikey]

[-- Attachment #1: Type: text/plain, Size: 6381 bytes --]

On Thursday 15 December 2005 09:58, Eric Edgar spammed:

> You also have to concider that you were doing things incorrectly.  We
> only have so much time in the day and have to go after the low hanging
> fruit first per say.  While you are correct that something is wrong in
> the tree the initial results did not indicate that.

It took me a few iterations to understand how the tool is _supposed_ to be 
used.  At no place in the documentation does it clearly explain that the 
syntax of stage4/use: does not migrate to stage2/use:, for example, or that 
USE flags are NOT to be used anywhere but stage4, or that USE flags are not 
to go into the envscript.  There is one generic stage example, which has 
been subsequently changed to explain that cflags and such only apply on 
stage2, but no mention of use flags and so on.  I assumed that:

"The catalyst tool is intended to be used by those who wish to create their 
own customized versions of Gentoo Linux, or their own customized LiveCDs. 
Our goal is to make catalyst a powerful tool that's a pleasure to use, and 
to ensure that the code we write is maintainable and of high-quality."

..meant that the tool was intended to be used to create customized versions, 
what I didn't understand was the customization only applies in very limited 
ways to stage4...

So I have had to learn the hard way and have had to bug the hell out of you 
guys.  I would be glad to work on some of the documentation after wrapping 
my mind around it more completely :)

> put out a new gentoo cd.  There are only 2 main developers working on
> catalyst at the moment, Myself and wolf31o2.

Geez, no wonder the fuse is kinda short.  I had no idea it was only you two.

> from my standpoint I have seen you as quite firm in your convictions
> that you were right.  I agree that the stage1 bug you posted is valid,
>      but you could have also performed the few requested steps to follow
>      wolf31o2's suggestions.

After making a few boneheaded mistakes I did follow Chris's suggestions, and 
arrived at the same results.  I also understood that the actual problem is 
not caused by those mistakes.  Stage1 is the only stage that installs into 
a separate ROOT as far as I can tell.  Catalyst is not taking into account 
the order that portage.grab_stacked() and portage.stack_lists() is 
returning package lists in.  Glibc requires headers to be installed first.  
The solution is either to hack portage to always sort lists that involve 
the toolchain correctly, or to account for it in catalyst.  I have never 
coded in python, so I don't quite know how to implement the solution, but I 
do have a guess at what the solution is.  Push any toolchain package to the 
top (or bottom, whichever) of the buildpkgs array in their proper order, 
don't rely on portage.grab_stacked() or portage.stack_lists() to order it 
correctly because it does not gracefully take into account the install is 
happening in a different ROOT.  The DEPEND check in the glibc ebuild is 
looking for headers in the running system, not what is installed in ROOT, 
so it does not see it as a dependency that needs to be installed first.  
<rant>As an anal purist, I believe the toolchain should be installed first 
and in the proper order explicitly one package at a time instead of in a 
stacked list.  To do otherwise is heaping problems on yourself on down the 
road if you ask me.  You also avoid this weird dependency problem when 
installing into a ROOT.</rant>


> Making a new profile is really very easy to do and with catalysts
> portage_overlay features quite painless to do.  It works just like an
> overlay on a host gentoo system.  I can understand that it might have
> sounded difficult or time consuming to follow the recommendation but a
> better course of action from you would have been how can I do this .. or
> how do you guys do this.  If it sounds like it sucks for you maybe it
> sucks for us too, but we just havent had the time to code a simpler way
> yet.  We are not out to make anyones life difficult, remember we are
> volunteers here with limited time.

Can you put profiles in overlays?  Duh, that would be much easier.

> you are also missing the fact that the profile stays consistent from one
> stage to the next.  custom use flags in a spec file can and will very
> easily be missed or dropped or typoed etc.  It really is a better design
> choice to do it with profiles.

Call this a feature request, or a suggestion.

Allow cflags, chosts, cxxflags, FEATURES, and use flags in only ONE place - 
stage2.  Stage2 can generate the entire make.conf that will be used through 
any future stages, including the user specified USE flags. Stage2 runs the 
bootstrap.sh, which correctly throws out non-applicable USE flags, so it is 
not a problem, all the catalyst developers would have to do is count on 
bootstrap.sh to work correctly.  

From that point forward every stage done by catalyst uses the USE flags 
generated in step 2 and never changes.  If a USE flag breaks the install, 
it would have broken during the install in stage4 anyway.  This would 
greatly simplify everything in my opinion.

Stage1 would continue to be the proper place to sanitize the build to 
pre-selected cflags and use flags, etc...

Am I crazy, or what?

> You assume that its a simple thing to compile a tool chain in order.
> But what you miss is the fact that upstream developers consistently
> change how things are done.  We have coded catalyst the best way for us.
> Using profiles allows us to leverage other developers time in the tree
> to catch a majority of the issues for us.  They can help reorganize the
> way things are built if necessary via a profile.  Otherwise we would be
> constantly updating our spec files and never know exactly what was going
> on.

It _is_ a simple thing to bootstrap a toolchain in order.  The order has 
always been the exact same and probably always will be.  binutils, gcc, 
headers, glibc.  Repeat, using tools from previous step.  This is the 
_only_ way as far as I know to do it correctly, and to do it correctly 
every time, and to be absolutely certain it has been done correctly.  It 
has been this way for over 10 years if I recall correctly, it is linux 
system building 101.


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-catalyst] Couple of questions

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 10572 bytes --]

On Thu, 2005-12-15 at 10:57 -0600, Mikey wrote:
> On Thursday 15 December 2005 09:58, Eric Edgar spammed:
> 
> > You also have to concider that you were doing things incorrectly.  We
> > only have so much time in the day and have to go after the low hanging
> > fruit first per say.  While you are correct that something is wrong in
> > the tree the initial results did not indicate that.
> 
> It took me a few iterations to understand how the tool is _supposed_ to be 
> used.  At no place in the documentation does it clearly explain that the 
> syntax of stage4/use: does not migrate to stage2/use:, for example, or that 
> USE flags are NOT to be used anywhere but stage4, or that USE flags are not 
> to go into the envscript.  There is one generic stage example, which has 
> been subsequently changed to explain that cflags and such only apply on 
> stage2, but no mention of use flags and so on.  I assumed that:

The templates for stage2 and stage3 don't list stage?/use.  Those are
the documentation for the specs, so it is mentioned in the documentation
pretty explicitly.

I think the problem was that you assumed.

> "The catalyst tool is intended to be used by those who wish to create their 
> own customized versions of Gentoo Linux, or their own customized LiveCDs. 
> Our goal is to make catalyst a powerful tool that's a pleasure to use, and 
> to ensure that the code we write is maintainable and of high-quality."
> 
> ..meant that the tool was intended to be used to create customized versions, 
> what I didn't understand was the customization only applies in very limited 
> ways to stage4...

No.  You can completely customize your lower stages, it just needs to be
done within the confines of what portage allows.  This means you need a
profile to make modifications.  Since stage4 allows you to do things
beyond the "system" target, it can be customized outside of a profile.
This is a limitation in portage which will not be worked around in
catalyst simply because I 100% agree with the limitation.

> So I have had to learn the hard way and have had to bug the hell out of you 
> guys.  I would be glad to work on some of the documentation after wrapping 
> my mind around it more completely :)

I am working on the documentation.  Right now, catalyst 1.x is the
stable tree.  Anything and everything in catalyst 2.x is still pretty
dynamic, as we are working in final features and bug fixes before
release.  I'll be working on the documentation a bit more as time
permits once 2.0 final is released.

> > put out a new gentoo cd.  There are only 2 main developers working on
> > catalyst at the moment, Myself and wolf31o2.
> 
> Geez, no wonder the fuse is kinda short.  I had no idea it was only you two.

Until I brought rocket on as a developer, it was just myself maintaining
code that I had inherited, and the original developers were no longer
with the project.  Catalyst has come a long way since I took it over,
and it still has a long way to go.

> > from my standpoint I have seen you as quite firm in your convictions
> > that you were right.  I agree that the stage1 bug you posted is valid,
> >      but you could have also performed the few requested steps to follow
> >      wolf31o2's suggestions.
> 
> After making a few boneheaded mistakes I did follow Chris's suggestions, and 
> arrived at the same results.  I also understood that the actual problem is 
> not caused by those mistakes.  Stage1 is the only stage that installs into 
> a separate ROOT as far as I can tell.  Catalyst is not taking into account 
> the order that portage.grab_stacked() and portage.stack_lists() is 
> returning package lists in.  Glibc requires headers to be installed first.

No.  The problem was 100% a problem in the tree which has been fixed
(now).

> The solution is either to hack portage to always sort lists that involve 
> the toolchain correctly, or to account for it in catalyst.  I have never

Portage handles all dependencies for a reason.  The toolchain on Gentoo
is not static.  What if you're using catalyst to build a Gentoo/FreeBSD
target?  What if you're using it to build a hardened uclibc MIPS target?

> coded in python, so I don't quite know how to implement the solution, but I 
> do have a guess at what the solution is.  Push any toolchain package to the 
> top (or bottom, whichever) of the buildpkgs array in their proper order, 
> don't rely on portage.grab_stacked() or portage.stack_lists() to order it 
> correctly because it does not gracefully take into account the install is 
> happening in a different ROOT.  The DEPEND check in the glibc ebuild is 
> looking for headers in the running system, not what is installed in ROOT, 
> so it does not see it as a dependency that needs to be installed first.

See above.

> <rant>As an anal purist, I believe the toolchain should be installed first 
> and in the proper order explicitly one package at a time instead of in a 
> stacked list.  To do otherwise is heaping problems on yourself on down the 
> road if you ask me.  You also avoid this weird dependency problem when 
> installing into a ROOT.</rant>

We cannot enforce an order on an arbitrary set of packages.  That is
portage's job.  The problem that you were hitting was due to a bug in
the glibc ebuild which has been resolved.

> > Making a new profile is really very easy to do and with catalysts
> > portage_overlay features quite painless to do.  It works just like an
> > overlay on a host gentoo system.  I can understand that it might have
> > sounded difficult or time consuming to follow the recommendation but a
> > better course of action from you would have been how can I do this .. or
> > how do you guys do this.  If it sounds like it sucks for you maybe it
> > sucks for us too, but we just havent had the time to code a simpler way
> > yet.  We are not out to make anyones life difficult, remember we are
> > volunteers here with limited time.
> 
> Can you put profiles in overlays?  Duh, that would be much easier.

Yes.  You can even inherit from the profiles in the live tree.  A
portage overlay is a complete overlay.  You can provide any files you
require.

> > you are also missing the fact that the profile stays consistent from one
> > stage to the next.  custom use flags in a spec file can and will very
> > easily be missed or dropped or typoed etc.  It really is a better design
> > choice to do it with profiles.
> 
> Call this a feature request, or a suggestion.
> 
> Allow cflags, chosts, cxxflags, FEATURES, and use flags in only ONE place - 
> stage2.  Stage2 can generate the entire make.conf that will be used through 
> any future stages, including the user specified USE flags. Stage2 runs the 
> bootstrap.sh, which correctly throws out non-applicable USE flags, so it is 
> not a problem, all the catalyst developers would have to do is count on 
> bootstrap.sh to work correctly.

Most of this is done, except there are specific reasons for not doing as
you suggest.  For example, a stage1 can use differing FEATURES from a
stage2.  This is expected.  I will not ever allow customization of USE
flags in the lower stages (stages 1 through 3) as it breaks the static
concept of what each stage means.  The idea of a stage3 is it is a stage
that follows the profile's "system" target *to the letter* with *no
changes* in it.  Allowing for changes breaks this concept and will never
be accepted into catalyst.

> From that point forward every stage done by catalyst uses the USE flags 
> generated in step 2 and never changes.  If a USE flag breaks the install, 
> it would have broken during the install in stage4 anyway.  This would 
> greatly simplify everything in my opinion.

No.  A USE flag can break the "system" target but not break beyond that
point.  This means a USE flag may be safe during the creation of a
stage4 tarball but not during a stage3 tarball.  Why?  Because during
the creation of stages 1 though 3 we only have a limited set of packages
and even those packages are compiled and configured in minimal ways.
Until we reach the "system" target we do not have a guaranteed working
complete system to work with.

> Stage1 would continue to be the proper place to sanitize the build to 
> pre-selected cflags and use flags, etc...
> 
> Am I crazy, or what?

You said it, not I.  ;]

Remember that we've been using catalyst for more than 2 years now to do
releases and have seen what can happen.  Most of the restrictions that
currently exist in catalyst were not there when we first started using
it and have been added because they allow breakages and are generally
not safe.  We didn't set out to arbitrarily restrict building.  It
happened over time due to limitations in portage as we discovered them.

> > You assume that its a simple thing to compile a tool chain in order.
> > But what you miss is the fact that upstream developers consistently
> > change how things are done.  We have coded catalyst the best way for us.
> > Using profiles allows us to leverage other developers time in the tree
> > to catch a majority of the issues for us.  They can help reorganize the
> > way things are built if necessary via a profile.  Otherwise we would be
> > constantly updating our spec files and never know exactly what was going
> > on.
> 
> It _is_ a simple thing to bootstrap a toolchain in order.  The order has 
> always been the exact same and probably always will be.  binutils, gcc, 
> headers, glibc.  Repeat, using tools from previous step.  This is the

Except for uclibc... or libc on *BSD... or Mac OS... or Solaris...
remember that "Gentoo" is more than just Linux/glibc.

> _only_ way as far as I know to do it correctly, and to do it correctly 
> every time, and to be absolutely certain it has been done correctly.  It 
> has been this way for over 10 years if I recall correctly, it is linux 
> system building 101.

As a tool, catalyst utilizes portage for determining all building.  If
the portage tree is broken, it is broken.  It isn't catalyst's place to
sanitize it.  It is the Gentoo developers' job as a whole to do so.  A
"Gentoo" toolchain can be any number of various combinations of things.
It doesn't have to stick to the Linux kernel/GNU userland model that
other distributions maintain.

-- 
Chris Gianelloni
Release Engineering - Strategic Lead
x86 Architecture Team
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-catalyst] Couple of questions

[Mikey]

[-- Attachment #1: Type: text/plain, Size: 2486 bytes --]

On Monday 19 December 2005 10:06, Chris Gianelloni wrote:

> No.  A USE flag can break the "system" target but not break beyond that
> point.  This means a USE flag may be safe during the creation of a
> stage4 tarball but not during a stage3 tarball.  Why?  Because during
> the creation of stages 1 though 3 we only have a limited set of packages
> and even those packages are compiled and configured in minimal ways.
> Until we reach the "system" target we do not have a guaranteed working
> complete system to work with.

What legal USE flag, other than perhaps build/bootstrap, can break a stage3 
(system) build?  

If catalyst supports custom use flags via a profile, why not via make.conf?  
Is there an underlying difference between the way use flags are calculated 
depending on their presence in the profile versus make.conf?  

My primary motivation for wanting to be able to use USE flags in the lower 
stages is to shorten the build time and reduce bloat, by using USE flags 
that are perfectly valid for the toolchain packages.  I am building for 
servers, not desktops, and would rather avoid the extra step (stage3 -> 
stage4).  Catalyst would be an extremely useful tool for me if it were not 
for what seem to be arbitrary limitations.  All invalid USE flags are 
thrown out by bootstrap.sh, any standard USE flags should be available when 
emerging system, if they are not something is wrong with portage...  

It is much simpler in my mind to add USE="nptl -pam -nls" to make.conf than 
to create custom profiles, dick around with rsync-excludes to prevent them 
from being deleted when I sync, having to port them over to every new 
release, etc...

> Except for uclibc... or libc on *BSD... or Mac OS... or Solaris...
> remember that "Gentoo" is more than just Linux/glibc.

Which is kind of why the order of the bootstrap probably needs to be handled 
by the profile, not portage.  I would be willing to bet that the bootstrap 
process for every O/S supported by gentoo does, in fact, have to be done in 
a very specific order.  Probably off-topic for this list, but there really 
should be something along the lines of a toolchain set of packages in the 
profiles, similar to the method of using *category/package.  Probably in 
over my head here...

Anyhow, I won't bug you guys about it any more.  I'm sure you are probably 
getting a little tweaked about my whining when you are trying to get a 
release out :)


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-catalyst] Couple of questions

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 2831 bytes --]

On Tue, 2005-12-20 at 23:35 -0600, Mikey wrote:
> On Monday 19 December 2005 10:06, Chris Gianelloni wrote:
> 
> > No.  A USE flag can break the "system" target but not break beyond that
> > point.  This means a USE flag may be safe during the creation of a
> > stage4 tarball but not during a stage3 tarball.  Why?  Because during
> > the creation of stages 1 though 3 we only have a limited set of packages
> > and even those packages are compiled and configured in minimal ways.
> > Until we reach the "system" target we do not have a guaranteed working
> > complete system to work with.
> 
> What legal USE flag, other than perhaps build/bootstrap, can break a stage3 
> (system) build?

hal (or any other that requires kernel sources)

> If catalyst supports custom use flags via a profile, why not via make.conf?  
> Is there an underlying difference between the way use flags are calculated 
> depending on their presence in the profile versus make.conf?

Not so much as there is an underlying definition of what each stage
represents.

> My primary motivation for wanting to be able to use USE flags in the lower 
> stages is to shorten the build time and reduce bloat, by using USE flags 
> that are perfectly valid for the toolchain packages.  I am building for 
> servers, not desktops, and would rather avoid the extra step (stage3 -> 

THEN MAKE A PROFILE.

I'm not sure how I can make it any clearer without busting out the
<blink></blink> tags.

> stage4).  Catalyst would be an extremely useful tool for me if it were not 
> for what seem to be arbitrary limitations.  All invalid USE flags are 
> thrown out by bootstrap.sh, any standard USE flags should be available when 
> emerging system, if they are not something is wrong with portage...

What's a "standard" USE flag?  There are *tons* of examples of circular
dependencies and other issues with using certain USE flags during
"emerge system", which is the primary reason why we are removing the
lower stages from distribution.

> It is much simpler in my mind to add USE="nptl -pam -nls" to make.conf than 
> to create custom profiles, dick around with rsync-excludes to prevent them 
> from being deleted when I sync, having to port them over to every new 
> release, etc...

What is in your mind and what is truth don't exactly match up.  Why do
you need rsync-excludes?  Why do you need to port anything?  Either I am
not understanding or you don't exactly understand how a profile works.

You need exactly two files: parent and make.defaults

Point parent to the 2005.1 profile.  In make.defaults you need exactly 1
line, which is your changed USE from what 2005.1 defaults to.

-- 
Chris Gianelloni
Release Engineering - Strategic Lead
x86 Architecture Team
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]