[gentoo-dev] About linux-headers, making stages with catalyst

[gentoo-dev] About linux-headers, making stages with catalyst

[Lluís Batlle i Rossell]

Hi!

Well, I just got subscribed to this list... I don't know if I'm a gentoo
developer, but I think only here I'll find an answer to this question.

I'm a user of catalyst; I'm trying to make stages for some different
arquitectures, which use the newest portage and _at least_ linux 2.6 headers.
I'm using the profile portage/profiles/default-linux/x86/2004.2, and there are
virtuals which refer to linux 2.6.

When catalyst tries to build stage 1 (from a stage 3), it creates a "stage1root"
in /tmp. There are copied the portage profile, and little more. Then, "emerge"
is called for installing the base packages, using environment variable "ROOT".
That "emerge" call, as a dependence, tries to install "linux-headers". In a
similar case, using the same profile, it happens that "emerge" tries to install
"gentoo-sources-2.4..." when emerging "device-mapper" (which depends on
virtual/linux-source).

Where does "emerge" read that it has to install "2.4" things? There are only the
"virtuals" in the profile, isn't it? And they relate to 2.6... I really don't
know how to tell emerge not to use "2.4" things, as I know only that he reads
the "virtuals" for satisfying virtuals' dependencies.

I hope I wrote my problem clear enough. I'm not too good writting in English. :)

Thanks!

-- 
+-------------------------------------------------------+----------------------+
| Lluís Batlle i Rossell                                |Tel.Olot. 972 26 71 24|
| Membre de [s3os] (www.s3os.net)                       |    BCN.  93 16 22 680|
| viric@jabber.org / ICQ# 9658637                       |    Mòb.  654 08 67 35|
| +info personal: http://vicerveza.homeunix.net/~viric/ | Santa Pau / Catalunya|
+-------------------------------------------------------+----------------------+
Cita:
   - Press any key to accept the license.
   (Dell Computers)

--
gentoo-dev@gentoo.org mailing list

[gentoo-dev] Re: About linux-headers, making stages with catalyst

[Duncan]

Lluís Batlle i Rossell posted
<20041201222109.GA20954@vicerveza.homeunix.net>, excerpted below,  on Wed,
01 Dec 2004 23:21:09 +0100:

> That "emerge" call, as a dependence, tries to install "linux-headers".
> In a similar case, using the same profile, it happens that "emerge"
> tries to install "gentoo-sources-2.4..." when emerging "device-mapper"
> (which depends on virtual/linux-source).
> 
> Where does "emerge" read that it has to install "2.4" things? There are
> only the "virtuals" in the profile, isn't it? And they relate to 2.6...
> I really don't know how to tell emerge not to use "2.4" things, as I
> know only that he reads the "virtuals" for satisfying virtuals'
> dependencies.

OK, I know nothing about catalyst, and am just a user generally lurking on
the dev list, but... this one's a portage question at least in part, and I
/think/ I can answer that side of it.

Virtuals, in portage, mean any single package of a group of packages (all
with provides=<the virtual in question>) may meet the requirements.  If
one is already installed, great, that satisfies the requirement.  If no
such literal package fulfilling the requirement is yet installed, however,
portage falls back to a default choice.

What you have here is portage falling back to a 2.4 default choice, a 2.4
kernel and kernel-headers, while you want 2.6 versions.  For a live
install, you'd simply install your chosen 2.6 version which would then
provide the virtual you needed.

I'm on amd64 and as I said haven't worked with catalyst, but AFAIK, what
you need to do there is fix the profile such that the defaults are 2.6.
Why are you still using a 2004.2 profile for one thing?  The 2004.3
profile, if I'm not mistaken, defaults to kernel 2.6 along with updating
other requirements appropriately.  If you have no specific reason not to,
I'd suggest updating to the 2004.3 profile.

If there's a specific reason not to do 2004.3, keep in mind that the
profile you are using is a cascading profile (which means you should be
using portage 2.0.51 as .50 had issues with cascading profiles).  Thus,
the defaults from further up the tree are used if a profile itself doesn't
over-rule them.  Dirs further /down/ the tree are NOT used, but are there
for those who want them as a special case, therefore, the gcc34 subdir of
your profile is a special case of the 2004.2 profile, with 2.6 being a
special case of the gcc34 special case of the 2004.2 case of the x86 case
of the default-linux profile.  If you wish to use that 2004.2/gcc34/2.6
special case, you may do so, and it should change your requirements
accordingly.

If that still doesn't fit your rather customized case, then simply
customize the requirements.  Again, using cascading profiles, virtuals
from up the tree are used if nothing in the current profile dir overrides
them.  Thus, the virtuals file in x86 says use gentoo-sources (a 2.4
kernel) as the default virtual/linux-sources, while it falls back up to
default-linux to get the default for virtual/os-headers,
sys-kernel/linux-headers (kernel 2.4 headers).

Forcing 2.6 kernel and headers means placing a virtuals file in the
current profile, overriding those up the tree from it, with the defaults
you want, probably the same ones as in x86/2004.3, or in
x86/2004.2/gcc34/2.6, for those items.

Because I've never used catalyst, I'm not sure where you put edits to its
profile.  I assume you put them in the stage1root you mentioned, but
that's just a guess.  Read the documentation, or do a bit of experimenting.

-- 
Duncan - List replies preferred.   No HTML msgs.
"They that can give up essential liberty to obtain a little
temporary safety, deserve neither liberty nor safety." --
Benjamin Franklin



--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: About linux-headers, making stages with catalyst

[Lluís Batlle i Rossell]

Thanks a lot for the answers.
(I answer between your lines)

On Wed, Dec 01, 2004 at 06:21:20PM -0700, Duncan wrote:
> 
> OK, I know nothing about catalyst, and am just a user generally lurking on
> the dev list, but... this one's a portage question at least in part, and I
> /think/ I can answer that side of it.
> 
> Virtuals, in portage, mean any single package of a group of packages (all
> with provides=<the virtual in question>) may meet the requirements.  If
> one is already installed, great, that satisfies the requirement.  If no
> such literal package fulfilling the requirement is yet installed, however,
> portage falls back to a default choice.
The default choice is specified in the virtuals, isn't it?

> 
> What you have here is portage falling back to a 2.4 default choice, a 2.4
> kernel and kernel-headers, while you want 2.6 versions.  For a live
> install, you'd simply install your chosen 2.6 version which would then
> provide the virtual you needed.
> 
> I'm on amd64 and as I said haven't worked with catalyst, but AFAIK, what
> you need to do there is fix the profile such that the defaults are 2.6.
> Why are you still using a 2004.2 profile for one thing?  The 2004.3
> profile, if I'm not mistaken, defaults to kernel 2.6 along with updating
> other requirements appropriately.  If you have no specific reason not to,
> I'd suggest updating to the 2004.3 profile.
Ok, I'll just try the 2004.3 profile. I thought 2004.2 had 2.6 virtuals.
---
I just tried the profile. There isn't a virtuals definition in it!
(Portage 20041108)

> 
> If there's a specific reason not to do 2004.3, keep in mind that the
> profile you are using is a cascading profile (which means you should be
> using portage 2.0.51 as .50 had issues with cascading profiles).  Thus,
> the defaults from further up the tree are used if a profile itself doesn't
> over-rule them.  Dirs further /down/ the tree are NOT used, but are there
> for those who want them as a special case, therefore, the gcc34 subdir of
> your profile is a special case of the 2004.2 profile, with 2.6 being a
> special case of the gcc34 special case of the 2004.2 case of the x86 case
> of the default-linux profile.  If you wish to use that 2004.2/gcc34/2.6
> special case, you may do so, and it should change your requirements
> accordingly.

I'm using portage 2.0.51-r4. I didn't know how cascade profiles work... I
imagined that they worked that way similar...

> 
> If that still doesn't fit your rather customized case, then simply
> customize the requirements.  Again, using cascading profiles, virtuals
> from up the tree are used if nothing in the current profile dir overrides
> them.  Thus, the virtuals file in x86 says use gentoo-sources (a 2.4
> kernel) as the default virtual/linux-sources, while it falls back up to
> default-linux to get the default for virtual/os-headers,
> sys-kernel/linux-headers (kernel 2.4 headers).

Are you talking about using the profile "portage/profiles/default-linux" instead
of "portage/profiles/default-linux/x86/2004.2" ?
In which case is "portage/profiles/default-linux/virtuals" read? Will that file
be read, if I have the profile 2004.3 or 2004.2? And what about the
portage/profiles/default-linux/x86/virtuals?

> 
> Forcing 2.6 kernel and headers means placing a virtuals file in the
> current profile, overriding those up the tree from it, with the defaults
> you want, probably the same ones as in x86/2004.3, or in
> x86/2004.2/gcc34/2.6, for those items.
Hummm I think _now_ I understand the problem. I read the virtuals file in
x86/2004.2/gcc34/2.6, and there was 2.6 virtuals. So, I thought that the
x86/2004.2 profile had 2.6 virtuals. I thought that the cascade was going down
to directories, and not up! I thought that that was strange... :)

> 
> Because I've never used catalyst, I'm not sure where you put edits to its
> profile.  I assume you put them in the stage1root you mentioned, but
> that's just a guess.  Read the documentation, or do a bit of experimenting.

Aha. I was in confusion with the 'direction' of the cascade, so I thought 2004.2
had virtuals.

What I've tried, while answering this email:
I've tried using 2004.3 => Still keeps on using 2.4 (there are no virtual
definitions in 2004.3!)
I've tried using default-linux/x86/2004.2/gcc34/2.6/ as profile => Even this way
emerge still keeps on using 2.4
I've tried changing the content of the virtuals in default-linux and
default-linux/x86 => YES! Now it installs 2.6 headers.

But... Shouldn't the other tries work ??? At least, you say that about 2004.3,
and the profile ....gcc34/2.6/ seems to have virtuals defined there.

Thanks!

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] About linux-headers, making stages with catalyst

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 1769 bytes --]

On Wed, 2004-12-01 at 23:21 +0100, Lluís Batlle i Rossell wrote:
> I'm a user of catalyst; I'm trying to make stages for some different
> arquitectures, which use the newest portage and _at least_ linux 2.6 headers.
> I'm using the profile portage/profiles/default-linux/x86/2004.2, and there are
> virtuals which refer to linux 2.6.

I made this profile and I can assure you that 2.4 headers and a 2.4
kernel are the defaults for it and the 2004.3 profile.  There is not a
x86 profile in the portage tree that does not have 2.4 as the defaults
yet.

> When catalyst tries to build stage 1 (from a stage 3), it creates a "stage1root"
> in /tmp. There are copied the portage profile, and little more. Then, "emerge"
> is called for installing the base packages, using environment variable "ROOT".
> That "emerge" call, as a dependence, tries to install "linux-headers". In a
> similar case, using the same profile, it happens that "emerge" tries to install
> "gentoo-sources-2.4..." when emerging "device-mapper" (which depends on
> virtual/linux-source).

This is expected behavior thanks to the profile's virtuals.

> Where does "emerge" read that it has to install "2.4" things? There are only the
> "virtuals" in the profile, isn't it? And they relate to 2.6... I really don't
> know how to tell emerge not to use "2.4" things, as I know only that he reads
> the "virtuals" for satisfying virtuals' dependencies.

You are correct in that the virtuals file is read.  The easiest way
would be for you to create your own profile and use that instead.

> I hope I wrote my problem clear enough. I'm not too good writting in English. :)

-- 
Chris Gianelloni
Release Engineering - Operational/QA Manager
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] About linux-headers, making stages with catalyst

[Lluís Batlle i Rossell]

On Thu, Dec 02, 2004 at 09:26:03AM -0500, Chris Gianelloni wrote:
> You are correct in that the virtuals file is read.  The easiest way
> would be for you to create your own profile and use that instead.

Ok! Thanks a lot. You tried the same as me, and got the same results. Now I
understand the virtuals 'cascade'. But I think I could use profile
"default-linux/x86/2004.2/gcc34/2.6/", isn't it? Anyway, using it, emerge keeps
on trying to get linux-headers for 2.4.

(By now, I changed the virtuals in default-linux/x86 and default-linux). That
isn't very pretty... but that's what I did. ;)

-- 
+-------------------------------------------------------+----------------------+
| Lluís Batlle i Rossell                                |Tel.Olot. 972 26 71 24|
| Membre de [s3os] (www.s3os.net)                       |    BCN.  93 16 22 680|
| viric@jabber.org / ICQ# 9658637                       |    Mòb.  654 08 67 35|
| +info personal: http://vicerveza.homeunix.net/~viric/ | Santa Pau / Catalunya|
+-------------------------------------------------------+----------------------+
Cita:
   - Press any key to accept the license.
   (Dell Computers)

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] About linux-headers, making stages with catalyst

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 770 bytes --]

On Thu, 2004-12-02 at 09:26 -0500, Chris Gianelloni wrote:
> > When catalyst tries to build stage 1 (from a stage 3), it creates a "stage1root"
> > in /tmp. There are copied the portage profile, and little more. Then, "emerge"
> > is called for installing the base packages, using environment variable "ROOT".
> > That "emerge" call, as a dependence, tries to install "linux-headers". In a
> > similar case, using the same profile, it happens that "emerge" tries to install
> > "gentoo-sources-2.4..." when emerging "device-mapper" (which depends on
> > virtual/linux-source).

By the way, the catalyst experts are on the gentoo-releng mailing
list... ;]

-- 
Chris Gianelloni
Release Engineering - Operational/QA Manager
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] About linux-headers, making stages with catalyst

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 1111 bytes --]

On Thu, 2004-12-02 at 15:33 +0100, Lluís Batlle i Rossell wrote:
> On Thu, Dec 02, 2004 at 09:26:03AM -0500, Chris Gianelloni wrote:
> > You are correct in that the virtuals file is read.  The easiest way
> > would be for you to create your own profile and use that instead.
> 
> Ok! Thanks a lot. You tried the same as me, and got the same results. Now I
> understand the virtuals 'cascade'. But I think I could use profile
> "default-linux/x86/2004.2/gcc34/2.6/", isn't it? Anyway, using it, emerge keeps
> on trying to get linux-headers for 2.4.

There is no profile in portage using 2.6 headers as default that I am
aware of.

> (By now, I changed the virtuals in default-linux/x86 and default-linux). That
> isn't very pretty... but that's what I did. ;)

You probably would have done better to have created a sub-profile of
default-linux/x86/2004.3 to suit your needs.  You would have only have
had to have changed the virtuals file to include "virtual/os-headers
linux26-headers"

-- 
Chris Gianelloni
Release Engineering - Operational/QA Manager
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] About linux-headers, making stages with catalyst

[Lluís Batlle i Rossell]

On Thu, Dec 02, 2004 at 11:53:05AM -0500, Chris Gianelloni wrote:
> On Thu, 2004-12-02 at 15:33 +0100, Lluís Batlle i Rossell wrote:
> > Ok! Thanks a lot. You tried the same as me, and got the same results. Now I
> > understand the virtuals 'cascade'. But I think I could use profile
> > "default-linux/x86/2004.2/gcc34/2.6/", isn't it? Anyway, using it, emerge keeps
> > on trying to get linux-headers for 2.4.
> 
> There is no profile in portage using 2.6 headers as default that I am
> aware of.

Hummm
Take a look at /usr/portage/profiles/default-linux/x86/2004.2/gcc34/2.6/



-- 
+-------------------------------------------------------+----------------------+
| Lluís Batlle i Rossell                                |Tel.Olot. 972 26 71 24|
| Membre de [s3os] (www.s3os.net)                       |    BCN.  93 16 22 680|
| viric@jabber.org / ICQ# 9658637                       |    Mòb.  654 08 67 35|
| +info personal: http://vicerveza.homeunix.net/~viric/ | Santa Pau / Catalunya|
+-------------------------------------------------------+----------------------+
Cita:
   - Press any key to accept the license.
   (Dell Computers)

--
gentoo-dev@gentoo.org mailing list

[gentoo-dev] Re: Re: About linux-headers, making stages with catalyst

[Duncan]

Lluís Batlle i Rossell posted
<20041202110107.GA27739@vicerveza.homeunix.net>, excerpted below,  on Thu,
02 Dec 2004 12:01:07 +0100:

> What I've tried, while answering this email: I've tried using 2004.3 =>
> Still keeps on using 2.4 (there are no virtual definitions in 2004.3!)
> I've tried using default-linux/x86/2004.2/gcc34/2.6/ as profile => Even
> this way emerge still keeps on using 2.4
> I've tried changing the content of the virtuals in default-linux and
> default-linux/x86 => YES! Now it installs 2.6 headers.
> 
> But... Shouldn't the other tries work ??? At least, you say that about
> 2004.3, and the profile ....gcc34/2.6/ seems to have virtuals defined
> there.

OK, I see statements CG's statement that no x86 profile defaults to 2.6
headers for the virtuals, that he knows of, but you point out
profiles/default-linux/x86/2004.2/gcc34/2.6/, as I did.

Where a virtual is required, cascading profiles should read parent dirs
until they find the default, if it's not in the current profile dir. Thus,
the lack of a virtual definition file (or one not including all necessary
virtuals) shouldn't be alarming, as it should just look up-tree to fill
the requirement.

Because profiles/default-linux/x86/2004.3 has a 2.4 subdir, and I'd seen
discussion on the dev group/list on switching to 2.6 by default, I wrongly
concluded 2004.3 on x86 had done so.  It appears I was wrong.  As you
mention, there's no virtuals there, so it looks up-tree, to x86, where it
finds virtual/linux-sources defaults to sys-kernel/gentoo-sources.  That
covers the kernel virtual default, but not headers, so it looks up-tree
another notch, and finds them under default-linux, where
virtual/os-headers defaults to sys-kernel/linux-headers.  Note that it
/also/ mentions virtual/kernel, defaulting that to
sys-kernel/vanilla-sources.

This means that we have three separate virtuals provided.
virtual/os-headers defaults to the 2.4 series linux-headers.  However, we
have TWO DIFFERENT virtuals for kernel, virtual/kernel, and
virtual/linux-sources.  I'm GUESSING these should be one and the same, but
in any case, they both default to 2.4 series kernels altho different
branches of the series.  Which one (or both) would try to be merged would
therefore be dependent on which virtual individual packages called for.

I thought to simplify matters, quickly clarifying which /should/ be used,
by checking my arch, amd64.  However, the virtuals file there only ADDS to
the confusion, because instead of virtual/kernel pointing to a kernel, it
points to headers.  Of course the 2.4 kernel is depreciated on amd64, so
everything points to 2.6 versions, but both virtual/os-headers and
virtual/kernel point to sys-kernel/linux26-headers, while
virtual/linux-sources points to sys-kernel/gentoo-dev-sources.

Thus, virtual/kernel points to kernel sources in the one place, and only
kernel headers in another, which is certainly confusing me!

Anyway, back to x86, it indeed appears that
profiles/default-linux/x86/2004.2/gcc34/2.6 defaults to 2.6 versions of
both headers and kernel:
virtual/kernel                  sys-kernel/linux26-headers
virtual/os-headers              sys-kernel/linux26-headers
virtual/linux-sources           sys-kernel/gentoo-dev-sources

Hmm.. here too we have virtual/kernel pointing to headers, not sources, as
in amd64, but NOT as the default-linux virtuals file, where virtual/kernel
points to kernel sources instead of just kernel headers.

Anyway, it would seem that profiles/default-linux/x86/2004.2/gcc34/2.6
does what you want, PROVIDED you don't have any problems with gcc-3.4,
which is of course what the gcc34 refers to.  That sub-profile defines all
three virtuals, kernel, headers, and sources, to 2.6 defaults.

Note that catalyst most likely will *NOT* notice the changed profile,
unless you erase its previous tmp files.  I'm suspecting that's the other
problem you are experiencing now.  It has its config, and doesn't go
looking to change that, unless it disappears.  Therefore, after changing
your profile to the above 2004.2/gcc34/2.6, erase catalyst's temp files
and see if it recreates them using the desired 2.6 minimals, now.

Alternatively and the reason catalyst (again, presumably) works as it
does, is that it allows you to modify its temp file virtuals once it gets
past that point, thus allowing you to build a live-cd with a modified
profile, without affecting the build-system's profile.  Thus, instead of
erasing the tmp files so catalyst sees the new configuration, consider
editing the catalyst version directly instead.  However, I'd say do as
little editing here as possible, because it's possible tweaking one thing
there will cause something else problems.  An example would be tweaking
the headers virtual to linux26-headers, while leaving the kernel at 2.4
virtuals.  Gentoo's pre-built profiles should have those sort of
dependencies worked out, while editing them could cause problems due to
interlinking dependencies you weren't aware of when you did the editing. 
Thus, I'd recommend doing the above, change the system profile and erase
catalyst's tmp files so it regenerates from that, instead of manually
tweaking its profile, preventing potentially nasty surprises with
conflicting dependencies.

Also...  you mention changing the content of the virtuals.  You don't
mention specifically /where/ you changed that content, in the catalyst
temp files (OK), or directly in the portage tree itself (not so OK).  The
problem with changing it in the portage tree is that an emerge sync will
erase the changes you made (unless you have rsync-ignore set on that dir,
which creates problems of its own if Gentoo changes anything).  For a
one-shot catalyst build, and as long as you don't go doing an emerge sync
while it's running, you should be fine, but just be aware that emerge sync
/will/ erase changes you've made to the profiles in the portage tree
itself.

-- 
Duncan - List replies preferred.   No HTML msgs.
"They that can give up essential liberty to obtain a little
temporary safety, deserve neither liberty nor safety." --
Benjamin Franklin



--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] About linux-headers, making stages with catalyst

[Daniel Goller]

that is my doing back when gcc3.4 was profile masked, i could even 
boostrap with it at the time
it was only an escape to make gcc3.4 ~x86 w/o anyone pulling it but 
those who chose to switch to

default-linux/x86/2004.2/gcc34/2.6/


Chris Gianelloni wrote:

>On Thu, 2004-12-02 at 15:33 +0100, Lluís Batlle i Rossell wrote:
>  
>
>>On Thu, Dec 02, 2004 at 09:26:03AM -0500, Chris Gianelloni wrote:
>>    
>>
>>>You are correct in that the virtuals file is read.  The easiest way
>>>would be for you to create your own profile and use that instead.
>>>      
>>>
>>Ok! Thanks a lot. You tried the same as me, and got the same results. Now I
>>understand the virtuals 'cascade'. But I think I could use profile
>>"default-linux/x86/2004.2/gcc34/2.6/", isn't it? Anyway, using it, emerge keeps
>>on trying to get linux-headers for 2.4.
>>    
>>
>
>There is no profile in portage using 2.6 headers as default that I am
>aware of.
>
>  
>
>>(By now, I changed the virtuals in default-linux/x86 and default-linux). That
>>isn't very pretty... but that's what I did. ;)
>>    
>>
>
>You probably would have done better to have created a sub-profile of
>default-linux/x86/2004.3 to suit your needs.  You would have only have
>had to have changed the virtuals file to include "virtual/os-headers
>linux26-headers"
>
>  
>

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] About linux-headers, making stages with catalyst

[Lluís Batlle i Rossell]

Oh, ok. :)

So I better make my own profile for the linux 2.6 headers and sources.

Thanks!

On Thu, Dec 02, 2004 at 09:15:37PM -0600, Daniel Goller wrote:
> that is my doing back when gcc3.4 was profile masked, i could even 
> boostrap with it at the time
> it was only an escape to make gcc3.4 ~x86 w/o anyone pulling it but 
> those who chose to switch to
> 
> default-linux/x86/2004.2/gcc34/2.6/
> 
> 

--
gentoo-dev@gentoo.org mailing list