[gentoo-catalyst] Sort of Build Platform

[gentoo-catalyst] Sort of Build Platform

[Shinkan]

[-- Attachment #1: Type: text/plain, Size: 1708 bytes --]

Hi there !

I'm quitely new to Gentoo, but I'm loving it, so I have projects with it !
Some of them may require Catalyst in its process from what I understood
about Catalyst.

I'll copy/paste a forum post I've made before I knew there was a mailing
list for Catalyst :

OK, let me expose things differently since I know a little more what I would
like to do and what Gentoo could offer.

- I have a local mirror, which may have overlay packages.
- I have a host system, which have Gentoo setup on it.
- I want to build barely-usable minimal systems that I call "guests" from my
host. Guests would have to be setup on target machines using fdisk and tar
only, or put on a Live{CD,DVD,USB}.

Assuming that I can kinda cross-compile guests binaries from my host system
with emerge (using --root and --portage_configdir, guests have same arch
than host), I would like my host system to entirely build my guest BUT I
want my guest to be extremely minimal. My guest don't have to have
sys-devel/* things like binutils, gcc and so on. It just have to run a few
already compiled C++ binaries. Everything that my guest need will be emerged
from my host.

How can I proceed (the clean way !) from my host to emerge (or copy ?) a
fully minimal system to a host dir, like say /guest/ ?
Did I have to "create" a profile on my local mirror which tells that my
"system" meta-port have almost nothing in it ? How can I do this if it's the
clean way ?
Did I have to use Catalyst ? How can I do this if it's the clean way ?

Many thanks in advance.

-- 
Pierre.
"Sometimes when I'm talking, my words can't keep up with my thoughts. I
wonder why we think faster than we speak. Probably so we can think twice." -
Bill Watterson

[-- Attachment #2: Type: text/html, Size: 2257 bytes --]

Re: [gentoo-catalyst] Sort of Build Platform

[Peter Stuge]

Shinkan wrote:
> - I want to build barely-usable minimal systems that I call
> "guests" from my host. Guests would have to be setup on target
> machines using fdisk and tar only, or put on a Live{CD,DVD,USB}.
..
> Did I have to use Catalyst ? How can I do this if it's the clean
> way ?

You can certainly do it with a catalyst stage4 spec file. You'll also
need to prepare a kernel configuration for the guests, and reference
that in the spec file. You'll also spend some time on filling the
spec file with packages, files and directories that should be
unmerged and simply rm:ed from the final build.

To make changes (like add another package to a guest) you would run
catalyst again. It starts over, but keeps a cache of binpkgs that
have been built so it runs well under an hour even on oldish systems.

This is how I make custom distributions for customers.

Since the guests will be very different from a standard Gentoo system
it may also be worthwhile for you to create a new, custom, profile
for the guests.


//Peter

Re: [gentoo-catalyst] Sort of Build Platform

[Shinkan]

[-- Attachment #1: Type: text/plain, Size: 2834 bytes --]

Thank you for your answer.
By inspecting catalyst examples specs, especially stage4 one, I figured that
catalyst could do the job.

Right now I proceed by "cross-emerging" from a Gentoo host to a target dir
with emerge's --root and --config-dir opts.
With that, I can start with an empty target dir, and emerge into it
baselayout, bash and glibc. That's all I need for now.
Then I cross compile needed kernel to my target dir following Gentoo
cross-dev guide :
http://www.gentoo.org/proj/en/base/embedded/cross-development.xml#doc_chap6

That's what I need. But that's all hand-crafted scripts to always build the
same things.
Plus, I will have to make my target dir a LiveCD just after, so I'll have to
script more.

I thought Catalyst could do the exact same thing with a more Gentoo-ish way
and sort-of cache settings.
But I'm still not sure. Plus I don't get the "stage"-related thing in spec.
I read FAQ and get it a little more.

Here are the 2 crucial points that I don't want to evade :
1) I do not want to build with a "deploy base Gentoo system, then remove"
strategy (won't this break things ?). I really want to start from nothing,
or from a almost-empty root dir/tar I define. Because I want my target
system to be a "oneshot" system. It will not use portage, or gcc. It will
just run applications.
2) I do not want my target system to bootstrap itselfs by compilating its
binaries. I want my host system to build target things. Because of 1)
reasons.

Can catalyst still do the job ?
Why exactly should I make a profile ?

2009/11/24 Peter Stuge <peter@stuge.se>

> Shinkan wrote:
> > - I want to build barely-usable minimal systems that I call
> > "guests" from my host. Guests would have to be setup on target
> > machines using fdisk and tar only, or put on a Live{CD,DVD,USB}.
> ..
> > Did I have to use Catalyst ? How can I do this if it's the clean
> > way ?
>
> You can certainly do it with a catalyst stage4 spec file. You'll also
> need to prepare a kernel configuration for the guests, and reference
> that in the spec file. You'll also spend some time on filling the
> spec file with packages, files and directories that should be
> unmerged and simply rm:ed from the final build.
>
> To make changes (like add another package to a guest) you would run
> catalyst again. It starts over, but keeps a cache of binpkgs that
> have been built so it runs well under an hour even on oldish systems.
>
> This is how I make custom distributions for customers.
>
> Since the guests will be very different from a standard Gentoo system
> it may also be worthwhile for you to create a new, custom, profile
> for the guests.
>
>
> //Peter
>
>


-- 
Pierre.
"Sometimes when I'm talking, my words can't keep up with my thoughts. I
wonder why we think faster than we speak. Probably so we can think twice." -
Bill Watterson

[-- Attachment #2: Type: text/html, Size: 3576 bytes --]

Re: [gentoo-catalyst] Sort of Build Platform

[Peter Stuge]

Pierre,

Please don't top-post.


Shinkan wrote:
> I thought Catalyst could do the exact same thing with a more
> Gentoo-ish way and sort-of cache settings.

Yes, that is also a good summary of what I wrote.


> But I'm still not sure.

Be sure.


> Here are the 2 crucial points that I don't want to evade :
> 1) I do not want to build with a "deploy base Gentoo system, then
> remove" strategy
..
> 2) I do not want my target system to bootstrap itselfs
..
> Can catalyst still do the job ?

For the fourth time: catalyst does what you want.

It creates a stage tarball according to instructions in a spec file.
It uses emerge to build and install all packages in the stage. It
saves binary packages of everything that has been built to save time
if you want to make changes and run catalyst again. You specify how
you want the tarball (or livecd) to be produced, by adding
information to the spec file.


> Why exactly should I make a profile ?

Because your target or guest system will be sufficiently different
from what the Gentoo profiles are intended for.


In your case another idea is to create a simple script that runs
after the stage4 is completed. Then you will not unmerge, empty or rm
anything in the stage4 spec file, but use your script to create the
deployment tarball with only the specific binaries and libraries that
you want. Of course avoiding the script has the big advantage that
you can encapsulate everything in a spec file and have catalyst
generate the final product, but it will have a long list with lots of
files that should _not_ be included. Having your own profile helps in
shortening that list too.


//Peter

Re: [gentoo-catalyst] Sort of Build Platform

[Shinkan]

[-- Attachment #1: Type: text/plain, Size: 1713 bytes --]

2009/11/25 Peter Stuge

> Pierre,
>
> Please don't top-post.
>

Thanks again and sorry but I don't get what "top-post" means ... I guess
it's about answering on top of history mails ?
I'm kinda new to mailing-lists...


>  > Can catalyst still do the job ?
>
> For the fourth time: catalyst does what you want.
>

Sorry for that too, I'm also new to Gentoo, and for an unknown reason, I
trust it for a big project, so I must be sure of many things...


> It uses emerge to build and install all packages in the stage. It
>

It's not clear to me on that point: is the
"currently-being-built-stage-or-livecd" emerge used, or is it the
"catalyst-running-host" emerge ?


> Because your target or guest system will be sufficiently different
> from what the Gentoo profiles are intended for.
>

After many doc reads, what you say now seems obvious to me. I must create a
profile to specify base systems packages, and many "default" things.
My problem is that it appears that it cannot be done properly and easily :
Catalyst requires a /usr/portage related profile path, but putting a custom
profile here would kinda break original Gentoo portage tree.
I tried to put a custom profile in /usr/local/portage/profiles and
specifying a "PORTDIR_OVERLAY" in my make.conf, but the profile didn't show
up in "eselect profile list" so I guess Catalyst won't see it.
How can I tell catalyst to use my custom profile without putting
"will-be-deleted-on-next-sync" garbare in official portage tree ?
I didn't found any doc on this.

Thanks in advance.

-- 
Pierre.
"Sometimes when I'm talking, my words can't keep up with my thoughts. I
wonder why we think faster than we speak. Probably so we can think twice." -
Bill Watterson

[-- Attachment #2: Type: text/html, Size: 2667 bytes --]