[gentoo-embedded] generating root file systems

[gentoo-embedded] generating root file systems

[Christopher Harvey]

I was wondering if there was any interest in adding a script that could 
generate a bootable linux filesystem from a cross compiled target 
directory.
For example:

arm-unknown-linux-gnueabi-make-root   (or any arch)

which would do something like:

cp -ar /usr/arm-unknown-linux-gnueabi /tmp/target
mkdir /tmp/target/proc
mkdir -p /tmp/target/var/run
#generate an inittab
then perform a mix of the following:
checks to make sure that all required packages are present to make a 
system bootable
package specific setup like starting services if ssh servers are 
installed.
create device nodes
set root password
call hooks in ~/.crossdev-makeroot/
install kernel modules
warn about missing packages that could be useful.
package a cpio (or something else depending on passed options)

I've got a script that is a mix of my own stuff plus some stuff I 
ripped off buildroot. I find it really handy, but isn't a general enough 
to work on anything except my own boards.

It's kinda tricky to write a script that will work for most users out 
of the box since I only have a couple of embedded machines, so I was 
wondering how many people would be interested and how many people could 
test on their own boards. I was thinking I could create an ebuild and a 
git repo then let people hack at it and see where it goes, and how many 
boards it can support at once.

Re: [gentoo-embedded] generating root file systems

[Ed W]

On 07/07/2011 16:27, Christopher Harvey wrote:
> I was wondering if there was any interest in adding a script that could
> generate a bootable linux filesystem from a cross compiled target
> directory.

I would be happy to work with you on this - however, I also saw on
Planet Gentoo something about a gentoaster project - possibly related?
	http://hexxeh.net/?p=328117725

What I have is a very simple portage-a-like to support the case that I
have several stages to my script.

- Mounting all necessary sources
- Building a base system
- Copying base to target ready to be imaged
- Image
- Unmount

Additionally I am trying a packaging system using aufs + squashfs, so I
am building more than one target image and the whole gets stacked
together at the end

The scripting is fairly simple, but I guess this could develop into some
best practices..?

Arguably this is re-inventing catalyst, but speaking from inexperience,
catalyst appears to work best when the ebuilds are perfect and need to
micro patching - for embedded builds some flexibility is very helpful.

Patching ebuilds in mind: I have been experimenting with
/etc/portage/patches and also the bashrc for broad patching, eg where
some long standing patch or config customisation is necessary (eg delete
some openrc file which makes no sense, or customise some udev config, etc)

I guess if we could get this well written up then it would encourage
others to look at gentoo for embedded development?  At the moment it's
very powerful, but I sense there is a steep learning curve while you
develop your own build environment?

How do you think we should proceed?

Ed W

Re: [gentoo-embedded] generating root file systems

[Christopher Harvey]

On Wed, 13 Jul 2011 11:54:40 +0100, Ed W wrote:
> On 07/07/2011 16:27, Christopher Harvey wrote:
>> I was wondering if there was any interest in adding a script that 
>> could
>> generate a bootable linux filesystem from a cross compiled target
>> directory.
>
> I would be happy to work with you on this - however, I also saw on
> Planet Gentoo something about a gentoaster project - possibly 
> related?
> 	http://hexxeh.net/?p=328117725
possibly, but I don't understand the whole web based thing.
>
> What I have is a very simple portage-a-like to support the case that 
> I
> have several stages to my script.
>
> - Mounting all necessary sources
> - Building a base system
> - Copying base to target ready to be imaged
> - Image
> - Unmount
>
> Additionally I am trying a packaging system using aufs + squashfs, so 
> I
> am building more than one target image and the whole gets stacked
> together at the end
>
> The scripting is fairly simple, but I guess this could develop into 
> some
> best practices..?
yeah my script is a mess, but I still think it useful because it gets 
all the basic steps down which I know would help some people.
>
> Arguably this is re-inventing catalyst, but speaking from 
> inexperience,
> catalyst appears to work best when the ebuilds are perfect and need 
> to
> micro patching - for embedded builds some flexibility is very 
> helpful.
Well, catalyst was designed to build a gentoo image. I want to build 
embedded images that have nothing gentoo specific about them. No 
portage, no python, no eselect or anything else like that.
>
> Patching ebuilds in mind: I have been experimenting with
> /etc/portage/patches and also the bashrc for broad patching, eg where
> some long standing patch or config customisation is necessary (eg 
> delete
> some openrc file which makes no sense, or customise some udev config, 
> etc)
I've never heard of that file, '/etc/portage/patches', and can't find 
it in man portage.
>
> I guess if we could get this well written up then it would encourage
> others to look at gentoo for embedded development?  At the moment 
> it's
> very powerful, but I sense there is a steep learning curve while you
> develop your own build environment?
>
> How do you think we should proceed?
There always is a learning curve for embedded and it will be impossible 
to support every single configuration for every single board. Basically 
my plan was to try to logically split all the steps in making a 
filesystem image and put them into a clear well documented bash script, 
then let users tweak it by calling hooks at each step. Hopefully after a 
while if people start adding steps and options it would support most 
base filesystem images people want to boot a machine with.

I can drop what a have so far into a git repo and we can go from there 
taking the best from each of our scripts. Do you want to make the repo 
or should I?
>
> Ed W

-Chris

Re: [gentoo-embedded] generating root file systems

[Ed W]

> Well, catalyst was designed to build a gentoo image. I want to build
> embedded images that have nothing gentoo specific about them. No
> portage, no python, no eselect or anything else like that.

Actually (and speaking with no real experience), I believe it can build
exactly that...  However, it's big and complicated as solutions go (also
there is the tool from Funtoo)


>> Patching ebuilds in mind: I have been experimenting with
>> /etc/portage/patches and also the bashrc for broad patching, eg where
>> some long standing patch or config customisation is necessary (eg delete
>> some openrc file which makes no sense, or customise some udev config,
>> etc)
> I've never heard of that file, '/etc/portage/patches', and can't find it
> in man portage.

Just create /etc/portage/patches/net-dns/dnscache/somepatch.patch

In theory the docs said that if the file was called
dnscache-1.2.3-mypatch.patch, then it would only apply to that version
number, but for me it seems all patches are applied (I rename them to
exclude them)

Additionally the hooks for each stage of portage are accessible from
/etc/portage/bashrc


> There always is a learning curve for embedded and it will be impossible
> to support every single configuration for every single board. Basically
> my plan was to try to logically split all the steps in making a
> filesystem image and put them into a clear well documented bash script,

Sure - actually I just have a base file called "mod", that includes your
"recipe" file and then it calls functions: mount_deps(), build(),
target(), unmount_deps() from the recipe file.  It's the provider of the
recipe's job to fill in each of those functions

It means I can call "./mod build 0.1" repeatedly (the number is the
package version number) until I'm happy and then call "./mod target
0.1", etc

> I can drop what a have so far into a git repo and we can go from there
> taking the best from each of our scripts. Do you want to make the repo
> or should I?

I think you should knock up some repo (probably github is a good choice
because it's so easy to fork).

If you are interested I will email over some samples of my small scripts
and you can see if they are interesting to work into your basic environment?

Cheers

Ed W