[gentoo-catalyst] How is initrd built?

[gentoo-catalyst] How is initrd built?

[Paul Smith]

How are the modules to be included in the initrd computed?

I'm seeing something strange where some of the modules in the real
filesystem version of the /lib/modules/.../drivers/scsi are missing from
the initrd filesystem version of the same directory; in particular one
of the drivers for my SATA disks (ata_piix) is not there.  Others are
missing as well.


It's genkernel that's doing this, right?  I'm trying to follow along in
the gen_initrd.sh script but I can't quite see what it's doing WRT
modules.

Any hints?

-- 
-------------------------------------------------------------------------------
 Paul D. Smith <psmith@nortel.com>           HASMAT: HA Software Mthds & Tools
 "Please remain calm...I may be mad, but I am a professional." --Mad Scientist
-------------------------------------------------------------------------------
   These are my opinions---Nortel Networks takes no responsibility for them.
-- 
gentoo-catalyst@gentoo.org mailing list

Re: [gentoo-catalyst] How is initrd built?

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 1100 bytes --]

On Wed, 2005-07-06 at 09:45 -0400, Paul Smith wrote:
> How are the modules to be included in the initrd computed?
> 
> I'm seeing something strange where some of the modules in the real
> filesystem version of the /lib/modules/.../drivers/scsi are missing from
> the initrd filesystem version of the same directory; in particular one
> of the drivers for my SATA disks (ata_piix) is not there.  Others are
> missing as well.
> 
> 
> It's genkernel that's doing this, right?  I'm trying to follow along in
> the gen_initrd.sh script but I can't quite see what it's doing WRT
> modules.

genkernel's /usr/share/genkernel/$arch/modules_load

I recommend unmasking the latest genkernel 3.2.0_pre version and using
that, as it will be going stable shortly.  I also recommend doing the
same with the masked version of catalyst.

I *NEED* testers for these, as I plan on them going stable before 2005.1
is released and I would love to resolve as many bugs in them as
possible.

-- 
Chris Gianelloni
Release Engineering - Strategic Lead/QA Manager
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-catalyst] How is initrd built?

[Paul Smith]

%% Chris Gianelloni <wolf31o2@gentoo.org> writes:

  cg> On Wed, 2005-07-06 at 09:45 -0400, Paul Smith wrote:
  >> How are the modules to be included in the initrd computed?
  >> 
  >> I'm seeing something strange where some of the modules in the real
  >> filesystem version of the /lib/modules/.../drivers/scsi are missing from
  >> the initrd filesystem version of the same directory; in particular one
  >> of the drivers for my SATA disks (ata_piix) is not there.  Others are
  >> missing as well.
  >> 
  >> 
  >> It's genkernel that's doing this, right?  I'm trying to follow along in
  >> the gen_initrd.sh script but I can't quite see what it's doing WRT
  >> modules.

  cg> genkernel's /usr/share/genkernel/$arch/modules_load

Got it, thanks.  Is there any good way in catalyst to override that
file, or do I have to create my own ebuild for it in an overlay?

  cg> I recommend unmasking the latest genkernel 3.2.0_pre version and
  cg> using that, as it will be going stable shortly.

I'll look into that; thanks.

Just to be clear on this: I don't need to rebuild stage3 when I make a
change like that right?  Rebuilding livecd-stage1 (to get the new
genkernel) and livecd-stage2 (to build the kernel/initrd) is good
enough?  I know it's good enough normally, just making sure that the
kernel/initrd build isn't special in some way.

  cg> I also recommend doing the same with the masked version of
  cg> catalyst.

OK, I'll look into this as well.  I really, really need to get my
current iteration done today so I might not have time.  I'll kick off a
build in the background for this in another workspace though.

  cg> I *NEED* testers for these, as I plan on them going stable before
  cg> 2005.1 is released and I would love to resolve as many bugs in
  cg> them as possible.

OK, although as you'll recall some of the stuff I'm doing is probably
not anything you care much about supporting :-).

-- 
-------------------------------------------------------------------------------
 Paul D. Smith <psmith@nortel.com>           HASMAT--HA Software Mthds & Tools
 "Please remain calm...I may be mad, but I am a professional." --Mad Scientist
-------------------------------------------------------------------------------
   These are my opinions---Nortel Networks takes no responsibility for them.
-- 
gentoo-catalyst@gentoo.org mailing list

Re: [gentoo-catalyst] How is initrd built?

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 3134 bytes --]

On Wed, 2005-07-06 at 11:31 -0400, Paul Smith wrote:
> %% Chris Gianelloni <wolf31o2@gentoo.org> writes:
> 
>   cg> On Wed, 2005-07-06 at 09:45 -0400, Paul Smith wrote:
>   >> How are the modules to be included in the initrd computed?
>   >> 
>   >> I'm seeing something strange where some of the modules in the real
>   >> filesystem version of the /lib/modules/.../drivers/scsi are missing from
>   >> the initrd filesystem version of the same directory; in particular one
>   >> of the drivers for my SATA disks (ata_piix) is not there.  Others are
>   >> missing as well.
>   >> 
>   >> 
>   >> It's genkernel that's doing this, right?  I'm trying to follow along in
>   >> the gen_initrd.sh script but I can't quite see what it's doing WRT
>   >> modules.
> 
>   cg> genkernel's /usr/share/genkernel/$arch/modules_load
> 
> Got it, thanks.  Is there any good way in catalyst to override that
> file, or do I have to create my own ebuild for it in an overlay?

There is no good way for a reason.  We test them very extensively.  Like
I said, newer genkernel versions have this updated.  It's best to go
with the newer genkernel versions anyway, since the old ones will soon
be removed from portage and unsupported.

>   cg> I recommend unmasking the latest genkernel 3.2.0_pre version and
>   cg> using that, as it will be going stable shortly.
> 
> I'll look into that; thanks.
> 
> Just to be clear on this: I don't need to rebuild stage3 when I make a
> change like that right?  Rebuilding livecd-stage1 (to get the new
> genkernel) and livecd-stage2 (to build the kernel/initrd) is good
> enough?  I know it's good enough normally, just making sure that the
> kernel/initrd build isn't special in some way.

Actually, if you're using the latest catalyst, also, then you will only
need to rebuild livecd-stage2, since genkernel is explicitly merged in
livecd-stage2 with the proper USE flags.

>   cg> I also recommend doing the same with the masked version of
>   cg> catalyst.
> 
> OK, I'll look into this as well.  I really, really need to get my
> current iteration done today so I might not have time.  I'll kick off a
> build in the background for this in another workspace though.

Merging catalyst takes about 30 seconds.  The simple truth is that most
bugs you're going to run across in older catalyst versions will be fixed
in these pre-releases.  They really are more stable than the current
stable versions, as you can see, we've gone through quite a few of the
_pre versions of both catalyst and genkernel, working out all the bugs
we found.

>   cg> I *NEED* testers for these, as I plan on them going stable before
>   cg> 2005.1 is released and I would love to resolve as many bugs in
>   cg> them as possible.
> 
> OK, although as you'll recall some of the stuff I'm doing is probably
> not anything you care much about supporting :-).

Nothing wrong with that.  But more eyes and more users means less bugs
that slip through the cracks.

-- 
Chris Gianelloni
Release Engineering - Strategic Lead/QA Manager
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-catalyst] How is initrd built?

[Paul Smith]

%% Chris Gianelloni <wolf31o2@gentoo.org> writes:

  cg> Merging catalyst takes about 30 seconds.  The simple truth is that
  cg> most bugs you're going to run across in older catalyst versions
  cg> will be fixed in these pre-releases.  They really are more stable
  cg> than the current stable versions, as you can see, we've gone
  cg> through quite a few of the _pre versions of both catalyst and
  cg> genkernel, working out all the bugs we found.

Yep.  It's not the time needed to do it but the potential for
instability.

However, I've updated these now and I'll see how it goes...

-- 
-------------------------------------------------------------------------------
 Paul D. Smith <psmith@nortel.com>           HASMAT--HA Software Mthds & Tools
 "Please remain calm...I may be mad, but I am a professional." --Mad Scientist
-------------------------------------------------------------------------------
   These are my opinions---Nortel Networks takes no responsibility for them.
-- 
gentoo-catalyst@gentoo.org mailing list

Re: [gentoo-catalyst] How is initrd built?

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 1121 bytes --]

On Wed, 2005-07-06 at 12:36 -0400, Paul Smith wrote:
> %% Chris Gianelloni <wolf31o2@gentoo.org> writes:
> 
>   cg> Merging catalyst takes about 30 seconds.  The simple truth is that
>   cg> most bugs you're going to run across in older catalyst versions
>   cg> will be fixed in these pre-releases.  They really are more stable
>   cg> than the current stable versions, as you can see, we've gone
>   cg> through quite a few of the _pre versions of both catalyst and
>   cg> genkernel, working out all the bugs we found.
> 
> Yep.  It's not the time needed to do it but the potential for
> instability.

Like I said, they're more stable and probably better tested than
catalyst 1.1.9 or genkernel 3.1.6, since both were rushed out before the
last release.  These have been in testing across multiple architectures
over the past 2 months, and will likely be coming out of a _pre state
and directly to stable in the next day or so.

> However, I've updated these now and I'll see how it goes...

-- 
Chris Gianelloni
Release Engineering - Strategic Lead/QA Manager
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-catalyst] How is initrd built?

[Paul Smith]

OK, I finally got myself back to building--the latest emerge sync caused
a lot of ebuilds that I had made in my own portdir_overlay to suddenly
become supported in the default snapshot, which caused build issues.

Then, I ran into a problem with the very latest kernel (2.6.12-r4) where
the genpatches version it required (7) was not available on the gentoo
site or any of the mirrors.  I dropped back to the 2.6.11 kernel using a
package.mask file.


Anyway.  On booting I tried to add the unionfs flag to the boot line and
noticed this bug in linuxrc:

      unionfs)
	  if [ ! -x /sbin/unionctl ]
	  then
	      USE_UNIONFS_NORMAL=0
	      bad_msg 'Unionctl not found: aborting use of unionfs!'
	  else
	      USE_UNIONFS_NORMAL=1
	  fi
	  ;;

But, when I install this unionctl IS there: it lives in /usr/sbin, not
/sbin.

      unionfs\=*)
	  if [ ! -x /sbin/unionctl ]

Ditto here.

I know this is in genkernel not catalyst; should I file a bug or is this
good enough?

-- 
-------------------------------------------------------------------------------
 Paul D. Smith <psmith@nortel.com>           HASMAT--HA Software Mthds & Tools
 "Please remain calm...I may be mad, but I am a professional." --Mad Scientist
-------------------------------------------------------------------------------
   These are my opinions---Nortel Networks takes no responsibility for them.
-- 
gentoo-catalyst@gentoo.org mailing list

Re: [gentoo-catalyst] How is initrd built?

[Paul Smith]

Next question: what are the replacements for rcadd/rcdel in the
livecd.conf file in the new version of Catalyst?

My current settings look like this and worked fine in the previous
version of Catalyst:

livecd/rcadd: 
	autoconfig:boot
	iptables:default
	syslog-ng:default 
	vixie-cron:default 
	net.eth0:default
	sshd:default
	apache2:default
	ntp-client:default
	ntpd:default
	dhcp:default
	dnsmasq:default
livecd/rcdel:
	bootsplash:default
	iptables:boot


However, in my log file now when I'm building livecd-stage2 I see this:

        WARNING: No value set for key: livecd/rcadd
        deleting key: livecd/rcadd


        WARNING: No value set for key: livecd/rcdel
        deleting key: livecd/rcdel

and when the system comes up none of my added services are started, and
rc-status -u shows them all as unassigned.

Any help?


PS. Chris: this is the kind of thing I meant by "instability"--not that
    Catalyst was unstable but rather that my build would be unstable
    when I moved to the new version :-)

-- 
-------------------------------------------------------------------------------
 Paul D. Smith <psmith@nortel.com>           HASMAT--HA Software Mthds & Tools
 "Please remain calm...I may be mad, but I am a professional." --Mad Scientist
-------------------------------------------------------------------------------
   These are my opinions---Nortel Networks takes no responsibility for them.
-- 
gentoo-catalyst@gentoo.org mailing list

Re: [gentoo-catalyst] How is initrd built?

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 1191 bytes --]

On Wed, 2005-07-06 at 17:47 -0400, Paul Smith wrote:
> Then, I ran into a problem with the very latest kernel (2.6.12-r4) where
> the genpatches version it required (7) was not available on the gentoo
> site or any of the mirrors.  I dropped back to the 2.6.11 kernel using a
> package.mask file.

Hrrrmn... it is working here... strange.

> Anyway.  On booting I tried to add the unionfs flag to the boot line and
> noticed this bug in linuxrc:
> 
>       unionfs)
> 	  if [ ! -x /sbin/unionctl ]
> 	  then
> 	      USE_UNIONFS_NORMAL=0
> 	      bad_msg 'Unionctl not found: aborting use of unionfs!'
> 	  else
> 	      USE_UNIONFS_NORMAL=1
> 	  fi
> 	  ;;
> 
> But, when I install this unionctl IS there: it lives in /usr/sbin, not
> /sbin.
> 
>       unionfs\=*)
> 	  if [ ! -x /sbin/unionctl ]
> 
> Ditto here.
> 
> I know this is in genkernel not catalyst; should I file a bug or is this
> good enough?

So it is in /usr/sbin within the initrd?  Yeah, that definitely should
be filed as a bug and assigned to genkernel@gentoo.org, if you would.

-- 
Chris Gianelloni
Release Engineering - Strategic Lead/QA Manager
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-catalyst] How is initrd built?

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 1700 bytes --]

On Wed, 2005-07-06 at 18:08 -0400, Paul Smith wrote:
> Next question: what are the replacements for rcadd/rcdel in the
> livecd.conf file in the new version of Catalyst?
> 
> My current settings look like this and worked fine in the previous
> version of Catalyst:
> 
> livecd/rcadd: 
> 	autoconfig:boot
> 	iptables:default
> 	syslog-ng:default 
> 	vixie-cron:default 
> 	net.eth0:default
> 	sshd:default
> 	apache2:default
> 	ntp-client:default
> 	ntpd:default
> 	dhcp:default
> 	dnsmasq:default

Try removing the initial line break.

> livecd/rcdel:
> 	bootsplash:default
> 	iptables:boot

> However, in my log file now when I'm building livecd-stage2 I see this:
> 
>         WARNING: No value set for key: livecd/rcadd
>         deleting key: livecd/rcadd
> 
> 
>         WARNING: No value set for key: livecd/rcdel
>         deleting key: livecd/rcdel
> 
> and when the system comes up none of my added services are started, and
> rc-status -u shows them all as unassigned.
> 
> Any help?
> 
> 
> PS. Chris: this is the kind of thing I meant by "instability"--not that
>     Catalyst was unstable but rather that my build would be unstable
>     when I moved to the new version :-)

Understandable.  I just don't want to support the older versions and
plan on ripping them out of portage and making them unavailable as soon
as we stabilize the new version.  I absolutely hate having to support
already-resolved bugs... ;]

Anyway, this seems to be an issue with the new code that I need to look
into.  Thanks for reminding me about it.

-- 
Chris Gianelloni
Release Engineering - Strategic Lead/QA Manager
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-catalyst] How is initrd built?

[Paul Smith]

%% Chris Gianelloni <wolf31o2@gentoo.org> writes:

  cg> Try removing the initial line break.

Mm.  Is this going to be an issue for ALL the keys?  I don't have any
values on the same line as any of my keys, but these are the only two
that Catalyst is upset by.

Anyway, before I got your email I tried put everything on the same line
separated by whitespace and Catalyst didn't complain anymore (but I
haven't booted that image yet to make sure things are started properly).

  cg> Anyway, this seems to be an issue with the new code that I need to
  cg> look into.  Thanks for reminding me about it.

Do you want a bug?


Cheers!

-- 
-------------------------------------------------------------------------------
 Paul D. Smith <psmith@nortel.com>           HASMAT--HA Software Mthds & Tools
 "Please remain calm...I may be mad, but I am a professional." --Mad Scientist
-------------------------------------------------------------------------------
   These are my opinions---Nortel Networks takes no responsibility for them.
-- 
gentoo-catalyst@gentoo.org mailing list

Re: [gentoo-catalyst] How is initrd built?

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 936 bytes --]

On Wed, 2005-07-06 at 18:41 -0400, Paul Smith wrote:
> %% Chris Gianelloni <wolf31o2@gentoo.org> writes:
> 
>   cg> Try removing the initial line break.
> 
> Mm.  Is this going to be an issue for ALL the keys?  I don't have any
> values on the same line as any of my keys, but these are the only two
> that Catalyst is upset by.
> 
> Anyway, before I got your email I tried put everything on the same line
> separated by whitespace and Catalyst didn't complain anymore (but I
> haven't booted that image yet to make sure things are started properly).
> 
>   cg> Anyway, this seems to be an issue with the new code that I need to
>   cg> look into.  Thanks for reminding me about it.
> 
> Do you want a bug?

If you don't mind.  I'll try to get to it ASAP anyway... but a bug
always keeps one from forgetting.

-- 
Chris Gianelloni
Release Engineering - Strategic Lead/QA Manager
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-catalyst] How is initrd built?

[Paul Smith]

Next question... and it's back to the subject line even!! :-)

So, I'm using the latest Catalyst and genkernel.  In previous releases
if I needed to tweak the initrd image during development (typically of
the linuxrc script), rather than rebuilding everything I'd often unpack
the Catalyst tarball, uncompress the .igz file into a .ird file, then
mount it as an ext2 filesystem.  Then I could modify it as necessary,
sync, and rebuild the .iso (I'd usually change the .cfg file to use the
uncompressed version to avoid having to recompress it).

In the new version, the initrd doesn't appear to be an ext2 filesystem
anymore... it's some kind of compressed cpio file?!?!  Anyway, that's
what "file" tells me.


Is there any way I can work with this new format in a similar manner
(mount it, modify it, etc.)?

-- 
-------------------------------------------------------------------------------
 Paul D. Smith <psmith@nortel.com>           HASMAT--HA Software Mthds & Tools
 "Please remain calm...I may be mad, but I am a professional." --Mad Scientist
-------------------------------------------------------------------------------
   These are my opinions---Nortel Networks takes no responsibility for them.
-- 
gentoo-catalyst@gentoo.org mailing list

Re: [gentoo-catalyst] How is initrd built?

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 997 bytes --]

On Thu, 2005-07-07 at 09:50 -0400, Paul Smith wrote:
> In the new version, the initrd doesn't appear to be an ext2 filesystem
> anymore... it's some kind of compressed cpio file?!?!  Anyway, that's
> what "file" tells me.

It is an initramfs image.  You can work with it just like any other cpio
archive.

> Is there any way I can work with this new format in a similar manner
> (mount it, modify it, etc.)?

There's no need to mount it, as it isn't a filesystem image, but a cpio
archive.  It's about the same as a tarball, except different.  :P

We switched to initramfs because it allowed us to work around the
absolute *hack* that I had included in genkernel for gensplash support
and allows us to use something sane.  It also keeps us from having to
statically install the splash theme into the kernel bzImage itself,
saving a ton of space on the CD images created.

-- 
Chris Gianelloni
Release Engineering - Strategic Lead/QA Manager
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-catalyst] How is initrd built?

[Paul Smith]

%% Chris Gianelloni <wolf31o2@gentoo.org> writes:

  cg> On Thu, 2005-07-07 at 09:50 -0400, Paul Smith wrote:
  >> In the new version, the initrd doesn't appear to be an ext2 filesystem
  >> anymore... it's some kind of compressed cpio file?!?!  Anyway, that's
  >> what "file" tells me.

  cg> It is an initramfs image.  You can work with it just like any
  cg> other cpio archive.

Well, yeah, I tried to do that, but it doesn't work.  What am I doing
wrong (I haven't used cpio for years...)

I unpack the tarball and then:

  # gzcat isolinux/gentoo.igz | cpio -tv
  drwxr-xr-x  11 root     root            0 Jul  6 20:09 .
  drwxr-xr-x   2 root     root            0 Jul  6 20:09 dev
  crw-rw----   1 root     root       5,   1 Jul  6 20:09 dev/console
  crw-rw----   1 root     root       1,   3 Jul  6 20:09 dev/null
  drwxr-xr-x   2 root     root            0 Jul  6 20:09 bin
  drwxr-xr-x   2 root     root            0 Jul  6 20:09 etc
  -rw-r--r--   1 root     root           93 Jul  6 20:09 etc/fstab
  drwxr-xr-x   4 root     root            0 Jul  6 20:09 usr
  drwxr-xr-x   2 root     root            0 Jul  6 20:09 usr/bin
  drwxr-xr-x   2 root     root            0 Jul  6 20:09 usr/sbin
  drwxr-xr-x   2 root     root            0 Jul  6 20:09 proc
  drwxr-xr-x   2 root     root            0 Jul  6 20:09 temp
  drwxr-xr-x   2 root     root            0 Jul  6 20:09 sys
  drwxr-xr-x   3 root     root            0 Jul  6 20:09 var
  drwxr-xr-x   3 root     root            0 Jul  6 20:09 var/lock
  drwxr-xr-x   2 root     root            0 Jul  6 20:09 var/lock/dmraid
  drwxr-xr-x   2 root     root            0 Jul  6 20:09 sbin

Where's everything else?  If I actually extract the archive I get the
same results: just this small directory skeleton and no actual contents
other than the ones in bin.

The file, of course, is huge and the system does boot properly so it's
all in there somewhere, but...???


I feel dumb :-/.

-- 
-------------------------------------------------------------------------------
 Paul D. Smith <psmith@nortel.com>           HASMAT--HA Software Mthds & Tools
 "Please remain calm...I may be mad, but I am a professional." --Mad Scientist
-------------------------------------------------------------------------------
   These are my opinions---Nortel Networks takes no responsibility for them.
-- 
gentoo-catalyst@gentoo.org mailing list

Re: [gentoo-catalyst] How is initrd built?

[Eric Edgar]

[-- Attachment #1: Type: text/plain, Size: 3697 bytes --]

You technically cannot extract the cpio archive as far as I can tell
because it really is multiple cpio archives concatenated together.

the kernel takes each one and overlays them ontop of eachother.
this makes it easy to create a new cpio archive and cat it to the end of
the existing archive to override any file.  It does make the initramfs
file itself bigger, but shouldnt take any more ram.

There is an --initramfs-overlay option that will allow you to do this
automatically as it points to a directory and copies its structure into
the initramfs.

cd ${INITRAMFS_OVERLAY}
find . -print | cpio --quiet -o -H newc | gzip -9 > ${CACHE_DIR}/cpio/initramfs-overlay.cpio.gz
cat old-oldinitramfs.cpio.gz initramfs-overlay.cpio.gz 	> new-initramfs.cpio.gz	

We did it this way because its easier to add or subtract components from
the initramfs. because udev is its own cpio file, busybox is its own
cpio file etc.  It would work as well if everything was one big cpio,
but then genkernel would be much slower as it has to do a lot more
copy operations.

Dont feel dumb.

On 10:35 Thu 07 Jul     , Paul Smith wrote:
> %% Chris Gianelloni <wolf31o2@gentoo.org> writes:
> 
>   cg> On Thu, 2005-07-07 at 09:50 -0400, Paul Smith wrote:
>   >> In the new version, the initrd doesn't appear to be an ext2 filesystem
>   >> anymore... it's some kind of compressed cpio file?!?!  Anyway, that's
>   >> what "file" tells me.
> 
>   cg> It is an initramfs image.  You can work with it just like any
>   cg> other cpio archive.
> 
> Well, yeah, I tried to do that, but it doesn't work.  What am I doing
> wrong (I haven't used cpio for years...)
> 
> I unpack the tarball and then:
> 
>   # gzcat isolinux/gentoo.igz | cpio -tv
>   drwxr-xr-x  11 root     root            0 Jul  6 20:09 .
>   drwxr-xr-x   2 root     root            0 Jul  6 20:09 dev
>   crw-rw----   1 root     root       5,   1 Jul  6 20:09 dev/console
>   crw-rw----   1 root     root       1,   3 Jul  6 20:09 dev/null
>   drwxr-xr-x   2 root     root            0 Jul  6 20:09 bin
>   drwxr-xr-x   2 root     root            0 Jul  6 20:09 etc
>   -rw-r--r--   1 root     root           93 Jul  6 20:09 etc/fstab
>   drwxr-xr-x   4 root     root            0 Jul  6 20:09 usr
>   drwxr-xr-x   2 root     root            0 Jul  6 20:09 usr/bin
>   drwxr-xr-x   2 root     root            0 Jul  6 20:09 usr/sbin
>   drwxr-xr-x   2 root     root            0 Jul  6 20:09 proc
>   drwxr-xr-x   2 root     root            0 Jul  6 20:09 temp
>   drwxr-xr-x   2 root     root            0 Jul  6 20:09 sys
>   drwxr-xr-x   3 root     root            0 Jul  6 20:09 var
>   drwxr-xr-x   3 root     root            0 Jul  6 20:09 var/lock
>   drwxr-xr-x   2 root     root            0 Jul  6 20:09 var/lock/dmraid
>   drwxr-xr-x   2 root     root            0 Jul  6 20:09 sbin
> 
> Where's everything else?  If I actually extract the archive I get the
> same results: just this small directory skeleton and no actual contents
> other than the ones in bin.
> 
> The file, of course, is huge and the system does boot properly so it's
> all in there somewhere, but...???
> 
> 
> I feel dumb :-/.
> 
> -- 
> -------------------------------------------------------------------------------
>  Paul D. Smith <psmith@nortel.com>           HASMAT--HA Software Mthds & Tools
>  "Please remain calm...I may be mad, but I am a professional." --Mad Scientist
> -------------------------------------------------------------------------------
>    These are my opinions---Nortel Networks takes no responsibility for them.
> -- 
> gentoo-catalyst@gentoo.org mailing list
> 
> 

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-catalyst] How is initrd built?

[Eric Edgar]

[-- Attachment #1: Type: text/plain, Size: 1835 bytes --]


This is not a bug.  There are symlinks that point /usr/sbin to /sbin in
the initrd/initramfs.  It works for me.  If it doesnt work for you would
you post the output you are seeing so I can troubleshoot the issue?
Thanks

Rocket

On 17:47 Wed 06 Jul     , Paul Smith wrote:
> OK, I finally got myself back to building--the latest emerge sync caused
> a lot of ebuilds that I had made in my own portdir_overlay to suddenly
> become supported in the default snapshot, which caused build issues.
> 
> Then, I ran into a problem with the very latest kernel (2.6.12-r4) where
> the genpatches version it required (7) was not available on the gentoo
> site or any of the mirrors.  I dropped back to the 2.6.11 kernel using a
> package.mask file.
> 
> 
> Anyway.  On booting I tried to add the unionfs flag to the boot line and
> noticed this bug in linuxrc:
> 
>       unionfs)
> 	  if [ ! -x /sbin/unionctl ]
> 	  then
> 	      USE_UNIONFS_NORMAL=0
> 	      bad_msg 'Unionctl not found: aborting use of unionfs!'
> 	  else
> 	      USE_UNIONFS_NORMAL=1
> 	  fi
> 	  ;;
> 
> But, when I install this unionctl IS there: it lives in /usr/sbin, not
> /sbin.
> 
>       unionfs\=*)
> 	  if [ ! -x /sbin/unionctl ]
> 
> Ditto here.
> 
> I know this is in genkernel not catalyst; should I file a bug or is this
> good enough?
> 
> -- 
> -------------------------------------------------------------------------------
>  Paul D. Smith <psmith@nortel.com>           HASMAT--HA Software Mthds & Tools
>  "Please remain calm...I may be mad, but I am a professional." --Mad Scientist
> -------------------------------------------------------------------------------
>    These are my opinions---Nortel Networks takes no responsibility for them.
> -- 
> gentoo-catalyst@gentoo.org mailing list
> 
> 

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-catalyst] How is initrd built?

[Paul Smith]

%% Eric Edgar <rocket@gentoo.org> writes:

  ee> This is not a bug.  There are symlinks that point /usr/sbin to
  ee> /sbin in the initrd/initramfs.  It works for me.  If it doesnt
  ee> work for you would you post the output you are seeing so I can
  ee> troubleshoot the issue?  Thanks

I think I might have spoken too quickly... I didn't have
"livecd/gk_mainargs: --unionfs" in my Catalyst livecd.conf file.

Since I haven't succeeded in unpacking the initramfs I really didn't
know what was in there; I just kind of ass-u-me'd that since the option
was available, genkernel would have the right stuff.  Then I looked in
the runtime image and saw that unionctl is in /usr/sbin there, while
linuxrc was looking for it in /sbin, and leapt to a conclusion :-/.

I'm rebuilding now with the --unionfs option and I'll let you know.

-- 
-------------------------------------------------------------------------------
 Paul D. Smith <psmith@nortel.com>           HASMAT--HA Software Mthds & Tools
 "Please remain calm...I may be mad, but I am a professional." --Mad Scientist
-------------------------------------------------------------------------------
   These are my opinions---Nortel Networks takes no responsibility for them.
-- 
gentoo-catalyst@gentoo.org mailing list

Re: [gentoo-catalyst] How is initrd built?

[Eric Edgar]

[-- Attachment #1: Type: text/plain, Size: 1828 bytes --]

Catalyst doesnt really support unionfs at the moment. 

You will need to add unionfs to your isolinux loader script to get it to
work.

unionfs=/path/to/block device will let you use a formatted device as the
place to store changes.  This code is VERY experimental and is currently
just starting to be supported.  It will not be in official cds for
sometime as no one has tested it all besides myself.

Rocket

On 11:36 Thu 07 Jul     , Paul Smith wrote:
> %% Eric Edgar <rocket@gentoo.org> writes:
> 
>   ee> This is not a bug.  There are symlinks that point /usr/sbin to
>   ee> /sbin in the initrd/initramfs.  It works for me.  If it doesnt
>   ee> work for you would you post the output you are seeing so I can
>   ee> troubleshoot the issue?  Thanks
> 
> I think I might have spoken too quickly... I didn't have
> "livecd/gk_mainargs: --unionfs" in my Catalyst livecd.conf file.
> 
> Since I haven't succeeded in unpacking the initramfs I really didn't
> know what was in there; I just kind of ass-u-me'd that since the option
> was available, genkernel would have the right stuff.  Then I looked in
> the runtime image and saw that unionctl is in /usr/sbin there, while
> linuxrc was looking for it in /sbin, and leapt to a conclusion :-/.
> 
> I'm rebuilding now with the --unionfs option and I'll let you know.
> 
> -- 
> -------------------------------------------------------------------------------
>  Paul D. Smith <psmith@nortel.com>           HASMAT--HA Software Mthds & Tools
>  "Please remain calm...I may be mad, but I am a professional." --Mad Scientist
> -------------------------------------------------------------------------------
>    These are my opinions---Nortel Networks takes no responsibility for them.
> -- 
> gentoo-catalyst@gentoo.org mailing list
> 
> 

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-catalyst] How is initrd built?

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 809 bytes --]

On Thu, 2005-07-07 at 15:40 +0000, Eric Edgar wrote:
> Catalyst doesnt really support unionfs at the moment. 
> 
> You will need to add unionfs to your isolinux loader script to get it to
> work.
> 
> unionfs=/path/to/block device will let you use a formatted device as the
> place to store changes.  This code is VERY experimental and is currently
> just starting to be supported.  It will not be in official cds for
> sometime as no one has tested it all besides myself.

Translation:  Only the guy who wrote it can get it to work.  If it is
broken, we probably won't fix it any time soon, as it is low priority...
Unless you have a working patch, then we'll gladly apply it.

*grin*

-- 
Chris Gianelloni
Release Engineering - Strategic Lead/QA Manager
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-catalyst] How is initrd built?

[Paul Smith]

%% Eric Edgar <rocket@gentoo.org> writes:

  ee> Catalyst doesnt really support unionfs at the moment.  You will
  ee> need to add unionfs to your isolinux loader script to get it to
  ee> work.

"isolinux loader script"?

  ee> unionfs=/path/to/block device will let you use a formatted device
  ee> as the place to store changes.  This code is VERY experimental and
  ee> is currently just starting to be supported.  It will not be in
  ee> official cds for sometime as no one has tested it all besides
  ee> myself.

Hm.  I had it working with the previous genkernel/catalyst, with a
custom linuxrc, but took out what I had in favor of the new support;
maybe I was too hasty :-).


The way I had it working was twofold: first, I used it even in the
livecd situation where there was no formatted device; in this case the
"top" filesystem in the union, the RW one, was a ramdisk.  The advantage
here is that I could avoid the long tmpfs copy step (by far the slowest
step) during boot: instead of copying all the modifiable directory
structures (/etc, /var, etc.) to a ramdisk I just used the unionfs to
overlay the RO partition and I was done.

Secondly, as you say, it could be used as a persistent storage.  In this
case I had the linuxrc try to mount disk devices (using /sys/block/...)
until I found one with a special token file in the root of the
filesystem.  I had it hardcoded to ext3 so I didn't need to worry about
different filesystem types.  If I found an ext3 filesystem with that
token file in the root I would use that as the top (RW) element in the
unionfs.


Anyway, for the time being I'm mostly interested in the first item:
speeding up the boot.  As things are turning out it's going to be more
important in my project to be able to install the whole thing to disk
and boot off of disk, than to boot off of CD with a persistent storage.
However, unionfs really helps me in two ways anyway: first is the boot
time from CD as I mentioned above.  Second is the copy time to the HD
partition, which can also benefit.

-- 
-------------------------------------------------------------------------------
 Paul D. Smith <psmith@nortel.com>           HASMAT--HA Software Mthds & Tools
 "Please remain calm...I may be mad, but I am a professional." --Mad Scientist
-------------------------------------------------------------------------------
   These are my opinions---Nortel Networks takes no responsibility for them.
-- 
gentoo-catalyst@gentoo.org mailing list

Re: [gentoo-catalyst] How is initrd built?

[Paul Smith]

%% Eric Edgar <rocket@gentoo.org> writes:

  ee> Catalyst doesnt really support unionfs at the moment. 

  ee> You will need to add unionfs to your isolinux loader script to get
  ee> it to work.

Well, it pretty much works.  I wonder if anyone can be more clear about
in what way Catalyst doesn't support unionfs and what exactly "add
unionfs to my isolinux script" means?  I'm happy to continue moving
forward with this and even posting fixes etc. if I run into any.


What happens now: I'm booting a livecd image.  If I add --union to the
genkernel arguments when I build my livecd image with catalyst, then the
system does come up correctly.  I see it detect unionctl, I see it
create the tmpfs, I see it mount the tmpfs, etc.

However, after the system is booted there is one problem: all the
partitions that were mounted during the linuxrc are no longer visible
from within the unionfs.  Here are the unionfs-related steps genkernel
uses as far as I can determine them; I'm replacing variables with actual
values here to make things simpler to follow (I hope!):

   0) mkdir -p /newroot

   1) Create a tmpfs at /memory

   2) Create a unionfs at /union, where the only item in the union is
      /memory (mounted RW)

   3) Init /newroot (make directories, copy over /dev/null and /dev/console)

   4) Find the cdrom and mount it as /newroot/mnt/cdrom

   5) Mount the squashfs as /newroot/mnt/livecd (read-only)

   6) Add /newroot/mnt/livecd to the unionfs after the tmpfs (/memory)

   7) Create /union/proc and /union/sys

   8) Pivot_root to make /union the new /.

   9) Clean up the old root (in /tmp/.initrd)

  10) chroot . 'exec /sbin/init' (basically).

So, at the end of this our / is actually /union, and things are set up
properly, BUT if I run "df", for example, I get:

  Filesystme    1K-blocks       Used    Available   Use%    Mounted on
  df: `/memory': No such file or directory
  df: `/newroot/mnt/cdrom': No such file or directory
  df: `/newroot/mnt/livecd': No such file or directory
  udev             127724       2668       125056     3%    /dev

Is there a way to fix this?  Can we do some sort of fancy footwork
(maybe mount --move?) to get those things visible inside the union?
Creating these directories inside the union keeps df from complaining,
but those directories are still empty: you can't unmount them for
example (not that you'd be able to anyway for the unioned ones).

I'm mostly concerned with the cdrom mount.  I wonder if it would at
least be reasonable to mount /mnt/cdrom inside /union instead of inside
/newroot: that should be safe.  You can't do that with /memory since it
needs to exist before the union is created, and I doubt it would work to
do it with the livecd (union mount something that is mounted inside the
union...?  Hmm)

-- 
-------------------------------------------------------------------------------
 Paul D. Smith <psmith@nortel.com>           HASMAT--HA Software Mthds & Tools
 "Please remain calm...I may be mad, but I am a professional." --Mad Scientist
-------------------------------------------------------------------------------
   These are my opinions---Nortel Networks takes no responsibility for them.
-- 
gentoo-catalyst@gentoo.org mailing list

Re: [gentoo-catalyst] How is initrd built?

[Paul Smith]

%% Eric Edgar <rocket@gentoo.org> writes:

  ee> You technically cannot extract the cpio archive as far as I can tell
  ee> because it really is multiple cpio archives concatenated together.

Ah!  OK.

  ee> cd ${INITRAMFS_OVERLAY}
  ee> find . -print | cpio --quiet -o -H newc | gzip -9 > ${CACHE_DIR}/cpio/initramfs-overlay.cpio.gz
  ee> cat old-oldinitramfs.cpio.gz initramfs-overlay.cpio.gz 	> new-initramfs.cpio.gz	

Wait, that's not right is it?

Is the initramfs really a concatenation of compressed cpio files?  I
don't think it is... gzip can't handle that.

It must be a compression of a concatenation of cpio files, right?  In
that case the above command line is not correct... I'd need to create
the cpio file, concatenate it to an uncompressed copy of the existing
initramfs, then compress the whole thing again.

Yes?

-- 
-------------------------------------------------------------------------------
 Paul D. Smith <psmith@nortel.com>           HASMAT--HA Software Mthds & Tools
 "Please remain calm...I may be mad, but I am a professional." --Mad Scientist
-------------------------------------------------------------------------------
   These are my opinions---Nortel Networks takes no responsibility for them.
-- 
gentoo-catalyst@gentoo.org mailing list

Re: [gentoo-catalyst] How is initrd built?

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 3904 bytes --]

On Thu, 2005-07-07 at 14:42 -0400, Paul Smith wrote:
> %% Eric Edgar <rocket@gentoo.org> writes:
> 
>   ee> Catalyst doesnt really support unionfs at the moment. 
> 
>   ee> You will need to add unionfs to your isolinux loader script to get
>   ee> it to work.
> 
> Well, it pretty much works.  I wonder if anyone can be more clear about
> in what way Catalyst doesn't support unionfs and what exactly "add
> unionfs to my isolinux script" means?  I'm happy to continue moving
> forward with this and even posting fixes etc. if I run into any.

Actually, what rocket means in that catalyst has no built-in support for
unionfs.  To get unionfs, you need to use gk_mainargs (or gk_kernargs)
--unionfs.  You will also probably want livecd/bootargs: unionfs...
unless you know the device you want, in which case you can do
livecd/bootargs: unionfs=/dev/sda1 (or whatever).

Yeah, livecd/bootargs is new in the _pre series.  I even documented it
in the template spec files... ;]

Before livecd/bootargs, you would have had to edit
livecd/runscript/x86-archscript.sh yourself to add an option to the
isolinux boot line.

> What happens now: I'm booting a livecd image.  If I add --union to the
> genkernel arguments when I build my livecd image with catalyst, then the
> system does come up correctly.  I see it detect unionctl, I see it
> create the tmpfs, I see it mount the tmpfs, etc.
> 
> However, after the system is booted there is one problem: all the
> partitions that were mounted during the linuxrc are no longer visible
> from within the unionfs.  Here are the unionfs-related steps genkernel
> uses as far as I can determine them; I'm replacing variables with actual
> values here to make things simpler to follow (I hope!):
> 
>    0) mkdir -p /newroot
> 
>    1) Create a tmpfs at /memory
> 
>    2) Create a unionfs at /union, where the only item in the union is
>       /memory (mounted RW)
> 
>    3) Init /newroot (make directories, copy over /dev/null and /dev/console)
> 
>    4) Find the cdrom and mount it as /newroot/mnt/cdrom
> 
>    5) Mount the squashfs as /newroot/mnt/livecd (read-only)
> 
>    6) Add /newroot/mnt/livecd to the unionfs after the tmpfs (/memory)
> 
>    7) Create /union/proc and /union/sys
> 
>    8) Pivot_root to make /union the new /.
> 
>    9) Clean up the old root (in /tmp/.initrd)
> 
>   10) chroot . 'exec /sbin/init' (basically).
> 
> So, at the end of this our / is actually /union, and things are set up
> properly, BUT if I run "df", for example, I get:
> 
>   Filesystme    1K-blocks       Used    Available   Use%    Mounted on
>   df: `/memory': No such file or directory
>   df: `/newroot/mnt/cdrom': No such file or directory
>   df: `/newroot/mnt/livecd': No such file or directory
>   udev             127724       2668       125056     3%    /dev
> 
> Is there a way to fix this?  Can we do some sort of fancy footwork
> (maybe mount --move?) to get those things visible inside the union?
> Creating these directories inside the union keeps df from complaining,
> but those directories are still empty: you can't unmount them for
> example (not that you'd be able to anyway for the unioned ones).
> 
> I'm mostly concerned with the cdrom mount.  I wonder if it would at
> least be reasonable to mount /mnt/cdrom inside /union instead of inside
> /newroot: that should be safe.  You can't do that with /memory since it
> needs to exist before the union is created, and I doubt it would work to
> do it with the livecd (union mount something that is mounted inside the
> union...?  Hmm)

AFIAK, /mnt/cdrom would need to be done before the pivot_root, too.  So
you can't move that to later.

Other than that, well, we're into unknown territory for me.

-- 
Chris Gianelloni
Release Engineering - Strategic Lead/QA Manager
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-catalyst] How is initrd built?

[Eric Edgar]

[-- Attachment #1: Type: text/plain, Size: 1709 bytes --]


That code is correct.  an initramfs can be any number of 
uncompressed or compressed cpio files concatenated together.  
The kernel is handling figuring out
whats compressed or not etc.  So it works because of the code in the
kernel.  so no you dont need to do that extra work.

On 15:14 Thu 07 Jul     , Paul Smith wrote:
> %% Eric Edgar <rocket@gentoo.org> writes:
> 
>   ee> You technically cannot extract the cpio archive as far as I can tell
>   ee> because it really is multiple cpio archives concatenated together.
> 
> Ah!  OK.
> 
>   ee> cd ${INITRAMFS_OVERLAY}
>   ee> find . -print | cpio --quiet -o -H newc | gzip -9 > ${CACHE_DIR}/cpio/initramfs-overlay.cpio.gz
>   ee> cat old-oldinitramfs.cpio.gz initramfs-overlay.cpio.gz 	> new-initramfs.cpio.gz	
> 
> Wait, that's not right is it?
> 
> Is the initramfs really a concatenation of compressed cpio files?  I
> don't think it is... gzip can't handle that.
> 
> It must be a compression of a concatenation of cpio files, right?  In
> that case the above command line is not correct... I'd need to create
> the cpio file, concatenate it to an uncompressed copy of the existing
> initramfs, then compress the whole thing again.
> 
> Yes?
> 
> -- 
> -------------------------------------------------------------------------------
>  Paul D. Smith <psmith@nortel.com>           HASMAT--HA Software Mthds & Tools
>  "Please remain calm...I may be mad, but I am a professional." --Mad Scientist
> -------------------------------------------------------------------------------
>    These are my opinions---Nortel Networks takes no responsibility for them.
> -- 
> gentoo-catalyst@gentoo.org mailing list
> 
> 

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-catalyst] How is initrd built?

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 1236 bytes --]

On Thu, 2005-07-07 at 15:14 -0400, Paul Smith wrote:
> %% Eric Edgar <rocket@gentoo.org> writes:
> 
>   ee> You technically cannot extract the cpio archive as far as I can tell
>   ee> because it really is multiple cpio archives concatenated together.
> 
> Ah!  OK.
> 
>   ee> cd ${INITRAMFS_OVERLAY}
>   ee> find . -print | cpio --quiet -o -H newc | gzip -9 > ${CACHE_DIR}/cpio/initramfs-overlay.cpio.gz
>   ee> cat old-oldinitramfs.cpio.gz initramfs-overlay.cpio.gz 	> new-initramfs.cpio.gz	
> 
> Wait, that's not right is it?

That's right.

From the genkernel code:

cat ${MERGE_LIST} > ${TEMP}/initramfs-${KV}

Each of those files in ${MERGE_LIST} are cpio.gz files.

> Is the initramfs really a concatenation of compressed cpio files?  I
> don't think it is... gzip can't handle that.

Yes, it is.

> It must be a compression of a concatenation of cpio files, right?  In
> that case the above command line is not correct... I'd need to create
> the cpio file, concatenate it to an uncompressed copy of the existing
> initramfs, then compress the whole thing again.
> 
> Yes?

Nope.

=]

-- 
Chris Gianelloni
Release Engineering - Strategic Lead/QA Manager
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-catalyst] How is initrd built?

[Paul Smith]

%% Chris Gianelloni <wolf31o2@gentoo.org> writes:

  cg> Actually, what rocket means in that catalyst has no built-in
  cg> support for unionfs.  To get unionfs, you need to use gk_mainargs
  cg> (or gk_kernargs) --unionfs.

Oh, OK.  Yeah, I already found this in the new genkernel, and did do it
in livecd.conf (I thought I mentioned that in one of my mails).

  cg> You will also probably want livecd/bootargs: unionfs...  unless
  cg> you know the device you want, in which case you can do
  cg> livecd/bootargs: unionfs=/dev/sda1 (or whatever).

I'm actually using GNAP to create the ISO, so I override the kernel boot
params there and don't need this.

  cg> Yeah, livecd/bootargs is new in the _pre series.  I even documented it
  cg> in the template spec files... ;]

That's a good enhancement though! :-).

  >> 0) mkdir -p /newroot
  >> 
  >> 1) Create a tmpfs at /memory
  >> 
  >> 2) Create a unionfs at /union, where the only item in the union is
  >> /memory (mounted RW)
  >> 
  >> 3) Init /newroot (make directories, copy over /dev/null and /dev/console)
  >> 
  >> 4) Find the cdrom and mount it as /newroot/mnt/cdrom
  >> 
  >> 5) Mount the squashfs as /newroot/mnt/livecd (read-only)
  >> 
  >> 6) Add /newroot/mnt/livecd to the unionfs after the tmpfs (/memory)
  >> 
  >> 7) Create /union/proc and /union/sys
  >> 
  >> 8) Pivot_root to make /union the new /.
  >> 
  >> 9) Clean up the old root (in /tmp/.initrd)
  >> 
  >> 10) chroot . 'exec /sbin/init' (basically).

Hold on a second...  I just discovered something that I don't understand
at all.  The behavior above, in particular steps #8 & 9, is not actually
what happens.

If you look in the linuxrc you'll see it has an option to be invoked one
of two ways: where $0 is "/init", and where $0 is "/linuxrc".  Depending
on which one is used the script exhibits different behavior... very
different, in some cases.  Like, the "/init" branch doesn't use
pivot_root at all; it just does a chroot.  The whole end cleanup is very
different.

I've been thinking that the livecd is booting with $0 == "/linuxrc", but
in fact it's not: $0 == "/init".

Can someone give me an idea of where this ($0) is set up (it's not init=
on the boot line because my boot line says init=/linuxrc--but it must be
the kernel somehow because it execs the shell so it would set argv[0]),
and what the purpose is between the different behaviors, and why you'd
choose one over the other?

  cg> AFIAK, /mnt/cdrom would need to be done before the pivot_root,
  cg> too.  So you can't move that to later.

True, but you could do one of two things:

 a) Mount it in /unionfs/mnt/cdrom in the first place, not in
    /newroot/mnt/cdrom.  By the time we mount the cdrom we already have
    a /union filesystem created and working.

OR

 b) Try to "mount --move" it from /newroot to /union once the unionfs is
    set up.


I looked at (a) and while I think it would work, it would involve a lot
of changes.  Although the value of CHROOT is set to $NEW_ROOT (or $UNION,
if we're using unionfs) early in the script, almost all the commands
after that continue to use $NEW_ROOT explicitly rather than $CHROOT.
Sometimes this probably makes sense (such as mounting livecd) but in
other cases it doesn't, so much (such as in mounting cdrom).  I think
these variables need to be rationalized somewhat, so $CHROOT is used
instead of $NEW_ROOT where appropriate.

There are other things that probably don't hurt, but aren't required
either.  For example, even in the unionfs case the code still
initializes $NEW_ROOT with a bunch of stuff that is never needed, and
then is just cleaned up again afterwards.

-- 
-------------------------------------------------------------------------------
 Paul D. Smith <psmith@nortel.com>           HASMAT--HA Software Mthds & Tools
 "Please remain calm...I may be mad, but I am a professional." --Mad Scientist
-------------------------------------------------------------------------------
   These are my opinions---Nortel Networks takes no responsibility for them.
-- 
gentoo-catalyst@gentoo.org mailing list

Re: [gentoo-catalyst] How is initrd built?

[Paul Smith]

Got it.  Thanks Eric & Chris!

-- 
-------------------------------------------------------------------------------
 Paul D. Smith <psmith@nortel.com>           HASMAT--HA Software Mthds & Tools
 "Please remain calm...I may be mad, but I am a professional." --Mad Scientist
-------------------------------------------------------------------------------
   These are my opinions---Nortel Networks takes no responsibility for them.
-- 
gentoo-catalyst@gentoo.org mailing list

Re: [gentoo-catalyst] How is initrd built?

[Eric Edgar]

[-- Attachment #1: Type: text/plain, Size: 5711 bytes --]

On 16:01 Thu 07 Jul     , Paul Smith wrote:
> %% Chris Gianelloni <wolf31o2@gentoo.org> writes:
> 
>   cg> Actually, what rocket means in that catalyst has no built-in
>   cg> support for unionfs.  To get unionfs, you need to use gk_mainargs
>   cg> (or gk_kernargs) --unionfs.
> 
> Oh, OK.  Yeah, I already found this in the new genkernel, and did do it
> in livecd.conf (I thought I mentioned that in one of my mails).
> 
>   cg> You will also probably want livecd/bootargs: unionfs...  unless
>   cg> you know the device you want, in which case you can do
>   cg> livecd/bootargs: unionfs=/dev/sda1 (or whatever).
> 
> I'm actually using GNAP to create the ISO, so I override the kernel boot
> params there and don't need this.
> 
>   cg> Yeah, livecd/bootargs is new in the _pre series.  I even documented it
>   cg> in the template spec files... ;]
> 
> That's a good enhancement though! :-).
> 
>   >> 0) mkdir -p /newroot
>   >> 
>   >> 1) Create a tmpfs at /memory
>   >> 
>   >> 2) Create a unionfs at /union, where the only item in the union is
>   >> /memory (mounted RW)
>   >> 
>   >> 3) Init /newroot (make directories, copy over /dev/null and /dev/console)
>   >> 
>   >> 4) Find the cdrom and mount it as /newroot/mnt/cdrom
>   >> 
>   >> 5) Mount the squashfs as /newroot/mnt/livecd (read-only)
>   >> 
>   >> 6) Add /newroot/mnt/livecd to the unionfs after the tmpfs (/memory)
>   >> 
>   >> 7) Create /union/proc and /union/sys
>   >> 
>   >> 8) Pivot_root to make /union the new /.
>   >> 
>   >> 9) Clean up the old root (in /tmp/.initrd)
>   >> 
>   >> 10) chroot . 'exec /sbin/init' (basically).
> 
> Hold on a second...  I just discovered something that I don't understand
> at all.  The behavior above, in particular steps #8 & 9, is not actually
> what happens.
Initramfs's are special as in they eliminate many of the workarounds
that are present in an initrd.  You no longer need to specify init= on
the command line.  In fact the kernel ignores it when an initramfs is
detected.  Also the kernel expects the initial script to be called init.
So genkernel renames the linuxrc file to init inside the initramfs.
Because the script is called init the value of $0 is set to /init.

Initramfs are also different in that they no longer pivot_root and then
chroot.  You can just chroot directly.  The differing code is to handle
this behavior difference.  Ultimately its much simpler than worrying
about the pivot root.

> 
> If you look in the linuxrc you'll see it has an option to be invoked one
> of two ways: where $0 is "/init", and where $0 is "/linuxrc".  Depending
> on which one is used the script exhibits different behavior... very
> different, in some cases.  Like, the "/init" branch doesn't use
> pivot_root at all; it just does a chroot.  The whole end cleanup is very
> different.
> 
> I've been thinking that the livecd is booting with $0 == "/linuxrc", but
> in fact it's not: $0 == "/init".
> 
> Can someone give me an idea of where this ($0) is set up (it's not init=
> on the boot line because my boot line says init=/linuxrc--but it must be
> the kernel somehow because it execs the shell so it would set argv[0]),
> and what the purpose is between the different behaviors, and why you'd
> choose one over the other?
> 
>   cg> AFIAK, /mnt/cdrom would need to be done before the pivot_root,
>   cg> too.  So you can't move that to later.
> 
> True, but you could do one of two things:
> 
>  a) Mount it in /unionfs/mnt/cdrom in the first place, not in
>     /newroot/mnt/cdrom.  By the time we mount the cdrom we already have
>     a /union filesystem created and working.
> 
> OR
> 
>  b) Try to "mount --move" it from /newroot to /union once the unionfs is
>     set up.

I would lean toward --move if it works, but it will need to be tested.
> 
> I looked at (a) and while I think it would work, it would involve a lot
> of changes.  Although the value of CHROOT is set to $NEW_ROOT (or $UNION,
> if we're using unionfs) early in the script, almost all the commands
> after that continue to use $NEW_ROOT explicitly rather than $CHROOT.
> Sometimes this probably makes sense (such as mounting livecd) but in
> other cases it doesn't, so much (such as in mounting cdrom).  I think
> these variables need to be rationalized somewhat, so $CHROOT is used
> instead of $NEW_ROOT where appropriate.

Alot of this unionfs code is an after thought and so it was implemented
in such a way that it was minimally impacting to the normal booting
method.  Overtime I hope to have things cleaned up as we discover issues
and/or convert entirely to this boot method.

> 
> There are other things that probably don't hurt, but aren't required
> either.  For example, even in the unionfs case the code still
> initializes $NEW_ROOT with a bunch of stuff that is never needed, and
> then is just cleaned up again afterwards.

Again see the above comment about code impact and that will explain why
things are as they are.  

If you find a working solution for you that boots the system with or
without unionfs support please make a patch and we will look to have it
included.

Thanks.
> -- 
> -------------------------------------------------------------------------------
>  Paul D. Smith <psmith@nortel.com>           HASMAT--HA Software Mthds & Tools
>  "Please remain calm...I may be mad, but I am a professional." --Mad Scientist
> -------------------------------------------------------------------------------
>    These are my opinions---Nortel Networks takes no responsibility for them.
> -- 
> gentoo-catalyst@gentoo.org mailing list
> 
> 

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-catalyst] How is initrd built?

[Paul Smith]

%% Eric Edgar <rocket@gentoo.org> writes:

  ee> Initramfs are also different in that they no longer pivot_root and
  ee> then chroot.  You can just chroot directly.  The differing code is
  ee> to handle this behavior difference.  Ultimately its much simpler
  ee> than worrying about the pivot root.

OK, this is good stuff to know.

Thanks!

  >> a) Mount it in /unionfs/mnt/cdrom in the first place, not in
  >> /newroot/mnt/cdrom.  By the time we mount the cdrom we already have
  >> a /union filesystem created and working.
  >> 
  >> OR
  >> 
  >> b) Try to "mount --move" it from /newroot to /union once the unionfs is
  >> set up.

  ee> I would lean toward --move if it works, but it will need to be tested.

Unfortunately I can't get it working.  First, the mount in busybox
doesn't have a --move option.  Looking at the mount applet code, it
appears as though they support the move option as "mount -o move .. ..",
however, that doesn't work for me when I try it:

    mount -o move /newroot/mnt/cdrom /union/mnt/cdrom

complains that the first filename is not a block device, which it isn't
and shouldn't be: the move operation takes two mounted filesystems
(according to man mount).  So, I don't know whether this is a bug or what.
I'm going to check the busybox mailing list.


I was able to mount the cdrom twice, in both directories at the same
time.  Of course, I can't unmount /newroot/mnt/cdrom because it's held
there to provide the squashfs, but at least the cdrom is visible after
the chroot/during boot time.


If mount --move (or equivalent) can't be made to work, it seems the only
alternative is to mount it in $CHROOT instead of $NEW_ROOT, right from
the start.

  ee> Alot of this unionfs code is an after thought and so it was
  ee> implemented in such a way that it was minimally impacting to the
  ee> normal booting method.  Overtime I hope to have things cleaned up
  ee> as we discover issues and/or convert entirely to this boot method.

Yep, understood.


So, the next problem is that after the init is done and the system
starts to boot, various things fail because /etc/fstab is empty.

I don't see anywhere in the "normal" boot process that /etc/fstab is
built... am I missing something?  Is it as simple as "cp /proc/mounts
$CHROOT/etc/fstab"?

-- 
-------------------------------------------------------------------------------
 Paul D. Smith <psmith@nortel.com>           HASMAT--HA Software Mthds & Tools
 "Please remain calm...I may be mad, but I am a professional." --Mad Scientist
-------------------------------------------------------------------------------
   These are my opinions---Nortel Networks takes no responsibility for them.
-- 
gentoo-catalyst@gentoo.org mailing list

Re: [gentoo-catalyst] How is initrd built?

[Paul Smith]

%% I wrote:

  sp> So, the next problem is that after the init is done and the system
  sp> starts to boot, various things fail because /etc/fstab is empty.

  sp> I don't see anywhere in the "normal" boot process that /etc/fstab is
  sp> built... am I missing something?  Is it as simple as "cp /proc/mounts
  sp> $CHROOT/etc/fstab"?

OK, I found this problem.  This code in linuxrc in genkernel is bogus:

    cat /${UNION}/etc/fstab | sed -e 's|\(.*\s/\s*tmpfs\s*\)defaults\(.*\)|\1defaults,ro\2|' > /${UNION}/etc/fstab

We are cat'ing a file in a pipeline, and redirecting the output back to
that same file; that is not safe.  In fact, sometimes when I run this it
works but most of the time I get a zero-length fstab file.  When I get an
empty file the normal boot fails because it tries to do invalid things
like remount the union filesystem, which is not supported.

This is fixed easily enough by using sed -i instead:

    sed -i -e 's|\(.*\s/\s*tmpfs\s*\)defaults\(.*\)|\1defaults,ro\2|' /${UNION}/etc/fstab

However, it's actually not really clear to me what this is trying to do.
The comment is at best very confusing to me:

    # Make sure fstab notes livecd is mounted ro.  Makes system skip remount which fails on unionfs dirs.

The above sed doesn't touch anything related to livecd; it's managing
"/", which will be (when we chroot) the unionfs.  And, why changing it
to say "ro" in /etc/fstab will STOP the system from trying to remount it
"rw" is really a mystery to me...??

-- 
-------------------------------------------------------------------------------
 Paul D. Smith <psmith@nortel.com>           HASMAT--HA Software Mthds & Tools
 "Please remain calm...I may be mad, but I am a professional." --Mad Scientist
-------------------------------------------------------------------------------
   These are my opinions---Nortel Networks takes no responsibility for them.
-- 
gentoo-catalyst@gentoo.org mailing list

Re: [gentoo-catalyst] How is initrd built?

[Paul Smith]

OK, I'm sending a lot of mail.

But, I'm realizing that the version of busybox that's included in the
initramfs is really quite old, and that's (I think) why I'm seeing
various problems.

In my previous email I recommended using "sed -i", and I had already
looked that up in the busybox documentation and it is supported.  But,
the version of busybox that appears in my initramfs, 1.00-pre7,
apparently doesn't support that flag (I get errors when I use it in
linuxrc).


Now I'm wondering if this is the reason why my "mount -o move" didn't
work, either: maybe the version of busybox is old enough that this
option is not recognized.


Where does that busybox version come from?  It doesn't seem to be
built on my system; does it come in one of those prebuilt tarballs for
genkernel?  Is there any straightforward way to get it updated?

Or, am I totally out in left field again?

-- 
-------------------------------------------------------------------------------
 Paul D. Smith <psmith@nortel.com>           HASMAT--HA Software Mthds & Tools
 "Please remain calm...I may be mad, but I am a professional." --Mad Scientist
-------------------------------------------------------------------------------
   These are my opinions---Nortel Networks takes no responsibility for them.
-- 
gentoo-catalyst@gentoo.org mailing list

Re: [gentoo-catalyst] How is initrd built?

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 529 bytes --]

On Thu, 2005-07-07 at 18:39 -0400, Paul Smith wrote:
> So, the next problem is that after the init is done and the system
> starts to boot, various things fail because /etc/fstab is empty.
> 
> I don't see anywhere in the "normal" boot process that /etc/fstab is
> built... am I missing something?  Is it as simple as "cp /proc/mounts
> $CHROOT/etc/fstab"?

We never build a fstab.  We just manually mount stuff.

-- 
Chris Gianelloni
Release Engineering - Strategic Lead/QA Manager
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-catalyst] How is initrd built?

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 834 bytes --]

On Thu, 2005-07-07 at 21:44 -0400, Paul Smith wrote:
> OK, I found this problem.  This code in linuxrc in genkernel is bogus:

Right, right.  Ignore my previous comment about not building fstab.  I
was thinking something else.

> The above sed doesn't touch anything related to livecd; it's managing
> "/", which will be (when we chroot) the unionfs.  And, why changing it
> to say "ro" in /etc/fstab will STOP the system from trying to remount it
> "rw" is really a mystery to me...??

This one is easy.  Normally the bootloader sets root to ro, but the
fstab entry is for rw, so the init scripts remount it rw.  Now, if you
want to force the init scripts to not remount it, then it needs to state
ro in fstab.

-- 
Chris Gianelloni
Release Engineering - Strategic Lead/QA Manager
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-catalyst] How is initrd built?

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 941 bytes --]

On Thu, 2005-07-07 at 22:19 -0400, Paul Smith wrote:
> Where does that busybox version come from?  It doesn't seem to be
> built on my system; does it come in one of those prebuilt tarballs for
> genkernel?  Is there any straightforward way to get it updated?

It comes from genkernel.  To update it, you need to update the tarball
provided by genkernel, the /etc/genkernel.conf file, and possibly the
configuration file used to actually build busybox.

Remember that, as a rule, we follow the "if it isn't broken" rule with
genkernel.  We never update the software versions in genkernel unless
there is a good reason to do so.  This might be a good reason.  We would
have to evaluate if there are any custom patches that we have added, and
if they would need to be forward ported to a new version, then get it
tested.

-- 
Chris Gianelloni
Release Engineering - Strategic Lead/QA Manager
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]