[gentoo-user] My first initramfs

[gentoo-user] My first initramfs

[Tanstaafl]

Ok, looking into this now...

Reading the referenced wiki page shows this comment:

> Introduction and bootloader configuration
>
> To create an initramfs, it is important that you know what
> additional drivers, scripts and tools you need to boot your system.
> For instance, if you use LVM, then you will need to support LVM tools
> on the initramfs. Likewise, if you use software RAID, you need mdadm,
> etc.

I thought I'd mentioned/asked this before, but don't recall a 
satisfactory answer...

Ok, up until now, I haven't *had* to 'know' what additional drivers are 
needed by my system to boot.

So... how the heck am I supposed to find out? Trial and error?

I do know I need lvm, so have that one listed... but what else? There 
has *got* to be some kind of way to analyze an existing system to reveal 
the 'drivers, scripts and tools' I need for my initramfs so I know what 
to include?

And how to do this should be *fully* covered in that wiki page.

I really feel like I'm being thrown to the wolves here.

Re: [gentoo-user] My first initramfs

[Tanstaafl]

On 2013-11-12 4:49 PM, Tanstaafl <tanstaafl@libertytrek.org> wrote:
> Ok, looking into this now...
>
> Reading the referenced wiki page shows this comment:
>
>> Introduction and bootloader configuration
>>
>> To create an initramfs, it is important that you know what
>> additional drivers, scripts and tools you need to boot your system.
>> For instance, if you use LVM, then you will need to support LVM tools
>> on the initramfs. Likewise, if you use software RAID, you need mdadm,
>> etc.
>
> I thought I'd mentioned/asked this before, but don't recall a
> satisfactory answer...
>
> Ok, up until now, I haven't *had* to 'know' what additional drivers are
> needed by my system to boot.
>
> So... how the heck am I supposed to find out? Trial and error?

Further down under the genkernel section has:

> Depending on your system, you might want to add one or more of the following options:
> Option 	Description
> --disklabel 	Add support for LABEL= settings in your /etc/fstab
> --dmraid 	Add support for fake hardware RAID
> --firmware 	Add in firmware code found on the system
> --gpg 	Add in GnuPG support
> --iscsi 	Add support for iSCSI
> --luks 	Add support for luks encryption containers
> --lvm 	Add support for LVM
> --mdadm 	Add support for software RAID
> --multipath 	Add support for multiple I/O access towards a SAN
> --zfs 	Add support for ZFS

So, are these *all* of the potential options? If so, then I can probably 
safely say that all I need is lvm...

Re: [gentoo-user] My first initramfs

[Canek Peláez Valdés]

On Tue, Nov 12, 2013 at 3:55 PM, Tanstaafl <tanstaafl@libertytrek.org> wrote:
> On 2013-11-12 4:49 PM, Tanstaafl <tanstaafl@libertytrek.org> wrote:
>>
>> Ok, looking into this now...
>>
>> Reading the referenced wiki page shows this comment:
>>
>>> Introduction and bootloader configuration
>>>
>>> To create an initramfs, it is important that you know what
>>> additional drivers, scripts and tools you need to boot your system.
>>> For instance, if you use LVM, then you will need to support LVM tools
>>> on the initramfs. Likewise, if you use software RAID, you need mdadm,
>>> etc.
>>
>>
>> I thought I'd mentioned/asked this before, but don't recall a
>> satisfactory answer...
>>
>> Ok, up until now, I haven't *had* to 'know' what additional drivers are
>> needed by my system to boot.
>>
>> So... how the heck am I supposed to find out? Trial and error?
>
>
> Further down under the genkernel section has:
>
>> Depending on your system, you might want to add one or more of the
>> following options:
>> Option  Description
>> --disklabel     Add support for LABEL= settings in your /etc/fstab
>> --dmraid        Add support for fake hardware RAID
>> --firmware      Add in firmware code found on the system
>> --gpg   Add in GnuPG support
>> --iscsi         Add support for iSCSI
>> --luks  Add support for luks encryption containers
>> --lvm   Add support for LVM
>> --mdadm         Add support for software RAID
>> --multipath     Add support for multiple I/O access towards a SAN
>> --zfs   Add support for ZFS
>
>
> So, are these *all* of the potential options? If so, then I can probably
> safely say that all I need is lvm...

You *MUST* add the necessary modules/tools to mount root and/or /usr.
So if you have an XFS partition on a LVM volume on top of an mdraid,
and the partition is encrypted, then you need the kernel modules for
xfs, lvm, mdraid and crypt, and the corresponding userspace tools,
including fsck.xfs.

You *MAY* add whatever else you want in your initramfs (I add the
drivers for my USB keyboard, so in case of emergencies I can actually
type commands), but the only parts you *MUST* include are the ones you
need to mount (and check) your root and /usr partitions.

I strongly recommend dracut; the modules listed in DRACUT_MODULES are
usually self explanatory, and it Just Works™.

Regards.
-- 
Canek Peláez Valdés
Posgrado en Ciencia e Ingeniería de la Computación
Universidad Nacional Autónoma de México

Re: [gentoo-user] My first initramfs

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 1526 bytes --]

On Tue, 12 Nov 2013 16:49:33 -0500, Tanstaafl wrote:

> > To create an initramfs, it is important that you know what
> > additional drivers, scripts and tools you need to boot your system.
> > For instance, if you use LVM, then you will need to support LVM tools
> > on the initramfs. Likewise, if you use software RAID, you need mdadm,
> > etc.  
> 
> I thought I'd mentioned/asked this before, but don't recall a 
> satisfactory answer...
> 
> Ok, up until now, I haven't *had* to 'know' what additional drivers are 
> needed by my system to boot.
> 
> So... how the heck am I supposed to find out? Trial and error?
> 
> I do know I need lvm, so have that one listed... but what else? There 
> has *got* to be some kind of way to analyze an existing system to
> reveal the 'drivers, scripts and tools' I need for my initramfs so I
> know what to include?

You need to be able to mount your root filesystem, and possibly /usr.

So you need anything needed for that that isn't compiled not your kernel,
such as SATA drivers, filesystem etc. You would normally build anything
like this into the kernel, so you just need the userspace tools. For
standard operations, that is all covered by busybox, but if you use LVM,
RAID or dmcrypt for those filesystems, include the relevant tools.

There is a very capable tool for examining your system and deciding what
is needed... you. 


-- 
Neil Bothwick

"If you can't explain it simply, you don't understand it well enough."
 (Albert Einstein)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] My first initramfs

[covici]

Canek Peláez Valdés <caneko@gmail.com> wrote:

> On Tue, Nov 12, 2013 at 3:55 PM, Tanstaafl <tanstaafl@libertytrek.org> wrote:
> > On 2013-11-12 4:49 PM, Tanstaafl <tanstaafl@libertytrek.org> wrote:
> >>
> >> Ok, looking into this now...
> >>
> >> Reading the referenced wiki page shows this comment:
> >>
> >>> Introduction and bootloader configuration
> >>>
> >>> To create an initramfs, it is important that you know what
> >>> additional drivers, scripts and tools you need to boot your system.
> >>> For instance, if you use LVM, then you will need to support LVM tools
> >>> on the initramfs. Likewise, if you use software RAID, you need mdadm,
> >>> etc.
> >>
> >>
> >> I thought I'd mentioned/asked this before, but don't recall a
> >> satisfactory answer...
> >>
> >> Ok, up until now, I haven't *had* to 'know' what additional drivers are
> >> needed by my system to boot.
> >>
> >> So... how the heck am I supposed to find out? Trial and error?
> >
> >
> > Further down under the genkernel section has:
> >
> >> Depending on your system, you might want to add one or more of the
> >> following options:
> >> Option  Description
> >> --disklabel     Add support for LABEL= settings in your /etc/fstab
> >> --dmraid        Add support for fake hardware RAID
> >> --firmware      Add in firmware code found on the system
> >> --gpg   Add in GnuPG support
> >> --iscsi         Add support for iSCSI
> >> --luks  Add support for luks encryption containers
> >> --lvm   Add support for LVM
> >> --mdadm         Add support for software RAID
> >> --multipath     Add support for multiple I/O access towards a SAN
> >> --zfs   Add support for ZFS
> >
> >
> > So, are these *all* of the potential options? If so, then I can probably
> > safely say that all I need is lvm...
> 
> You *MUST* add the necessary modules/tools to mount root and/or /usr.
> So if you have an XFS partition on a LVM volume on top of an mdraid,
> and the partition is encrypted, then you need the kernel modules for
> xfs, lvm, mdraid and crypt, and the corresponding userspace tools,
> including fsck.xfs.
> 
> You *MAY* add whatever else you want in your initramfs (I add the
> drivers for my USB keyboard, so in case of emergencies I can actually
> type commands), but the only parts you *MUST* include are the ones you
> need to mount (and check) your root and /usr partitions.
> 
> I strongly recommend dracut; the modules listed in DRACUT_MODULES are
> usually self explanatory, and it Just Works™.
Hi.  I am using genkernel and openrc and it boots correctly using lvm,
(I have ext2-3 modules built in), but what it does never do is check the
/usr which is on a separate partition before mounting, so /usr is never
checked unless I boot from a live cd and do it myself.  I wonder if
there is any way to get this to be done?



-- 
Your life is like a penny.  You're going to lose it.  The question is:
How do
you spend it?

         John Covici
         covici@ccs.covici.com

Re: [gentoo-user] My first initramfs

[Tanstaafl]

On 2013-11-12 4:55 PM, Tanstaafl <tanstaafl@libertytrek.org> wrote:
>> Depending on your system, you might want to add one or more of the
>> following options:
>> Option     Description
>> --disklabel     Add support for LABEL= settings in your /etc/fstab
>> --dmraid     Add support for fake hardware RAID
>> --firmware     Add in firmware code found on the system
>> --gpg     Add in GnuPG support
>> --iscsi     Add support for iSCSI
>> --luks     Add support for luks encryption containers
>> --lvm     Add support for LVM
>> --mdadm     Add support for software RAID
>> --multipath     Add support for multiple I/O access towards a SAN
>> --zfs     Add support for ZFS
>
> So, are these *all* of the potential options? If so, then I can probably
> safely say that all I need is lvm...

Never got a response to this...

Maybe I can ask another way...

Is this a complete list of the items that may break BOOTING? I 
understand now that you can add pretty much any drivers that you want, 
but on my server (that has the separate /usr), modules are disabled and 
so I have every driver and filesystem built into the kernel.

Going just by the above list of items, the only thing I use is lvm, so 
I'm guessing (I *hate* guessing) that this is the only thing I need in 
my initramfs.

Another question - is there any reason I couldn't just enable everything 
just in case? Or could that cause other problems?

On 2013-11-12 5:54 PM, Neil Bothwick <neil@digimed.co.uk> wrote:
 > There is a very capable tool for examining your system and deciding
 > what is needed... you.

Yes, but again, I have never had to worry about this - EVER - in the 
past. The question simply never came up.

Now, I am being FORCED to worry about it... and I want to understand it 
as much as possible.

Re: [gentoo-user] My first initramfs

[Tanstaafl]

On 2013-11-12 5:50 PM, Canek Peláez Valdés <caneko@gmail.com> wrote:
> You*MUST*  add the necessary modules/tools to mount root and/or /usr.
> So if you have an XFS partition on a LVM volume on top of an mdraid,
> and the partition is encrypted, then you need the kernel modules for
> xfs, lvm, mdraid and crypt, and the corresponding userspace tools,
> including fsck.xfs.

Ok, re-reading these messages, this explanation is probably the most 
significant - thanks Canek - but I still have a few questions...

Maybe part of my confusion stems from use of the word 'modules' above? 
Remember, the system I am talking about has always had kernel module 
support completely disabled (everything is built into the kernel).

So, my /usr is reiserfs (built into the kernel - and please no 
filesystem flame-war comments, it is what it is) on LVM (built into the 
kernel) on hardware RAID (driver built into the kernel), with no 
encryption involved.

So, am I correct that I don't need ANY 'modules' (never have, and don't 
want them), leaving only the userspace tools for lvm and reiserfsck?

If so then the only other question is, how to build an initramfs with 
just these...

> I strongly recommend dracut; the modules listed in DRACUT_MODULES are
> usually self explanatory, and it Just Works™.

Ok, well, emerge -pvuDN dracut gives:

> [ebuild  N    ~] sys-kernel/dracut-034-r1  USE="-debug -device-mapper -net (-selinux)" DRACUT_MODULES="-biosdevname -bootchart -btrfs -caps -cifs -crypt -crypt-gpg -crypt-loop -dash -dmraid -dmsquash-live -gensplash -iscsi -livenet -lvm -mdraid -multipath -nbd -nfs -plymouth -ssh-client -syslog -systemd" 272 kB

First question: I don't see reference to reiserfs there anywhere, so how 
do I get reiserfsck (and is that all I need?) support into the initrams?

Second question: I guess I'd need to enable the device-mapper USE flag 
for dracut, but, since I do not use ANY kernel modules, can I ignore all 
of the 'DRACUT_MODULES'? Or do 'DRACUT_MODULES' have nothing to do with 
kernel modules?

Thanks to all for your patience...

Re: [gentoo-user] My first initramfs

[Tanstaafl]

On 2013-11-17 7:57 AM, Tanstaafl <tanstaafl@libertytrek.org> wrote:
> First question: I don't see reference to reiserfs there anywhere,

Or XFS for that matter...

Re: [gentoo-user] My first initramfs

[thegeezer]

[-- Attachment #1: Type: text/plain, Size: 2969 bytes --]

On 11/17/2013 12:15 PM, Tanstaafl wrote:
> On 2013-11-12 4:55 PM, Tanstaafl <tanstaafl@libertytrek.org> wrote:
>>> Depending on your system, you might want to add one or more of the
>>> following options:
>>> Option     Description
>>> --disklabel     Add support for LABEL= settings in your /etc/fstab
>>> --dmraid     Add support for fake hardware RAID
>>> --firmware     Add in firmware code found on the system
>>> --gpg     Add in GnuPG support
>>> --iscsi     Add support for iSCSI
>>> --luks     Add support for luks encryption containers
>>> --lvm     Add support for LVM
>>> --mdadm     Add support for software RAID
>>> --multipath     Add support for multiple I/O access towards a SAN
>>> --zfs     Add support for ZFS
>>
>> So, are these *all* of the potential options? If so, then I can probably
>> safely say that all I need is lvm...
>

these options say do additional tasks. i.e. initialise lvm volume groups
or search and start all mdraid devices.  or all of the above.
you only need lvm if root/usr is on lvm.
you only need mdadm if root/usr is on a mdraid
if your root/usr is on a normal device  like /dev/sda3  but all data
drives are on lvm then you still do not need these options.
they are just what to do in the initramfs to get root mounted.

> Never got a response to this...
>
> Maybe I can ask another way...
>
> Is this a complete list of the items that may break BOOTING? I
> understand now that you can add pretty much any drivers that you want,
> but on my server (that has the separate /usr), modules are disabled
> and so I have every driver and filesystem built into the kernel.
>
> Going just by the above list of items, the only thing I use is lvm, so
> I'm guessing (I *hate* guessing) that this is the only thing I need in
> my initramfs.
>
do you have kernel modules that are not compiled into the kernel in
order to mount root ? 
i.e. is root on ext4 and ext4 is <M> in menuconfig
i.e. is root on an adaptec scsi card and the scsi drivers is  <M> in
menuconfig
then you need  to edit genkernel's modules_load
https://forums.gentoo.org/viewtopic-t-497370.html

for split /usr check out initramfs.mounts
http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/sys-kernel/genkernel/files/initramfs.mounts?view=markup
i don't do this so can't comment if it works, maybe someone else can?
> Another question - is there any reason I couldn't just enable
> everything just in case? Or could that cause other problems?
>
if you have them selected anyway, it jus tmakes the boot process slower
as it scans for raid devices to find none..
> On 2013-11-12 5:54 PM, Neil Bothwick <neil@digimed.co.uk> wrote:
> > There is a very capable tool for examining your system and deciding
> > what is needed... you.
>
> Yes, but again, I have never had to worry about this - EVER - in the
> past. The question simply never came up.
>
> Now, I am being FORCED to worry about it... and I want to understand
> it as much as possible.
>
hope this helps

[-- Attachment #2: Type: text/html, Size: 5499 bytes --]

Re: [gentoo-user] My first initramfs

[Tanstaafl]

On 2013-11-17 8:39 AM, thegeezer <thegeezer@thegeezer.net> wrote:
> you only need lvm if root/usr is on lvm.

root is on a normal partition, /usr is on a separate lvm partition.

> do you have kernel modules that are not compiled into the kernel in
> order to mount root ?

Ummm... I think I was *extremely* clear on this point?

Directly from the quoted text in my message that you responded to:

"but on my server (that has the separate /usr), modules are disabled and 
so I have every driver and filesystem built into the kernel."

So, again, no modules, zero, nada, everything is compiled into the kernel.

So, I'm not even certain that I need lvm support (since it is built into 
the kernel)?

And then, how to make sure I have reiserfsck (anything else reiserfs 
related?) support?

Re: [gentoo-user] My first initramfs

[thegeezer]

On 11/17/2013 02:34 PM, Tanstaafl wrote:
> On 2013-11-17 8:39 AM, thegeezer <thegeezer@thegeezer.net> wrote:
>> you only need lvm if root/usr is on lvm.
>
> root is on a normal partition, /usr is on a separate lvm partition.
>
you will need the intramfs.mount and lvm
>> do you have kernel modules that are not compiled into the kernel in
>> order to mount root ?
>
> Ummm... I think I was *extremely* clear on this point?
sorry missed in the thread
>
> Directly from the quoted text in my message that you responded to:
>
> "but on my server (that has the separate /usr), modules are disabled
> and so I have every driver and filesystem built into the kernel."
>
> So, again, no modules, zero, nada, everything is compiled into the
> kernel.
>
ya no need for modules in intramfs
> So, I'm not even certain that I need lvm support (since it is built
> into the kernel)?
>
you do in order to premount usr
> And then, how to make sure I have reiserfsck (anything else reiserfs
> related?) support?
>
>
reiser support would appear beacuse you have it compiled in the kernel
any additional tools you would need to add into genkernel.

for reiserfsck, i would suggest looking into your busybox config.  i
know there is an option you can pass to genkernel --busybox-config 
you can look at /usr/share/genkernel/defaults/busy-config but the exact
options i don't know.
I did google it though because i hate not knowing. 
here a suggestion is to use genkernel --initramfs-overlay=<dir>  to add
<dir> into the initramfs
http://forums.gentoo.org/viewtopic-t-928398-start-0.html
so you could mkdir -p /srv/bin/genkernelextras/tools, put all the tools
you want to use in that folder and pass
--initramfs-overlay=/srv/bin/genkernelextras

but the initramfs is supposed to be the minimal for getting the system
booted,
genkernel is a tool to help get this done.
it's not really for rescue. for rescue i'd suggest a seperate boot,
perhaps if using grub2 boot an iso.

however, a quick google search away and it looks like dracut has a much
more modular approach.  i have never used dracut maybe someone else can
chip in with it
http://wiki.gentoo.org/wiki/Dracut#Other_Filesystems

you may also consider creating your own initramfs, which gives you the
granular control you perhaps need.
http://gentoo-en.vfose.ru/wiki/Initramfs

my own 2c would be to suggest to create an lvm using not reiser that you
sync your existing root and usr into. but this is way offtopic

finally a quick google of some of the issues you raise found me this guy
doing something almost identical to you with the suggestion that busybox
will be able to support reiser out of the box
https://forums.gentoo.org/viewtopic-t-912716-start-0.html

Re: [gentoo-user] My first initramfs

[Canek Peláez Valdés]

On Sun, Nov 17, 2013 at 6:57 AM, Tanstaafl <tanstaafl@libertytrek.org> wrote:
> On 2013-11-12 5:50 PM, Canek Peláez Valdés <caneko@gmail.com> wrote:
>>
>> You*MUST*  add the necessary modules/tools to mount root and/or /usr.
>>
>> So if you have an XFS partition on a LVM volume on top of an mdraid,
>> and the partition is encrypted, then you need the kernel modules for
>> xfs, lvm, mdraid and crypt, and the corresponding userspace tools,
>> including fsck.xfs.
>
>
> Ok, re-reading these messages, this explanation is probably the most
> significant - thanks Canek - but I still have a few questions...
>
> Maybe part of my confusion stems from use of the word 'modules' above?
> Remember, the system I am talking about has always had kernel module support
> completely disabled (everything is built into the kernel).

In dracut, you can add kernel modules ("drivers" in dracut
terminology, which you don't require since you compile everything
in-kernel), and dracut modules. I was talking about the firsts kind.

If you have the modules compiled in the kernel, you only need the
corresponding userspace tools.

> So, my /usr is reiserfs (built into the kernel - and please no filesystem
> flame-war comments, it is what it is) on LVM (built into the kernel) on
> hardware RAID (driver built into the kernel), with no encryption involved.

I've never used a hardware raid. I don't know if it needs mdraid; if
it does, you need the *dracut* module for mdraid, otherwise you are
good. In either case, you need the *dracut* module for LVM: the kernel
alone is not able to mount LVM volumes, AFAIU; you need the userspace
tools.

> So, am I correct that I don't need ANY 'modules' (never have, and don't want
> them), leaving only the userspace tools for lvm and reiserfsck?

You need the *dracut* module for LVM, you don't need any kernel
module. Or better stated: you already have the kernel modules, inside
your kernel. What dracut does when you specify "drivers" (kernel
modules), is only to include them in the initramfs. So even if you
specify them in your dracut config file, since they are not available,
dracut will print a warning about the driver not being avaible, and it
will continue without it.

> If so then the only other question is, how to build an initramfs with just
> these...
>
>
>> I strongly recommend dracut; the modules listed in DRACUT_MODULES are
>> usually self explanatory, and it Just Works™.
>
>
> Ok, well, emerge -pvuDN dracut gives:
>
>> [ebuild  N    ~] sys-kernel/dracut-034-r1  USE="-debug -device-mapper -net
>> (-selinux)" DRACUT_MODULES="-biosdevname -bootchart -btrfs -caps -cifs
>> -crypt -crypt-gpg -crypt-loop -dash -dmraid -dmsquash-live -gensplash -iscsi
>> -livenet -lvm -mdraid -multipath -nbd -nfs -plymouth -ssh-client -syslog
>> -systemd" 272 kB
>
>
> First question: I don't see reference to reiserfs there anywhere, so how do
> I get reiserfsck (and is that all I need?) support into the initrams?

Filesystems are inferred from fstab and the fscks key in dracut.conf.
If you use the -H (host) option, you don't even need to touch fscks,
the information will be parsed from fstab.

> Second question: I guess I'd need to enable the device-mapper USE flag for
> dracut, but, since I do not use ANY kernel modules, can I ignore all of the
> 'DRACUT_MODULES'? Or do 'DRACUT_MODULES' have nothing to do with kernel
> modules?

DRACUT_MODULES have nothing to do with kernel modules (those are
called "drivers" inside dracut). The dracut modules are little scripts
which copy stuff inside the initramfs.

By the way, if you install dracut, it will do nothing in your system;
it's only for populate an initramfs. You can install it and it will
not affect your system in any way.

Regards.
-- 
Canek Peláez Valdés
Posgrado en Ciencia e Ingeniería de la Computación
Universidad Nacional Autónoma de México