[gentoo-dev] Need design help/input for eclean-kernel

[gentoo-dev] Need design help/input for eclean-kernel

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1538 bytes --]

Hello, everyone.

Back in 2011 I started a project called eclean-kernel. The idea was
pretty simple -- to have a tool that would clean the old kernels for
me since their install is not controlled by the package manager. This
little project of mine seems to have gained a lot of popularity.

Sadly, over time a lot of people had trouble with it. Aside to minor
Python problems, eclean-kernel proved too simple to handle multitude of
user systems with varying /boot layouts. In fact, even I don't use it
on all of my systems since it doesn't handle them properly.

After being buried in another set of bug reports, I'd like to
officially ask Gentoo developers and users for help. I think it's
impossible to solve most of the bugs reported so far in the current
program design. Therefore, I'd like to rewrite it in a more flexible
manner.

For this reason, I would like to ask you to provide me with
different /boot layouts you may have, had or seen. Basically, the idea
is to collect as many different layouts as necessary, and use that to
design eclean-kernel in a way making it possible to easily configure it
to handle proper variant -- or even possibly make it capable of
autoconfiguration.

So if you have some time, please reply to this thread with
a specific /boot layout that you think needs to be handled, with
as much helpful information as possible -- including possible
distinctive features and pitfalls.

Thanks in advance.

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] Need design help/input for eclean-kernel

[Daniel Campbell]

[-- Attachment #1.1: Type: text/plain, Size: 2575 bytes --]

On 06/30/2016 05:38 AM, Michał Górny wrote:
> Hello, everyone.
> 
> Back in 2011 I started a project called eclean-kernel. The idea was
> pretty simple -- to have a tool that would clean the old kernels for
> me since their install is not controlled by the package manager. This
> little project of mine seems to have gained a lot of popularity.
> 
> Sadly, over time a lot of people had trouble with it. Aside to minor
> Python problems, eclean-kernel proved too simple to handle multitude of
> user systems with varying /boot layouts. In fact, even I don't use it
> on all of my systems since it doesn't handle them properly.
> 
> After being buried in another set of bug reports, I'd like to
> officially ask Gentoo developers and users for help. I think it's
> impossible to solve most of the bugs reported so far in the current
> program design. Therefore, I'd like to rewrite it in a more flexible
> manner.
> 
> For this reason, I would like to ask you to provide me with
> different /boot layouts you may have, had or seen. Basically, the idea
> is to collect as many different layouts as necessary, and use that to
> design eclean-kernel in a way making it possible to easily configure it
> to handle proper variant -- or even possibly make it capable of
> autoconfiguration.
> 
> So if you have some time, please reply to this thread with
> a specific /boot layout that you think needs to be handled, with
> as much helpful information as possible -- including possible
> distinctive features and pitfalls.
> 
> Thanks in advance.
> 
I'm not sure if this is the info you're looking for, but I'll give it a
shot:

I have grub-static installed to /boot/. I like to organize my kernels
with the filenames as linux-${version}-gentoo-${buildno}. So my first
build of 4.5.0, for example, would be 'linux-4.5.0-gentoo-1'. It has all
the info I need for reference should something go awry.

I have three symlinks: current, last, backup

I wrote scripts that will update those symlinks for me, which makes the
process of kernel management pretty painless. Now that I'm thinking
about it, it could be simple in my case to simply clean any kernel that
wasn't linked to.

My /boot/:

grub
lost+found
backup -> linux-4.4.1-gentoo-2
boot
current -> linux-4.4.6-gentoo-1
initrd
last -> linux-4.4.1-gentoo-3
linux-4.4.1-gentoo-2
linux-4.4.1-gentoo-3
linux-4.4.6-gentoo-1
-- 
Daniel Campbell - Gentoo Developer
OpenPGP Key: 0x1EA055D6 @ hkp://keys.gnupg.net
fpr: AE03 9064 AE00 053C 270C  1DE4 6F7A 9091 1EA0 55D6


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-dev] Need design help/input for eclean-kernel

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2930 bytes --]

On Thu, 30 Jun 2016 05:55:42 -0700
Daniel Campbell <zlg@gentoo.org> wrote:

> On 06/30/2016 05:38 AM, Michał Górny wrote:
> > Hello, everyone.
> > 
> > Back in 2011 I started a project called eclean-kernel. The idea was
> > pretty simple -- to have a tool that would clean the old kernels for
> > me since their install is not controlled by the package manager. This
> > little project of mine seems to have gained a lot of popularity.
> > 
> > Sadly, over time a lot of people had trouble with it. Aside to minor
> > Python problems, eclean-kernel proved too simple to handle multitude of
> > user systems with varying /boot layouts. In fact, even I don't use it
> > on all of my systems since it doesn't handle them properly.
> > 
> > After being buried in another set of bug reports, I'd like to
> > officially ask Gentoo developers and users for help. I think it's
> > impossible to solve most of the bugs reported so far in the current
> > program design. Therefore, I'd like to rewrite it in a more flexible
> > manner.
> > 
> > For this reason, I would like to ask you to provide me with
> > different /boot layouts you may have, had or seen. Basically, the idea
> > is to collect as many different layouts as necessary, and use that to
> > design eclean-kernel in a way making it possible to easily configure it
> > to handle proper variant -- or even possibly make it capable of
> > autoconfiguration.
> > 
> > So if you have some time, please reply to this thread with
> > a specific /boot layout that you think needs to be handled, with
> > as much helpful information as possible -- including possible
> > distinctive features and pitfalls.
> > 
> > Thanks in advance.
> >   
> I'm not sure if this is the info you're looking for, but I'll give it a
> shot:
> 
> I have grub-static installed to /boot/. I like to organize my kernels
> with the filenames as linux-${version}-gentoo-${buildno}. So my first
> build of 4.5.0, for example, would be 'linux-4.5.0-gentoo-1'. It has all
> the info I need for reference should something go awry.
> 
> I have three symlinks: current, last, backup
> 
> I wrote scripts that will update those symlinks for me, which makes the
> process of kernel management pretty painless. Now that I'm thinking
> about it, it could be simple in my case to simply clean any kernel that
> wasn't linked to.
> 
> My /boot/:
> 
> grub
> lost+found
> backup -> linux-4.4.1-gentoo-2
> boot

What's 'boot' here? Is that relevant?

> current -> linux-4.4.6-gentoo-1
> initrd

Is that a single initrd for all kernels?

> last -> linux-4.4.1-gentoo-3
> linux-4.4.1-gentoo-2
> linux-4.4.1-gentoo-3
> linux-4.4.6-gentoo-1

And most importantly, how are all those files referenced in grub? I
suspect you are using the symlinks in grub.conf but want to confirm.

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] Need design help/input for eclean-kernel

[Joseph Booker]

[-- Attachment #1: Type: text/plain, Size: 2735 bytes --]

As someone using eclean-kernel somewhat regularly, I figured I should
mention what running `make all modules_install install` in the kernel
source looks like:

$ ls -F1 /boot
config-4.6.0-gentoo
config-4.6.1-gentoo
config-4.6.2-gentoo
config-4.6.3-gentoo
efi/
grub/
initramfs-4.6.0-gentoo.img
initramfs-4.6.1-gentoo.img
initramfs-4.6.2-gentoo.img
initramfs-4.6.3-gentoo.img
lost+found/
System.map-4.6.0-gentoo
System.map-4.6.1-gentoo
System.map-4.6.2-gentoo
System.map-4.6.3-gentoo
vmlinuz-4.6.0-gentoo
vmlinuz-4.6.1-gentoo
vmlinuz-4.6.2-gentoo
vmlinuz-4.6.3-gentoo

This is from running `make install` with files in /etc/kernel/postinst.d to
run the following:
cd /boot; dracut --force --kver=$1 --install "<list of firmware>
grub2-mkconfig -o /boot/grub/grub.cfg

/etc/grub.d/10_linux (in sys-boot/grub:2) does have a collection of names
it searches through as well, including 13 variants of initrd for the same
kernel version.

Thanks for maintaining this tool, it's pretty useful in this
straightforward case.

On Thu, Jun 30, 2016 at 8:38 AM, Michał Górny <mgorny@gentoo.org> wrote:

> Hello, everyone.
>
> Back in 2011 I started a project called eclean-kernel. The idea was
> pretty simple -- to have a tool that would clean the old kernels for
> me since their install is not controlled by the package manager. This
> little project of mine seems to have gained a lot of popularity.
>
> Sadly, over time a lot of people had trouble with it. Aside to minor
> Python problems, eclean-kernel proved too simple to handle multitude of
> user systems with varying /boot layouts. In fact, even I don't use it
> on all of my systems since it doesn't handle them properly.
>
> After being buried in another set of bug reports, I'd like to
> officially ask Gentoo developers and users for help. I think it's
> impossible to solve most of the bugs reported so far in the current
> program design. Therefore, I'd like to rewrite it in a more flexible
> manner.
>
> For this reason, I would like to ask you to provide me with
> different /boot layouts you may have, had or seen. Basically, the idea
> is to collect as many different layouts as necessary, and use that to
> design eclean-kernel in a way making it possible to easily configure it
> to handle proper variant -- or even possibly make it capable of
> autoconfiguration.
>
> So if you have some time, please reply to this thread with
> a specific /boot layout that you think needs to be handled, with
> as much helpful information as possible -- including possible
> distinctive features and pitfalls.
>
> Thanks in advance.
>
> --
> Best regards,
> Michał Górny
> <http://dev.gentoo.org/~mgorny/>
>

[-- Attachment #2: Type: text/html, Size: 3382 bytes --]

Re: [gentoo-dev] Need design help/input for eclean-kernel

[R0b0t1]

On Thu, Jun 30, 2016 at 7:38 AM, Michał Górny <mgorny@gentoo.org> wrote:
> So if you have some time, please reply to this thread with
> a specific /boot layout that you think needs to be handled, with
> as much helpful information as possible -- including possible
> distinctive features and pitfalls.

/boot on removable media.

I am busy with work now, but I had a set of scripts to compile,
package, and install a kernel and initrd. At some point I realized I
was recreating portage.

I have a mostly complete plan for a replacement, but it is not quite
presentable yet.

Re: [gentoo-dev] Need design help/input for eclean-kernel

[Marc Schiffbauer]

[-- Attachment #1: Type: text/plain, Size: 481 bytes --]

* Michał Górny schrieb am 30.06.16 um 15:19 Uhr:
> > My /boot/:
> > 
> > grub
> > lost+found
> > backup -> linux-4.4.1-gentoo-2
> > boot
> 
> What's 'boot' here? Is that relevant?

This might be a symlink to '.' which I too have on many systems. You 
need it, so grub can find files that start with /boot/ if you have /boot 
in its own partition. (IIRC)

-Marc


-- 
0xCA3E7BF67F979BE5 - F7FB 78F7 7CC3 79F6 DF07
                     6E9E CA3E 7BF6 7F97 9BE5

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-dev] Need design help/input for eclean-kernel

[Mike Gilbert]

On Thu, Jun 30, 2016 at 8:38 AM, Michał Górny <mgorny@gentoo.org> wrote:
> So if you have some time, please reply to this thread with
> a specific /boot layout that you think needs to be handled, with
> as much helpful information as possible -- including possible
> distinctive features and pitfalls.

Standard "make install" layout with dracut and grub2:

/boot/vmlinuz-$KV{,.old}
/boot/config-$KV
/boot/System.map-${KV}
/boot/initramfs-$KV.img
/boot/grub/grub.cfg generated by grub2-mkconfig contains menu entries
for each pairing.

The other layout I use on my UEFI laptop is the typical systemd-boot
(gummiboot) layout, which you are probably familiar with. I don't use
dracut on that, so I'm not certain where the initramfs lives.

Re: [gentoo-dev] Need design help/input for eclean-kernel

[Daniel Campbell (zlg)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On June 30, 2016 6:19:23 AM PDT, "Michał Górny" <mgorny@gentoo.org> wrote:
>On Thu, 30 Jun 2016 05:55:42 -0700
>Daniel Campbell <zlg@gentoo.org> wrote:
>
>> On 06/30/2016 05:38 AM, Michał Górny wrote:
>> > Hello, everyone.
>> >
>> > Back in 2011 I started a project called eclean-kernel. The idea was
>> > pretty simple -- to have a tool that would clean the old kernels
>for
>> > me since their install is not controlled by the package manager.
>This
>> > little project of mine seems to have gained a lot of popularity.
>> >
>> > Sadly, over time a lot of people had trouble with it. Aside to
>minor
>> > Python problems, eclean-kernel proved too simple to handle
>multitude of
>> > user systems with varying /boot layouts. In fact, even I don't use
>it
>> > on all of my systems since it doesn't handle them properly.
>> >
>> > After being buried in another set of bug reports, I'd like to
>> > officially ask Gentoo developers and users for help. I think it's
>> > impossible to solve most of the bugs reported so far in the current
>> > program design. Therefore, I'd like to rewrite it in a more
>flexible
>> > manner.
>> >
>> > For this reason, I would like to ask you to provide me with
>> > different /boot layouts you may have, had or seen. Basically, the
>idea
>> > is to collect as many different layouts as necessary, and use that
>to
>> > design eclean-kernel in a way making it possible to easily
>configure it
>> > to handle proper variant -- or even possibly make it capable of
>> > autoconfiguration.
>> >
>> > So if you have some time, please reply to this thread with
>> > a specific /boot layout that you think needs to be handled, with
>> > as much helpful information as possible -- including possible
>> > distinctive features and pitfalls.
>> >
>> > Thanks in advance.
>> >
>> I'm not sure if this is the info you're looking for, but I'll give it
>a
>> shot:
>>
>> I have grub-static installed to /boot/. I like to organize my kernels
>> with the filenames as linux-${version}-gentoo-${buildno}. So my first
>> build of 4.5.0, for example, would be 'linux-4.5.0-gentoo-1'. It has
>all
>> the info I need for reference should something go awry.
>>
>> I have three symlinks: current, last, backup
>>
>> I wrote scripts that will update those symlinks for me, which makes
>the
>> process of kernel management pretty painless. Now that I'm thinking
>> about it, it could be simple in my case to simply clean any kernel
>that
>> wasn't linked to.
>>
>> My /boot/:
>>
>> grub
>> lost+found
>> backup -> linux-4.4.1-gentoo-2
>> boot
>
>What's 'boot' here? Is that relevant?
>
>> current -> linux-4.4.6-gentoo-1
>> initrd
>
>Is that a single initrd for all kernels?
>
>> last -> linux-4.4.1-gentoo-3
>> linux-4.4.1-gentoo-2
>> linux-4.4.1-gentoo-3
>> linux-4.4.6-gentoo-1
>
>And most importantly, how are all those files referenced in grub? I
>suspect you are using the symlinks in grub.conf but want to confirm.

'boot' is a symlink to '.'. Not really sure why it's there but if I remove it, things break. Probably a minor misconfiguration.

Yes, the same initrd is used for all kernels. I use LUKS and LVM, so I need the initrd to boot. I produced the initrd using genkernel and it's worked ever since.

Yes, grub.conf indeed references the symlinks and never an explicit kernel path.

Sorry I wasn't clear in my first mail.
- --
Sent from my Android device with K-9 Mail. Please excuse my brevity.
-----BEGIN PGP SIGNATURE-----

iQJRBAEBCgA7NBxEYW5pZWwgQ2FtcGJlbGwgKEdlbnRvbyBEZXZlbG9wZXIpIDx6
bGdAZ2VudG9vLm9yZz4FAld1m50ACgkQASQOlFA54XCEOg//W1uQNSgnGxu7OAUe
13gLTgDN3+DMolhB/peyNRW3MxMyYaIr3PiG3DscLF538wevyx6ANp4eOHrEsANg
bFoA4BR1rPeB55A5zcQg4rnZnD23EHPkg56MDq6mtnib1ewK09sK6XbhrEMQ+eKr
LnAAUgwvkJab2Dd1q/thi3fGaIdJ8OgFAQLWnW4frqyIM7XgY+jLJCtSf7gaVKHx
7X/ZF9WyvZaGxQK64b6wuMQ4OdCGaQA6cCz4z3CYnFGh6bvAvKhQ+vaoTCJweCDS
ik0VuExsS9ILjMD8L5eQ2wYKmv2Ip/ua2fg+rV+8DzMzqQglYwkyrirjgVAXsnGb
/qBlJnuM7FCbTxjJ+eVjBAWvj8Iy8L4UPiFV62Qzvxr8jtPsjs6048x8tRLKaA8R
0XcH2zbujymz4Tbj+wwZtzNmdXLinOsFdU9O+0QEvQr7D4jaRNheNLtgExBVe8ho
kS3+DmgUL+GgKLUKiXTV6bRnrNHFNFZpQjuy6FMxqR4UZ+95r+YSORRDsJw5dg2M
1Esa4tyPjvezeGwdnZceFyok2G/qSQoYKYP766p9JmT2KegwCCiQNReRlVyEADz3
rpklcgBivkX7XkPlC94u9vEFbXpY3CG73eWczFFfrKPH9C9lwE3d1NQcNcEqRDiu
O1T2ae+TKH2d379E2Rn/sTQOVN4=
=4A8G
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Need design help/input for eclean-kernel

[Gordon Pettey]

[-- Attachment #1: Type: text/plain, Size: 583 bytes --]

On Thu, Jun 30, 2016 at 5:22 PM, Daniel Campbell (zlg) <zlg@gentoo.org>
wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA512
>
> 'boot' is a symlink to '.'. Not really sure why it's there but if I remove
> it, things break. Probably a minor misconfiguration.
>

/boot/boot -> /boot allows you to prefix every reference to a kernel image
with /boot, regardless of whether you are running something in userspace on
a fully mounted system or in GRUB or syslinux or the kernel where "/boot"
is actually /dev/sda1 or some such and effectively /. Likely irrelevant to
eclean tools.

[-- Attachment #2: Type: text/html, Size: 938 bytes --]

[gentoo-dev] Re: Need design help/input for eclean-kernel

[Jonathan Callen]

[-- Attachment #1.1: Type: text/plain, Size: 2041 bytes --]

On 06/30/2016 08:38 AM, Michał Górny wrote:
> Hello, everyone.
> 
> Back in 2011 I started a project called eclean-kernel. The idea was
> pretty simple -- to have a tool that would clean the old kernels for
> me since their install is not controlled by the package manager. This
> little project of mine seems to have gained a lot of popularity.
> 
> Sadly, over time a lot of people had trouble with it. Aside to minor
> Python problems, eclean-kernel proved too simple to handle multitude of
> user systems with varying /boot layouts. In fact, even I don't use it
> on all of my systems since it doesn't handle them properly.
> 
> After being buried in another set of bug reports, I'd like to
> officially ask Gentoo developers and users for help. I think it's
> impossible to solve most of the bugs reported so far in the current
> program design. Therefore, I'd like to rewrite it in a more flexible
> manner.
> 
> For this reason, I would like to ask you to provide me with
> different /boot layouts you may have, had or seen. Basically, the idea
> is to collect as many different layouts as necessary, and use that to
> design eclean-kernel in a way making it possible to easily configure it
> to handle proper variant -- or even possibly make it capable of
> autoconfiguration.
> 
> So if you have some time, please reply to this thread with
> a specific /boot layout that you think needs to be handled, with
> as much helpful information as possible -- including possible
> distinctive features and pitfalls.
> 
> Thanks in advance.
> 

https://wiki.freedesktop.org/www/Specifications/BootLoaderSpec/

${MACHINE_ID} is the contents of /etc/machine-id

Boot loader config:
    /boot/loader/entries/${MACHINE_ID}-${KV}.conf

Kernel, initramfs, etc.:
    /boot/${MACHINE_ID}/${KV}/linux
    /boot/${MACHINE_ID}/${KV}/initrd
    /boot/${MACHINE_ID}/${KV}/config
    /boot/${MACHINE_ID}/${KV}/System.map

Can be generated by kernel-install(8) (part of systemd).

-- 
Jonathan Callen


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 847 bytes --]

[gentoo-dev] Re: Need design help/input for eclean-kernel

[Duncan]

Michał Górny posted on Thu, 30 Jun 2016 14:38:26 +0200 as excerpted:

> [P]lease reply to this thread with
> a specific /boot layout that you think needs to be handled, with as much
> helpful information as possible -- including possible distinctive
> features and pitfalls.

This is surely more info than you need, but I imagine it's one of the 
more unique and complex layouts you'll see and thus might need additional 
explanation for some bits.  Trouble is I don't know which bits, so...

I have never used eclean-kernel and probably won't as I have my own 
scripts to handle things (as I was doing even back over a decade ago on 
mandrake, before I switched to gentoo), but here's the layout, in case 
you find it useful (or a fun challenge? =:^) to support.

So what I have for boot:

/boot itself is a symlink, normally pointing at /bt, the mountpoint for 
my working boot partition, but setup with /boot as a symlink so I can 
point it at /bk/bt, where I can mount my backup boots on other devices, 
when I want to install grub to them.

/bt, LABEL=bt0238gcn1+35l0 : working boot mountpoint and filesystem when 
mounted, where the /boot symlink normally points.  Filesystem is btrfs 
mixed-blockgroup dup mode.  This is a partition on one of a pair of 238 
GiB (256 GB) Corsair Neutron SSDs.


/bk/bt: mountpoint for my backup boots.  The /boot symlink can be 
adjusted to point here when I want to install grub to one of the backup 
boots.

LABEL=bt0238gcn0+35l1 : primary backup boot filesystem, also btrfs mixed-
blockgroup dup, a partition on the other one of the pair of 238 GiB SSDs.

LABEL=bt0465gsg0+47f0 : secondary backup boot filesystem, reiserfs on a 
partition on a 465 GiB spinning rust seagate.

Various other removable drives and their labels for further backups...

All bootable storage (including removable) is GPT partitioned with both a 
legacy-BIOS partition to which grub2 is installed and an EFI partition 
reserved for future use.  Each installed grub points at the /bt boot 
partition on that device, so I can at least get both a grub prompt and a 
bootable kernel on any of them, even with all other storage 
disconnected.  From there it's up to where I point the kernel using root= 
on the kernel commandline, but FWIW, each grub is configured with a menu 
from which I can choose any of my working or primary or secondary backup 
root.

On each of these boot partitions the filesystem layout is:

amd64/
	64-bit kernels, configs, system-maps, symlinks...

boot -> .

grub/

[x86/]
	[when I had the 32-bit netbook, this was its kernels, etc.]

[A single zero-length file named working, backup, backup2, etc, so I can 
easily confirm which one I actually booted to when testing an updated 
grub install or the like.]


The kernel dirs are laid out as...

$$ ls -1 /bt/amd64

config@
config-4.5.0-dirty
config-4.6.0-dirty
config-4.6.0-rc7-00096-g685764b10-dirty
config.old@
config.stable@
System.map@
System.map-4.5.0-dirty
System.map-4.6.0-dirty
System.map-4.6.0-rc7-00096-g685764b10-dirty
System.map.old@
System.map.stable@
vmlinuz@
vmlinuz-4.5.0-dirty
vmlinuz-4.6.0-dirty
vmlinuz-4.6.0-rc7-00096-g685764b10-dirty
vmlinuz.old@
vmlinuz.stable@


That's three symlinks for each of the map, config and kernel.  They point 
to current, old (previous) and stable.  Since I often test/run and 
occasionally bisect git kernels, current and old often point to git and 
perhaps bisect-bad kernels, and during a bisect I may have nearly a dozen 
kernels and associated files, tho my scripts only maintain the current/
old symlinks.  I update the stable symlink manually, when I consider a 
released version tested well enough locally to be confident doing so.

Only the working boot gets the git kernels.  I update the primary backup 
with the new release-stables each kernel cycle, and only update the 
secondary backup every few kernel cycles.

The git kernel testing and bisects are why I prefer to do my own kernel 
cleanups.  That way I can keep the first git kernel I saw the problem in 
around until I get around to doing a bisect, if the problem hasn't been 
caught and fixed upstream by then, making the bisect easier than it would 
be if I kept updating to see if the problem was fixed, losing track of 
the first bad kernel I saw in the process and thus perhaps forcing 
another round or two of bisection to track down the problem.

And for sure I don't want anything touching the stable symlinks but me, 
manually, when I am sufficiently confident I can do so and won't be left 
in a hole without an easy way to dig myself out as a result.

I use a dracut-built initr*, compressed and appended as an initramfs to 
each kernel built and tested, so once I know the kernel/initramfs 
combination work, I don't have to worry about a buggy initr* update 
breaking older kernels as they have their initramfs builtin.  The dracut-
built initr* is kept on a different filesystem in ordered to leave more 
room in the boot and backups for kernels.


More detailed explanation of likely unique characteristics, and why...:

Most of my primary filesystems are btrfs raid1 (and were on mdraid1 
before that), with a working copy and a primary and secondary backup, all 
on similarly sized partitions, working and primary backup on a pair of 
SSDs, secondary backup on spinning rust.

That doesn't work for /boot, since grub1 (where I originally setup the 
scheme) can only point at one with no way to point at the backup instead, 
and while grub2 can manually load a different /boot, it's extra trouble, 
and I already had the scheme setup and working by then, so I kept what 
worked.

In ordered to facilitate all this, /boot itself is a symlink, that points 
to /bt normally, or to /bk/bt, where I mount the backups if I want to 
install grub to them.  (I simply copy-backup /boot to the backups as 
appropriate, but grub has to be installed to the legacy bios partition on 
each device separately, so I need to switch the /boot symlink temporarily 
when I'm installing grub to the backup devices.)

And I keep track of all working and backups of the various filesystems 
via standardized 15-character LABEL (which I list in fstab to mount by) 
that looks something like this:

bt0238gcn1+35l0

bt=boot (rt=root, hm=home, etc)

0238g=0238 gig

cn=corsair neutron (brand)

1=second 238-gig corsair neutron (0-base)

+=workstation permanent installed (it's also a visual separator)

35l=2013 (last repartitioning), May (5th month), l=day-of-month

0=working copy (on that device at least).

FWIW, filesystem and gpt partition labels are kept in sync (but for the 
device number on raid, of course) so I can uniquely ID within my own 
storage inventory on sight by either one.

bt0238gcn0+35l1 is the backup boot (the tailing 1) on the other 238-gig 
corsair neutron ssd. (Yes, I did put the working boot on device 1, the 
first backup on device 0, oh, well...)  bt0465gsg0+47f0 is the secondary 
backup boot, on the first spinning rust 465-gig seagate, last 
repartitioned in July of 2014.  FWIW, the three roots are rt0238gcnx+35l0 
(working, x in the device number slot as it's a pair-device btrfs raid1), 
rt0238gcnx+35l1 (primary backup), and rt0465gsg0+47f0 (secondary backup, 
but the first rt on the first 465-gig seagate, thus the trailing 0).

Additionally, given the 32-bit netbook I had at one point, and the fact 
that I wanted a standardized /boot layout that would work with both 32-
bit and 64-bit kernels, on both permanently installed drives and USB 
devices that might be used to boot either one, my boot layouts have the 
kernels in dedicated subdirs, in my case, /boot/amd64, and when I had the 
netbook, /boot/x86.  (I used the 64-bit machine with a 32-bit chroot as 
the 32-bit build, so my kernel build scripts had logic/config that looked 
at $HOSTTYPE, IIRC, to decide which kernel to build, which working output 
dir to use, and where to install the kernel.)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Need design help/input for eclean-kernel

[Patrick McLean]

On Thu, 30 Jun 2016 14:38:26 +0200
Michał Górny <mgorny@gentoo.org> wrote:
> So if you have some time, please reply to this thread with
> a specific /boot layout that you think needs to be handled, with
> as much helpful information as possible -- including possible
> distinctive features and pitfalls.
> 

All of our systems have this:
/boot/${KV}/{vmlinuz,initramfs,System.map,perf,config}-${KV}

With these symlinks
/boot/vmlinuz -> ${KV}/vmlinuz-${KV}
/boot/initramfs -> ${KV}/initramfs-${KV}
/boot/config -> ${KV}/config-${KV}
/boot/System.map -> ${KV}/System.map-${KV}

Some systems also have:
/boot/${KV}/vmlinux-${KV}

/boot/vmlinux -> ${KV}/vmlinux-${KV}

When updating to a new kernel, we generally unpack a tarball
containing the new kernel to /boot and update the symlinks to point to
the new versions. All files related to a kernel are in that kernel's
directory, which makes cleanup somewhat easier.

The values of KV look like one of these:
4.4.14-vanilla-base-1
4.4.14-gentoo-r1-base-1
4.7.0-rc5-vanilla-base-1
4.7.0-rc5-vanilla-base-1+
4.7.0-rc5-vanilla-base-1-00254-g1a0a02d

Mostly, it's a version, sources version, configuration type and
version. These are generated via setting CONFIG_LOCALVERSION, and
whatever else gets spit out by the build system.

Re: [gentoo-dev] Need design help/input for eclean-kernel

[Guilherme Amadio]

On Thu, Jun 30, 2016 at 02:38:26PM +0200, Michał Górny wrote:
> So if you have some time, please reply to this thread with
> a specific /boot layout that you think needs to be handled, with
> as much helpful information as possible -- including possible
> distinctive features and pitfalls.

Looks like not too many people are booting kernels like me, so
I'm going to add my crude setup to the pile :)

I compile most stuff into the kernel (at least enough so I don't
need to use an initrd/initramfs to mount / or /usr). I do not use a boot
loader either, I use CONFIG_EFI_STUB=y and compile in the command
line arguments to avoid trouble. Then, I use efibootmgr to boot the
kernel at /boot/EFI/Gentoo/bootx64.efi

I usually keep several kernels in the same directory with a version
suffix (e.g. bootx64-4.6.2.efi, etc) and overwrite the bootx64.efi
file with whatever kernel I want to boot with cp bootx64{-4.x.x,}.efi

I keep a few older kernels around in case I screw up the configuration
of a new one (usually video drivers), but for the most part, that's it.

My compiled kernel command line is also pretty simple:

quiet console=tty1 root=/dev/sda4 init=/usr/lib/systemd/systemd

I never really though about writing scripts to manage this, since it's
quite a simple setup, but having tools to manage it would be nice.

Just think about supporting a simple EFI/UEFI setup, I would say, in
addition to the more common setups using a boot loader.

Cheers,
—Guilherme

Re: [gentoo-dev] Need design help/input for eclean-kernel

[Raymond Jennings]

[-- Attachment #1: Type: text/plain, Size: 1796 bytes --]

Just to give kudos, I would not be able to keep my system tidy without
eclean-kernel.  It takes care of lots of stuff portage does not.

On Fri, Jul 1, 2016 at 10:58 AM, Guilherme Amadio <amadio@gentoo.org> wrote:

> On Thu, Jun 30, 2016 at 02:38:26PM +0200, Michał Górny wrote:
> > So if you have some time, please reply to this thread with
> > a specific /boot layout that you think needs to be handled, with
> > as much helpful information as possible -- including possible
> > distinctive features and pitfalls.
>
> Looks like not too many people are booting kernels like me, so
> I'm going to add my crude setup to the pile :)
>
> I compile most stuff into the kernel (at least enough so I don't
> need to use an initrd/initramfs to mount / or /usr). I do not use a boot
> loader either, I use CONFIG_EFI_STUB=y and compile in the command
> line arguments to avoid trouble. Then, I use efibootmgr to boot the
> kernel at /boot/EFI/Gentoo/bootx64.efi
>
> I usually keep several kernels in the same directory with a version
> suffix (e.g. bootx64-4.6.2.efi, etc) and overwrite the bootx64.efi
> file with whatever kernel I want to boot with cp bootx64{-4.x.x,}.efi
>
> I keep a few older kernels around in case I screw up the configuration
> of a new one (usually video drivers), but for the most part, that's it.
>
> My compiled kernel command line is also pretty simple:
>
> quiet console=tty1 root=/dev/sda4 init=/usr/lib/systemd/systemd
>
> I never really though about writing scripts to manage this, since it's
> quite a simple setup, but having tools to manage it would be nice.
>
> Just think about supporting a simple EFI/UEFI setup, I would say, in
> addition to the more common setups using a boot loader.
>
> Cheers,
> —Guilherme
>
>
>

[-- Attachment #2: Type: text/html, Size: 2214 bytes --]

Re: [gentoo-dev] Need design help/input for eclean-kernel

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 833 bytes --]

On Thu, Jun 30, 2016 at 06:18:11PM -0500, Gordon Pettey wrote:
> On Thu, Jun 30, 2016 at 5:22 PM, Daniel Campbell (zlg) <zlg@gentoo.org>
> wrote:
> 
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA512
> >
> > 'boot' is a symlink to '.'. Not really sure why it's there but if I remove
> > it, things break. Probably a minor misconfiguration.
> >
> 
> /boot/boot -> /boot allows you to prefix every reference to a kernel image
> with /boot, regardless of whether you are running something in userspace on
> a fully mounted system or in GRUB or syslinux or the kernel where "/boot"
> is actually /dev/sda1 or some such and effectively /. Likely irrelevant to
> eclean tools.

I have to bring up a question for clarification. When we are talking
about grub, are we talking about grub legasy or grub2?

William


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] Need design help/input for eclean-kernel

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 733 bytes --]

On Thu, Jun 30, 2016 at 01:46:27PM -0400, Mike Gilbert wrote:
> On Thu, Jun 30, 2016 at 8:38 AM, Michał Górny <mgorny@gentoo.org> wrote:
> > So if you have some time, please reply to this thread with
> > a specific /boot layout that you think needs to be handled, with
> > as much helpful information as possible -- including possible
> > distinctive features and pitfalls.
> 
> Standard "make install" layout with dracut and grub2:
> 
> /boot/vmlinuz-$KV{,.old}
> /boot/config-$KV
> /boot/System.map-${KV}
> /boot/initramfs-$KV.img
> /boot/grub/grub.cfg generated by grub2-mkconfig contains menu entries
> for each pairing.

This is what I do on this machine, with no initramfs. It just works
(tm).

William


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] Need design help/input for eclean-kernel

[Gordon Pettey]

[-- Attachment #1: Type: text/plain, Size: 1138 bytes --]

On Fri, Jul 1, 2016 at 2:52 PM, William Hubbs <williamh@gentoo.org> wrote:

> On Thu, Jun 30, 2016 at 06:18:11PM -0500, Gordon Pettey wrote:
> > On Thu, Jun 30, 2016 at 5:22 PM, Daniel Campbell (zlg) <zlg@gentoo.org>
> > wrote:
> >
> > > -----BEGIN PGP SIGNED MESSAGE-----
> > > Hash: SHA512
> > >
> > > 'boot' is a symlink to '.'. Not really sure why it's there but if I
> remove
> > > it, things break. Probably a minor misconfiguration.
> > >
> >
> > /boot/boot -> /boot allows you to prefix every reference to a kernel
> image
> > with /boot, regardless of whether you are running something in userspace
> on
> > a fully mounted system or in GRUB or syslinux or the kernel where "/boot"
> > is actually /dev/sda1 or some such and effectively /. Likely irrelevant
> to
> > eclean tools.
>
> I have to bring up a question for clarification. When we are talking
> about grub, are we talking about grub legasy or grub2?

Which bootloader doesn't matter. Instead of having to remember to delete
/boot from the beginning of the path, it just lets you reference everything
the same way you would from a shell on your running mounted system.

[-- Attachment #2: Type: text/html, Size: 1625 bytes --]

Re: [gentoo-dev] Need design help/input for eclean-kernel

[Daniel Campbell]

[-- Attachment #1.1: Type: text/plain, Size: 1203 bytes --]

On 07/01/2016 12:52 PM, William Hubbs wrote:
> On Thu, Jun 30, 2016 at 06:18:11PM -0500, Gordon Pettey wrote:
>> On Thu, Jun 30, 2016 at 5:22 PM, Daniel Campbell (zlg) <zlg@gentoo.org>
>> wrote:
>>
>>> -----BEGIN PGP SIGNED MESSAGE-----
>>> Hash: SHA512
>>>
>>> 'boot' is a symlink to '.'. Not really sure why it's there but if I remove
>>> it, things break. Probably a minor misconfiguration.
>>>
>>
>> /boot/boot -> /boot allows you to prefix every reference to a kernel image
>> with /boot, regardless of whether you are running something in userspace on
>> a fully mounted system or in GRUB or syslinux or the kernel where "/boot"
>> is actually /dev/sda1 or some such and effectively /. Likely irrelevant to
>> eclean tools.
> 
> I have to bring up a question for clarification. When we are talking
> about grub, are we talking about grub legasy or grub2?
> 
> William
> 
I can't speak for the others but I'm using
sys-boot/grub-static-0.97-r12. I doubt I have a specific need for it to
be statically-built, but it works!

-- 
Daniel Campbell - Gentoo Developer
OpenPGP Key: 0x1EA055D6 @ hkp://keys.gnupg.net
fpr: AE03 9064 AE00 053C 270C  1DE4 6F7A 9091 1EA0 55D6


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-dev] Need design help/input for eclean-kernel

[Christopher Head]

[-- Attachment #1: Type: text/plain, Size: 1204 bytes --]

On Thu, 30 Jun 2016 14:38:26 +0200
Michał Górny <mgorny@gentoo.org> wrote:

> So if you have some time, please reply to this thread with
> a specific /boot layout that you think needs to be handled, with
> as much helpful information as possible -- including possible
> distinctive features and pitfalls.

I use genkernel to build my kernels and initramfsen. Most of my
machines don’t support EFI, so my /boot contains a bunch of
{initramfs,kernel,System.map}-$KV files plus the
{initramfs,kernel,System.map}.old symlinks. My GRUB2 grub.cfg points at
the symlinks. Once in a while (usually when I get disk full errors on
the dedicated partition) I delete the old files by hand.

A couple of my machines do support EFI. On those, I set INSTALL=no in
genkernel.conf. I then run a custom script afterwards which maintains
directories /boot/EFI/gentoo and /boot/EFI/gentoo-old. Each of these
contains a file kernel.efi and another file initramfs. This is FAT, so
no symlinks. The script deletes gentoo-old, copies gentoo to
gentoo-old, and finally puts the new files in gentoo. I have EFI boot
records pointing to both, using initrd=\gentoo\initramfs syntax.
-- 
Christopher Head

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 648 bytes --]