[gentoo-desktop] System problems

[gentoo-desktop] System problems

[Lindsay Haisley]

I'm caught between a rock and a hard place.

I've been running this desktop box using kernel 2.6.23-gentoo-r3 and
have come to the point at which there are too many dependencies and
reverse dependencies, so I _have_ to upgrade to kernel 2.6.29-gentoo-r5
and have been unable to bring the system up in the new kernel.  Here's
what's happening.

The newer kernel requires a newer version of udev, which I emerged.  The
system came up with a root device of some sort mounted, I think in
single-user mode, but couldn't mount other devices.

So I changed the main drive designations to UUID's in /etc/fstab,
re-emerged the newer udev, and tried again.  This time I got a message
that the kernel needed a root parameter at boot time.  It seems that all
my /dev/hda? drives have been renamed /dev/sda? so I set gave
"root=/dev/sda4" as a kernel parameter and got a little further.  After
"Checking root filesystem" in the boot sequence, I got a message that
the UUID for the root filesystem wasn't understood in /etc/fstab.

So I set the root filesystem in /etc/fstab to /dev/sda4, and got the
same error - that "/dev/sda4" was not understood either, although the
kernel seemed to understand this just fine as a boot parameter, and once
again, I'm dumped into a very limited single-user mode.

So I'm stuck!  I had to boot from a rescue disk, back-version to
udev-141 and revert to kernel 2.6.23-gentoo-r3 to get my desktop back.

What do I need to put into /etc/fstab to satisfy the kernel?  I need to
move forward with this, but I need my desktop system to run my business.
Any _real_ suggestions will be welcome.  Please be aware that I'm no
Linux novice, so don't give me novice advice.  I've been building,
running, and getting paid to admin Linux systems since 1995.

-- 
Lindsay Haisley       | "Everything works if you let it"
FMP Computer Services |   (The Roadie)
512-259-1190          |
http://www.fmp.com    |

Re: [gentoo-desktop] System problems

[Jean-Marc Beaune]

[-- Attachment #1: Type: text/plain, Size: 2475 bytes --]

Hi,

This is due to ATA/ATAPI (DEPRECATED) being disabled in newer kernels,
replaced by Serial ATA and Parallel ATA drivers.

Make sure you enabled this support properly.

In my case that happened to me as well, on a remote computer, which was my
mother's box...

Anyway, in fstab /dev/sdXX shoud work, at least I made this change on a
couple of machines and that went fine.

Cheers,
JM

On Sun, Mar 20, 2011 at 4:46 AM, Lindsay Haisley <fmouse-gentoo@fmp.com>wrote:

> I'm caught between a rock and a hard place.
>
> I've been running this desktop box using kernel 2.6.23-gentoo-r3 and
> have come to the point at which there are too many dependencies and
> reverse dependencies, so I _have_ to upgrade to kernel 2.6.29-gentoo-r5
> and have been unable to bring the system up in the new kernel.  Here's
> what's happening.
>
> The newer kernel requires a newer version of udev, which I emerged.  The
> system came up with a root device of some sort mounted, I think in
> single-user mode, but couldn't mount other devices.
>
> So I changed the main drive designations to UUID's in /etc/fstab,
> re-emerged the newer udev, and tried again.  This time I got a message
> that the kernel needed a root parameter at boot time.  It seems that all
> my /dev/hda? drives have been renamed /dev/sda? so I set gave
> "root=/dev/sda4" as a kernel parameter and got a little further.  After
> "Checking root filesystem" in the boot sequence, I got a message that
> the UUID for the root filesystem wasn't understood in /etc/fstab.
>
> So I set the root filesystem in /etc/fstab to /dev/sda4, and got the
> same error - that "/dev/sda4" was not understood either, although the
> kernel seemed to understand this just fine as a boot parameter, and once
> again, I'm dumped into a very limited single-user mode.
>
> So I'm stuck!  I had to boot from a rescue disk, back-version to
> udev-141 and revert to kernel 2.6.23-gentoo-r3 to get my desktop back.
>
> What do I need to put into /etc/fstab to satisfy the kernel?  I need to
> move forward with this, but I need my desktop system to run my business.
> Any _real_ suggestions will be welcome.  Please be aware that I'm no
> Linux novice, so don't give me novice advice.  I've been building,
> running, and getting paid to admin Linux systems since 1995.
>
> --
> Lindsay Haisley       | "Everything works if you let it"
> FMP Computer Services |   (The Roadie)
> 512-259-1190          |
> http://www.fmp.com    |
>
>
>


-- 
Jean-Marc

[-- Attachment #2: Type: text/html, Size: 3202 bytes --]

Re: [gentoo-desktop] System problems

[Brent Busby]

On Sun, 20 Mar 2011, Jean-Marc Beaune wrote:

> This is due to ATA/ATAPI (DEPRECATED) being disabled in newer kernels,
> replaced by Serial ATA and Parallel ATA drivers.
>
> Make sure you enabled this support properly.
>
> In my case that happened to me as well, on a remote computer, which was my
> mother's box...
>
> Anyway, in fstab /dev/sdXX shoud work, at least I made this change on a
> couple of machines and that went fine.

Yes, I went through this also awhile back.  All your drivers should now 
come from the Sata section, both the Sata drivers for your hard drives, 
plus the Pata drivers for any Pata DVD drives you may have.  There are 
now Pata support drivers living in the Sata section.  You should turn 
off the toggle for the entire old Pata driver section in menuconfig. 
If you have only Pata drives, you should just use the Pata drivers from 
the Sata section by themselves.  No need for anything in the old Pata 
section in any case anymore.

The newer Sata subsystem Pata drivers had been included before in the 
kernel for a long time as deprecated drivers for people to beta test, 
and the regular Pata drivers were recommended for use.  Now it's 
reversed -- the Pata drivers in the Sata section are recommended, and 
the old Pata drivers from the Pata section are deprecated...expecially 
because they don't work anymore with current versions of udev!

-- 
+ Brent A. Busby	 + "We've all heard that a million monkeys
+ UNIX Systems Admin	 +  banging on a million typewriters will
+ University of Chicago	 +  eventually reproduce the entire works of
+ Physical Sciences Div. +  Shakespeare.  Now, thanks to the Internet,
+ James Franck Institute +  we know this is not true." -Robert Wilensky

Re: [gentoo-desktop] System problems

[Lindsay Haisley]

On Sun, 2011-03-20 at 08:24 +0000, Jean-Marc Beaune wrote:
> This is due to ATA/ATAPI (DEPRECATED) being disabled in newer kernels,
> replaced by Serial ATA and Parallel ATA drivers. 
> 
> Make sure you enabled this support properly. 
> 
> In my case that happened to me as well, on a remote computer, which
> was my mother's box... 
> 
> Anyway, in fstab /dev/sdXX shoud work, at least I made this change on
> a couple of machines and that went fine. 

Jean-Marc, thanks.  This gives me a bit of insight.

I've been compiling kernels for this box for some time, carrying over
the .config file between kernel versions and updating them.  Last night
I built IDE functionality as a module (it was previously compiled into
the kernel, as per my carried-over .config files) thinking that I would
simply not load it at run-time.  As I noted in my post, while the the
kernel recognized "root=/dev/sda4" as a kernel param, the boot process
crashed with a note that "/dev/sda4" wasn't recognized.  It's entirely
possible that this module got auto-loaded, or that there's some other
anomaly in my kernel config that's throwing things off.

My next step is to completely rebuild the kernel, using the config built
into the distributed kernel source, making only the necessary mods for
the box's hardware needs.  I'll find a time window during the next few
days to work on this.

-- 
Lindsay Haisley       |"Windows .....
FMP Computer Services |    life's too short!"
512-259-1190          |
http://www.fmp.com    |              - Brad Johnston

Re: [gentoo-desktop] System problems

[Donnie Berkholz]

[-- Attachment #1: Type: text/plain, Size: 2243 bytes --]

On 11:36 Sun 20 Mar     , Lindsay Haisley wrote:
> On Sun, 2011-03-20 at 08:24 +0000, Jean-Marc Beaune wrote:
> > This is due to ATA/ATAPI (DEPRECATED) being disabled in newer kernels,
> > replaced by Serial ATA and Parallel ATA drivers. 
> > 
> > Make sure you enabled this support properly. 
> > 
> > In my case that happened to me as well, on a remote computer, which
> > was my mother's box... 
> > 
> > Anyway, in fstab /dev/sdXX shoud work, at least I made this change on
> > a couple of machines and that went fine. 
> 
> Jean-Marc, thanks.  This gives me a bit of insight.
> 
> I've been compiling kernels for this box for some time, carrying over
> the .config file between kernel versions and updating them.  Last night
> I built IDE functionality as a module (it was previously compiled into
> the kernel, as per my carried-over .config files) thinking that I would
> simply not load it at run-time.  As I noted in my post, while the the
> kernel recognized "root=/dev/sda4" as a kernel param, the boot process
> crashed with a note that "/dev/sda4" wasn't recognized.  It's entirely
> possible that this module got auto-loaded, or that there's some other
> anomaly in my kernel config that's throwing things off.

I also suspect What Jean-Marc said is the problem. I'd recommend 
completely disabling everything in the old ATA section to ensure it 
doesn't attempt to control any devices, while building the PATA driver 
into the kernel and using root=/dev/sdXN in the grub parameters.

The module approach should also work, but I always get a little 
suspicious that it might build something else into the kernel 
unnecessarily that causes problems.

> My next step is to completely rebuild the kernel, using the config built
> into the distributed kernel source, making only the necessary mods for
> the box's hardware needs.  I'll find a time window during the next few
> days to work on this.

It might be a worthwhile step to boot from a LiveCD and run `lspci -k` 
to identify the kernel modules. If the pciutils version on the CD is too 
old, `pcimodules` might work instead.

-- 
Thanks,
Donnie

Donnie Berkholz
Desktop project lead
Gentoo Linux
Blog: http://dberkholz.com

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-desktop] System problems

[Lindsay Haisley]

On Sun, 2011-03-20 at 21:13 -0500, Donnie Berkholz wrote:
> I also suspect What Jean-Marc said is the problem. I'd recommend 
> completely disabling everything in the old ATA section to ensure it 
> doesn't attempt to control any devices, while building the PATA driver 
> into the kernel and using root=/dev/sdXN in the grub parameters.

If I use the stock configuration from the 2.6.36-gentoo-r5 kernel, won't
this have the correct basic kernel facilities built in, at least as far
as the deprecated IDE capabilities are concerned and the libata
replacement?  I assume the Gentoo devs modify kernels so that the
default config settings are more appropriate than those which come with
the vanilla kernel from the kernel devs, yes?

Putting "root=/dev/sda4" on the kernel cmd line in grub actually worked,
and got me a bit further in the boot process.  The kernel obviously
understood it.  However later in the boot process, I got "Checking the
root filesystem", following by an error message that the root filesystem
spec of /dev/sda4 wasn't understood.  This is a complaint about the root
fs spec is in /etc/fstab, since I had been using a UUID spec there, and
got an error at the same point in the boot-up about the UUID instead.

> The module approach should also work, but I always get a little 
> suspicious that it might build something else into the kernel 
> unnecessarily that causes problems.

I don't know.  According to the the ebuild notes with udev, the IDE
facility in the kernel is completely depricated and needs to be turned
off entirely to prevent unexpected results.  If I build IDE as a module,
my concern is that the module will get auto-loaded and I'll be back in
the same boat.  So I'll use a more recent kernel and turn IDE off
altogether and we'll see what happens.

> 
> > My next step is to completely rebuild the kernel, using the config built
> > into the distributed kernel source, making only the necessary mods for
> > the box's hardware needs.  I'll find a time window during the next few
> > days to work on this.
> 
> It might be a worthwhile step to boot from a LiveCD and run `lspci -k` 
> to identify the kernel modules.

lsmod will probably give the same useful information.

>  If the pciutils version on the CD is too 
> old, `pcimodules` might work instead.

Nope.  The my rescue CD is up-to-date.

Thanks!

-- 
Lindsay Haisley       | "Everything works if you let it"
FMP Computer Services |   (The Roadie)
512-259-1190          |
http://www.fmp.com    |

Re: [gentoo-desktop] System problems

[Brent Busby]

On Mon, 21 Mar 2011, Lindsay Haisley wrote:

> I don't know.  According to the the ebuild notes with udev, the IDE 
> facility in the kernel is completely depricated and needs to be turned 
> off entirely to prevent unexpected results.  If I build IDE as a 
> module, my concern is that the module will get auto-loaded and I'll be 
> back in the same boat.  So I'll use a more recent kernel and turn IDE 
> off altogether and we'll see what happens.

Yes, turn the whole Pata section off.  After that, everything should be 
recognizable with "sd" rather than "hd" device names, in both Grub and 
/etc/fstab.  Also, to save yourself trouble, build the drivers for the 
Sata subsystem and your particular controller chipset into the kernel, 
and forget about initrd's.  The only thing they're good for is distros 
that have to run on whatever they're booting up on.  The minute you have 
to compile your own kernel to suit yourself, you might as well get rid 
of the hassle of initrd's while you're at it.

-- 
+ Brent A. Busby	 + "We've all heard that a million monkeys
+ UNIX Systems Admin	 +  banging on a million typewriters will
+ University of Chicago	 +  eventually reproduce the entire works of
+ Physical Sciences Div. +  Shakespeare.  Now, thanks to the Internet,
+ James Franck Institute +  we know this is not true." -Robert Wilensky

Re: [gentoo-desktop] System problems

[Donnie Berkholz]

[-- Attachment #1: Type: text/plain, Size: 2226 bytes --]

On 02:38 Mon 21 Mar     , Lindsay Haisley wrote:
> On Sun, 2011-03-20 at 21:13 -0500, Donnie Berkholz wrote:
> > I also suspect What Jean-Marc said is the problem. I'd recommend 
> > completely disabling everything in the old ATA section to ensure it 
> > doesn't attempt to control any devices, while building the PATA 
> > driver into the kernel and using root=/dev/sdXN in the grub 
> > parameters.
> 
> If I use the stock configuration from the 2.6.36-gentoo-r5 kernel, 
> won't this have the correct basic kernel facilities built in, at least 
> as far as the deprecated IDE capabilities are concerned and the libata 
> replacement?  I assume the Gentoo devs modify kernels so that the 
> default config settings are more appropriate than those which come 
> with the vanilla kernel from the kernel devs, yes?

Things are pretty vanilla, as per Gentoo philosophy, unless you run 
genkernel to build your kernel. I wouldn't rely on anything being built 
in for a manually configured kernel.

Take a look at the kernel config guide if you want some pointers:

http://www.gentoo.org/doc/en/handbook/handbook-amd64.xml?part=1&chap=7

> Putting "root=/dev/sda4" on the kernel cmd line in grub actually 
> worked, and got me a bit further in the boot process.  The kernel 
> obviously understood it.  However later in the boot process, I got 
> "Checking the root filesystem", following by an error message that the 
> root filesystem spec of /dev/sda4 wasn't understood.  This is a 
> complaint about the root fs spec is in /etc/fstab, since I had been 
> using a UUID spec there, and got an error at the same point in the 
> boot-up about the UUID instead.

That is symptomatic of a missing driver for an ATA controller or root 
filesystem.

> > It might be a worthwhile step to boot from a LiveCD and run `lspci 
> > -k` to identify the kernel modules.
> 
> lsmod will probably give the same useful information.

The useful thing about `lspci -k` is it only shows modules actually used 
by hardware on your system, rather than the huge superset of modules 
that are loaded.

-- 
Thanks,
Donnie

Donnie Berkholz
Desktop project lead
Gentoo Linux
Blog: http://dberkholz.com

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-desktop] System problems

[Roman Zilka]

Please send us your `emerge --info`, /etc/fstab, grub/lilo/smthng
config and kernel config for the 2.6.29.

By the way, why not gentoo-sources-2.6.36-r5? I hope you have a good
reason to run a system as ancient as that. Your system is swarming with
widely known security holes. I suppose it's theoretically possible to
have a desktop protected by some special means from all the real threats
an actively used desktop is facing, but I wonder if you use any such
means. Also, by upgrading to a little less ancient versions than 2.6.29
you won't have the same situation like now boomerang back at you in the
near future.

-rz


Lindsay Haisley (Sat, 19 Mar 2011 23:46:06 -0500):
> I'm caught between a rock and a hard place.
> 
> I've been running this desktop box using kernel 2.6.23-gentoo-r3 and
> have come to the point at which there are too many dependencies and
> reverse dependencies, so I _have_ to upgrade to kernel 2.6.29-gentoo-r5
> and have been unable to bring the system up in the new kernel.  Here's
> what's happening.
> 
> The newer kernel requires a newer version of udev, which I emerged.  The
> system came up with a root device of some sort mounted, I think in
> single-user mode, but couldn't mount other devices.
> 
> So I changed the main drive designations to UUID's in /etc/fstab,
> re-emerged the newer udev, and tried again.  This time I got a message
> that the kernel needed a root parameter at boot time.  It seems that all
> my /dev/hda? drives have been renamed /dev/sda? so I set gave
> "root=/dev/sda4" as a kernel parameter and got a little further.  After
> "Checking root filesystem" in the boot sequence, I got a message that
> the UUID for the root filesystem wasn't understood in /etc/fstab.
> 
> So I set the root filesystem in /etc/fstab to /dev/sda4, and got the
> same error - that "/dev/sda4" was not understood either, although the
> kernel seemed to understand this just fine as a boot parameter, and once
> again, I'm dumped into a very limited single-user mode.
> 
> So I'm stuck!  I had to boot from a rescue disk, back-version to
> udev-141 and revert to kernel 2.6.23-gentoo-r3 to get my desktop back.
> 
> What do I need to put into /etc/fstab to satisfy the kernel?  I need to
> move forward with this, but I need my desktop system to run my business.
> Any _real_ suggestions will be welcome.  Please be aware that I'm no
> Linux novice, so don't give me novice advice.  I've been building,
> running, and getting paid to admin Linux systems since 1995.
> 

Re: [gentoo-desktop] System problems

[Lindsay Haisley]

On Sun, 2011-03-20 at 09:50 +0100, Roman Zilka wrote:
> By the way, why not gentoo-sources-2.6.36-r5? I hope you have a good
> reason to run a system as ancient as that. Your system is swarming with
> widely known security holes. 

Yes, I have a good reason for this, and I'll be responsible for security
on the box, which I'm well able to do.  Security, and the relative
antiquity of the portage tree are not issues for which I'm seeking
advice, so your observations are totally beside the point, and not
particularly welcome.  I'm well aware of both issues.

The challenge here is to get the system to boot with the newer kernel
and version of udev which I cited in my previous post.

/etc/fstab has been edited several times, as I noted in my post.  The
kernel, udev and /etc/fstab have been now been reverted, as I also noted, so
I could get the desktop working.  Considering that, posting any of the
information you've asked for would probably be useless.

Roman, if you don't have any useful insights based on the information I
already posted, then please don't post on thread and leave it to others
who may.

Having said this, I'll ask you one question:

> Also, by upgrading to a little less ancient versions than 2.6.29
> you won't have the same situation like now boomerang back at you in the
> near future.

Can you cite a source or sources for this assertion?  Is there a known
problem with kernel 2.6.29, or the portage tree which spec'd that
kernel?  Is there any discussion, bug report, or anything that you can
cite noting that this was a known problem at one point and addressed at
a later date?

In almost every case, I've found that people who lecture me online about
my system admin practices don't really have a handle on the issue about
which I'm writing.  Please prove me wrong :-)

-- 
Lindsay Haisley       |"What if the Hokey Pokey really IS all it  
FMP Computer Services |    really is about?"
512-259-1190          |
http://www.fmp.com    |          -- Jimmy Buffett


-- 
Lindsay Haisley       |"Windows .....
FMP Computer Services |    life's too short!"
512-259-1190          |
http://www.fmp.com    |              - Brad Johnston

Re: [gentoo-desktop] System problems

[Roman Zilka]

> /etc/fstab has been edited several times, as I noted in my post.  The
> kernel, udev and /etc/fstab have been now been reverted, as I also noted, so
> I could get the desktop working.  Considering that, posting any of the
> information you've asked for would probably be useless.

OK, so be it, fstab is not that important.

> Roman, if you don't have any useful insights based on the information I
> already posted, then please don't post on thread and leave it to others
> who may.

I may have useful insights that are different from the insights posted
previously by other people. But I need your `emerge --info` and kernel
conf for that first. To give you a hint of explanation: I need the
kernel conf to look for whatever may be wrong in there. There's no
point in sending you a working conf for my (i.e., different) machine -
there's plenty of those lying around the net, as we both know. I assume
you have either already tried one of those or simply don't want to use
one for some reason. Thus, it's possible that you keep making a
recurring mistake while modifying default / borrowed / your own old
configs. And I need to see your conf to discover such potential
mistakes. As for `emerge --info`, it may uncover problems relevant in
this case too.

Please, cooperate with those whom you'd asked for help. Writing these
several paragraphs worth of e-mail text as a reply was a waste of time
for you - it clearly hasn't produced any help at all regarding your
booting issue. On the other hand, sending me what I'd asked for right
away would not only eat up much less of your time, it might have
yielded a solution by now. I suppose you're asking for help because you
understand that others may be more knowledgeable than yourself.

> > Also, by upgrading to a little less ancient versions than 2.6.29
> > you won't have the same situation like now boomerang back at you in the
> > near future.
> 
> Can you cite a source or sources for this assertion?

The source is the very reality of change of things in the world over
time. Software evolves and because hardly anything in nature has
infinite duration, it is only a matter of time before compatibility of
udev (or something else) with the 2.6.29 ends. In fact, this is true
for any two pieces of software that coexist, not just the
kernel+something, of course.

> Is there a known
> problem with kernel 2.6.29, or the portage tree which spec'd that
> kernel?

There are so many known problems with that kernel that it'd take me a
lifetime to remember and copy them all. See:
ftp://ftp.kernel.org/pub/linux/kernel/v2.6/ChangeLog-2.6.3?*

And some of those are relatively serious security holes and it'd take a
really special handling of the system to avoid them. And I'm talking
about handling that'd probably render an Internet-connected desktop box
with a web browser unusable. I'm not gonna google those specific ones
for you, we don't need that to get ahead; every active admin will
remember them.

And yes, Gentoo devs deem 2.6.29 dangerous too - that's why it isn't in
the current Portage tree at all (vanilla-sources and gentoo-sources).
Kernel devs themselves deem it dangerous and they don't maintain that
branch anymore. Of what's maintained (in terms of security patches),
2.6.27 and 2.6.32 are nearest to 2.6.29. And I wouldn't expect at least
one of them to linger around for a very long time.

> In almost every case, I've found that people who lecture me online about
> my system admin practices don't really have a handle on the issue about
> which I'm writing.  Please prove me wrong :-)

I suppose one can say I've done just that, having written what I've
written. At least I hope did so in a sensitive way. There's no need to
defend your admin skills in case you happen to feel offended by
something above. Why is there no need? Because failing in an honest
effort is not a reason for disregard for a human being. So there's
actually no harm for you from that.

Well, in fact, it is a reason for disregard for a few people, but let's
not have our lives spoiled by those.:)

-rz

Re: [gentoo-desktop] System problems

[Roman Zilka]

> Well, in fact, it is a reason for disregard for a few people, but let's
> not have our lives spoiled by those.:)

*aehm* Sorry for being ambiguous here. In other words: there are a few
people who take it as a reason for disregard. But let's not have our
lives spoiled by those said few people.:)

-rz

Re: [gentoo-desktop] System problems

[Lindsay Haisley]

On Mon, 2011-03-21 at 00:10 +0100, Roman Zilka wrote:
> > /etc/fstab has been edited several times, as I noted in my post.  The
> > kernel, udev and /etc/fstab have been now been reverted, as I also noted, so
> > I could get the desktop working.  Considering that, posting any of the
> > information you've asked for would probably be useless.
> 
> OK, so be it, fstab is not that important.

Actually, /etc/fstab has been central to the problem, since the system
seems to be unable to interpret it during the boot process, although the
kernel correctly interprets the same drive spec when it's on the kernel
cmd line in menu.lst.

> > Roman, if you don't have any useful insights based on the information I
> > already posted, then please don't post on thread and leave it to others
> > who may.
> 
> I may have useful insights that are different from the insights posted
> previously by other people. But I need your `emerge --info` and kernel
> conf for that first. To give you a hint of explanation: I need the
> kernel conf to look for whatever may be wrong in there.

What I'll do is this, Roman.  I've emerged the linux-2.6.36-gentoo-r5
kernel and built it with the _stock_ Gentoo settings.  I'll add the
specific drivers for my hardware, such as my NIC and dual sound cards,
rebuild the kernel again and take another shot at it when my time
allows.  If the problem persists, _then_ my kernel .config may be a
candidate for more eyes to look at than mine.

As you kind of point out, it really doesn't make sense to work with
trying to whip into shape a kernel that's no longer even in the portage
tree, and probably shouldn't be used in any event, and which has been
configured with my legacy .config setup.

This will take some of the variables out of the problem and if
necessary, perhaps we can look at the situation more cleanly.

>  There's no
> point in sending you a working conf for my (i.e., different) machine -
> there's plenty of those lying around the net, as we both know. I assume
> you have either already tried one of those or simply don't want to use
> one for some reason.

I'm going to start with the _stock_ Gentoo kernel config, which should
at very least bring up the drives.  If I can get the drives and boot
process to work, then I can add modules and facilities after that.

>  Thus, it's possible that you keep making a
> recurring mistake while modifying default / borrowed / your own old
> configs.

This is absolutely true, as noted above.

>  And I need to see your conf to discover such potential
> mistakes. As for `emerge --info`, it may uncover problems relevant in
> this case too.

"emerge --info" is the the stock Gentoo system profile, and I'll be
happy to share it, but in this case I'm looking at what's almost a
"pre-Gentoo" issue, involving the kernel and the boot-up.

> Please, cooperate with those whom you'd asked for help. Writing these
> several paragraphs worth of e-mail text as a reply was a waste of time
> for you - it clearly hasn't produced any help at all regarding your
> booting issue.

Taking the desktop system off-line, re-emerging udev, bringing it up
into its failure mode with a newer kernel and pulling the necessary
pieces together, then backing out and putting everything back so the
system is actually fairly usable is a major hassle.  I have had _zero_
time to work on this problem since I posted this morning, but will be
able to take another run at it this evening, hopefully.  Writing is no
effort for me, and doesn't disable my desktop ;)  

>  On the other hand, sending me what I'd asked for right
> away would not only eat up much less of your time, it might have
> yielded a solution by now. I suppose you're asking for help because you
> understand that others may be more knowledgeable than yourself.

We are all have different skill and knowledge sets, and sometimes, as
everyone has found out, the very process of organizing the presentation
of a problem to others leads one to a solution.

> > Can you cite a source or sources for this assertion?
> 
> The source is the very reality of change of things in the world over
> time. Software evolves and because hardly anything in nature has
> infinite duration,

I was hoping for something a little less nebulous ;-)

> And some of those are relatively serious security holes and it'd take a
> really special handling of the system to avoid them. And I'm talking
> about handling that'd probably render an Internet-connected desktop box
> with a web browser unusable.

This desktop box is on an RFC-1918 masqueraded network.  It has zero
exposure to the Internet, except insofar as the firewall will permit
traffic from related and established connections, as per the firewall
NAT rules.  The only other person on the LAN is my sweetie, and as far
as I know I can trust her not to black-hat hack my desktop system :-)
All my professional work is done via VPNs to my client's systems.

> And yes, Gentoo devs deem 2.6.29 dangerous too - that's why it isn't in
> the current Portage tree at all (vanilla-sources and gentoo-sources).
> Kernel devs themselves deem it dangerous and they don't maintain that
> branch anymore.

Thanks, Roman.  This is very useful lore.  As I noted, I've moved on to
2.6.36-gentoo-r5.

> > In almost every case, I've found that people who lecture me online about
> > my system admin practices don't really have a handle on the issue about
> > which I'm writing.  Please prove me wrong :-)
> 
> I suppose one can say I've done just that, having written what I've
> written. At least I hope did so in a sensitive way. There's no need to
> defend your admin skills in case you happen to feel offended by
> something above. Why is there no need? Because failing in an honest
> effort is not a reason for disregard for a human being. So there's
> actually no harm for you from that.

No problem, sir.  I've already moved on.

Tonight or tomorrow evening I'll add the necessary minimal mods to the
stock build of the Gentoo kernel noted above, and take another shot at
this problem.  _If_ I continue to have this problem then I'll post my
results to this list in a somewhat more ordered fashion.  Rather than
posting my entire kernel .config, emerge --info and /etc/fstab to this
list, which I consider questionable netiquette, I'll put it on my
personal file space on one of my servers and post the URL.  We can take
it from there.

Thanks for your help.  Unless you have specific suggestions for me to
try out, you might want to stand by until I've had a chance to take a
shot at the problem with the newer kernel.

lh

-- 
Lindsay Haisley       | SUPPORT NETWORK NEUTRALITY
FMP Computer Services | --------------------------
512-259-1190          | Boycott Yahoo, RoadRunner, AOL
http://www.fmp.com    | and Verison

Re: [gentoo-desktop] System problems

[Dale]

Lindsay Haisley wrote:
> << SNIP >>
> lh
>
>    
One other thing since this appears to be a PATA/IDE driver issue, make 
sure you remove the old IDE part in the kernel.  I forgot that on my 
system when I switched from the old IDE drivers to PATA.

Also, if you use grub, you may be able to learn if things are laid out 
the way you think they are.  I had two IDE drives attached to the mobo 
and one SATA drive attached to a SATA PCI card.  I expected the kernel 
to see the drives attached to the mobo first then the drive connected to 
the SATA card.  It didn't work that way.  I used grub to figure out that 
it was seeing the drive attached to the card first then the drives 
attached to the mobo.

This info may not help but thought it worth mentioning just in case.

Dale

:-)  :-)

Re: [gentoo-desktop] System problems

[Lindsay Haisley]

On Sun, 2011-03-20 at 21:39 -0500, Dale wrote:
> One other thing since this appears to be a PATA/IDE driver issue, make 
> sure you remove the old IDE part in the kernel.  I forgot that on my 
> system when I switched from the old IDE drivers to PATA.

I turned it into a module, but the module may have been auto-loaded.  I
didn't check.

> Also, if you use grub, you may be able to learn if things are laid out 
> the way you think they are.  I had two IDE drives attached to the mobo 
> and one SATA drive attached to a SATA PCI card.  I expected the kernel 
> to see the drives attached to the mobo first then the drive connected to 
> the SATA card.  It didn't work that way.  I used grub to figure out that 
> it was seeing the drive attached to the card first then the drives 
> attached to the mobo.

I don't think this is the issue.  The rescue disk brings up all of the
drives, including some LVM drives on the mapper device, which are built
on top of RAID-1 pairs.  Although in a boot w.o. the rescue disk, the
kernel recognizes the root filesystem when I spec it with /dev/sda4 in
grub.conf, this recognitions seems to be lost later in the boot process.

There may be some conflicts.  /dev/hda1 is a VMware partition, which
becomes /dev/sda1 in newer kernels.  /dev/sda1 is also a SATA partition
which is part of a RAID-1 array.  The Linux RAID-1 and LVM stuff seem to
pretty much take care of themselves, as long as the RAID and
device-mapper drivers are available in the kernel or as modules, and
this seems tangential to the problem.

-- 
Lindsay Haisley       | "Everything works if you let it"
FMP Computer Services |   (The Roadie)
512-259-1190          |
http://www.fmp.com    |

Re: [gentoo-desktop] System problems

[Roman Zilka]

[reordered the paragraphs]

> Unless you have specific suggestions for me to
> try out, you might want to stand by until I've had a chance to take a
> shot at the problem with the newer kernel.

I do have a few. I hope you'll make a good use of them to make your
life more productive.

> >  And I need to see your conf to discover such potential
> > mistakes. As for `emerge --info`, it may uncover problems relevant in
> > this case too.
> 
> "emerge --info" is the the stock Gentoo system profile, and I'll be
> happy to share it, but in this case I'm looking at what's almost a
> "pre-Gentoo" issue, involving the kernel and the boot-up.

This sounds like you're sure that there is nothing interesting to
see in `emerge --info`. Well, let's change it into something more
correct: according to your knowledge there is nothing interesting to
see. But your knowledge isn't perfect (just like anyone's). Keeping
that in mind while consulting with others is the first of my specific
suggestions into the future. It will save your+our time and may get you
the solution faster.

> > Please, cooperate with those whom you'd asked for help. Writing these
> > several paragraphs worth of e-mail text as a reply was a waste of time
> > for you - it clearly hasn't produced any help at all regarding your
> > booting issue.
> 
> Taking the desktop system off-line, re-emerging udev, bringing it up
> into its failure mode with a newer kernel and pulling the necessary
> pieces together, then backing out and putting everything back so the
> system is actually fairly usable is a major hassle.  I have had _zero_
> time to work on this problem since I posted this morning, but will be
> able to take another run at it this evening, hopefully.  Writing is no
> effort for me, and doesn't disable my desktop ;)  

The second of my specific suggestions into a more fruitful future is:
read what people ask of you. Sending me what I'd asked for would've
taken about a few dozen seconds (I suppose you have the config for
2.6.29 stored somewhere). It certainly wouldn't entail taking your
desktop offline, reemerging udev, etc. etc. It's just attaching two
text files to a piece of e-mail. So if your current attempt at 2.6.36
(was it?) fails, you know what to do right away.

> > And some of those are relatively serious security holes and it'd take a
> > really special handling of the system to avoid them. And I'm talking
> > about handling that'd probably render an Internet-connected desktop box
> > with a web browser unusable.
> 
> This desktop box is on an RFC-1918 masqueraded network.  It has zero
> exposure to the Internet, except insofar as the firewall will permit
> traffic from related and established connections, as per the firewall
> NAT rules.  The only other person on the LAN is my sweetie, and as far
> as I know I can trust her not to black-hat hack my desktop system :-)
> All my professional work is done via VPNs to my client's systems.

The third suggestion is probably the most important one: being NAT'd
and being behind any iptables configuration (that allows for operations
such as sending mail and browsing the web) doesn't make your PC
invulnerable or anything near that. In other words, active break-in
attempts via open ports is by far not the only option hackers have.

> Rather than
> posting my entire kernel .config, emerge --info and /etc/fstab to this
> list, which I consider questionable netiquette, I'll put it on my
> personal file space on one of my servers and post the URL.

In fact, some people who've appeared on this list over the years would
consider it unacceptably bad netiquette not to include `emerge --info`.
I also recall people who would consider it bad netiquette, but would
still answer your questions (perhaps with some remarks). And I suppose
most others consider it at least a good idea and a potential time-saver
to include it, unless the topic in question is "what laptop should I
buy to run Gentoo". So there goes my last specific suggestion to help
you make a more efficient use of this list: include your `emerge
--info` and relevant config files, if any, in the opening post to a
mailinglist. It's not like you're telling us your card number.

-rz

Re: [gentoo-desktop] System problems

[Lindsay Haisley]

On Mon, 2011-03-21 at 09:57 +0100, Roman Zilka wrote:
> I do have a few. I hope you'll make a good use of them to make your
> life more productive.

Good heavens, Roman!  Please let's back off on the lectures and stick to
the technical stuff.  When I have a chance to fully a address the
problem again, if I still have a problem I will take the information you
request and post it, as I said, to a private URL, which I will post to
this list, and NOT send it out to a gazillion subscribers who aren't
interested.  This is the way patches, configs, emerge --info, etc is
done on many forums and mailing lists.  This is the way it WILL be done
in this case, and I'm sure you can deal with it.

This is NOT an operation of a few dozen seconds.  I will post to this
list when I have time to fully focus on the problem.  In the meantime, I
don't intend to argue with you or anyone else about what does or does
not constitute proper netiquette.  I'm 69 years old, have been
programming in numerous languages and working with Unix since the mid
80s, have been working with Linux since kernel v1.x.x in the 90s, am
past president of the oldest Unix pofessional society in Austin, TX,
co-founder of the oldest Linux Users Group in Austin, have moderated
more mailing lists than I care to remember, run a small but well
respected hosting service here, and am not without knowledge and
resources.  I come to this list with this problem not because you, or
anyone else "knows more than I do", but because people may have
technical insights or suggestions that may help me understand what's
going on.  Let's keep it to that going forward.

> The second of my specific suggestions into a more fruitful future is:
> read what people ask of you.

Jesus, Roman!  BACK OFF!!  I'm sorry if I offended you, but your
response is offensive to me.  All it does is annoy me and encourage the
script kiddies on the list to take cheap shots at me.  I appreciate your
insights, but please let's stick to the technical issues.  _IF_ I need
to, I will post the material you request in the manner I've noted
previously, at such time as I have my work on the problem better
organized.  Don't waste my time, and I won't waste yours.  OK?  

Again, I sincerely apologize if I've wasted your time, or anyone else's
on this list, and I'll try to keep my posts to a minimum going forward
until I have more to report.  Again, going forward, PLEASE stick to
technical matters and let's let issues of style, netiquette, etc. alone.
Thank you, and 'nuff said about that.  And again, please accept my
apology.

-- 
Lindsay Haisley       | "Never expect the people who caused a problem
FMP Computer Services |  to solve it."  - Albert Einstein
512-259-1190          |        
http://www.fmp.com    |

Re: [gentoo-desktop] System problems

[Lindsay Haisley]

Roman, et al, since you, and maybe others seem to be incredibly hot to
trot on my desktop booting issue, the files you requested are at

<http://www.fmp.com/~fmouse/vishnu/>

/etc/fstab (vishnu_fstab.txt) is configured as per the _working_ kernel
on the box, which is 2.6.21-gentoo-r1.  When booting from a more recent
kernel, the 1st three lines will be changed to spec /dev/sdXn instead
of /dev/hdXn.

The kernel config (vishnu_kernel-2.6.29-gentoo-r5_config.txt) is suspect
at this point, as per Roman's information re. 2.6.29 being a thoroughly
discredited kernel version.  This configuration will _not_ be used going
forward into a more recent kernel.

vishnu_emerge_info.txt is "emerge --info" from the box.  I'm not sure
how this will give you any insight into the booting problem on the box,
but since you seem to have your knickers in a knot about my _not_
posting it, here it is :-)

Kernels on the box are built manually, using "make bzImage" and friends,
not using genkernel.

When I have some time to take the box down, upgrade the kernel and take
another shot at this problem, I'll update.

-- 
Lindsay Haisley       | "Real programmers use butterflies"
FMP Computer Services |
512-259-1190          |     - xkcd
http://www.fmp.com    |

Re: [gentoo-desktop] System problems

[eamjr56]

Jesus, take your meds already. Roman has only been trying to help you throughout this long sorry episode. This is NOT about your past Unix yada yada, but how well you follow kernel devel, won't look on the Gentoo forums for answers and generally take help.                     It is not make bzimage for one, it is make, make modules_install, make install. Boot with a Live CD and chroot into yer install, remove ALL options for IDE period. Check SATA.   Build, change all HDxxx to SDxx run your bootloader and TADA fixed
Sent from my Verizon Wireless BlackBerry

-----Original Message-----
From: Lindsay Haisley <fmouse-gentoo@fmp.com>
Date: Mon, 21 Mar 2011 11:18:07 
To: <gentoo-desktop@lists.gentoo.org>
Reply-to: gentoo-desktop@lists.gentoo.org
Subject: Re: [gentoo-desktop] System problems

On Mon, 2011-03-21 at 09:57 +0100, Roman Zilka wrote:
> I do have a few. I hope you'll make a good use of them to make your
> life more productive.

Good heavens, Roman!  Please let's back off on the lectures and stick to
the technical stuff.  When I have a chance to fully a address the
problem again, if I still have a problem I will take the information you
request and post it, as I said, to a private URL, which I will post to
this list, and NOT send it out to a gazillion subscribers who aren't
interested.  This is the way patches, configs, emerge --info, etc is
done on many forums and mailing lists.  This is the way it WILL be done
in this case, and I'm sure you can deal with it.

This is NOT an operation of a few dozen seconds.  I will post to this
list when I have time to fully focus on the problem.  In the meantime, I
don't intend to argue with you or anyone else about what does or does
not constitute proper netiquette.  I'm 69 years old, have been
programming in numerous languages and working with Unix since the mid
80s, have been working with Linux since kernel v1.x.x in the 90s, am
past president of the oldest Unix pofessional society in Austin, TX,
co-founder of the oldest Linux Users Group in Austin, have moderated
more mailing lists than I care to remember, run a small but well
respected hosting service here, and am not without knowledge and
resources.  I come to this list with this problem not because you, or
anyone else "knows more than I do", but because people may have
technical insights or suggestions that may help me understand what's
going on.  Let's keep it to that going forward.

> The second of my specific suggestions into a more fruitful future is:
> read what people ask of you.

Jesus, Roman!  BACK OFF!!  I'm sorry if I offended you, but your
response is offensive to me.  All it does is annoy me and encourage the
script kiddies on the list to take cheap shots at me.  I appreciate your
insights, but please let's stick to the technical issues.  _IF_ I need
to, I will post the material you request in the manner I've noted
previously, at such time as I have my work on the problem better
organized.  Don't waste my time, and I won't waste yours.  OK?  

Again, I sincerely apologize if I've wasted your time, or anyone else's
on this list, and I'll try to keep my posts to a minimum going forward
until I have more to report.  Again, going forward, PLEASE stick to
technical matters and let's let issues of style, netiquette, etc. alone.
Thank you, and 'nuff said about that.  And again, please accept my
apology.

-- 
Lindsay Haisley       | "Never expect the people who caused a problem
FMP Computer Services |  to solve it."  - Albert Einstein
512-259-1190          |        
http://www.fmp.com    |

Re: [gentoo-desktop] System problems

[Lindsay Haisley]

On Mon, 2011-03-21 at 20:08 +0000, eamjr56@bellsouth.net wrote:
> Jesus, take your meds already. Roman has only been trying to help you
> throughout this long sorry episode.

Please excuse yourself from any further postings to this thread.  I have
asked you before to do this.  If you would shut up, and if Roman would
stick to tech information rather than dispensing free social advice,
then this wouldn't be a "sorry episode", would it?

> This is NOT about your past Unix yada yada, but how well you follow
> kernel devel, won't look on the Gentoo forums for answers and
> generally take help.

Y'know, eamjr56, I have a life.  I don't sit around all day and play
with my operating system.  I have a lot of things to do _with_ my office
desktop system.  I don't have a lot of spare time to mess with and fix
stuff that shouldn't need messing with and fixing.  I came to this forum
because I thought I might get some insights into a specific problem I'm
having, and have been met mainly with rudeness by you and Roman,
although a couple of other people had useful/thougtful suggestions.
Isn't there a moderator on this list who understands how a technical
forum should be run?

I've posted the files that Roman wanted to see, and they're there for
anyone else to look at.  So far, Roman hasn't offered me any "help" with
this problem, but has blown a lot of hot air on this forum suggesting
how I should run my life.  I'm beginning to think that there aren't many
technically literate people on this forum.  They've probably all
abandoned Gentoo and are using Debian or Ubuntu.

> It is not make bzimage for one, it is make, make modules_install, make
> install. 

You need to read up on kernel building.  "make bzImage && make modules"
has worked for kernel building since probably Linux kernel 1.x.x.  Yes,
"make" will also work, and will combine these two operations into one,
but on occasion it's convenient to separate these.

-- 
Lindsay Haisley       | "Fighting against human creativity is like 
FMP Computer Services |   trying to eradicate dandelions"
512-259-1190          |    (Pamela Jones)
http://www.fmp.com    |

Re: [gentoo-desktop] System problems

[Tiago Marques]

[-- Attachment #1: Type: text/plain, Size: 2337 bytes --]

On Mon, Mar 21, 2011 at 10:24 PM, Lindsay Haisley <fmouse-gentoo@fmp.com>wrote:

> On Mon, 2011-03-21 at 20:08 +0000, eamjr56@bellsouth.net wrote:
> > Jesus, take your meds already. Roman has only been trying to help you
> > throughout this long sorry episode.
>
> Please excuse yourself from any further postings to this thread.  I have
> asked you before to do this.  If you would shut up, and if Roman would
> stick to tech information rather than dispensing free social advice,
> then this wouldn't be a "sorry episode", would it?
>
> > This is NOT about your past Unix yada yada, but how well you follow
> > kernel devel, won't look on the Gentoo forums for answers and
> > generally take help.
>
> Y'know, eamjr56, I have a life.


Please go troll elsewhere. Please...




> I don't sit around all day and play
> with my operating system.  I have a lot of things to do _with_ my office
> desktop system.  I don't have a lot of spare time to mess with and fix
> stuff that shouldn't need messing with and fixing.  I came to this forum
> because I thought I might get some insights into a specific problem I'm
> having, and have been met mainly with rudeness by you and Roman,
> although a couple of other people had useful/thougtful suggestions.
> Isn't there a moderator on this list who understands how a technical
> forum should be run?
>
> I've posted the files that Roman wanted to see, and they're there for
> anyone else to look at.  So far, Roman hasn't offered me any "help" with
> this problem, but has blown a lot of hot air on this forum suggesting
> how I should run my life.  I'm beginning to think that there aren't many
> technically literate people on this forum.  They've probably all
> abandoned Gentoo and are using Debian or Ubuntu.
>
> > It is not make bzimage for one, it is make, make modules_install, make
> > install.
>
> You need to read up on kernel building.  "make bzImage && make modules"
> has worked for kernel building since probably Linux kernel 1.x.x.  Yes,
> "make" will also work, and will combine these two operations into one,
> but on occasion it's convenient to separate these.
>
> --
> Lindsay Haisley       | "Fighting against human creativity is like
> FMP Computer Services |   trying to eradicate dandelions"
> 512-259-1190          |    (Pamela Jones)
> http://www.fmp.com    |
>
>
>

[-- Attachment #2: Type: text/html, Size: 3176 bytes --]

Re: [gentoo-desktop] System problems

[Lindsay Haisley]

On Mon, 2011-03-21 at 22:35 +0000, Tiago Marques wrote:
> Please go troll elsewhere. Please... 

Let's just bring this back to technical discussion of the problem, shall
we?  I think that's what's needed here.

Roman wanted to see some of my system information and config files,
which I've made available.  He thinks he may be able to shed some light
on the problem, so the ball is in his court, or in anyone else's who has
thoughts or observations on them related to the boot problem here.

-- 
Lindsay Haisley       |  "We are all broken toasters, but we still
FMP Computer Services |    manage to make toast"
512-259-1190          |
http://www.fmp.com    |        - Cheryl Dehut
                      |

[gentoo-desktop] Re: System problems

[Duncan]

Lindsay Haisley posted on Mon, 21 Mar 2011 17:44:32 -0500 as excerpted:

> On Mon, 2011-03-21 at 22:35 +0000, Tiago Marques wrote:
>> Please go troll elsewhere. Please...
> 
> Let's just bring this back to technical discussion of the problem, shall
> we?  I think that's what's needed here.

Let me try one more time, then.  On my original reply you only responded 
to the top half.  In the second part I asked a question about whether 
you're running an initrd/initramfs, which I repeated again in a second 
reply when there was no answer to the first, both times explaining why 
it's important and the two different paths one would take from there 
depending on the answer.

I'm still waiting on an answer to that, but as I spent quite some time on 
the details the first AND second time, I'll simply refer you back to them 
this time.  You can go back and reply to them, or not.  I won't bother 
pursuing it further if you don't believe it's worth the followup.  No hard 
feelings either way, but I believe it could be of help or I'd have not 
bothered.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-desktop] Re: System problems

[Lindsay Haisley]

On Tue, 2011-03-22 at 07:07 +0000, Duncan wrote:
> Lindsay Haisley posted on Mon, 21 Mar 2011 17:44:32 -0500 as excerpted:
> 
> > On Mon, 2011-03-21 at 22:35 +0000, Tiago Marques wrote:
> >> Please go troll elsewhere. Please...
> > 
> > Let's just bring this back to technical discussion of the problem, shall
> > we?  I think that's what's needed here.
> 
> Let me try one more time, then.  On my original reply you only responded 
> to the top half.  In the second part I asked a question about whether 
> you're running an initrd/initramfs, which I repeated again in a second 
> reply when there was no answer to the first, both times explaining why 
> it's important and the two different paths one would take from there 
> depending on the answer.

Duncan, I'm very sorry to have overlooked your question (twice)!  Please
accept my apology.  The answer is no, on the box is question, I'm not
running an initrd/initramfs, and unless it's necessary I would rather
not do so.  If I were, this would be one explanation for the problem.

> I'm still waiting on an answer to that, but as I spent quite some time on 
> the details the first AND second time, I'll simply refer you back to them 
> this time.

Thanks for following up.  I'll re-read your posts and get back on this.
I haven't been able to follow up on the problem.  It's planting season
here and beyond doing email I've been kinda tired and not up for late
night hacking after a day spent building a greenhouse or whatever.  At
this point, with a cooler head and a verified back out path, I'm 99%
sure I can solve the problem myself.  I was kind of freaked out when
this problem originally happened since I depend so heavily on the
desktop system for company billing, website development, customer
support, etc.  I guess my original post to this list on this problem
showed this and got me and everyone else off on the wrong foot.

-- 
Lindsay Haisley       |"Windows .....
FMP Computer Services |    life's too short!"
512-259-1190          |
http://www.fmp.com    |              - Brad Johnston

[gentoo-desktop] Re: System problems

[Duncan]

Lindsay Haisley posted on Tue, 22 Mar 2011 10:41:24 -0500 as excerpted:

>> > Let's just bring this back to technical discussion of the problem,
>> > shall we?  I think that's what's needed here.
>> 
>> Let me try one more time, then.  On my original reply you only
>> responded to the top half.  In the second part I asked a question about
>> whether you're running an initrd/initramfs
> 
> Duncan, I'm very sorry to have overlooked your question (twice)!  Please
> accept my apology.  The answer is no, on the box is question, I'm not
> running an initrd/initramfs, and unless it's necessary I would rather
> not do so.  If I were, this would be one explanation for the problem.

I'm absolutely with you on that.  If you don't need an initr*, it 
definitely makes the whole thing simpler to avoid it (as I too have done).

(FWIW, my style is to make my opinion known, but go ahead and deal with 
the question too, to the extent that I can.  You saw the opinion part and 
skipped the rest, which is understandable, I suppose...  In any case it's 
straight now.)

>> I'm still waiting on an answer to that, but as I spent quite some time
>> on the details the first AND second time, I'll simply refer you back to
>> them this time.
> 
> Thanks for following up.  I'll re-read your posts and get back on this.
> I haven't been able to follow up on the problem.  It's planting season
> here and beyond doing email I've been kinda tired and not up for late
> night hacking after a day spent building a greenhouse or whatever.  At
> this point, with a cooler head and a verified back out path, I'm 99%
> sure I can solve the problem myself.  I was kind of freaked out when
> this problem originally happened since I depend so heavily on the
> desktop system for company billing, website development, customer
> support, etc.

Panicking is certainly understandable, particularly coming at an already 
busy time of year.  But that's behind us now.  And simply having no time 
to deal with it for a few days is something I'm sure we all deal with, 
especially when there ends up a social crisis to deal with too.  Sometimes 
it's all just too much and something has to give.  When that's finally 
realized and done... it's like venting steam from an overheated nuclear 
reactor (apropos comparison ATM, if I do say so =:^), it might mean a 
slightly elevated immediate issue, but tends to deescalate the entire 
situation.

Meanwhile, now we know the fork in the path to take.  Without an initr*, 
the fact that you get even a limited userspace (not just a kernel panic) 
means that the kernel has the necessary drivers to get to and do the 
initial load of the rootfs.  The problem must be beyond that, in the 
userspace config, initscripts or binaries.  

At this point I'd guess something like the udev/kernel-config-deprecated-
sysfs issue someone else mentioned.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-desktop] Re: System problems

[Lindsay Haisley]

On Tue, 2011-03-22 at 19:15 +0000, Duncan wrote:
> I'm absolutely with you on that.  If you don't need an initr*, it 
> definitely makes the whole thing simpler to avoid it (as I too have done).

I have an initrd on our firewall/fileserver here at the SOHO since I'm
I'm loading the root fs from a RAID-supported, EVMS drive.  I did this
as a kind of proof of concept to learn how to do it, but at one point a
kernel upgrade forced me to rebuild the initrd manually with a newer
glibc.  It was a strange problem, and a lot of work to fix.  If I could
diagnose and fix _that_ I reckon I can quit whining and fix the
(probably relatively easy) one with my desktop boot.

It's a pity that EVMS is an orphaned project, or was the last time I
checked.  IBM dropped it.  It was and is a pretty slick system.

Ubuntu uses an initrd in all their stuff.  I believe it's involved in
the display of their boot graphic and such.  They've achieved _very_
fast boot times, though, and the loaded system is disk-based.  I know a
lot of Gentoo people look down on Ubuntu, but every distribution I've
ever worked with has its advantages and disadvantages.  Open Source is
about choice :-)

> Panicking is certainly understandable, particularly coming at an already 
> busy time of year.  But that's behind us now.  And simply having no time 
> to deal with it for a few days is something I'm sure we all deal with, 
> especially when there ends up a social crisis to deal with too.

I'm going to nominate Jorge for sainthood!  Everybody's moved on.  Kudos
to everybody :-)

> Meanwhile, now we know the fork in the path to take.  Without an initr*, 
> the fact that you get even a limited userspace (not just a kernel panic) 
> means that the kernel has the necessary drivers to get to and do the 
> initial load of the rootfs.

I was getting a kernel panic for a while until I added a root spec to
the kernel invocation in menu.lst.

> The problem must be beyond that, in the 
> userspace config, initscripts or binaries.

This is probably close to the point.  Still working on our greenhouse,
but soon I'll get the time to get back on the case with this problem.

> At this point I'd guess something like the udev/kernel-config-deprecated-
> sysfs issue someone else mentioned.

Excellent shot, Duncan.  I'll check it.  I won't be using the 2.6.29
kernel again, as per Roman's suggestion, but 2.6.36 instead.

-- 
Lindsay Haisley       |  "We are all broken toasters, but we still
FMP Computer Services |    manage to make toast"
512-259-1190          |
http://www.fmp.com    |        - Cheryl Dehut
                      |

Re: [gentoo-desktop] Re: System problems

[Lindsay Haisley]

On Tue, 2011-03-22 at 19:15 +0000, Duncan wrote:
> I'm absolutely with you on that.  If you don't need an initr*, it 
> definitely makes the whole thing simpler to avoid it (as I too have done).

I have an initrd on our firewall/fileserver here at the SOHO since I'm
I'm loading the root fs from a RAID-supported, EVMS drive.  I did this
as a kind of proof of concept to learn how to do it, but at one point a
kernel upgrade forced me to rebuild the initrd manually with a newer
glibc.  It was a strange problem, and a lot of work to fix.  If I could
diagnose and fix _that_ I reckon I can quit whining and fix the
(probably relatively easy) one with my desktop boot.

It's a pity that EVMS is an orphaned project, or was the last time I
checked.  IBM dropped it.  It was and is a pretty slick system.

Ubuntu uses an initrd in all their stuff.  I believe it's involved in
the display of their boot graphic and such.  They've achieved _very_
fast boot times, though, and the loaded system is disk-based.  I know a
lot of Gentoo people look down on Ubuntu, but every distribution I've
ever worked with has its advantages and disadvantages.  Open Source is
about choice :-)

> Panicking is certainly understandable, particularly coming at an already 
> busy time of year.  But that's behind us now.  And simply having no time 
> to deal with it for a few days is something I'm sure we all deal with, 
> especially when there ends up a social crisis to deal with too.  Sometimes 
> it's all just too much and something has to give.  When that's finally 
> realized and done... it's like venting steam from an overheated nuclear 
> reactor (apropos comparison ATM, if I do say so =:^), it might mean a 
> slightly elevated immediate issue, but tends to deescalate the entire 
> situation.

I'm going to nominate Jorge for sainthood!

> Meanwhile, now we know the fork in the path to take.  Without an initr*, 
> the fact that you get even a limited userspace (not just a kernel panic) 
> means that the kernel has the necessary drivers to get to and do the 
> initial load of the rootfs.  The problem must be beyond that, in the 
> userspace config, initscripts or binaries.

This is probably close to the point.  Still working on our greenhouse,
but soon I'll get the time to get back on the case with this problem.

> At this point I'd guess something like the udev/kernel-config-deprecated-
> sysfs issue someone else mentioned.

Excellent shot, Duncan.  I'll check it.  I won't be using the 2.6.29
kernel again, as per Roman's suggestion, but 2.6.36 instead.

-- 
Lindsay Haisley       | "Everything works if you let it"
FMP Computer Services |   (The Roadie)
512-259-1190          |
http://www.fmp.com    |

Re: [gentoo-desktop] Re: System problems

[Lindsay Haisley]

>From the config for the 2.6.29 kernel on my desktop box.

CONFIG_SYSFS_DEPRECATED_V2=y

Oops!

Very possibly the nucleus of the problem.  Thanks, Jorge.

-- 
Lindsay Haisley       | "Everything works if you let it"
FMP Computer Services |   (The Roadie)
512-259-1190          |
http://www.fmp.com    |

Re: [gentoo-desktop] System problems

[Lindsay Haisley]

OK, Roman,

I've posted a new kernel config for 2.6.36-gentoo-r5.  I haven't had a
chance to back down the system and try it out yet.

<http://www.fmp.com/~fmouse/vishnu/vishnu_kernel-2.6.36-gentoo-r5_config.txt>

This should be everything you wanted to see in order to impart some of
your great wisdom and infinite Linux knowledge to me ;-)

Other files you requested are shown in the index at
<http://www.fmp.com/~fmouse/vishnu/>, as previously noted.

Have at!

-- 
Lindsay Haisley       |  "Humor will get you through times of no humor
FMP Computer Services |      better than no humor will get you through
512-259-1190          |         times of humor."
http://www.fmp.com    |            - Butch Hancock

Re: [gentoo-desktop] System problems

[Roman Zilka]

I've gone through the 2.6.36 config. It's indeed overflowing with
exotic options, some of which I have never even read the description
for. I've gone through them, but let's keep their pruning on the
programme in case tackling anything else doesn't lead anywhere.

As for the common config options, I too suggest removing initramfs
completely. The target CPU is set as a generic 64b. Judging by your
`emerge --info`, you probably have a Pentium 4. The config will also
start SELinux by default. God knows I'm not proficient with SELinux,
but I suppose it requires extended attributes support in the
filesystem. Ext_attr is enabled for ext3 (you /boot), but disabled for
reiserfs, which in turn seems to be on your /, judging by the fstab. At
any rate, removing SELinux out of the kernel completely will be far from
harmful (unless you know you need it and actually use it for something
specific, of course). ATA-related options seem to be OK. The /dev nodes
should be called sda*.

Your `emerge --info` implies that the system isn't all that old. That's
very good. I think it's well possible to run at least an 'emerge -uD
world' on the system without messing it up irreversibly. That's one big
(albeit somewhat off-topic) suggestion, in case you're not switching to
another distro really soon.

I can also imagine some highly unlikely, yet possible collision between
the relatively recent in-kernel ext3/reiserfs drivers and the old
userland. If it doesn't cause an avalanche of updates, try emerging the
latest stable udev, e2fsprogs, reiserfsprogs and perhaps also
util-linux and baselayout. Just to be sure. Don't rely on dependencies
absolutely (such as in case of the intended udev semi-update).

Another unlikely, but possible problem: long-forgotten custom udev rules
that are in some way incompatible with something new. Of course, this
may / may not apply depending on how far you get during the boot-up.

So far for the first-hand stuff. Let's wait here for your updates on the
latest events.

-rz


Lindsay Haisley (Mon, 21 Mar 2011 17:38:53 -0500):
> OK, Roman,
> 
> I've posted a new kernel config for 2.6.36-gentoo-r5.  I haven't had a
> chance to back down the system and try it out yet.
> 
> <http://www.fmp.com/~fmouse/vishnu/vishnu_kernel-2.6.36-gentoo-r5_config.txt>
> 
> This should be everything you wanted to see in order to impart some of
> your great wisdom and infinite Linux knowledge to me ;-)
> 
> Other files you requested are shown in the index at
> <http://www.fmp.com/~fmouse/vishnu/>, as previously noted.
> 
> Have at!
> 

Re: [gentoo-desktop] System problems

[eamjr56]

Sorry I won't because I keep getting spammed by your crap. And IF you were technical and logical, AND your reading comprehension skills were above normal you would have figured this problem on your own.                          You have a problem because you have'nt kept up with how the new Linux kernels work-(in ALL the distros) and we are the jerks because we have?  I've been using Gentoo since 2001-(just letting you know how long I've been doing this)  and I don't spend all my time admining my boxes either. BUT when the kernel change came about I was smart enough to look for answers on the forums and it was a smeamless upgrade.                Your attitude and general interaction with others. Needs work. See Ya, now install Ubuntu and leave us in peace for god's sake. 
Sent from my Verizon Wireless BlackBerry

-----Original Message-----
From: Lindsay Haisley <fmouse-gentoo@fmp.com>
Date: Mon, 21 Mar 2011 17:24:27 
To: <gentoo-desktop@lists.gentoo.org>
Reply-to: gentoo-desktop@lists.gentoo.org
Subject: Re: [gentoo-desktop] System problems

On Mon, 2011-03-21 at 20:08 +0000, eamjr56@bellsouth.net wrote:
> Jesus, take your meds already. Roman has only been trying to help you
> throughout this long sorry episode.

Please excuse yourself from any further postings to this thread.  I have
asked you before to do this.  If you would shut up, and if Roman would
stick to tech information rather than dispensing free social advice,
then this wouldn't be a "sorry episode", would it?

> This is NOT about your past Unix yada yada, but how well you follow
> kernel devel, won't look on the Gentoo forums for answers and
> generally take help.

Y'know, eamjr56, I have a life.  I don't sit around all day and play
with my operating system.  I have a lot of things to do _with_ my office
desktop system.  I don't have a lot of spare time to mess with and fix
stuff that shouldn't need messing with and fixing.  I came to this forum
because I thought I might get some insights into a specific problem I'm
having, and have been met mainly with rudeness by you and Roman,
although a couple of other people had useful/thougtful suggestions.
Isn't there a moderator on this list who understands how a technical
forum should be run?

I've posted the files that Roman wanted to see, and they're there for
anyone else to look at.  So far, Roman hasn't offered me any "help" with
this problem, but has blown a lot of hot air on this forum suggesting
how I should run my life.  I'm beginning to think that there aren't many
technically literate people on this forum.  They've probably all
abandoned Gentoo and are using Debian or Ubuntu.

> It is not make bzimage for one, it is make, make modules_install, make
> install. 

You need to read up on kernel building.  "make bzImage && make modules"
has worked for kernel building since probably Linux kernel 1.x.x.  Yes,
"make" will also work, and will combine these two operations into one,
but on occasion it's convenient to separate these.

-- 
Lindsay Haisley       | "Fighting against human creativity is like 
FMP Computer Services |   trying to eradicate dandelions"
512-259-1190          |    (Pamela Jones)
http://www.fmp.com    |

Re: [gentoo-desktop] System problems

[Dale]

Lindsay Haisley wrote:
> On Mon, 2011-03-21 at 20:08 +0000, eamjr56@bellsouth.net wrote:
>    
>> Jesus, take your meds already. Roman has only been trying to help you
>> throughout this long sorry episode.
>>      
> Please excuse yourself from any further postings to this thread.  I have
> asked you before to do this.  If you would shut up, and if Roman would
> stick to tech information rather than dispensing free social advice,
> then this wouldn't be a "sorry episode", would it?
>
>    

I have a feeling that a lot of others have already excused themselves 
from this thread as well.  I always try to read things in the most 
positive way that I can but very little of your posts can have a 
positive way to read them.  You came to this list to get help, then seem 
to refuse to take the help that is offered as if you already know the 
answer, which is sort of odd since you came here for help.

If you want help, try being a little more grateful for it.  I'm a lot 
younger than you are but I learned a long time ago, if I ask a question, 
I obviously don't have the answer and need to listen to other peoples 
suggestions.  I have learned this about Linux, it ain't always the 
solution I think it is.  Having someone thinking differently about a 
problem is the reason I come to a mailing list for help.  If I am asking 
for help, I have already tried what I think should fix the problem.  I 
need to listen to someone else's suggestion or just don't post at all.

As far as I know, there are no mods here.  You can email 
gentoo-desktop+owner@lists.gentoo.org if you wish to complain tho.  I 
have not seen anything that is wrong but maybe you can convince them 
otherwise.  If someone has done something to violate some rule, I'm sure 
they will take care of it.

Since you most likely won't like this post, I'll excuse myself now.  
Good luck.

Dale

:-)  :-)

Re: [gentoo-desktop] System problems

[Jorge Manuel B. S. Vicetto]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello.

On 21-03-2011 21:46, Dale wrote:
> Lindsay Haisley wrote:
>> On Mon, 2011-03-21 at 20:08 +0000, eamjr56@bellsouth.net wrote:
>>   
>>> Jesus, take your meds already. Roman has only been trying to help you
>>> throughout this long sorry episode.
>>>      
>> Please excuse yourself from any further postings to this thread.  I have
>> asked you before to do this.  If you would shut up, and if Roman would
>> stick to tech information rather than dispensing free social advice,
>> then this wouldn't be a "sorry episode", would it?

First of all, let me start by recalling that the Gentoo Council[1] has
approved some years ago the CoC (Code of Conduct)[2] that should be
followed by anyone posting to Gentoo's public communication mediums.

 [1] - http://www.gentoo.org/proj/en/council/
 [2] - http://www.gentoo.org/proj/en/council/coc.xml

As such, I would kindly like to ask everyone on this thread to "calm
down" and to please try to reply in a respectful manner to any poster on
this ml.

> I have a feeling that a lot of others have already excused themselves
> from this thread as well.  I always try to read things in the most
> positive way that I can but very little of your posts can have a
> positive way to read them.  You came to this list to get help, then seem
> to refuse to take the help that is offered as if you already know the
> answer, which is sort of odd since you came here for help.

Lindsay,

I have to agree with other posters in this ml that your tone when you
started this thread asking for help wasn't the best one.
Thank you for the apology you've sent to the list about your tone. Let's
try to move forward and see if / how we can help you.

As I've stated above, I kindly ask everyone participating in this thread
to please watch the tone.

> As far as I know, there are no mods here.  You can email
> gentoo-desktop+owner@lists.gentoo.org if you wish to complain tho.  I
> have not seen anything that is wrong but maybe you can convince them
> otherwise.  If someone has done something to violate some rule, I'm sure
> they will take care of it.

We have some people with the ability to moderate the mailing lists, but
we prefer to leave that for last resort only.
Many Gentoo developers are subscribed to the many lists and they may at
any time intervene as they see fit.


PS - Lindsay, two last notes about your issue and your time constraints.
This type of issue might be easier to diagnose on IRC (one good place is
#gentoo in the freenode network) where a short intense session may prove
to be quicker in the end. Also, I haven't seen anyone mention that the
latest udev versions react *very* badly to CONFIG_SYSFS_DEPRECATED_V2.
So be sure to check if you disable that as iirc udev will refuse to
create the proper device nodes if that kernel option is active.

PPS - If you decide to test our IRC support, feel free to poke me in any
channel or to /msg me in private.

- -- 
Regards,

Jorge Vicetto (jmbsvicetto) - jmbsvicetto at gentoo dot org
Gentoo- forums / Userrel / Devrel / KDE / Elections / RelEng
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQIcBAEBAgAGBQJNh/1EAAoJEC8ZTXQF1qEPTloP/Avdlio/gJVvqE4tFp+0N7OK
/lgVSB3Vo5emEsXHkyQjOpvHq5jtASLQNQ37e/AD2FnE7w5iVqQ1kHkm8RPHcsWE
n739VGlnnAtO8ktsAeO3yxKF55uHE718joHZ4nkI9pjFnW5VDEWDMxp+rj4kSKq5
QVT8InG6wrVKVIrSzwdyQgMwzV0F0L1pSjzJiQ2xG2ZM7pNL5FEeZ8BEOJwFdiS7
zVPupu4jD98uDBaaSl6gUB9zjBAAvhSf07JTY7DLLrNk4XqyfO5HJjOhvCa2OtPH
WFlUo/WWoue7cuz6QqTPdHLOBYrCaMmmGVGBmJlnlTyAvNjpACY4BrHkFjui9Ukl
NgPmrc+kKJwShNDe7jBIZVibVD0d+5Kxc6KWwdRo7kuIjCW8pq2y67p8bprPsnFA
RzxZwwDjzCJuIOuyliy/UgkgtOqDN3VRqQa7KRRobN4Kkq4evMcxlxPNpKD+kgrL
ytGvQZ+aDh7L37INn3QGmxxx4Ww7GfA9H/rQAhjc8adnbLgEiH7wub+NU1bQxdK4
SnzmEJm4MhNikH7eakfdD+lKIiOYKRmp1n3s/tsCiyykGgzizgjzF2VKKYMmVxrW
vynLdkIq2ckm5V8InHcbbFkFfHU9bGbtOwC7qUf2Y3ABn5YqDjkf2RCW5XDIG2Mh
KF7cwCWvUOvVCTIpmiuI
=RxDY
-----END PGP SIGNATURE-----

Re: [gentoo-desktop] System problems

[Lindsay Haisley]

On Tue, 2011-03-22 at 00:37 -0100, Jorge Manuel B. S. Vicetto wrote:
> As such, I would kindly like to ask everyone on this thread to "calm
> down" and to please try to reply in a respectful manner to any poster on
> this ml.

Thank you, Jorge.  I will do so, going forward.

> I have to agree with other posters in this ml that your tone when you
> started this thread asking for help wasn't the best one.

I was kinda freaked out.  I depend on my desktop system for so many
things in a day, and it's _way_ behind in updates, I know.  I got pushed
into a kernel update by a simple software mod, which required a perl
update, which required that I run perl-cleaner, which has issues of its
own, and somewhere along the way I ran into a dependency on a newer
version of udev, which depended on a newer kernel, which I'd tried to
install before with the same or similar problems.  I couldn't make
things work and had to resort to a rescue disk to back out.  I felt
cornered, and I guess my plea for help showed it.

I actually have enough knowledge and experience with Linux that I can
probably solve this problem myself if I approach it methodically with a
clear head.

> Thank you for the apology you've sent to the list about your tone. Let's
> try to move forward and see if / how we can help you.

I can eat crow with the best of 'em, Jorge.

> PS - Lindsay, two last notes about your issue and your time constraints.
> This type of issue might be easier to diagnose on IRC (one good place is
> #gentoo in the freenode network) where a short intense session may prove
> to be quicker in the end. Also, I haven't seen anyone mention that the
> latest udev versions react *very* badly to CONFIG_SYSFS_DEPRECATED_V2.

Ah!  Thanks for the heads-up on this.  This switch is _not_ on in my
newer kernel, and doesn't exist in my older kernel.  CONFIG_SYSFS,
however, _is_ on as I assume it should be for /sys on which udev
depends.

I don't know why udev hasn't been completely integrated into kernel
development.  It's become so much a part of modern Linux and is so
interdependent with the kernel.  Yes, it's in user space, but so are
other kernel hooks.

> So be sure to check if you disable that as iirc udev will refuse to
> create the proper device nodes if that kernel option is active.

Question:  There are switch and value dependencies in the kernel that
are Gentoo specific, or specific to system configurations favored by
Gentoo (similar to what you mentioned above).  If I emerge a Gentoo
kernel, are the default configuration options for that kernel set by
Gentoo devs so as to build a kernel which will avoid these kinds of
"gotchas"?   In other words, if I build a gentoo kernel out of the box,
as opposed to a vanilla kernel from kernel.org, do I gain an advantage
with regard to possibly problematic configuration options on a fairly
standard Gentoo system?  I hope this makes sense.

> PPS - If you decide to test our IRC support, feel free to poke me in any
> channel or to /msg me in private.

Jorge, thank you SO MUCH, for this, (and for being an adult in the
sandbox.  I wasn't at the top of my game today :-) 

-- 
Lindsay Haisley       |  "Humor will get you through times of no humor
FMP Computer Services |      better than no humor will get you through
512-259-1190          |         times of humor."
http://www.fmp.com    |            - Butch Hancock

Re: [gentoo-desktop] System problems

[eamjr56]

Now that you explained more fully the situation  and why and what led to your problems I understand some of your frustrations. Sometimes wanting to fix one thing leads to other complex issues. Maybe (being Gentoo-specific) would be first to build your toolchain (linux-headers, glibc, binutils,and gcc) to today's versions then a "emerge -e world" so all of your stuff will be new and compatable-(your original problem). All of this can be done in the background while you keep working.                                                This is only one option. Feel free to use it or not. Sometimes I cuss the Gentoo devs for things they do that screws up my installs too.         As to your kernel question as to "default" switches in Gentoo kernel I really don't think there are. Gen-kernel is one answer, but it never worked for me, I always had to get dirty with "make menuconfig".         Hope you figure it out.  Cheers
Sent from my Verizon Wireless BlackBerry

-----Original Message-----
From: Lindsay Haisley <fmouse-gentoo@fmp.com>
Date: Mon, 21 Mar 2011 21:31:26 
To: <gentoo-desktop@lists.gentoo.org>
Reply-to: gentoo-desktop@lists.gentoo.org
Cc: Jorge Manuel B. S. Vicetto<jmbsvicetto@gentoo.org>
Subject: Re: [gentoo-desktop] System problems

On Tue, 2011-03-22 at 00:37 -0100, Jorge Manuel B. S. Vicetto wrote:
> As such, I would kindly like to ask everyone on this thread to "calm
> down" and to please try to reply in a respectful manner to any poster on
> this ml.

Thank you, Jorge.  I will do so, going forward.

> I have to agree with other posters in this ml that your tone when you
> started this thread asking for help wasn't the best one.

I was kinda freaked out.  I depend on my desktop system for so many
things in a day, and it's _way_ behind in updates, I know.  I got pushed
into a kernel update by a simple software mod, which required a perl
update, which required that I run perl-cleaner, which has issues of its
own, and somewhere along the way I ran into a dependency on a newer
version of udev, which depended on a newer kernel, which I'd tried to
install before with the same or similar problems.  I couldn't make
things work and had to resort to a rescue disk to back out.  I felt
cornered, and I guess my plea for help showed it.

I actually have enough knowledge and experience with Linux that I can
probably solve this problem myself if I approach it methodically with a
clear head.

> Thank you for the apology you've sent to the list about your tone. Let's
> try to move forward and see if / how we can help you.

I can eat crow with the best of 'em, Jorge.

> PS - Lindsay, two last notes about your issue and your time constraints.
> This type of issue might be easier to diagnose on IRC (one good place is
> #gentoo in the freenode network) where a short intense session may prove
> to be quicker in the end. Also, I haven't seen anyone mention that the
> latest udev versions react *very* badly to CONFIG_SYSFS_DEPRECATED_V2.

Ah!  Thanks for the heads-up on this.  This switch is _not_ on in my
newer kernel, and doesn't exist in my older kernel.  CONFIG_SYSFS,
however, _is_ on as I assume it should be for /sys on which udev
depends.

I don't know why udev hasn't been completely integrated into kernel
development.  It's become so much a part of modern Linux and is so
interdependent with the kernel.  Yes, it's in user space, but so are
other kernel hooks.

> So be sure to check if you disable that as iirc udev will refuse to
> create the proper device nodes if that kernel option is active.

Question:  There are switch and value dependencies in the kernel that
are Gentoo specific, or specific to system configurations favored by
Gentoo (similar to what you mentioned above).  If I emerge a Gentoo
kernel, are the default configuration options for that kernel set by
Gentoo devs so as to build a kernel which will avoid these kinds of
"gotchas"?   In other words, if I build a gentoo kernel out of the box,
as opposed to a vanilla kernel from kernel.org, do I gain an advantage
with regard to possibly problematic configuration options on a fairly
standard Gentoo system?  I hope this makes sense.

> PPS - If you decide to test our IRC support, feel free to poke me in any
> channel or to /msg me in private.

Jorge, thank you SO MUCH, for this, (and for being an adult in the
sandbox.  I wasn't at the top of my game today :-) 

-- 
Lindsay Haisley       |  "Humor will get you through times of no humor
FMP Computer Services |      better than no humor will get you through
512-259-1190          |         times of humor."
http://www.fmp.com    |            - Butch Hancock

Re: [gentoo-desktop] System problems

[Lindsay Haisley]

Thanks, eamjr56!

On Tue, 2011-03-22 at 07:43 +0000, eamjr56@gmail.com wrote:
> Now that you explained more fully the situation  and why and what led
> to your problems I understand some of your frustrations. Sometimes
> wanting to fix one thing leads to other complex issues. Maybe (being
> Gentoo-specific) would be first to build your toolchain
> (linux-headers, glibc, binutils,and gcc) to today's versions then a
> "emerge -e world" so all of your stuff will be new and
> compatable-(your original problem).

The box is scheduled for replacement, as I've noted.  The hardware is
old, relatively, and has limitations.  Additionally, Gentoo has its
limitations as a desktop environment, although it's much easier to
maintain on servers.  As Linux has to some extent penetrated the
marketplace, manufacturers are distributing drivers for things like
printers as .deb or .rpm files - no joy for Gentoo users :-(  My sweetie
bought such a printer a while back, and I can't use it for this reason.
Pulling packages apart and installing the contents manually is possible,
but not simple, and I have other things to do.

I'm reluctant to "emerge -e world" on the box because X is seriously out
of date, and I've been trying to avoid having to deal with the issue.
revdep-rebuild takes days!  And it bails if _any_ ebuild fails, so I
have to go to the office, figure out the problem, and restart the
process.  When I get the box so it'll boot properly with a more recent
kernel I can at least move in that direction.


>  All of this can be done in the background while you keep working.
> This is only one option. Feel free to use it or not. Sometimes I cuss
> the Gentoo devs for things they do that screws up my installs too.

Gentoo is substantially more complex as a distribution than are others.
Subtle screw-ups in ebuilds are more than just an annoyance on my
servers, where things break in the background after an emerge and I
don't find out until one or more customers calls me and says "**** isn't
working!"

>          As to your kernel question as to "default" switches in Gentoo
> kernel I really don't think there are. Gen-kernel is one answer, but
> it never worked for me, I always had to get dirty with "make
> menuconfig".

I tried genkernel for a while and didn't like it.  For one thing, I
recall that it installed an initrd, which introduced a level of
complexity that I didn't feel was necessary.  I use an initrd on our
SOHO gateway box since the root filesystem is managed by EVMS, but I had
to manually re-write the initrd filesystem at one point pursuant to a
glibc upgrade!  Too much work.

I assume that there must some mods to or initial setting of the default
kernel switches, somewhere, to avoid things such as enabling CONFIG_IDE
in a kernel build where it conflicts with recent releases of udev.

When I get a few moments to work on it again I'm starting with a new,
much more recent kernel and _not_ massaging my .config from a previous
kernel for it.  I assume the default settings will at very least produce
a bootable system and I need only add drivers/modules for my hardware -
NIC, sound cards, etc. to get it up and running.  I can go in and add
stuff later as needed.

>          Hope you figure it out.  Cheers

Thanks, really.  I mean it!  With a cooler head and a verified back down
plan for any changes I may make (plus the few points of light I received
on this list in spite of the misunderstandings), I'm 99% sure I can jump
in an fix the problem myself.  If I post here with it again, I'll have
my presentation of the issue MUCH better organized and thought out.

-- 
Lindsay Haisley       |"Windows .....
FMP Computer Services |    life's too short!"
512-259-1190          |
http://www.fmp.com    |              - Brad Johnston

Re: [gentoo-desktop] System problems

[Paul Hartman]

On Tue, Mar 22, 2011 at 11:35 AM, Lindsay Haisley <fmouse-gentoo@fmp.com> wrote:
> And it bails if _any_ ebuild fails, so I
> have to go to the office, figure out the problem, and restart the
> process.

With a new-enough portage you can use the --keep-going switch to cause
it to emerge everything it possibly can emerge which is not dependent
on the failed package.

If your system is new enough to support bootable CD/DVD/USB, I'd try
to find a linux live CD which runs the same kernel version that you're
hoping to switch to, and (assuming it works) look at its kernel
config, dmesg, lspci -k, that sort of thing, and try to use it as a
hint for configuring your own kernel on your hardware. You can chroot
into gentoo from the liveCD and do all the kernel compilation stuff
from there so it might be relatively painless.

Re: [gentoo-desktop] System problems

[Nicholas E. Andrade]

Quoting Paul Hartman <paul.hartman+gentoo@gmail.com>:

> On Tue, Mar 22, 2011 at 11:35 AM, Lindsay Haisley  
> <fmouse-gentoo@fmp.com> wrote:
>> And it bails if _any_ ebuild fails, so I
>> have to go to the office, figure out the problem, and restart the
>> process.
>
> With a new-enough portage you can use the --keep-going switch to cause
> it to emerge everything it possibly can emerge which is not dependent
> on the failed package.
>
> If your system is new enough to support bootable CD/DVD/USB, I'd try
> to find a linux live CD which runs the same kernel version that you're
> hoping to switch to, and (assuming it works) look at its kernel
> config, dmesg, lspci -k, that sort of thing, and try to use it as a
> hint for configuring your own kernel on your hardware. You can chroot
> into gentoo from the liveCD and do all the kernel compilation stuff
> from there so it might be relatively painless.
>

Also if you do choose to go to a kernel newer than 2.6.32, consider  
trying the "make localyesconfig" which takes a loaded, working kernel  
(e.g. from a live CD) and creates a .config based on it.   For  
additional info:  
http://kernelnewbies.org/Linux_2_6_32#head-11f54cdac41ad6150ef817fd68597554d9d05a5f




----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.

Re: [gentoo-desktop] System problems

[Lindsay Haisley]

On Tue, 2011-03-22 at 12:48 -0700, Nicholas E. Andrade wrote:
> Also if you do choose to go to a kernel newer than 2.6.32, consider  
> trying the "make localyesconfig" which takes a loaded, working kernel  
> (e.g. from a live CD) and creates a .config based on it.   For  
> additional info:  
> http://kernelnewbies.org/Linux_2_6_32#head-11f54cdac41ad6150ef817fd68597554d9d05a5f

Thanks, that's an interesting possibility.  Live CD kernels do tend to
be somewhat overloaded with features/modules since they're designed to
cover all possible systems on which they might be used.  Nonetheless,
that's good to know.  It might be useful.

-- 
Lindsay Haisley <fmouse-gentoo@fmp.com>
FMP Computer Services

Re: [gentoo-desktop] System problems

[eamjr56]

Portage now has a very useful switch that you can use "emerge --keep-going" that will not stop when a package will not compile and will continue on to the next leaving output at the end of the emerge what programs did not compile.     Cheers
Sent from my Verizon Wireless BlackBerry

-----Original Message-----
From: Lindsay Haisley <fmouse-gentoo@fmp.com>
Date: Tue, 22 Mar 2011 11:35:47 
To: <gentoo-desktop@lists.gentoo.org>
Reply-to: gentoo-desktop@lists.gentoo.org
Subject: Re: [gentoo-desktop] System problems

Thanks, eamjr56!

On Tue, 2011-03-22 at 07:43 +0000, eamjr56@gmail.com wrote:
> Now that you explained more fully the situation  and why and what led
> to your problems I understand some of your frustrations. Sometimes
> wanting to fix one thing leads to other complex issues. Maybe (being
> Gentoo-specific) would be first to build your toolchain
> (linux-headers, glibc, binutils,and gcc) to today's versions then a
> "emerge -e world" so all of your stuff will be new and
> compatable-(your original problem).

The box is scheduled for replacement, as I've noted.  The hardware is
old, relatively, and has limitations.  Additionally, Gentoo has its
limitations as a desktop environment, although it's much easier to
maintain on servers.  As Linux has to some extent penetrated the
marketplace, manufacturers are distributing drivers for things like
printers as .deb or .rpm files - no joy for Gentoo users :-(  My sweetie
bought such a printer a while back, and I can't use it for this reason.
Pulling packages apart and installing the contents manually is possible,
but not simple, and I have other things to do.

I'm reluctant to "emerge -e world" on the box because X is seriously out
of date, and I've been trying to avoid having to deal with the issue.
revdep-rebuild takes days!  And it bails if _any_ ebuild fails, so I
have to go to the office, figure out the problem, and restart the
process.  When I get the box so it'll boot properly with a more recent
kernel I can at least move in that direction.


>  All of this can be done in the background while you keep working.
> This is only one option. Feel free to use it or not. Sometimes I cuss
> the Gentoo devs for things they do that screws up my installs too.

Gentoo is substantially more complex as a distribution than are others.
Subtle screw-ups in ebuilds are more than just an annoyance on my
servers, where things break in the background after an emerge and I
don't find out until one or more customers calls me and says "**** isn't
working!"

>          As to your kernel question as to "default" switches in Gentoo
> kernel I really don't think there are. Gen-kernel is one answer, but
> it never worked for me, I always had to get dirty with "make
> menuconfig".

I tried genkernel for a while and didn't like it.  For one thing, I
recall that it installed an initrd, which introduced a level of
complexity that I didn't feel was necessary.  I use an initrd on our
SOHO gateway box since the root filesystem is managed by EVMS, but I had
to manually re-write the initrd filesystem at one point pursuant to a
glibc upgrade!  Too much work.

I assume that there must some mods to or initial setting of the default
kernel switches, somewhere, to avoid things such as enabling CONFIG_IDE
in a kernel build where it conflicts with recent releases of udev.

When I get a few moments to work on it again I'm starting with a new,
much more recent kernel and _not_ massaging my .config from a previous
kernel for it.  I assume the default settings will at very least produce
a bootable system and I need only add drivers/modules for my hardware -
NIC, sound cards, etc. to get it up and running.  I can go in and add
stuff later as needed.

>          Hope you figure it out.  Cheers

Thanks, really.  I mean it!  With a cooler head and a verified back down
plan for any changes I may make (plus the few points of light I received
on this list in spite of the misunderstandings), I'm 99% sure I can jump
in an fix the problem myself.  If I post here with it again, I'll have
my presentation of the issue MUCH better organized and thought out.

-- 
Lindsay Haisley       |"Windows .....
FMP Computer Services |    life's too short!"
512-259-1190          |
http://www.fmp.com    |              - Brad Johnston

Re: [gentoo-desktop] System problems

[Jorge Manuel B. S. Vicetto]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 22-03-2011 01:29, Lindsay Haisley wrote:
> On Tue, 2011-03-22 at 00:37 -0100, Jorge Manuel B. S. Vicetto wrote:
>> PS - Lindsay, two last notes about your issue and your time constraints.
>> This type of issue might be easier to diagnose on IRC (one good place is
>> #gentoo in the freenode network) where a short intense session may prove
>> to be quicker in the end. Also, I haven't seen anyone mention that the
>> latest udev versions react *very* badly to CONFIG_SYSFS_DEPRECATED_V2.
> 
> Ah!  Thanks for the heads-up on this.  This switch is _not_ on in my
> newer kernel, and doesn't exist in my older kernel.  CONFIG_SYSFS,
> however, _is_ on as I assume it should be for /sys on which udev
> depends.

That symbol shows up here in my 2.6.32-gentoo and 2.6.35-gentoo-r8
kernel trees. On a 2.6.37-gentoo kernel tree I see
CONFIG_SYSFS_DEPRECATED. Whatever your kernel version use, make sure you
disable it before booting into the kernel again.

> I don't know why udev hasn't been completely integrated into kernel
> development.  It's become so much a part of modern Linux and is so
> interdependent with the kernel.  Yes, it's in user space, but so are
> other kernel hooks.
> 
>> So be sure to check if you disable that as iirc udev will refuse to
>> create the proper device nodes if that kernel option is active.
> 
> Question:  There are switch and value dependencies in the kernel that
> are Gentoo specific, or specific to system configurations favored by
> Gentoo (similar to what you mentioned above).  If I emerge a Gentoo
> kernel, are the default configuration options for that kernel set by
> Gentoo devs so as to build a kernel which will avoid these kinds of
> "gotchas"?   In other words, if I build a gentoo kernel out of the box,
> as opposed to a vanilla kernel from kernel.org, do I gain an advantage
> with regard to possibly problematic configuration options on a fairly
> standard Gentoo system?  I hope this makes sense.

The vanilla-sources in Gentoo are the kernel.org tree, so they're the
same as getting a kernel version from the kernel site. The
gentoo-sources package include minimal gentoo patches for specific
features. You can read more about our patches at Mike's page[1].

 [1] - http://dev.gentoo.org/~mpagano/genpatches/

If I understood your question correctly, you were asking about the
.config options set in the kernel packages, right? My experience is that
the options enabled by default are the ones the person releasing the
kernel tested or finds more useful.
I don't think we can / should claim that the options set are necessarily
the best or the most appropriate for Gentoo or even for a general Linux
kernel. I can say I've at times wondered why certain options were active
and others disabled. Oh and fwiw, I've done an update or two to the
kernel specs of the weekly live-cds for amd64 / x86.
In any case, as I don't work in the kernel team, I'm not the best
qualified person to answer the above question. You may have more luck on
#gentoo-kernel or asking the kernel alias.
In case I misunderstood your question, please let me know and try to
explain it to me so I can try to answer it.

- -- 
Regards,

Jorge Vicetto (jmbsvicetto) - jmbsvicetto at gentoo dot org
Gentoo- forums / Userrel / Devrel / KDE / Elections / RelEng
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQIcBAEBAgAGBQJNiA+tAAoJEC8ZTXQF1qEPsrcP/1Lfq3frtjqwloHp9YAS12fF
lIJUr495G2m5fv8DKxs6ag6SpInG6ErXlAAnCaz9S9nSz42Lw8VaWbjF/aErWwP5
CTKOz4cWFDk1xlRjqA84lv7KzueDuo9cZwS6ggdwDFmtRasuTxEUqfRJUZSJqnBE
myfbrm383+A6ld+aKqbXvfO+pBnEJYVeGw3ciHtQemx9D2/iNU+GbFUgldl8Lcpn
2qs+MJirmCG2wzZXCU6TpbOmt30keTUaw0OgIXJDScu4Mgja3biuqAKXhEPSxSvn
l6kyIb4yFmUoxNcDH5R1fjGeDqmpxL7JMK99NRIkiw/8f71L9dHkn4B+YbYIKC2P
byVJXurZ6CpEYu8n+1FWg+Z+9wnxBqkaZt5XJZVc6W+4BGygv/+nuaCcWovRpH5p
IWeMI+W9J/PfPParXh270ojgw4hZDVI98tvtNS04tT77kBZ9eCC/LQDLQD9CygsY
E9R6kw5dL6IcU+NPeIi12sRSYPBnnZtgHtLxYMYpHMBtipKNlcpvodUBlKKCZOPF
8s2TS45FAyrI4LIadKdrXs+WzaNSHK3oLrh/vrisIlPiKGXe9Xqaoci+9hB2+9LH
W9PgCsJH+4hS8Bi7iUk5JB3ONdzhVfupv+i4oUx8KQoPXkFoJrD2jSFtXnwp6OT7
iB2qc2F5b41q6HEbMlRA
=55xG
-----END PGP SIGNATURE-----

Re: [gentoo-desktop] System problems - some progress

[Lindsay Haisley]

On Tue, 2011-03-22 at 00:37 -0100, Jorge Manuel B. S. Vicetto wrote:
> I haven't seen anyone mention that the
> latest udev versions react *very* badly to CONFIG_SYSFS_DEPRECATED_V2.
> So be sure to check if you disable that as iirc udev will refuse to
> create the proper device nodes if that kernel option is active.

Jorge, thanks VERY MUCH for the heads-up on CONFIG_SYSFS_DEPRECATED_V2.
Making sure that this, and CONFIG_SYSFS_DEPRECATED are off in my kernel
config has at least gotten the system to a terminal prompt - no single
user mode and no kernel panic.  Nvidia support is broken, so no X, but
that's always a hassle with new kernels and can probably be fixed,
assuming that support for my ancient Nvidia card is still available in
the portage tree and compatible with kernel 2.6.36.  I've done this many
times before and I'll deal with it later for this upgrade.

Device mapper configuration seems to be broken, which is where I'm at
now.  Kernel config settings for RAID autodetect and device mapper
support are exactly as before, but the RAID subsystem appears to be
stumbling on a device name conflict, which has prevented the LVM VG from
being constituted.

Looking into the kernel log file for the 2.6.36 boot up ...

Mar 24 12:18:53 [kernel] [    1.229464] md: Autodetecting RAID arrays.
Mar 24 12:18:53 [kernel] [    1.281388] md: Scanned 2 and added 2 devices.
Mar 24 12:18:53 [kernel] [    1.281566] md: autorun ...
Mar 24 12:18:53 [kernel] [    1.281739] md: considering sdc1 ...
Mar 24 12:18:53 [kernel] [    1.281918] md:  adding sdc1 ...
Mar 24 12:18:53 [kernel] [    1.282096] md:  adding sdb1 ...
Mar 24 12:18:53 [kernel] [    1.282273] md: created md0
Mar 24 12:18:53 [kernel] [    1.282453] md: bind<sdb1>
Mar 24 12:18:53 [kernel] [    1.282639] md: bind<sdc1>
Mar 24 12:18:53 [kernel] [    1.282826] md: running: <sdc1><sdb1>
Mar 24 12:18:53 [kernel] [    1.283219] md: personality for level 1 is not loaded!
Mar 24 12:18:53 [kernel] [    1.283401] md: do_md_run() returned -22
Mar 24 12:18:53 [kernel] [    1.283581] md: md0 still in use.
Mar 24 12:18:53 [kernel] [    1.283756] md: ... autorun DONE.

This is fubar!  There are two RAID arrays which support the missing LVM
volume group, and they're seriously out of kilter here.  The correct
configuration, booting from my old kernel, is:

$ cat /proc/mdstat
Personalities : [raid1] 
md1 : active raid1 sdc1[0] sdd1[1]
      36146112 blocks [2/2] [UU]
      
md0 : active raid1 sdb1[1] sda1[0]
      117218176 blocks [2/2] [UU]

The Linux RAID subsystem is trying to pair drives drives from two
different RAID arrays into a single md!  I see the possibility for
serious data corruption here :-(  Although it looks as if the RAID
subsystem may have bailed on the RAID array, sensing that something was
awry.

The root of this problem is that on the old kernel, there are both
a /dev/hda1 and a /dev/sda1.  The former is a partition on an old PATA
drive, while the latter is a proper component of md0, but when
everything becomes /dev/sdNx, there's an obvious conflict and the RAID
subsystem is getting confused and is obviously not seeing it's sda1.

I suspect that this is why the main VG isn't getting set up.  What can
be done about this?

-- 
Lindsay Haisley       | "Fighting against human creativity is like 
FMP Computer Services |   trying to eradicate dandelions"
512-259-1190          |    (Pamela Jones)
http://www.fmp.com    |

Re: [gentoo-desktop] System problems - some progress

[Edward Martinez]

On 03/24/11 11:16, Lindsay Haisley wrote:
> On Tue, 2011-03-22 at 00:37 -0100, Jorge Manuel B. S. Vicetto wrote:
>> I haven't seen anyone mention that the
>> latest udev versions react *very* badly to CONFIG_SYSFS_DEPRECATED_V2.
>> So be sure to check if you disable that as iirc udev will refuse to
>> create the proper device nodes if that kernel option is active.
> Jorge, thanks VERY MUCH for the heads-up on CONFIG_SYSFS_DEPRECATED_V2.
> Making sure that this, and CONFIG_SYSFS_DEPRECATED are off in my kernel
> config has at least gotten the system to a terminal prompt - no single
> user mode and no kernel panic.  Nvidia support is broken, so no X, but
> that's always a hassle with new kernels and can probably be fixed,
> assuming that support for my ancient Nvidia card is still available in
> the portage tree and compatible with kernel 2.6.36.  I've done this many
> times before and I'll deal with it later for this upgrade.
>
> Device mapper configuration seems to be broken, which is where I'm at
> now.  Kernel config settings for RAID autodetect and device mapper
> support are exactly as before, but the RAID subsystem appears to be
> stumbling on a device name conflict, which has prevented the LVM VG from
> being constituted.
>
> Looking into the kernel log file for the 2.6.36 boot up ...
>
> Mar 24 12:18:53 [kernel] [    1.229464] md: Autodetecting RAID arrays.
> Mar 24 12:18:53 [kernel] [    1.281388] md: Scanned 2 and added 2 devices.
> Mar 24 12:18:53 [kernel] [    1.281566] md: autorun ...
> Mar 24 12:18:53 [kernel] [    1.281739] md: considering sdc1 ...
> Mar 24 12:18:53 [kernel] [    1.281918] md:  adding sdc1 ...
> Mar 24 12:18:53 [kernel] [    1.282096] md:  adding sdb1 ...
> Mar 24 12:18:53 [kernel] [    1.282273] md: created md0
> Mar 24 12:18:53 [kernel] [    1.282453] md: bind<sdb1>
> Mar 24 12:18:53 [kernel] [    1.282639] md: bind<sdc1>
> Mar 24 12:18:53 [kernel] [    1.282826] md: running:<sdc1><sdb1>
> Mar 24 12:18:53 [kernel] [    1.283219] md: personality for level 1 is not loaded!
> Mar 24 12:18:53 [kernel] [    1.283401] md: do_md_run() returned -22
> Mar 24 12:18:53 [kernel] [    1.283581] md: md0 still in use.
> Mar 24 12:18:53 [kernel] [    1.283756] md: ... autorun DONE.
>
> This is fubar!  There are two RAID arrays which support the missing LVM
> volume group, and they're seriously out of kilter here.  The correct
> configuration, booting from my old kernel, is:
>
> $ cat /proc/mdstat
> Personalities : [raid1]
> md1 : active raid1 sdc1[0] sdd1[1]
>        36146112 blocks [2/2] [UU]
>
> md0 : active raid1 sdb1[1] sda1[0]
>        117218176 blocks [2/2] [UU]
>
> The Linux RAID subsystem is trying to pair drives drives from two
> different RAID arrays into a single md!  I see the possibility for
> serious data corruption here :-(  Although it looks as if the RAID
> subsystem may have bailed on the RAID array, sensing that something was
> awry.
>
> The root of this problem is that on the old kernel, there are both
> a /dev/hda1 and a /dev/sda1.  The former is a partition on an old PATA
> drive, while the latter is a proper component of md0, but when
> everything becomes /dev/sdNx, there's an obvious conflict and the RAID
> subsystem is getting confused and is obviously not seeing it's sda1.
>
> I suspect that this is why the main VG isn't getting set up.  What can
> be done about this?
>
    Hi,

     I think it would be faster with less headaches  by reinstalling 
Gentoo from the latest  stage3 and portage:-)

Re: [gentoo-desktop] System problems - some progress

[Lindsay Haisley]

On Thu, 2011-03-24 at 12:15 -0700, Edward Martinez wrote:
>      I think it would be faster with less headaches  by reinstalling 
> Gentoo from the latest  stage3 and portage:-)

Possibly, but that's unlikely to make this problem go away since it
appears to come from a device naming conflict.  This is inherent in the
hardware setup.  I've inquired elsewhere regarding moving this matter to
some other list where the participants are more versed in Linux kernel
and system issues.  This list seems to have been a poor choice for this
thread, since kernel issues aren't the specialty of the participants,
although several people have been helpful.

The problem should be soluble without reloading the OS.  I've made some
progress.  If I can solve the dev name conflict issue and get my RAID
arrays reconstituted, then I can deal with the video card issues and I'm
home free :-)  I'm sure the naming conflict issue has come up before,
for other people, and someone, somewhere has a solution for it.

Ultimately the path forward will be to build a new desktop box, which
probably won't be running Gentoo.  It'll have a much more sane disk
configuration, and won't be subject to other legacy hardware issues
(which I haven't discussed here) than this one is.  I'd like to make one
more push with this box, though, to get the immediate problem resolved.

-- 
Lindsay Haisley       | "The difference between a duck is because
FMP Computer Services |    one leg is both the same"
512-259-1190          |     - Anonymous
http://www.fmp.com    |

Re: [gentoo-desktop] System problems - some progress

[Edward Martinez]

[-- Attachment #1: Type: text/plain, Size: 880 bytes --]

On 03/24/11 12:44, Lindsay Haisley wrote:
> I can deal with the video card issues and I'm
> home free
    Cool,:-) are you aware that the nvidia kernel module needs to be 
reinstall every time a new linux kernel or current kernel is compiled?

     code:*Important: * Every time you compile a new kernel 
<http://www.gentoo.org/doc/en/kernel-upgrade.xml> or recompile the 
current one, you will need to reinstall the nVidia kernel modules. An 
easy way to keep track of modules installed by ebuilds (such as 
nvidia-drivers) is to install sys-kernel/module-rebuild. Once you've 
installed it, simply run module-rebuild populate to populate its 
database with a list of packages to be rebuilt. Once you've finished 
compiling or recompiling a kernel, just run module-rebuild rebuild to 
rebuild the drivers for your new kernel./code


http://www.gentoo.org/doc/en/nvidia-guide.xml

[-- Attachment #2: Type: text/html, Size: 1710 bytes --]

Re: [gentoo-desktop] System problems - some progress

[Lindsay Haisley]

On Thu, 2011-03-24 at 13:17 -0700, Edward Martinez wrote:
>    Cool, :-) are you aware that the nvidia kernel module needs to be
> reinstall every time a new linux kernel or current kernel is compiled?
> 
>     code: Important: Every time you compile a new kernel or recompile
> the current one, you will need to reinstall the nVidia kernel modules.
> An easy way to keep track of modules installed by ebuilds (such as
> nvidia-drivers) is to install sys-kernel/module-rebuild. Once you've
> installed it, simply run module-rebuild populate to populate its
> database with a list of packages to be rebuilt. Once you've finished
> compiling or recompiling a kernel, just run module-rebuild rebuild to
> rebuild the drivers for your new kernel./code

Thanks, Edward.  Been there, done that, bought the T-shirt (several
times :-).  The legacy nVidia stuff is a hassle, and I'm looking forward
to dumping it on the new desktop I'm building.

-- 
Lindsay Haisley       | SUPPORT NETWORK NEUTRALITY
FMP Computer Services | --------------------------
512-259-1190          | Boycott Yahoo, RoadRunner, AOL
http://www.fmp.com    | and Verison

Re: [gentoo-desktop] System problems - some progress

[Paul Hartman]

On Thu, Mar 24, 2011 at 1:16 PM, Lindsay Haisley <fmouse-gentoo@fmp.com> wrote:
> The root of this problem is that on the old kernel, there are both
> a /dev/hda1 and a /dev/sda1.  The former is a partition on an old PATA
> drive, while the latter is a proper component of md0, but when
> everything becomes /dev/sdNx, there's an obvious conflict and the RAID
> subsystem is getting confused and is obviously not seeing it's sda1.

Possible alternative is to disable raid autodetection and define the
arrays by UUID in /etc/mdadm.conf so hopefully the device names become
irrelevant at that point.

Re: [gentoo-desktop] System problems - some progress

[Lindsay Haisley]

On Thu, 2011-03-24 at 15:15 -0500, Paul Hartman wrote:
> On Thu, Mar 24, 2011 at 1:16 PM, Lindsay Haisley <fmouse-gentoo@fmp.com> wrote:
> > The root of this problem is that on the old kernel, there are both
> > a /dev/hda1 and a /dev/sda1.  The former is a partition on an old PATA
> > drive, while the latter is a proper component of md0, but when
> > everything becomes /dev/sdNx, there's an obvious conflict and the RAID
> > subsystem is getting confused and is obviously not seeing it's sda1.
> 
> Possible alternative is to disable raid autodetection and define the
> arrays by UUID in /etc/mdadm.conf so hopefully the device names become
> irrelevant at that point.

This is a good idea.  I can turn off RAID autodetection in the kernel
config and spec RAID1 instead, since the root fs isn't on a RAID array.

I've found a number of references to putting UUIDs in ARRAY lines
in /etc/mdadm.conf to define the UUID of an array, but none yet to using
UUID specs in DEVICE lines, all of which I've found so far in the online
literature use /dev/xxxx specs.  Before I take this step I'm going to
find a more kernel-specific list and ask if this would be appropriate.
I've tripped on RAID array errors before at the expense of days of work
to reconstitute systems and their data.  I want to make sure this is
kosher before I go there.

Thanks! 

-- 
Lindsay Haisley       | "In an open world,    |     PGP public key
FMP Computer Services |    who needs Windows  |      available at
512-259-1190          |      or Gates"        | http://pubkeys.fmp.com
http://www.fmp.com    |                       |

Re: [gentoo-desktop] System problems - some progress

[Paul Hartman]

On Thu, Mar 24, 2011 at 3:38 PM, Lindsay Haisley <fmouse-gentoo@fmp.com> wrote:
> On Thu, 2011-03-24 at 15:15 -0500, Paul Hartman wrote:
>> On Thu, Mar 24, 2011 at 1:16 PM, Lindsay Haisley <fmouse-gentoo@fmp.com> wrote:
>> > The root of this problem is that on the old kernel, there are both
>> > a /dev/hda1 and a /dev/sda1.  The former is a partition on an old PATA
>> > drive, while the latter is a proper component of md0, but when
>> > everything becomes /dev/sdNx, there's an obvious conflict and the RAID
>> > subsystem is getting confused and is obviously not seeing it's sda1.
>>
>> Possible alternative is to disable raid autodetection and define the
>> arrays by UUID in /etc/mdadm.conf so hopefully the device names become
>> irrelevant at that point.
>
> This is a good idea.  I can turn off RAID autodetection in the kernel
> config and spec RAID1 instead, since the root fs isn't on a RAID array.
>
> I've found a number of references to putting UUIDs in ARRAY lines
> in /etc/mdadm.conf to define the UUID of an array, but none yet to using
> UUID specs in DEVICE lines, all of which I've found so far in the online
> literature use /dev/xxxx specs.  Before I take this step I'm going to
> find a more kernel-specific list and ask if this would be appropriate.
> I've tripped on RAID array errors before at the expense of days of work
> to reconstitute systems and their data.  I want to make sure this is
> kosher before I go there.

I was actually referring to the ARRAY lines and the array UUIDs. In
fact I don't even have a DEVICE line, man mdadm.conf says:
If no DEVICE line is present, then "DEVICE partitions containers" is assumed.

My mdadm.conf only contains 2 ARRAY lines, for my 2 raid arrays. I
also specify the metadata version, I assume you're using superblock
0.90 since you've been using autodetect and autodetect isn't supported
for newer versions.

So, mdadm scans all partitions (doesn't matter what they are named)
looking for superblocks containing the UUID of the arrays I specified.
Anything that doesn't match gets ignored for this purpose.

The mdadm manpage has this example command:

 mdadm --examine --brief --scan --config=partitions
Create a list of devices by reading /proc/partitions, scan these for
RAID superblocks, and printout a  brief listing of all that were
found.

Hopefully you can find your array UUIDs with that command (and if it
finds them, that's a good sign for it's ability to assemble the arrays
once the config file is made)

Good luck :)

Re: [gentoo-desktop] System problems - some progress

[Lindsay Haisley]

Thanks, Paul.

On Thu, 2011-03-24 at 16:42 -0500, Paul Hartman wrote:
> I was actually referring to the ARRAY lines and the array UUIDs. In
> fact I don't even have a DEVICE line, man mdadm.conf says:
> If no DEVICE line is present, then "DEVICE partitions containers" is
> assumed.
> 
> My mdadm.conf only contains 2 ARRAY lines, for my 2 raid arrays. I
> also specify the metadata version, I assume you're using superblock
> 0.90 since you've been using autodetect and autodetect isn't supported
> for newer versions.

Newer versions?  Kernel 2.6.36 has a config option for RAID autodetect.
What are you referring to here, mdadm?

mdadm is at 2.6.8 on this box.  If I upgrade to v3.1.4 will I lose the
ability to autodetect the arrays, on which the system depends even on
the 2.6.23 kernel on which I'm currently depending?

> So, mdadm scans all partitions (doesn't matter what they are named)
> looking for superblocks containing the UUID of the arrays I specified.
> Anything that doesn't match gets ignored for this purpose.

> The mdadm manpage has this example command:

>  mdadm --examine --brief --scan --config=partitions

So I get:

# mdadm --examine --brief --scan --config=partitions
ARRAY /dev/md0 level=raid1 num-devices=2 UUID=d3176595:06cb3677:46406ca7:d12d146f
ARRAY /dev/md1 level=raid1 num-devices=2 UUID=9463a434:24dbfcb6:a25ffb08:d8ab7c18

... which is what I would expect.

Does this mean that the UUID of the _array_ has been pushed onto the
component drives?  If so, why does the RAID assembly fail so miserably
with kernel 2.6.36?  I'm lost here.  It looks to me, from the boot log,
as if the problem is that there are _two_ partitions named /dev/sda1 and
the RAID subsystem can't see the one that's a component of
md0.  /etc/mdadm.conf contains:

DEVICE /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1
ARRAY /dev/md0 devices=/dev/sda1,/dev/sdb1
ARRAY /dev/md1 devices=/dev/sdc1,/dev/sdd1

> Create a list of devices by reading /proc/partitions, scan these for
> RAID superblocks, and printout a  brief listing of all that were
> found.

This gives me the UUIDs of the arrays, but my question here is whether I
can spec the component devices using UUIDs, and I'm not finding any
clear guidance on that.  The mdadm man page talks about the former, but
doesn't mention the latter.  In other words, can I put into mdadm.conf a
line such as the following:

ARRAY /dev/md0 devices=UUID=d3176595-06cb-3677-4640-6ca7d12d146f,UUID=d3176595-06cb-3677-4640-6ca7d12d146f

> Hopefully you can find your array UUIDs with that command (and if it
> finds them, that's a good sign for it's ability to assemble the arrays
> once the config file is made)

Finding the ARRAY UUIDs isn't the problem, it's assigning the array
components using _their_ respective UUIDs.  If I can do this, the
problem may be solved.

I don't know that this will work, I don't know that it won't.  I have
everything on the arrays, and the LVMs built on them, backed up.  I
probably should just try it and back out of it if it doesn't, since I
don't see any potential for data loss if it fails, in which case the
RAID arrays simply won't be built and I'll be dumped into the workable
but not very useful non-RAID configuration.

-- 
Lindsay Haisley       | "The difference between a duck is because
FMP Computer Services |    one leg is both the same"
512-259-1190          |     - Anonymous
http://www.fmp.com    |

Re: [gentoo-desktop] System problems - some progress

[Lindsay Haisley]

On Thu, 2011-03-24 at 17:33 -0500, Lindsay Haisley wrote:
> It looks to me, from the boot log,
> as if the problem is that there are _two_ partitions named /dev/sda1 and
> the RAID subsystem can't see the one that's a component of
> md0.  /etc/mdadm.conf contains:
> 
> DEVICE /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1
> ARRAY /dev/md0 devices=/dev/sda1,/dev/sdb1
> ARRAY /dev/md1 devices=/dev/sdc1,/dev/sdd1

@Paul, Ah, I see!

The component drives in a RAID-1 array have the _same_ UUID, so I would
assume that a line in /etc/mdadm.conf such as:

ARRAY /dev/md0 UUID=d3176595:06cb3677:46406ca7:d12d146f

would identify _both_ component drives.  This is what the output of
mdadm --examine --brief --scan --config=partitions would imply.

I'll try this.

I'm not fond of UUID's.  They're hard to read and impossible to copy by
hand without making mistakes!

-- 
Lindsay Haisley       | "We have met the enemy, and it is us."
FMP Computer Services |
512-259-1190          |          -- Pogo
http://www.fmp.com    |

Re: [gentoo-desktop] System problems - some progress

[Paul Hartman]

On Thu, Mar 24, 2011 at 5:33 PM, Lindsay Haisley <fmouse-gentoo@fmp.com> wrote:
> Newer versions?  Kernel 2.6.36 has a config option for RAID autodetect.
> What are you referring to here, mdadm?

Even the newest kernel supports autodetect, but autodetect only works
with a specific kind of RAID superblock, I think version 0.90.
Different versions of mdadm create arrays with different versions of
superblock by default. Newer versions of superblocks cannot
(presently) be autodetected by the kernel, so anyone using a newer
type of superblock will have to do the "manual" config like this
anyway.

As for why it's not working in your case, I really don't know, but
hopefully you can at least get it working /somehow/ so that you can
use your system normally to get real work done, and can investigate
why auto-detect doesn't work the way you'd like it to with less
urgency. I've got an old Gentoo system that takes days to update, but
if the system is usable during that time it's not really a big deal to
me. It's the days-long updates when the system is in an unusable state
that are a real nightmare.

> @Paul, Ah, I see!
>
> The component drives in a RAID-1 array have the _same_ UUID, so I would
> assume that a line in /etc/mdadm.conf such as:
>
> ARRAY /dev/md0 UUID=d3176595:06cb3677:46406ca7:d12d146f

Right, exactly. Sorry I didn't make it clear before.

I consider it somewhat of a miracle that I ever got any of it working
on my computer in the first place, so I'm definitely speaking from an
"as far as I know" point of view here. It's something I set up when
building the computer and never had to think about it again.

Re: [gentoo-desktop] System problems - some progress

[Lindsay Haisley]

On Thu, 2011-03-24 at 18:20 -0500, Paul Hartman wrote:
> On Thu, Mar 24, 2011 at 5:33 PM, Lindsay Haisley <fmouse-gentoo@fmp.com> wrote:
> > Newer versions?  Kernel 2.6.36 has a config option for RAID autodetect.
> > What are you referring to here, mdadm?
> 
> Even the newest kernel supports autodetect, but autodetect only works
> with a specific kind of RAID superblock, I think version 0.90.
> Different versions of mdadm create arrays with different versions of
> superblock by default. Newer versions of superblocks cannot
> (presently) be autodetected by the kernel, so anyone using a newer
> type of superblock will have to do the "manual" config like this
> anyway.

Ah. So it follows that if the array was created with an earlier version
of mdadm, and mdadm -D tells me that the superblock is persistent and is
version 0.90 then autodetection should work.  It would also follow that
if I turn off RAID autodetection in the kernel, and spec'd ARRAYs by
UUID in /etc/mdadm.conf, I should be OK.

> As for why it's not working in your case, I really don't know, but
> hopefully you can at least get it working /somehow/ so that you can
> use your system normally to get real work done, and can investigate
> why auto-detect doesn't work the way you'd like it to with less
> urgency. 

If I can't, it's not the end of the world, since I can just let it be
and build up a new box and move stuff to it.  I need to emerge -u mdadm
since I'm currently at v2.6.8 and the portage tree recommends v3.1.4.  I
need to really make sure that this upgrade will work, since, unlike
udev-141, I can't back-version if the newer mdadm causes a problem.

> I've got an old Gentoo system that takes days to update, but
> if the system is usable during that time it's not really a big deal to
> me. It's the days-long updates when the system is in an unusable state
> that are a real nightmare.

Yeah, there's some brilliant programming in Gentoo, and I really like
the concept of what Duncan calls the "rolling upgrade" design
philosophy, but it's a slow and complex process.  I'd rather deal with a
fixed version distribution these days and let others deal with the
builds.

-- 
Lindsay Haisley       |  "We are all broken toasters, but we still
FMP Computer Services |    manage to make toast"
512-259-1190          |
http://www.fmp.com    |        - Cheryl Dehut
                      |

Re: [gentoo-desktop] System problems - some progress

[Lindsay Haisley]

On Thu, 2011-03-24 at 18:20 -0500, Paul Hartman wrote:
> As for why it's not working in your case, I really don't know, but
> hopefully you can at least get it working /somehow/ so that you can
> use your system normally to get real work done, and can investigate
> why auto-detect doesn't work the way you'd like it to with less
> urgency.

A colleague of mine here in Austin pointed out to me that although I had
autodetect enabled in my kernel, I didn't have RAID-1 mode enabled.  He
figured this out from looking at my log file excerpt!  I thought I had
the necessary kernel config options copied from my old kernel to my new
one, but this one was overlooked.  Another pass at it will be in order.

-- 
Lindsay Haisley       |"Windows .....
FMP Computer Services |    life's too short!"
512-259-1190          |
http://www.fmp.com    |              - Brad Johnston

[gentoo-desktop] Re: System problems - some progress

[Duncan]

Lindsay Haisley posted on Thu, 24 Mar 2011 21:10:13 -0500 as excerpted:

> A colleague of mine here in Austin pointed out to me that although I had
> autodetect enabled in my kernel, I didn't have RAID-1 mode enabled.  He
> figured this out from looking at my log file excerpt!  I thought I had
> the necessary kernel config options copied from my old kernel to my new
> one, but this one was overlooked.  Another pass at it will be in order.

I noticed that immediately too ("personality for level 1 is not loaded", 
dead give-away!), and was going to post a response to that effect, but 
decided to check the rest of the thread in case someone else got to it 
first.

You (your colleague) got to it before I did! =:^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-desktop] Re: System problems - some progress

[Lindsay Haisley]

On Fri, 2011-03-25 at 08:57 +0000, Duncan wrote:
> Lindsay Haisley posted on Thu, 24 Mar 2011 21:10:13 -0500 as excerpted:
> 
> > A colleague of mine here in Austin pointed out to me that although I had
> > autodetect enabled in my kernel, I didn't have RAID-1 mode enabled.  He
> > figured this out from looking at my log file excerpt!  I thought I had
> > the necessary kernel config options copied from my old kernel to my new
> > one, but this one was overlooked.  Another pass at it will be in order.
> 
> I noticed that immediately too ("personality for level 1 is not loaded", 
> dead give-away!), and was going to post a response to that effect, but 
> decided to check the rest of the thread in case someone else got to it 
> first.
> 
> You (your colleague) got to it before I did! =:^)

Yeah, I missed it the first time around.

I posted to linuxforums.org and referenced the thread to the Central TX
LUG list, and Wayne Walker, one of our members, jumped on and saw it
right away.  We have some very smart and Linux-savvy folks here in
Austin!  The CTLUG tech list is really good.  We have IBM, AMD, Dell,
and a bunch of other tech companies in the area and the level of Linux
tech expertise here is exceptional.  You'd be welcome to join the list
if you're interested.  See <http://www.ctlug.org>.  We have people on
the list from all over the world.

I expect the LVM system will come up now, and it only remains to be seen
if I can get the legacy nVidia driver to build.

-- 
Lindsay Haisley       |"What if the Hokey Pokey really IS all it  
FMP Computer Services |    really is about?"
512-259-1190          |
http://www.fmp.com    |          -- Jimmy Buffett

[gentoo-desktop] Re: System problems - some progress

[Duncan]

Lindsay Haisley posted on Fri, 25 Mar 2011 07:48:12 -0500 as excerpted:

> Yeah, I missed it the first time around.
> 
> I posted to linuxforums.org and referenced the thread to the Central TX
> LUG list, and Wayne Walker, one of our members, jumped on and saw it
> right away.  We have some very smart and Linux-savvy folks here in
> Austin!  The CTLUG tech list is really good.  We have IBM, AMD, Dell,
> and a bunch of other tech companies in the area and the level of Linux
> tech expertise here is exceptional.  You'd be welcome to join the list
> if you're interested.  See <http://www.ctlug.org>.  We have people on
> the list from all over the world.
> 
> I expect the LVM system will come up now, and it only remains to be seen
> if I can get the legacy nVidia driver to build.

This might be a bit more of the "I don't want to hear it" stuff, which you 
can ignore if so, but for your /next/ system, consider the following, 
speaking from my own experience...

I ran LVM(2) on md-RAID here for awhile, but ultimately decided that the 
case of lvm on top of md-raid was too complex to get my head around well 
enough to be reasonably sure of recovery in the event of a problem.

Originally, I (thought I) needed lvm on top because md-raid didn't support 
partitioned-RAID all that well.  I migrated to that setup just after what 
was originally separate support for mdp, partitioned md-raid, was 
introduced, and the documentation for it was scarce indeed!  But md-raid's 
support for partitions has VASTLY improved, as has the documentation, with 
partitions now supported just fine on ordinary md-raid, making the 
separate mdp legacy.

So at some point I backed everything up and reorganized, cutting out the 
lvm.  Now I simply run partitioned md-raid.  Note that here, I have / on 
md-raid as well.  That was one of the problems with lvm, I could put / on 
md-raid and even have /boot on md-raid as long as it was RAID-1, but lvm 
requires userspace, so either / had to be managed separately, or I had to 
run an initrd/initramfs to manage the early userspace and do a pivot_root 
to my real / after lvm had brought it up.  I was doing the former, not 
putting / on lvm, but that defeated much of the purpose for me as now I 
was missing out on the flexibility of lvm for my / and root-backup 
partitions!

So especially now that partitioned md-raid is well supported and 
documented, you may wish to consider dropping the lvm layer, thus avoiding 
the complexity of having to recover both the md-raid and the lvm if 
something goes wrong, with the non-zero chance of admin-flubbing the 
recovery increasing dramatically due to the extra complexity of additional 
layers and having to keep straight which commands to run at each layer, in 
an emergency situation when you're already under pressure because things 
aren't working!

Here, I decided that extra layer was simply NOT worth the extra worry and 
hassle in a serious recovery scenario, and I'm glad I did.  I'm *FAR* more 
confident in my ability to recover from disaster now than I was before, 
because I can actually get my head around the whole, not just a step at a 
time, and thus am FAR less likely to screw things up with a stupid fat-
finger mistake.

But YMMV, as they say.  Given that the capacities of LVM2 have been 
improving as well (including its own RAID support, in some cases sharing 
code with md-raid), AND the fact that the device-mapper services (now part 
of the lvm2 package on the userspace side) used by lvm2 are now used by 
udisks and etc, the replacements for hal for removable disk detection and 
automounting, etc, switching to lvm exclusively instead of md-raid 
exclusively, is another option.  Of course lvm still requires userspace 
while md-raid doesn't, so it's a tradeoff of initr* if you put / on it 
too, vs using the same device-mapper technology for both lvm and udisks/
auto-mount.

There's a third choice as well, or soon will be, as the technology is 
available but still immature.  btrfs has built-in raid support (as with 
lvm2, sharing code at the kernel level, where it makes sense).  The two 
biggest advantages to btrfs are that (1) it's the designated successor to 
ext2/3/4 and will thus be EXTREMELY well supported when it matures, AND 
(2) because it's a filesystem as well, (2a) you're dealing with just the 
one (multi-faceted) technology, AND (2b) it knows what's valuable data and 
what's not, so recoveries are shorter because it doesn't have to deal with 
"empty" space, like md-raid does because it's on a layer of its own, not 
knowing what's valuable data and what's simply empty space.  The biggest 
disadvantage of course is that btrfs isn't yet mature.  In particular (1) 
the on-disk format isn't officially cast in stone yet (tho changes now are 
backward compatible, so you should have no trouble loading older btrfs 
with newer kernels, but might not be able to mount it with the older 
kernel once you do, if there was a change, AFAIK there have been two disk 
format changes so far, one with the no-old-kernels restriction, the latest 
without, as long as the filesystem was created with the older kernel), AND 
(2) as of now there's not yet a proper fsck.btrfs, tho that's currently 
very high priority and there very likely will be one within months, within 
a kernel or two, so likely available for 2.6.40.

Booting btrfs can be a problem currently as well.  As you may well know, 
grub-1 (0.9x) is officially legacy and hasn't had any official new 
features for years, /despite/ the fact that last I knew, grub2's on-disk 
format wasn't set in stone either.  However, being GPLv2-ed, the various 
distributions have been applying feature patches to bring it upto date for 
years, including a number of patches used routinely for ext2/3 
filesystems.  There is a grub-1 patch adding btrfs support, but I'm not 
sure whether it's in gentoo's version yet or not.  (Of course, that 
wouldn't be an issue for your new system as you mentioned it probably 
won't be gentoo-based anyway, but others will be reading this too.)

The newer grub-2 that many distributions are now using (despite the fact 
that it's still immature) has a btrfs patch as well.  However, there's an 
additional complication there as grub-2 is GPLv3, while the kernel and 
thus btrfs is GPLv2, specifically /without/ the "or later version" 
clause.  The existing grub-2 btrfs support patch is said to have worked 
around that thru reverse engineering, etc, but that is an issue for 
further updates, given that btrfs' on-disk-format is NOT yet declared 
final.  Surely the issue will eventually be resolved, but these are the 
sorts of "teething problems" that the immature btrfs is having, as it 
matures.

The other aspect of booting to btrfs that I've not yet seen covered in any 
detail is the extent to which "advanced" btrfs features such as built-in 
RAID and extensible sub-volumes will be boot-supported.  It's quite 
possible that only a quite basic and limited btrfs will be supported for 
/boot, with advanced features only supported from the kernel (thus on /) 
or even, possibly, userspace (thus not on / without an initr*).

Meanwhile, btrfs is already the default for some distributions despite all 
the issues.  How they can justify that even without a proper fsck.btrfs 
and without official on-disk format lock-down, among other things, I don't 
know, but anyway...  I believe at present, most of them are using 
something else (ext2 or even vfat) for /boot, tho, thus eliminating the 
grub/btrfs issues.

But I do believe 2011 is the year for btrfs, and by year-end (or say the 
first 2012 kernel, so a year from now, leaving a bit more wiggle room), 
the on-disk format will be nailed-down, a working fsck.btrfs will be 
available, and the boot problems solved to a large extent.  With those 
three issues gone, people will be starting the mass migration, altho 
conservative users will remain on ext3/ext4 for quite some time, just as 
many haven't yet adopted ext4, today.  (FWIW, I'm on reiserfs, and plan on 
staying there until I can move to btrfs and take advantage of both its 
tail-packing and built-in RAID support, so the ext3/ext4 thing doesn't 
affect me that much.)

That leaves the two choices for now now, md-raid and lvm2 including its 
raid features, with btrfs as a maturing third choice, likely reasonable by 
year-end.  Each has its strong and weak points and I'd recommend 
evaluating the three and making one's choice of /one/ of them.  By 
avoiding layering one on the other, significant complexity is avoided, 
simplifying both routine administration and disaster recovery, with the 
latter a BIG factor since it reduces by no small factor the chance of 
screwing things up /in/ that recovery.

Simply my experience-educated opinion.  YMMV, as they say.  And of course, 
it applies to new installations more than your current situation, but as 
you mentioned that you are planning such a new installation...

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-desktop] Re: System problems - some progress

[Lindsay Haisley]

On Fri, 2011-03-25 at 22:59 +0000, Duncan wrote:
> Simply my experience-educated opinion.  YMMV, as they say.  And of course, 
> it applies to new installations more than your current situation, but as 
> you mentioned that you are planning such a new installation...

Duncan, thanks for your very thorough discussion of current technologies
disk/RAID/filesystem/etc. technologies.  Wow!  I'm going to have to read
it through several times to absorb it.  I've gotten to the point at
which I'm more involved with what I can _do_ with the Linux boxes I set
up than what I can do that's cool and cutting edge with Linux in setting
them up, but playing with bleeding-edge stuff has always been tempting.
Some of the stuff you've mentioned, such as btrfs, are totally new to me
since I haven't kept up with the state of the art.  Some years ago we
had EVMS, which was developed by IBM here in Austin.  I was a member of
the Capital Area Central Texas UNIX Society (CACTUS) and we had the EVMS
developers come and talk about it.  EVMS was great.  It was a layered
technology with an API for a management client, so you could have a CLI,
a GUI, a web-based management client, whatever, and a all of them useing
the same API to the disk management layer.  It was an umbrella
technology which covered several levels of Linux MD Raid plus LVM.  You
could put fundamental storage elements together like tinker-toys and
slice and dice them any way you wanted to.

EVMS was started from an initrd, which set up the EVMS platform and then
did a pivot_root to the EVMS-supported result.  I have our SOHO
firewall/gateway and file server set up with it.  The root fs is on a
Linux MD RAID-1 array, and what's on top of that I've rather forgotten
but the result is a drive and partition layout that makes sense for the
purpose of the box.  I set this up as a kind of proof-of-concept
exercise because I was taken with EVMS and figured it would be useful,
which it was.  The down side of this was that some time after that, IBM
dropped support for the EVMS project and pulled their developers off of
it.  I was impressed with the fact that IBM was actually paying people
to develop open source stuff, but when they pulled the plug on it, EVMS
became an orphaned project.  The firewall/gateway box runs Gentoo, so I
proceeded with regular updates until one day the box stopped booting.
The libraries, notably glibc, embedded in the initrd system got out of
sync, version wise, with the rest of the system, and I was getting some
severely strange errors early in the boot process followed by a kernel
panic.  It took a bit of work to even _see_ the errors, since they were
emitted in the boot process earlier than CLI scroll-back comes into
play, and then there was further research to determine what I needed to
do to fix the problem.  I ended up having to mount and then manually
repair the initrd internal filesystem, manually reconstituting library
symlinks as required.

I've built some Linux boxes for one of my clients - 1U servers and the
like.  These folks are pretty straight-forward in their requirements,
mainly asking that the boxes just work.  The really creative work goes
into the medical research PHP application that lives on the boxes, and
I've learned beaucoup stuff about OOP in PHP, AJAX, etc. from the main
programming fellow on the project.  We've standardized on Ubuntu server
edition on SuperMicro 4 drive 1U boxes.  These boxes generally come with
RAID supported by a proprietary chipset or two, which never works quite
right with Linux, so the first job I always do with these is to rip out
the SATA cabling from the back plane and replace the on-board RAID with
an LSI 3ware card.  These cards don't mess around - they JUST WORK.
LSI/3ware has been very good about supporting Linux for their products.
We generally set these up as RAID-5 boxes.  There's a web-based
monitoring daemon for Linux that comes with the card, and it just works,
too, although it takes a bit of dickering.  The RAID has no component in
user-space (except for the monitor daemon) and shows up as a single SCSI
drive, which can be partitioned and formatted just as if it were a
single drive.  The 3ware cards are nice!  If you're using a redundant
array such as RAID-1 or RAID-5, you can designate a drive as a hot-spare
and if one of the drives in an array fails, the card will fail over to
the hot-spare, rebuild the array, and the monitor daemon will send you
an email telling you that it happened.  Slick!

The LSI 3ware cards aren't cheap, but they're not unreasonable either,
and I've never had one fail.  I'm thinking that the my drive setup on my
new desktop box will probably use RAID-1 supported by a 3ware card.
I'll probably use an ext3 filesystem on the partitions.  I know ext4 is
under development, but I'm not sure if it would offer me any advantages.
I used reiserfs on some of the partitions on my servers, and on some
partitions on my desktop box too.  Big mistake!  There was a bug in
reiserfs support the current kernel when I built the first server and
the kernel crapped all over the hard drive one night and the box
crashed!  I was able to fix it and salvage customer data, but it was
pretty scary.  Hans Reiser is in prison for life for murder, and there's
like one person on the Linux kernel development group who maintains
reiserfs.  Ext3/4, on the other hand, is solid, maybe not quite as fast,
but supported by a dedicated group of developers.

So I'm thinking that this new box will have a couple of
professional-grade (the 5-year warranty type) 1.5 or 2 TB drives and a
3ware card.  I still haven't settled on the mainboard, which will have
to support the 3ware card, a couple of sound cards and a legacy Adaptec
SCSI card for our ancient but extremely well-built HP scanner.  The
chipset will have to be well supported in Linux.  I'll probably build
the box myself once I decide on the hardware.

I'm gonna apply the KISS principle to the OS design for this, and stay
away from bleeding edge software technologies, although, especially
after reading your essay, it's very tempting to try some of this stuff
out to see what the the _really_ smart people are coming up with!  I'm
getting off of the Linux state-of-the art train for a while and go
walking in the woods.  The kernel will have to be low-latency since I
may use the box for recording work with Jack and Ardour2, and almost
certainly for audio editing, and maybe video editing at some point.
That's where my energy is going to go for this one.

-- 
Lindsay Haisley       |"Windows .....
FMP Computer Services |    life's too short!"
512-259-1190          |
http://www.fmp.com    |              - Brad Johnston

[gentoo-desktop] Re: System problems - some progress

[Duncan]

Lindsay Haisley posted on Fri, 25 Mar 2011 21:46:32 -0500 as excerpted:

> On Fri, 2011-03-25 at 22:59 +0000, Duncan wrote:
>> Simply my experience-educated opinion.  YMMV, as they say.  And of
>> course,
>> it applies to new installations more than your current situation, but
>> as you mentioned that you are planning such a new installation...
> 
> Duncan, thanks for your very thorough discussion of current technologies
> disk/RAID/filesystem/etc. technologies.  Wow!  I'm going to have to read
> it through several times to absorb it.  I've gotten to the point at
> which I'm more involved with what I can _do_ with the Linux boxes I set
> up than what I can do that's cool and cutting edge with Linux in setting
> them up, but playing with bleeding-edge stuff has always been tempting.

By contrast, Linux is still my hobby, tho really, a full time one in that 
I spend hours a day at it, pretty much 7 days a week.  I'm thinking I 
might switch to Linux as a job at some point, perhaps soon, but it's not a 
switch I'll make lightly, and it's not something I'll even consider 
"selling my soul for" to take -- it'll be on my terms or I might as well 
stay with Linux as a hobby -- an arrangement that works and that suits me 
fine.

Because Linux is a hobby, I go where my interest leads me.  Even tho I'm 
not a developer, I test, bisect and file bugs on the latest git kernels 
and am proud to be able to say that a number of bugs were fixed before 
release (and one, a Radeon AGP graphics bug, after, it hit stable and two 
kernel releases before it was ultimately fixed, as reasonably new graphics 
cards on AGP busses aren't as common as they once were...) because of my 
work.

Slowly, one at a time, I've tackled Bind DNS, NTPD, md/RAID, direct 
netfilter/iptables (which interestingly enough were *SIGNIFICANTLY* easier 
for me to wrap my mind around than the various so-called "easier" firewall 
tools that ultimately use netfilter/iptables at the back end anyway, 
perhaps because I already understood network basics and all the "simple" 
ones simply obscured the situation for me) and other generally considered 
"enterprise" tools.  But while having my head around these normally 
enterprise technologies well enough to troubleshoot them may well help me 
with a job in the field in the future, that's not why I learned them.  As 
a Linux hobbyist, I learned them for much the same reason mountain climber 
hobbyists climb mountains, because they were there, and for the personal 
challenge.

Meanwhile, as I alluded to earlier, I tried LVM2 (successor to both EVMS 
and the original LVM, as you likely already know) on top of md/RAID, but 
found that for me, they layering of technologies obscured my 
understanding, to the point where I was no longer comfortable with my 
ability to recover in a disaster situation in which both the RAID and LVM2 
levels were damaged.

Couple that with an experience where I had a broken LVM2 that needed 
rebuilt, but with the portage tree on LVM2, and I realized that for what I 
was doing, especially since md/raid's partitioned-raid support was now 
quite robust, the LVM2 layer just added complexity for very little real 
added flexibility or value, particularly since I couldn't put / on it 
anyway, without an initr* (one of the technologies I've never taken time 
to understand to the point I'm comfortable using it).

That's why I recommended that you pick a storage layer technology that 
fits your needs as best you can, get comfortable with it, and avoid if 
possible multi-layering.  The keep-it-simple rule really /does/ help avoid 
admin-level fat-fingering, which really /is/ a threat to data and system 
integrity.  Sure, there's a bit of additional flexibility by layering, but 
it's worth the hard look at whether the benefit really does justify the 
additional complexity.  In triple-digit or even higher double-digit 
storage device situations, basically enterprise level, there's certainly 
many scenarios where the multi-level layering adds significant value, but 
part of being a good admin, ultimately, is recognizing where that's not 
the case, and with a bit of experience under my belt, I realized it wasn't 
a good tradeoff for my usage.

Here, I picked md/raid over lvm2 (and over hardware RAID) for a number of 
reasons.  First, md/raid for / can be directly configured on the kernel 
command line.  No initr* needed.  That allowed me to put off learning 
initr* tech for another day, as well as reducing complexity.  As for
md/raid over hardware RAID, there's certainly a place for both, 
particularly when md/raid may be layered on hardware raid, but for low-
budget emphasis of the R(edundant) in Redundant Array of Independent 
Devices (RAID), there's nothing like being able to plug in not just 
another drive, but any standard (SATA in my case) controller, and/or any 
mobo, and with at worst a kernel rebuild with the new drivers (since I 
choose to build-in what I need and not build what I don't, so a kernel 
rebuild would be necessary if it were different controllers), be back up 
and running.  No having to find a compatible RAID card...  Plus, the Linux 
kernel md/RAID stack has FAR FAR more testing under all sorts of corner-
case situations than any hardware RAID is **EVER** going to get.

But as I mentioned, the times they are a changin', and with the latest 
desktop environments (kde4.6, gnome3 and I believe the latest gnome2, and 
xfce?, at minimum) leaving the deprecated hal behind in favor of udev/
udisks/upower and etc, and with udisks in particular depending on device-
mapper, now part of lvm2, and the usual removable device auto-detect/auto-
mount functionality of the desktops dependent in turn on udisks, while it 
probably won't affect non-X server based installations and arguably 
doesn't /heavily/ (other than non-optional dependencies) affect *ix 
traditionalist desktop users who aren't comfortable with auto-mount in the 
first place (I'm one, there's known security tradeoffs involved, see 
recent stories on auto-triggered vulns due to gnome scanning for icons on 
auto-mounted thumbdrives and/or cds, for instance), it's a fact that 
within the year, most new releases will be requiring that device-mapper 
and thus lvm2 be installed and device-mapper enabled in the kernel, to 
support their automount functionality.  As such, and because lvm2 has at 
least basic raid-0 and raid-1 support (tho not the more advanced stuff, 
raid5/6/10/50/60 etc, last I checked, but I may well be behind) of its 
own, particularly for distributions relying on prebuilt kernels, therefore 
modules, therefore initr*s, already, so lvm2's initr* requirement isn't a 
factor, lvm2 is likely to be a better choice for many than md/raid.

Meanwhile, while btrfs isn't /yet/ a major choice unless you want still 
clearly experimental, by first distro releases next year, it's very likely 
to be.  And because it's the clear successor to ext*, and has built-in 
multi-device and volume management flexibility of its own, come next year, 
both lvm2 and md/raid will lose their place in the spotlight to a large 
degree.  Yet still, btrfs is not yet mature and while tantalizing in its 
closeness, remains still an impractical immediate choice.  Plus, there's 
likely to be some limitations to its device management abilities that 
aren't clear yet, at least not to those not intimately following its 
development, and significant questions remain on what filesystem-supported 
features will be boot-time-supported as well.

> Some of the stuff you've mentioned, such as btrfs, are totally new to me
> since I haven't kept up with the state of the art.  Some years ago we
> had EVMS, which was developed by IBM here in Austin.  I was a member of
> the Capital Area Central Texas UNIX Society (CACTUS) and we had the EVMS
> developers come and talk about it.  EVMS was great.  It was a layered
> technology with an API for a management client, so you could have a CLI,
> a GUI, a web-based management client, whatever, and a all of them useing
> the same API to the disk management layer.  It was an umbrella
> technology which covered several levels of Linux MD Raid plus LVM.  You
> could put fundamental storage elements together like tinker-toys and
> slice and dice them any way you wanted to.

The technology is truly wonderful.  Unfortunately, the fact that running / 
on it requires an initr* means it's significantly less useful than it 
might be.  Were that one requirement removed, the whole equation would be 
altered and I may still be running LVM instead of md/raid.  Not that it's 
much of a problem for those running a binary-based distribution already 
dependent on an initr*, but even in my early days on Linux, back on 
Mandrake, I was one of the outliers who learned kernel config 
customization and building within months of switching to Linux, and never 
looked back.  And once you're doing that, why have an initr* unless you're 
absolutely forced into it, which in turn puts a pretty strong negative on 
any technology that's going to force you into it...

> EVMS was started from an initrd, which set up the EVMS platform and then
> did a pivot_root to the EVMS-supported result.  I have our SOHO
> firewall/gateway and file server set up with it.  The root fs is on a
> Linux MD RAID-1 array, and what's on top of that I've rather forgotten
> but the result is a drive and partition layout that makes sense for the
> purpose of the box.  I set this up as a kind of proof-of-concept
> exercise because I was taken with EVMS and figured it would be useful,
> which it was.  The down side of this was that some time after that, IBM
> dropped support for the EVMS project and pulled their developers off of
> it.  I was impressed with the fact that IBM was actually paying people
> to develop open source stuff, but when they pulled the plug on it, EVMS
> became an orphaned project.  The firewall/gateway box runs Gentoo, so I
> proceeded with regular updates until one day the box stopped booting.
> The libraries, notably glibc, embedded in the initrd system got out of
> sync, version wise, with the rest of the system, and I was getting some
> severely strange errors early in the boot process followed by a kernel
> panic.  It took a bit of work to even _see_ the errors, since they were
> emitted in the boot process earlier than CLI scroll-back comes into
> play, and then there was further research to determine what I needed to
> do to fix the problem.  I ended up having to mount and then manually
> repair the initrd internal filesystem, manually reconstituting library
> symlinks as required.

That's interesting.  I thought most distributions used or recommended use 
of an alternative libc in their initr*, one that either fully static-
linked so didn't need included if the binaries were static-linked, or at 
least was smaller and more fit for the purpose of a dedicated limited-
space-ram-disk early-boot environment.

But if you're basing the initr* on glibc, which would certainly be easier 
and is, now that I think of it, probably the way gentoo handles it, yeah, 
I could see the glibc getting stale in the initrd.

... Because if it was an initramfs, it'd be presumably rebuilt and thus 
synced with the live system when the kernel was updated, since an 
initramfs is appended to the kernel binary file itself.  That seems to me 
to be a big benefit to the initramfs system, easier syncing with the built 
kernel and the main system, tho it would certainly take longer, and I 
expect for that reason that the initramfs rebuild could be short-
circuited, thus allowed to get stale, if desired.  But it should at least 
be easier to keep updated if desired, because it /is/ dynamically attached 
to the kernel binary at each kernel build.

But as I said I haven't really gotten into initr*s.  In fact, I don't even 
build busybox, etc, here, sticking it in package.provided.  If my working 
system gets so screwed up that I'd end up with busybox, I simply boot to 
the backup / partition instead.  That backup is actually a fully 
operational system snapshot taken at the point the backup was made, so it 
includes everything the system did at that point.  As such, unlike most 
people's limited recovery environments, I have a full-featured X, KDE, 
etc, all fully functional and working just as they were on my main system 
at the time of the backup.  So if it comes to it, I can simply switch to 
it as my main root, and go back to work, updating or building from a new 
stage-3 as necessary, at my leisure, not because I have to before I can 
get a working system again, because the backup /is/ a working system, as 
fully functional (if outdated) as it was on the day I took the backup.

> I've built some Linux boxes for one of my clients - 1U servers and the
> like.  These folks are pretty straight-forward in their requirements,
> mainly asking that the boxes just work.  The really creative work goes
> into the medical research PHP application that lives on the boxes, and
> I've learned beaucoup stuff about OOP in PHP, AJAX, etc. from the main
> programming fellow on the project.  We've standardized on Ubuntu server
> edition on SuperMicro 4 drive 1U boxes.  These boxes generally come with
> RAID supported by a proprietary chipset or two, which never works quite
> right with Linux, so the first job I always do with these is to rip out
> the SATA cabling from the back plane and replace the on-board RAID with
> an LSI 3ware card.  These cards don't mess around - they JUST WORK.
> LSI/3ware has been very good about supporting Linux for their products.
> We generally set these up as RAID-5 boxes.  There's a web-based
> monitoring daemon for Linux that comes with the card, and it just works,
> too, although it takes a bit of dickering.  The RAID has no component in
> user-space (except for the monitor daemon) and shows up as a single SCSI
> drive, which can be partitioned and formatted just as if it were a
> single drive.  The 3ware cards are nice!  If you're using a redundant
> array such as RAID-1 or RAID-5, you can designate a drive as a hot-spare
> and if one of the drives in an array fails, the card will fail over to
> the hot-spare, rebuild the array, and the monitor daemon will send you
> an email telling you that it happened.  Slick!

> The LSI 3ware cards aren't cheap, but they're not unreasonable either,
> and I've never had one fail.  I'm thinking that the my drive setup on my
> new desktop box will probably use RAID-1 supported by a 3ware card.

That sounds about as standard as a hardware RAID card can be.  I like my 
md/raid because I can literally use any standard SATA controller, but 
there are certainly tradeoffs.  I'll have to keep this in mind in case I 
ever do need to scale an installation to where hardware RAID is needed 
(say if I were layering md/kernel RAID-0 on top of hardware RAID-1 or 
RAID-5/6).

FWIW, experience again.  I don't believe software RAID-5/6 to be worth 
it.  md/raid 0 or 1 or 10, yes, but if I'm doing RAID-5 or 6, it'll be 
hardware, likely underneath a kernel level RAID-0 or 10, for a final 
RAID-50/60/500/600/510/610, with the left-most digit as the hardware 
implementation.  This because software RAID-5/6 is simply slow.

Similar experience, Linux md/RAID-1 is /surprisingly/ efficient, MUCH more 
so than one might imagine, as the kernel I/O scheduler makes *VERY* good 
use of parallel scheduling.  In fact, in many cases it beats RAID-0 
performance, unless of course you need the additional space of RAID-0.  
Certainly, that has been my experience, at least.

> I'll
> probably use an ext3 filesystem on the partitions.  I know ext4 is under
> development, but I'm not sure if it would offer me any advantages.

FWIW, ext4 is no longer "under development", it's officially mature and 
ready for use, and has been for a number of kernels, now.  In fact, 
they're actually planning on killing separate ext2/3 driver support as the 
ext4 driver implements it anyway.

As such, I'd definitely recommend considering ext4, noting of course that 
you can specifically enable/disable various ext4 features at mkfs and/or 
mount time, if desired.  Thus there's really no reason to stick with ext3 
now.  Go with ext4, and if your situation warrants, disable one or more of 
the ext4 features, making it more ext3-like.

OTOH, I'd specifically recommend evaluating the journal options, 
regardless of ext3/ext4 choice.  ext3 defaulted to "ordered" for years, 
then for a few kernels, switched to "writeback" by default, then just 
recently (2.6.38?) switched back to "ordered".  AFAIK, ext4 has always 
defaulted to the faster but less corner-case crash safe "writeback".  The 
third and most conservative option is of course "journal" (journal the 
data too, as opposed to metadata only, with the other two).

Having lived thru the reiserfs "writeback" era and been OH so glad when 
they implemented and defaulted to "ordered" for it, I don't believe I'll 
/ever/ trust anything beyond what I'd trust on a RAID-0 without backups, 
to "writeback" again, regardless of /what/ the filesystem designers say or 
what the default is.

And, I know of at least one person that experienced data integrity issues 
with writeback on ext3 when the kernel was defaulting to that, that 
immediately disappeared when he switched back to ordered.

Bottom line, yeah I believe ext4 is safe, but ext3 or ext4, unless you 
really do /not/ care about your data integrity or are going to the extreme 
and already have data=journal, DEFINITELY specify data=ordered, both in 
your mount options, and by setting the defaults via tune2fs.

If there's one bit of advice in all these posts that I'd have you take 
away, it's that.  It's NOT worth the integrity of your data!  Use 
data=ordered unless you really do NOT care, to the same degree that you 
don't put data you care about on RAID-0, without at least ensuring that 
it's backed up elsewhere.  I've seen people lose data needlessly over 
this; I've lost it on reiserfs myself before they implemented data=ordered 
by default, and truly, just as with RAID-0, data=writeback is NOT worth 
whatever performance increase it might bring, unless you really do /not/ 
care about the data integrity on that filesystem!

> I used reiserfs on some of the partitions on my servers, and on some
> partitions on my desktop box too.  Big mistake!  There was a bug in
> reiserfs support the current kernel when I built the first server and
> the kernel crapped all over the hard drive one night and the box
> crashed!

IDR the kernel version but there's one that's specifically warned about.  
That must have been the one...

But FWIW, I've had no problems, even thru a period of bad-ram resulting in 
kernel crashes and the like, since the introduction of journal=ordered.  
Given the time mentioned above when ext3 defaulted to data=writeback, I'd 
even venture to say that for that period and on those kernels, reiserfs 
may well have been safer than ext3!

> I was able to fix it and salvage customer data, but it was
> pretty scary.  Hans Reiser is in prison for life for murder, and there's
> like one person on the Linux kernel development group who maintains
> reiserfs.  Ext3/4, on the other hand, is solid, maybe not quite as fast,
> but supported by a dedicated group of developers.

For many years, the kernel person doing most of the reiserfs maintenance 
and the one who introduced the previously mentioned data=ordered mode by 
default and data=journal mode as an option, was Chris Mason.  I believe he 
was employed by SuSE, for years the biggest distribution to default to 
reiserfs, even before it was in mainline, I believe.  I'm not sure if he's 
still the official reiserfs maintainer due to his current duties, but he 
DEFINITELY groks the filesystem.  Those current duties?  He's employed by 
Oracle now, and is the lead developer of btrfs.

Now reiserfs does have its warts.  It's not particularly fast any more, 
and has performance issues on multi-core systems due to its design around 
the BKL (big kernel lock, deprecated for years with users converting to 
other lock methods, no current in-tree users with 2.6.38 and set to be 
fully removed with 2.6.39), which tho reiserfs was converted to other 
locking a few kernels ago, the single-access-at-a-time assumption and 
bottleneck lives on.  However, it is and has been quite stable for many 
years now, since the intro of data=ordered, to the point that as mentioned 
above, I believe it was safer than ext3 during the time ext3 defaulted to 
writeback, because reiserfs still had the saner default of data=ordered.

But to each his own.  I'd still argue that a data=writeback default is 
needlessly risking data, however, and far more dangerous regardless of 
whether it's ext3, ext4, or reiserfs, than any of the three themselves 
are, otherwise.

> So I'm thinking that this new box will have a couple of
> professional-grade (the 5-year warranty type) 1.5 or 2 TB drives and a
> 3ware card.  I still haven't settled on the mainboard, which will have
> to support the 3ware card, a couple of sound cards and a legacy Adaptec
> SCSI card for our ancient but extremely well-built HP scanner.  The
> chipset will have to be well supported in Linux.  I'll probably build
> the box myself once I decide on the hardware.

FWIW, my RAID is 4x SATA 300 gig Seagates, 5 year warranty I expect now 
either expired or soon to.  Most of the system is RAID-1 across all four, 
however, and I'm backed up to external as well altho I'll admit that 
backup's a dated, now.  I bought them after having a string of bad luck 
with ~1 year failures on both Maxtor (which had previously been quite 
dependable for me) and Western Digital (which I had read bad things about 
but thought I'd try after Maxtor, only to have the same ~1 year issues).  
Obviously, they've long outlasted those, so I've been satisfied.

As I said, I'll keep the 3ware RAID cards in mind.

Mainboard:  If a server board fits your budget, I'd highly recommend 
getting a Tyan board that's Linux certified.  The one I'm running in my 
main machine is now 8 years old, /long/ out of warranty and beyond further 
BIOS updates, but still running solid.  It was a $400 board back then, 
reasonable for a dual-socket Opteron.  Not only did it come with Linux 
certifications for various distributions, but they had Linux specific 
support.  Further, how many boards do /you/ know of that have a pre-
customized sensors.conf file available for the download? =:^)  And when 
the dual-cores came out, a BIOS update was made available that supported 
them.  As a result, while it was $400, that then leading edge dual socket 
Opteron board from eight years ago, while it's no longer leading edge by 
any means, eight years later still forms the core for an acceptably decent 
system, dual-dual-core Opteron 290s @ 2.8 GHz (topped out the sockets), 
currently 6 gig RAM, 3x2-gig as I had one stick die that I've not 
replaced, but 8 sockets so I could run 16 gig if I wanted, 4xSATA drives, 
only SATA-150, but they're RAIDED, Radeon hd4650 AGP (no PCI-E, tho it 
does have PCI-X), etc.  No PCI-E, limited to SATA-150, and no hardware 
virtualization instruction support on the CPUs, so it's definitely dated, 
but after all, it's an 8 years' old system!

It's likely to be a decade old by the time I actually upgrade it.  Yes, 
it's definitely a server-class board and the $400 I paid reflected that, 
but 8 years and shooting for 10!  And with the official Linux support 
including a custom sensors.conf.  I'm satisfied that I got my money's 
worth.

But I don't believe all Tyan's boards are as completely Linux supported as 
that one was, so do your research.

> I'm gonna apply the KISS principle to the OS design for this, and stay
> away from bleeding edge software technologies, although, especially
> after reading your essay, it's very tempting to try some of this stuff
> out to see what the the _really_ smart people are coming up with!  I'm
> getting off of the Linux state-of-the art train for a while and go
> walking in the woods.  The kernel will have to be low-latency since I
> may use the box for recording work with Jack and Ardour2, and almost
> certainly for audio editing, and maybe video editing at some point.
> That's where my energy is going to go for this one.

Well, save btrfs for a project a couple years down the line, then.  But 
certainly, investigate md/raid vs lvm2 and make your choice, keeping in 
mind that while nowdays they overlap features, md/raid doesn't require an 
initr* to run / on it, while lvm2 will likely be pulled in as a dependency 
for your X desktop, at least kde/gnome/xfce, by later this year, whether 
you actually use its lvm features or not.

And do consider ext4, but regardless of ext3/4, be /sure/ you either 
choose data=ordered or can give a good reason why you didn't.  (Low-
latency writing just might be a reasonable excuse for data=writeback, but 
be sure you keep backed up if you do!)  Because /that/ one may well save 
your data, someday!

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-desktop] Re: System problems - some progress

[Lindsay Haisley]

On Sat, 2011-03-26 at 08:40 +0000, Duncan wrote:
> By contrast, Linux is still my hobby, tho really, a full time one in that 
> I spend hours a day at it, pretty much 7 days a week.  I'm thinking I 
> might switch to Linux as a job at some point, perhaps soon, but it's not a 
> switch I'll make lightly, and it's not something I'll even consider 
> "selling my soul for" to take -- it'll be on my terms or I might as well 
> stay with Linux as a hobby -- an arrangement that works and that suits me 
> fine.

What professional work I've gotten with Linux has been a real lesson in
synergy.  It seems as if every time I've gone out and experimented with
some facet of Linux technology - setting up iptables, learning routing
fundamentals, setting up and using OpenVPN, etc., I've been called upon
to use it, and get paid for using it, for a client.  My main client, in
return, has increased my understanding of higher level programming stuff
tremendously!  

> Slowly, one at a time, I've tackled Bind DNS, NTPD, md/RAID, direct 
> netfilter/iptables (which interestingly enough were *SIGNIFICANTLY* easier 
> for me to wrap my mind around than the various so-called "easier" firewall 
> tools that ultimately use netfilter/iptables at the back end anyway, 
> perhaps because I already understood network basics and all the "simple" 
> ones simply obscured the situation for me) and other generally considered 
> "enterprise" tools.

Yep, I know where you're coming from there.  Iptables isn't all that
hard to understand, and I've become pretty conversant with it in the
process of using for my own and others' systems.  I'd always rather deal
with the "under the hood" CLI tools than with some GUI tool that does
little more than obfuscate the real issue.  That way lies Windows!

> Bottom line, yeah I believe ext4 is safe, but ext3 or ext4, unless you 
> really do /not/ care about your data integrity or are going to the extreme 
> and already have data=journal, DEFINITELY specify data=ordered, both in 
> your mount options, and by setting the defaults via tune2fs.

So does this turn off journaling?  What's a good reference on the
advantages of ext4 over ext3, or can you just summarize them for me?

> But if you're basing the initr* on glibc, which would certainly be easier 
> and is, now that I think of it, probably the way gentoo handles it, yeah, 
> I could see the glibc getting stale in the initrd.

The problem with Gentoo was that because EVMS was an orphaned project, I
believe the ebuild wasn't updated.  The initrd file was specific for
EVMS.

> If there's one bit of advice in all these posts that I'd have you take 
> away, it's that.  It's NOT worth the integrity of your data!  Use 
> data=ordered unless you really do NOT care, to the same degree that you 
> don't put data you care about on RAID-0, without at least ensuring that 
> it's backed up elsewhere.

I've never used, or had much use for RAID-0.  LVM provides the same
capabilities.  For me, RAID is a way of insuring data integrity, and
large drives are getting cheaper and cheaper.  I've only used RAID-1 and
RAID-5.

I'm not a speed-freak on disk I/O, and am generally quite willing to
sacrifice a bit of speed for reliability.  data=writeback has been a
tweak, and I believe I've read up on it previously and decided against
it for probably the same reasons you cite.  data=ordered has been the
default, but apparently upgrading to 2.6.36 I'm going to have to spec
this explicitly in /etc/fstab unless I upgrade to 2.6.38.

> FWIW, my RAID is 4x SATA 300 gig Seagates, 5 year warranty I expect now 
> either expired or soon to.  Most of the system is RAID-1 across all four, 
> however, and I'm backed up to external as well altho I'll admit that 
> backup's a dated, now.  I bought them after having a string of bad luck 
> with ~1 year failures on both Maxtor (which had previously been quite 
> dependable for me)

I had a Maxtor drive actually *smoke* on me once, years ago.  There was
a "pop", and smoke, and a big burned spot on the circuit board on the
drive!  I never bought another Maxtor!  It's the smoke inside the little
colored thingies on printed circuit boards that make them work!  When
they break, and the smoke gets away, the thingies are useless.

I generally go with Seagates these days too, although the quality of
drives, and which brand is best, seems to change over time.  I used to
swear by IBM drives until they had a bad run of them with a high failure
rate, and before this got sorted out they sold their drive biz to
Fujitsu.  

>  and Western Digital (which I had read bad things about 
> but thought I'd try after Maxtor, only to have the same ~1 year issues).  
> Obviously, they've long outlasted those, so I've been satisfied.
> 
> As I said, I'll keep the 3ware RAID cards in mind.

After having had all kinds of trouble trying to get hardware RAID
working on one of my servers, I discovered the 3ware cards after asking
the advice of the hardware fellow here who works with one of my favorite
tech outfits in Austin, Outernet Connection Strategies.  He builds a lot
of servers and doesn't even _try_ to get the native RAID chipsets to
work.  He just slaps a 3ware card in them and moves on.  It's _real_
RAID, all the useful levels, not "fakeraid".  

> Mainboard:  If a server board fits your budget, I'd highly recommend 
> getting a Tyan board that's Linux certified.  The one I'm running in my 
> main machine is now 8 years old, /long/ out of warranty and beyond further 
> BIOS updates, but still running solid.

Hmmm.  I'll look into Tyan.  I hadn't heard of them, but it sounds as if
they bend over backwards to work with Linux.  That's always a plus.

> It's likely to be a decade old by the time I actually upgrade it.  Yes, 
> it's definitely a server-class board and the $400 I paid reflected that, 
> but 8 years and shooting for 10!  And with the official Linux support 
> including a custom sensors.conf.  I'm satisfied that I got my money's 
> worth.
> 
> But I don't believe all Tyan's boards are as completely Linux supported as 
> that one was, so do your research.

Of course.  I like technology that _lasts_!  We have a clock in our
house that's about 190 years old, and came to me through my family.  The
works are made of wood, and it keeps impeccable time - loses or gains
maybe 30 seconds a week if I wind it every day, which I need to.  Some
years ago one of the wooden gears gave out from over a century of
stress.  There's a label in the clock that says "warrented if well
used", and since I'd used it very well, I called up the Seth Thomas
company and told them that I had one of their clocks and it was broken,
and since I'd used it very well, I figured that it was still under
warranty.  The gal with whom I talked was amused and intrigued, and
turned me on to the Connecticut Clock and Watch museum, run by one
George Bruno.  It seems that Mr. Bruno also makes working replicas of
exactly the model of clock I have and was able to send me an exact
replacement part!  Try _THAT_ with your 1990's era computer ;-)  Every
time this nice old clock strikes the hour it reminds me that although I
work with computers where hardware is out of date in 5 years or so,
there are some things that were built to last!

But this is OT for this forum.  Sorry, folks.  I couldn't resist telling
a good story.

> Well, save btrfs for a project a couple years down the line, then.  But 
> certainly, investigate md/raid vs lvm2 and make your choice, keeping in 
> mind that while nowdays they overlap features, md/raid doesn't require an 
> initr* to run / on it, while lvm2 will likely be pulled in as a dependency 
> for your X desktop, at least kde/gnome/xfce, by later this year, whether 
> you actually use its lvm features or not.

Thanks, Duncan.  Good advice, that.

> And do consider ext4, but regardless of ext3/4, be /sure/ you either 
> choose data=ordered or can give a good reason why you didn't.  (Low-
> latency writing just might be a reasonable excuse for data=writeback, but 
> be sure you keep backed up if you do!)  Because /that/ one may well save 
> your data, someday!

I'm going to read up on btrfs and ext4, whether or not I use them.

-- 
Lindsay Haisley       |"Windows .....
FMP Computer Services |    life's too short!"
512-259-1190          |
http://www.fmp.com    |              - Brad Johnston

[gentoo-desktop] Re: System problems - some progress

[Duncan]

Lindsay Haisley posted on Sat, 26 Mar 2011 10:57:33 -0500 as excerpted:

> Yep, I know where you're coming from there.  Iptables isn't all that
> hard to understand, and I've become pretty conversant with it in the
> process of using for my own and others' systems.  I'd always rather deal
> with the "under the hood" CLI tools than with some GUI tool that does
> little more than obfuscate the real issue.  That way lies Windows!

Indeed, the MSWindows way is the GUI way.  But I wasn't even thinking 
about that.  I was thinking about the so-called "easier" firewalling CLI/
text-editing tools that have you initially answer a number of questions to 
setup the basics, then have you edit files to do any "advanced" tweaking 
the questions didn't have the foresight to cover.

But my (first) problem was that while I could answer the questions easy 
enough, I lacked sufficient understanding of the real implementation to 
properly do the advanced editing.  And if I were to properly dig into 
that, I might as well have mastered the IPTables/Netfilter stuff on which 
it was ultimately based in the first place.

The other problem, when building your own kernel, was that the so-called 
simpler tools apparently expect all the necessary Netfilter/IPTable kernel 
options to be available as pre-built modules (or built-in) -- IOW, they're 
designed for the binary distributions where that's the case.  Neither the 
questions nor the underlying config file comments mentioned their kernel 
module dependencies.  One either had to pre-build them all and hope they 
either got auto-loaded as needed, or delve into the scripts to figure out 
the dependencies and build/load the required modules.

Now keep in mind that I first tried this on Mandrake, where I was building 
my own kernel within 90 days of first undertaking the switch, while I was 
still booting to MS to do mail and news in MSOE, because I hadn't yet had 
time to look at user level apps well enough to make my choices and set 
them up.  So it's certainly NOT just a Gentoo thing.  It's a build-your-
own-kernel thing, regardless of the distro.

The problem ultimately boiled down to having to understand IPTables itself 
well enough to know what kernel options to enable, either built-in or as 
modules which would then need to be loaded.  But if I were to do that, why 
would I need the so-called "easier" tool, that only complicated things.  
Honestly, the tools made me feel like I was trying to remote-operate some 
NASA probe from half-way-across-the-solar-system, latency and all, instead 
of using the direct-drive, since what I was operating on was actually 
right there next to me!

At that time I simply punted.  I had (or could have and did have, by 
(wise) choice on MS) a NAPT based router between me and the net anyway, 
and already knew how to configure /it/.  So I just kept it and ran the 
computer itself without a firewall for a number of years.  Several years 
later, after switching to Gentoo, when I was quite comfortable on Linux in 
general, I /did/ actually learn netfilter/iptables, configure my computer 
firewall accordingly, and direct-connect for a year or two -- until my 
local config changed and I actually had the need for a NAPT device as I 
had multiple local devices to connect to the net.

Which brings up a nice point about Gentoo.  With Mandrake (and most other 
distributions of the era, from what I read), there were enough ports open 
by default that having a firewall of /some/ sort, either on-lan NAPT 
device or well configured on-computer IPChains/IPTables based, was wise.  
IOW, keeping that NAPT device was a good choice, even if it /was/ an MS-
based view of things, because the Linux distros of the time still ran with 
various open ports (whether they still do or not I don't know, I suspect 
most do, tho they probably do it with an IPTables firewall up now too).

Gentoo's policy by contrast has always (well, since before early 2004, 
when I switched to it) been:

1) Just because it's installed does NOT mean it should have its initscript 
activated so it runs automatically in the default runlevel -- Gentoo ships 
by default with the initscripts for net-active services in /etc/init.d, 
but does NOT automatically add them to the default runlevel.

2) Even when a net-active service IS activated, Gentoo's default 
configuration normally has it active on the loopback localhost address 
only.

3) Gentoo ships X itself with IP-forwarding disabled, only the local Unix 
domain socket active.

As such, by the time I actually got around to learning IPTables/netfilter 
and setting it up on my Gentoo box, it really wasn't as necessary as it 
would be on other distributions, anyway, because firewall or no firewall, 
the only open ports were ports I had deliberately opened myself and thus 
already knew about.

But of course defense in depth is a VERY important security principle, 
correlating as it does with the parallel "never trust yourself not to fat-
finger SOMETHING!"  (Now, if the so-called security services HBGary, et. 
al., only practiced it! ...  I think that's what galled most of the world 
most, not that they screwed up a couple things so badly, but that they so 
blatantly violated the basic defense-in-depth, or we'd have never read 
about the screw-ups in the first place as they'd have not amounted to 
anything if the proper layers of defense had been there... and for a 
SECURITY firm, no less, to so utterly and completely miss it!)  So 
regardless of the fact that in theory I didn't actually need the firewall 
by then since the only open ports were the ones I intended to be open, I 
wasn't going to run direct-connected without /some/ sort of firewall, and 
I learned and activated IPTables/netfilter before I did direct-connect.  
And now that I have NAPT again, I still keep it running, as that's simply 
another layer of that defense in depth, and I can use the NAPT router for 
multiplexing several devices on a single IP, not its originally accidental 
side-effect of inbound firewalling, tho again, I keep that too as it's 
another layer of that defense in depth, I just don't /count/ on it.

>> Bottom line, yeah I believe ext4 is safe, but ext3 or ext4, unless you
>> really do /not/ care about your data integrity or are going to the
>> extreme and already have data=journal, DEFINITELY specify data=ordered,
>> both in your mount options, and by setting the defaults via tune2fs.
> 
> So does this turn off journaling?  What's a good reference on the
> advantages of ext4 over ext3, or can you just summarize them for me?

No, this doesn't turn off journaling.

Briefly...

There's the actual data, the stuff in the files we care about, and 
metadata, the stuff the filesystem tracks behind the scenes so we don't 
have to worry about it.  Metadata includes stuff like the filename, the 
dates (create/modify/access, the latter of which isn't used that much any 
more and is often disabled), permissions (both traditional *ix set*/user/
group/world and if active SELinux perms, etc), INODE AND DIRECTORY TABLES 
(most important in this context, thus the CAPS, as without them, your data 
is effectively reduced to semi-random binary sequences), etc.

It's the metadata, in particular, the inode and directory tables, that fsck 
concerns itself with, that's potentially damaged in the event of a chaotic 
shutdown, that fsck checks and tries to restore on remount after such a 
shutdown, etc.

Because the original purpose of journaling was to shortcut the long fscks 
after a chaotic shutdown, traditionally it concerns itself only with 
metadata.  In practice, however, due to reordered disk operations at both 
the OS and disk hardware/firmware level, the result of a recovery with 
strict meta-data-only journaling on a filesystem can be perfectly restored 
filesystem metadata, but with incorrect real DATA in those files, because 
the metadata was already written to disk but the data itself hadn't been, 
at the time of the chaotic shutdown.

Due to important security implications (it's possible that the previous 
contents of that inode was an unlinked but not secure-erased file 
belonging to another user, UNAUTHORIZED DATA LEAK!!!), such restored 
metadata-only files where the data itself is questionable, are normally 
truncated to zero-length, thus the post-restore zero-length "empty" file 
phenomenon common with early journaled filesystems and still occasionally 
seen today.

The data= journaling option controls data/metadata handling.

data=writeback is "bare" metadata journaling.  It's the fastest but 
riskiest in terms of real data integrity for the reasons explained above.  
As such, it's often used where performance matters more than strict data 
integrity in the event of chaotic shutdown -- where data is backed up and 
changes since the backup tend to be trivial and/or easy to recover, where 
the data's easily redownloaded from the net (think the gentoo packages 
tree, source tarballs, etc), and/or where the filesystem is wiped at boot 
anyway (as /tmp is in many installations/).  Zeroed out files on recovery 
can and do happen in writeback mode.

data=ordered is the middle ground, "good enough" for most people, both in 
performance and in data integrity.  The system ensures that the commit of 
the real data itself is "ordered" before the metadata that indexes it, 
telling the filesystem where it's located.  This comes at a slight 
performance cost as some write-order-optimization must be skipped, but it 
GREATLY enhances the integrity of the data in the event of a chaotic 
shutdown and subsequent recovery.  There are corner-cases where it's still 
possible at least in theory to get the wrong behavior, but in practice, 
these don't happen very often, and when they do, the loss tends to be that 
of reverting to the pre-update version of the file, losing only the 
current edit, rather than zeroing out of the file (or worse yet, data 
leakage) entirely.

data=journal is the paranoid option.  With this you'll want a much larger 
journal, because not only the metadata, but the data itself, is 
journaled.  (And here most people thought that's what journaling did /all/ 
the time!)  Because ALL data is ultimately written TWICE in this mode, 
first to the journal and then from there to its ultimate location, by 
definition it's a factor of two slower, but provided the hardware is 
working correctly, the worst-case in a chaotic shutdown is loss of the 
current edit, reverting to the previous edition of the file.

FWIW and rather ironically, my original understanding of all this came 
from a series of IBM DeveloperWorks articles written in the early kernel 
2.4 series era, explaining the main filesystem choices, many of them then 
new, available in kernel 2.4.  While the performance data and some 
filesystem implementation detail (plus lack of mention of ext4 and btrfs 
as this was before their time) is now somewhat dated, the theory and 
general filesystem descriptions remain solid, and as such, the series 
remains a reasonably good intro to Linux filesystems to this day.  As 
such, parts of it are still available as linked from the Gentoo 
Documentation archived copy of those IBM DeveloperWorks articles.  In 
particular, two parts covering ext3 and the data= options remain available:

http://www.gentoo.org/doc/en/articles/afig-ct-ext3-intro.xml
http://www.gentoo.org/doc/en/articles/l-afig-p8.xml

The ironic bit is who the author was, one Daniel Robbins, the same DRobbins 
who founded the then Enoch Linux, now Gentoo.  But I read them long before 
I ever considered Gentoo, when I was first switching to Linux and using 
Mandrake.  It was thus with quite some amazement a number of years later, 
after I'd been on Gentoo for awhile, that I discovered that the *SAME* 
DRobbins who founded Gentoo (and was still active tho on his way out in 
early 2004 when I started on Gentoo), was the guy who wrote the Advanced 
Filesystem Implementor's Guide in IBM DeveloperWorks, the guide I'd found 
so *INCREDIBLY* helpful years before, when I hadn't a /clue/ who he was or 
what distribution I'd chose years later, as I just starting with Mandrake 
and trying to figure out what filesystems to choose.

As to the ext3/ext4 differences... AFAIK the (second) biggest one is that 
ext4 uses extents by default, thus fragmenting files somewhat less over 
time.  (Extents are a subject worth their own post, which I won't attempt 
as while I understand the basics I don't understand all the implications 
thereof myself.  But one effect is better efficiency in filesystem layout, 
when the filesystem was created with them anyway... it won't help old 
files on upgraded-to-ext4-from ext2/3 that much.  Google's available for 
more. =:^)

There's a lot of smaller improvements as well.  ext4 is native large-
filesystem by default.  A number of optimizations discovered since ext3 
are implemented in ext4 that can't be in ext3 for stability and/or old-
kernel backward compatibility reasons.  ext4 has a no-journal option 
that's far better on flash-based thumb-drives, etc.  There are a number of 
options that can make it better on SSDs and flash in general than ext3.

And the biggest advantage is that ext4 is actively supported in the kernel 
and supports ext2/3 as well, while ext2/3, as separate buildable kernel 
options, are definitely considered legacy, with talk, as I believe I 
mentioned, of removing them as separate implementations entirely, relying 
on ext4's backward compatibility for ext2/3 support.  In that regard, ext3 
as a separate option is in worse shape than reiserfs, since it's clearly 
legacy and targeted for removal.  As part of ext4, support will 
*DEFINITELY* continue for YEARS, more likely DECADES, so is in no danger 
in that regard (more so than reiserfs support, which will continue to be 
supported as well for at least years), but the focus is definitely on ext4 
now, and as ext3 becomes more and more legacy, the chances of corner-case 
bugs appearing in ext3-only code in the ext4 driver do logically 
increase.  In that regard, reiserfs could actually be argued to be in 
better shape, since it's not implemented as a now out-of-focus older-
brother to a current filesystem, so while it has less focus in general, it 
also has less chances of being accidentally affected by a change to the 
current-focus code.

Which can be argued to have already happened with the default ext3 
switching to data=writeback for a number of kernels, before being switched 
back to the data=ordered it always had before.  A number of kernels ago 
(2.6.29 IIRC), ext4 was either officially just out of or being discussed 
for bringing out of experimental.  I believe it was Ubuntu that first made 
it a rootfs system install option, in that same time period.  Shortly 
thereafter, a whole slew of Ubuntu on ext4 users, most of whom it turned 
out later were using the closed nVidia driver, which was unstable in that 
version against that Ubuntu version and kernel, thus provoking many cases 
of "chaotic shutdown", a classic worst-case trial-by-fire test for the 
then still coming out of experimental ext4, began experiencing the classic 
"zeroed out file" problems on reboot after their chaotic shutdowns.

*Greatly* compounding the problem were some seriously ill-advised Gnome 
config-file behaviors.  Apparently, they were opening config-files for 
read-write simply to READ them and get the config in the process of 
initializing GNOME.  Of course, the unstable nVidia driver was 
initializing in parallel to all this, with the predictable-in-hindsight 
results...  As gnome was only READING the config values, it SHOULD have 
opened those files READ-ONLY, if necessary later opening them read-write 
to write new values to them.  As with the security defense-in-depth 
mentioned in the HBGary parenthetical above, this is pretty basic 
filesystem principles, but the gnome folks had it wrong.  The were opening 
the files read/write when they only needed read, and the system was 
crashing with them in that state.  As a result, these files were open for 
writing in the crash, and as is standard security practice as explained 
above, the ext4 journaling system, defaulting to write-back mode, restored 
them as zeroed out files to prevent any possibility of data leak.  
Actually, there were a few other technicalities involved as well (file 
renaming on write, failure to call fsync, due in part to ext3's historic 
bad behavior on fsync, which it treated as whole-filesystem-sync, etc), 
but that's the gist of things.

So due to ext4's data=writeback and the immaturity of the filesystem such 
that it didn't take additional precautions, these folks were getting 
critical parts of their gnome config zeroed out every time they crashed, 
and due to the unstable nVidia drivers, they were crashing frequently!!

*NOT* a good situation, and that's a classic understatement!!

The resulting investigation discovered not only the obvious gnome problem, 
but several code tweaks that could be done to ext4 to reduce the 
likelihood of this sort of situation in the future.

All fine and good, so far.  But they quickly realized that the same sort 
of code tweak issues existed with ext3, except that because ext3 defaulted 
to data=ordered, only those specifically setting data=writeback were 
having problems, and because those using data=writeback were expected to 
have /some/ problems anyway, the issues had been attributed to that and 
thus hadn't been fully investigated and fixed, all these years.

So they fixed the problems in ext3 as well.  Again, all fine and good -- 
the problems NEEDED fixed.  *BUT*, and here's where the controversy comes 
in, they decided that data=writeback was now dependable enough for BOTH 
ext3 and ext4, thus changing the default for ext3.

To say that was hugely controversial is an understatement (multiple 
threads on LKML, LWN, elsewhere where the issue was covered at the time, 
often several hundreds of posts long each), and my feelings on 
data=writeback should be transparent by now so where I stand on the issue 
should be equally transparent, but Linus never-the-less merged the commit 
that switched ext3 to data=writeback by default, AFAIK in 2.6.31.  (AFAIK, 
they discovered the problem in 2.6.29, 2.6.30 contained temporary work-
around-fixes, 2.6.31 contained the permanent fixes and switched ext3 to 
data=writeback.)

Here's the critical point.  Because reiserfs isn't so closely related to 
the ext* family, it retained the data=ordered default it had gotten years 
early, the same kernel Chris Mason committed the code for reiserfs to do 
data=ordered at all.  ext3 got the change due to its relationship with 
ext4, despite the fact that it's officially an old and stable filesystem 
where arguably such major policy changes should not occur.  If the seperate 
kernel option for ext3 is removed in ordered to remove the duplicate 
functionality already included in ext4 for backward compatibility reasons, 
by definition, this sort of change to ext4 *WILL* change the ext3 it also 
supports, unless deliberate action is taken to avoid it.  That makes such 
issues far more likely to occur again in ext3, than in the relatively 
obscure ext4.

Meanwhile, as mentioned, with newer kernels (2.6.36, 37, or 38, IDR which, 
tho it won't matter for those specifying the data=option either via 
filesystem defaults using tune2fs, or via specific mount option), ext3 
reverted again to the older and safer default, data=ordered.

And as I said, it's my firm opinion that the data= option has a stronger 
effect on filesystem stability than any possibly remaining issues with 
ext4, which is really quite stable by now.  Thus, ext3, ext4, or reiserfs, 
I'd **STRONGLY** recommend data=ordered, regardless of whether it's the 
default as it is with old and new (but with a gap) ext3 and reiserfs as it 
has been for years, or not, as I believe ext4 still defaults to 
data=writeback.  If you value your data, "just do it!"

Meanwhile, I believe the default on the definitely still experimental 
btrfs is data=writeback too.  While I plan on switching to it eventually, 
you can be quite sure I'll be examining that default and as of this point, 
have no intentions of letting it be data=writeback, when I do.

....

> The problem with Gentoo was that because EVMS was an orphaned project, I
> believe the ebuild wasn't updated.  The initrd file was specific for
> EVMS.

That's quite likely, indeed.

> Of course.  I like technology that _lasts_!  We have a clock in our
> house that's about 190 years old [...] turned me on to the Connecticut
> Clock and Watch museum, run by one George Bruno [who] also makes working
> replicas [and] was able to send me an exact replacement part!  Try
> _THAT_ with your 1990's era computer ;-)

That reminds me...  I skipped it as irrelevant to the topic at hand, but 
due to kernel sensors and ACPI changes, I decided to try the last BIOS 
upgrade available for this Tyan, after having run an earlier BIOS for some 
years.  Along about 2.6.27, I had to start using a special boot parameter 
to keep the sensors working, as apparently the sensor address regions 
overlap ACPI address regions (not an uncommon issue in boards of that era, 
the kernel folks say).  The comments on the kernel bug I filed suggested 
that a BIOS update might straighten that out (it didn't, BIOS still too 
old and board EOLed, even if it is still working), so I decided to try it.

The problem was that I had a bad memory stick.  Now the kernel has 
detectors for that and I had them active, but the kernel drivers for that 
were introduced long after I got the hardware, and while it was logging an 
issue with the memory, since it had been doing that since I activated the 
kernel drivers for it, I misinterpreted that as simply how it worked, so 
wasn't aware of the bad memory it was trying to tell me about.

So I booted to the FreeDOS floppy I used for BIOS upgrades (I've used 
FreeDOS for BIOS upgrades for years, without incident before this) and 
began the process.

It crashed half-way thru the flash-burn, apparently when it hit that bad 
memory!!

Bad situation, but there's supposed to be a failsafe direct-read-recover 
mode built-in, that probably would have worked had I known about it.  
Unfortunately I didn't, and by the time I figured it out, I'd screwed that 
up as well.

But luckily I have a netbook, that I had intended to put Gentoo on but had 
never gotten around to at that point (tho it's running Gentoo now, 2.6.38 
kernel, kde 4.6.1, fully updated as of mid-March).  It was still running 
the Linpus Linux it shipped with (first full system I've bought since my 
original 486SX25 w/ 2MB memory and 130 MB hard drive in 1993, or so, and 
I'd have sooner done without the netbook than pay the MS tax, I DID have 
to order it from Canada and have it shipped to the US).  I was able to get 
online with that, grab a yahoo webmail account since my mail logins were 
stuck on the main system without a BIOS, and use that to order a new BIOS 
chip shipped to me, the target BIOS pre-installed.

That new BIOS chip rescued my system!

I suspect my feelings after that BIOS chip did the trick rather mirror 
yours after that gear did the trick for your clock.  The computer might 
not be 190 years old, but 2003 is old enough in computer years, and I 
suspect I have rather more of my life wound up in that computer than you 
do in that clock, 190 years old or not.

Regardless, tho, you'll surely agree,

WHAT A RELIEF TO SEE IT RUNNING AGAIN!  =:^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-desktop] System problems

[eamjr56]

Evidently your not all that good a sysadmin as you claim to be and you surely can't admin a Gentoo box. The solution is easy, you are making it hard. Wipe your drive and install ubuntu. 
Sent from my Verizon Wireless BlackBerry

-----Original Message-----
From: Lindsay Haisley <fmouse-gentoo@fmp.com>
Date: Sun, 20 Mar 2011 11:08:37 
To: <gentoo-desktop@lists.gentoo.org>
Reply-to: gentoo-desktop@lists.gentoo.org
Subject: Re: [gentoo-desktop] System problems

On Sun, 2011-03-20 at 09:50 +0100, Roman Zilka wrote:
> By the way, why not gentoo-sources-2.6.36-r5? I hope you have a good
> reason to run a system as ancient as that. Your system is swarming with
> widely known security holes. 

Yes, I have a good reason for this, and I'll be responsible for security
on the box, which I'm well able to do.  Security, and the relative
antiquity of the portage tree are not issues for which I'm seeking
advice, so your observations are totally beside the point, and not
particularly welcome.  I'm well aware of both issues.

The challenge here is to get the system to boot with the newer kernel
and version of udev which I cited in my previous post.

/etc/fstab has been edited several times, as I noted in my post.  The
kernel, udev and /etc/fstab have been now been reverted, as I also noted, so
I could get the desktop working.  Considering that, posting any of the
information you've asked for would probably be useless.

Roman, if you don't have any useful insights based on the information I
already posted, then please don't post on thread and leave it to others
who may.

Having said this, I'll ask you one question:

> Also, by upgrading to a little less ancient versions than 2.6.29
> you won't have the same situation like now boomerang back at you in the
> near future.

Can you cite a source or sources for this assertion?  Is there a known
problem with kernel 2.6.29, or the portage tree which spec'd that
kernel?  Is there any discussion, bug report, or anything that you can
cite noting that this was a known problem at one point and addressed at
a later date?

In almost every case, I've found that people who lecture me online about
my system admin practices don't really have a handle on the issue about
which I'm writing.  Please prove me wrong :-)

-- 
Lindsay Haisley       |"What if the Hokey Pokey really IS all it  
FMP Computer Services |    really is about?"
512-259-1190          |
http://www.fmp.com    |          -- Jimmy Buffett


-- 
Lindsay Haisley       |"Windows .....
FMP Computer Services |    life's too short!"
512-259-1190          |
http://www.fmp.com    |              - Brad Johnston

Re: [gentoo-desktop] System problems

[Lindsay Haisley]

On Sun, 2011-03-20 at 23:59 +0000, eamjr56@gmail.com wrote:
> Evidently your not all that good a sysadmin as you claim to be and you
> surely can't admin a Gentoo box. The solution is easy, you are making
> it hard. Wipe your drive and install ubuntu. 

<LOL!>  As a matter of fact, my new desktop box _will_ be running Ubuntu
desktop.  I'm not out to prove my Linux cajones by making work for
myself that I don't need ;-)  And I'm not going to insult everyone on
this forum by running down Gentoo, which has displayed some absolutely
brilliant programming skills on the part of many of the developers.  I
expect the same respect in return, eamjr56.

-- 
Lindsay Haisley       | "The difference between a duck is because
FMP Computer Services |    one leg is both the same"
512-259-1190          |     - Anonymous
http://www.fmp.com    |

Re: [gentoo-desktop] System problems

[Lindsay Haisley]

On Sun, 2011-03-20 at 23:59 +0000, eamjr56@gmail.com wrote:
> your not all that good a sysadmin as you claim to be and you surely
> can't admin a Gentoo box.

As a matter of fact, eamjr56, in addition to my desktop system, I admin
a Linux firewall and two commercial public servers, all running Gentoo
Linux.  All of them work well, and have never been hacked or
compromised.  I've been doing this for about 7 years, and have been
working professionally with Linux for about 16 years.  I would guess
from your attitude, eamjr56, that I've probably _forgotten_ more about
Linux than you know, and I really don't need to prove myself to you, or
anyone else.

'nuff said.  Please do me a favor and excuse yourself from any further
postings to this thread.

-- 
Lindsay Haisley       | "Real programmers use butterflies"
FMP Computer Services |  - xkcd
512-259-1190          |
http://www.fmp.com    |

[gentoo-desktop] Re: System problems

[Duncan]

Lindsay Haisley posted on Sat, 19 Mar 2011 23:46:06 -0500 as excerpted:

> I'm caught between a rock and a hard place.
> 
> I've been running this desktop box using kernel 2.6.23-gentoo-r3 and
> have come to the point at which there are too many dependencies and
> reverse dependencies, so I _have_ to upgrade to kernel 2.6.29-gentoo-r5
> and have been unable to bring the system up in the new kernel.  Here's
> what's happening.

You're still running 2.6.23 on a /desktop/?  I can see how it could be 
argued for a server that had /years/ of uptime, but for a desktop... I 
/really/ think you should upgrade a bit more often (as it would seem 
you're unfortunately finding out)...

> The newer kernel requires a newer version of udev, which I emerged.  The
> system came up with a root device of some sort mounted, I think in
> single-user mode, but couldn't mount other devices.
> 
> So I changed the main drive designations to UUID's in /etc/fstab,
> re-emerged the newer udev, and tried again.  This time I got a message
> that the kernel needed a root parameter at boot time.  It seems that all
> my /dev/hda? drives have been renamed /dev/sda? so I set gave
> "root=/dev/sda4" as a kernel parameter and got a little further.

Yes.  The old IDE subsystem used hdX device names.  The newer libata based 
subsystem handles both PATA and SATA, using the traditionally SCSI device 
names of sdX instead of the traditionally IDE names of hdX.

Of course, that's years'-old news by now, for anyone taking the "rolling" 
in rolling update at anything close to face value...  Seriously.  If you 
/are/ going to let things get that outdated, I'd recommend something like 
CentOS.  At least there, such big updates so far apart are semi-standard, 
and thus, both the distribution itself and its community are better 
equipped to handle them.  

Gentoo, OTOH, is a /rolling/ /update/ distribution.  They actually find it 
necessary to warn people not to (routinely) sync more than once a day, and 
updating daily to perhaps at the outside monthly, really is how it works 
best.  I believe you've seen this "sermon" before so I'll keep it short, 
but honestly, I'm not just saying this, if you're not going to go rolling 
update with it, and rolling update is NOT for everybody, you really
/should/ reevaluate whether Gentoo is the really the best matching choice 
for your needs.  An "impedance mismatch" of that magnitude may be possible 
to work around, but just because it's possible doesn't mean it's the most 
efficient choice available, for sure.

(The two following paragraphs are recent personal experience with an
8-month-out update.  Skip if you wish...

FWIW, I just upgraded my netbook from about 8 months out, kde 4.4.5, 
skipping 4.5 entirely, to 4.6.1.  I was able to do it because I understand 
reasonably well both the kernel and the Gentoo distribution as well as the 
hardware, following developments quite closely even when I'm not actually 
updating, AND because I kept the usual incremental rolling updates on my 
workstation during the period, so I'd done all except the hardware-
specific updates before, just not all at once, but I'll tell you what, I'd 
have a whole lot rather kept it updated at least every 2-3 months maximum. 

... Further, while it's probably fair to say it was easier for me to do 
the 8-month update than to backup /etc, /home and the kernel config, and 
start from a clean stage-3 tarball, it wasn't so by much, and for people 
who don't keep such close track of Gentoo and Linux developments in 
general and/or who hadn't been regularly updating other systems so as to 
have gone thru most of the updates already, just not all at once, starting 
from that clean stage-3 probably would have been easier!  And I can say 
that as I recently helped a friend install from a stage-3, too, so I've 
done both recently.  Thus, I now have personal experience to confirm what 
has been previously stated in other threads, that a six-month update is 
approaching the outer limits of practicality for an ordinary Gentooer, and 
that somewhere between that six months and a year, installing from a fresh 
stage-3 does become the most practical option.)

> After
> "Checking root filesystem" in the boot sequence, I got a message that
> the UUID for the root filesystem wasn't understood in /etc/fstab.

> So I set the root filesystem in /etc/fstab to /dev/sda4, and got the
> same error - that "/dev/sda4" was not understood either, although the
> kernel seemed to understand this just fine as a boot parameter, and once
> again, I'm dumped into a very limited single-user mode.
> 
> So I'm stuck!  I had to boot from a rescue disk, back-version to
> udev-141 and revert to kernel 2.6.23-gentoo-r3 to get my desktop back.
> 
> What do I need to put into /etc/fstab to satisfy the kernel?  I need to
> move forward with this, but I need my desktop system to run my business.
> Any _real_ suggestions will be welcome.  Please be aware that I'm no
> Linux novice, so don't give me novice advice.  I've been building,
> running, and getting paid to admin Linux systems since 1995.

Do you run an initrd/initramfs?  (AFAIK you do if you use genkernel, since 
it pretty much compiles everything as modules, putting what it thinks 
necessary to load the real-root in the initrd/initramfs.)

I'll skip the detail on init(rd|ramfs) both because I don't use it myself 
and because as you mention, you /aren't/ a novice, so if you do use one, 
you can probably tell me more about how it works than I can tell you, but..

Either...

1)  it's loading the kernel and dropping you in the initrd/initramfs, 
because it can't load real-root,

OR

2) It's loading root initially (either you don't have an initrd/initramfs 
or it gets past that), which normally defaults to read-only mode, and is 
failing when it tries to do the initial rootfs fsck and/or when it tries 
to do the remount read/write, which occurs just after that.

In either case, since you specifically mention that you get a limited 
single-user-mode, NOT a kernel panic as it tries to read the initial root 
(whether that's an initrd/initramfs or the real root), the kernel 
definitely DOES know how to read and mount that initial root and drop you 
into /some/ sort of user mode, even if limited single-user.

(It's worth noting here that one of the issues I experienced with that
8-month update above, was that somehow, when I updated the kernel from 
2.35-rc6-gitsomething, what I'd been running before, to 2.6.38, what I'm 
running now on both workstation and netbook, the config option for my SATA 
chipset vanished when I did make oldconfig, and my first kernel rebuild 
panicked when it couldn't find root.  I'm familiar enough with the 
kernel's boot messages and the hardware that I realized that it couldn't 
see sda at all, which meant either the hardware was dying and that didn't 
seem to be the case, or it wasn't loading the driver, so I deduced the 
problem almost immediately, and sure enough, when I checked the kernel 
config, it wasn't building that driver (I don't use modules, only the 
options I need, built-in), so change the option to build it, rebuild and 
install the kernel, and try again.  It worked once it had the driver for 
the sata controller, but the point is, I use all built-ins and no init(rd|
ramfs), so it kernel panicked before loading any userspace at all.)

If you do NOT use an initr*, the fact that it's loading even the single-
user userspace indicates that it not only properly loads the PATA driver 
and detects the hardware, but that it ALSO groks the rootfs and loads it 
read-only.  In that case, your problem really is one with fsck/mount/fstab 
or the scripts that invoke them.

OTOH, if you DO use an initr*, the first thing to determine is whether 
it's getting past that and dropping you into the real-root, in which case 
you have the same as above fsck/mount/fstab or script issues to look at, 
*OR*, whether you're getting dropped into that limited userspace while 
it's still initr*-based-only, in which case your problem is with the 
initr*, NOT with the real-root that the system later pivot-root-s to.

Given the info you posted, I'll predict that the most likely case, and I 
state this again with the caveat that I don't use initr* personally so 
don't know that much about it, is that you DO use an initr*, and are stuck 
in it, as it either doesn't have the correct kernel PATA driver and thus 
can't see the drive at all, or it has that but is missing the correct 
filesystem module, so it sees the drive but can't understand the 
filesystem you're pointing it at.

From what I've /read/ about initr*, back when initrds were the norm, the 
most common problem here was that either the updated kernel entry line 
still pointed at the old initrd, or that the initrd hadn't been updated at 
all.  In either case, the actually loaded initrd had the modules for the 
old kernel, which the new kernel would refuse to load.

Initramfs made that problem much less common as the initramfs is now 
appended to the end of the kernel file making it far more difficult to 
have a kernel/initr* mismatch, but the same effect sometimes happens, if 
the new kernel build doesn't build the appropriate modules or for whatever 
reason they're not included in the initramfs.

But beyond that you'll need to find someone with more initr* knowledge to 
help if it's an initr* issue, which I suspect it is.  It's a reasonably 
common problem, but one I have no experience with and don't know the 
details of the fix on.

If you're actually getting your real-root loaded, either because you don't 
use an initr* or because it's correct and you're successfully getting the 
pivot-root, then as I said, it would appear to be a problem with fsck, 
mount, fstab, or possibly the initscripts invoking them.  However, those 
are less common and there's not enough data in the original post to really 
go further with it at this point.

But I bet (and hope) it's an initr* problem... simply because those are 
most common, and normally easy to solve -- simply make sure you're loading 
the one corresponding to the new kernel and that it has the correct 
drivers and config...

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-desktop] Re: System problems

[Lindsay Haisley]

On Sun, 2011-03-20 at 09:32 +0000, Duncan wrote:
> Gentoo, OTOH, is a /rolling/ /update/ distribution.  They actually find it 
> necessary to warn people not to (routinely) sync more than once a day, and 
> updating daily to perhaps at the outside monthly, really is how it works 
> best.  I believe you've seen this "sermon" before so I'll keep it short, 
> but honestly, I'm not just saying this, if you're not going to go rolling 
> update with it, and rolling update is NOT for everybody, you really
> /should/ reevaluate whether Gentoo is the really the best matching choice 
> for your needs.  An "impedance mismatch" of that magnitude may be possible 
> to work around, but just because it's possible doesn't mean it's the most 
> efficient choice available, for sure.

Duncan, thank you.  I'm well aware of the advantages and disadvantages
of Gentoo's "rolling update" distribution design, and really appreciate
the advantages.  My problem isn't knowledge, of which I have enough, but
time.  Maintaining a Gentoo system is a relatively time consuming
activity, and I'm spread extremely thin these days.  This problem box
has been running Gentoo since 2004, but my work and priorities on my
time have changed over the years and it's probably time to move on.
Gentoo has gotten a lot more complex since then - arguably better, but
certainly more time-consuming to maintain.

In the meantime, I need to address the kernel | fstab | udev issue here.
Insights and suggestions to address _this_ problem would be most
helpful.  Lectures on the age of my system, my qualifications as an
admin for my own desktop, or my continued us of Gentoo as a desktop
distribution are not helpful.

-- 
Lindsay Haisley       |"Windows .....
FMP Computer Services |    life's too short!"
512-259-1190          |
http://www.fmp.com    |              - Brad Johnston

Re: [gentoo-desktop] Re: System problems

[Lindsay Haisley]

On Sun, 2011-03-20 at 11:50 -0500, Lindsay Haisley wrote:
> This problem box
> has been running Gentoo since 2004, but my work and priorities on my
> time have changed over the years and it's probably time to move on.

To be a bit more precise here, I'm actively working on replacing this
box, and will be running a Linux distribution on it other than Gentoo.
But in the meantime, I do need to solve the problem about which I
posted.

-- 
Lindsay Haisley       |"Friends are like potatoes.
FMP Computer Services |    If you eat them, they die"
512-259-1190          |
http://www.fmp.com    |              - Aaron Edmund

[gentoo-desktop] Re: System problems

[Duncan]

Lindsay Haisley posted on Sun, 20 Mar 2011 13:48:43 -0500 as excerpted:

> On Sun, 2011-03-20 at 11:50 -0500, Lindsay Haisley wrote:
>> This problem box has been running Gentoo since 2004, but my work and
>> priorities on my time have changed over the years and it's probably
>> time to move on.
> 
> To be a bit more precise here, I'm actively working on replacing this
> box, and will be running a Linux distribution on it other than Gentoo.
> But in the meantime, I do need to solve the problem about which I
> posted.

I appreciate that you /have/ decided to ultimately switch, and obviously 
believe it best or I'd have not recommend it.

I also appreciate that you need a solution for now.  That's what I suspect 
I may have provided (at least to a point) further down that post, which 
you may not have read given the order.

I was hoping to read all these replies and have you tell me whether I was 
on the right track or not, answering the question about initrd/initramfs 
so we could go from there if need be.  However, I don't see that answer, 
which suggests you didn't read that far down my post, for which I can't 
really blame you.

But you can always go back and do it now... =:^)

Briefly repeating so you can see if it's worth your trouble going back to 
get the details.

If the kernel could not load its initial root, be that initr* or the real-
root, it'd fail to load any userspace at all as it couldn't get to it, 
panicking instead when it couldn't get to a root at all.  (I know this 
from experience.)

Thus, that it's getting even a /limited/ userspace indicates it's getting 
to it's initial root.  That's a very significant point.

The question from there is whether that limited userspace is loading from 
the initrd/initramfs if you have one, indicating it's not successfully 
doing the initial real-root mount and pivot-root from the initr*, taking 
you down one path toward a solution (missing or incorrect drivers in the 
initr* or wrong initrd), or whether it's loading the real-root (as just 
getting to userspace at all indicates if you don't have an initr*), thus 
indicating that it has the necessary drivers (both pata and fs) to do so, 
and the problem is later, with the fsck or remount, taking you down a 
different path toward a solution (userspace issue).

If that sounds useful, check the earlier post for more.  If it doesn't, 
don't bother, as that's what I was detailing in the earlier post.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-desktop] System problems

[James Cloos]

>>>>> "LH" == Lindsay Haisley <fmouse-gentoo@fmp.com> writes:

LH> It seems that all
LH> my /dev/hda? drives have been renamed /dev/sda? so I set gave
LH> "root=/dev/sda4" as a kernel parameter and got a little further.  After
LH> "Checking root filesystem" in the boot sequence, I got a message that
LH> the UUID for the root filesystem wasn't understood in /etc/fstab.

LH> So I set the root filesystem in /etc/fstab to /dev/sda4, and got the
LH> same error - that "/dev/sda4" was not understood either, although the
LH> kernel seemed to understand this just fine as a boot parameter, and once
LH> again, I'm dumped into a very limited single-user mode.

It sounds like you do not have the required block special files in /dev.

With current udev, I use devtmpfs for /dev.  It looks like having
devtmpfs support compiled into the kernel is enough.

If you don't have that, or if there is something in /etc/fstab competing
against /etc/init.d/udev-mount, that might explain the problem.

-JimC
-- 
James Cloos <cloos@jhcloos.com>         OpenPGP: 1024D/ED7DAEA6

Re: [gentoo-desktop] System problems

[Lindsay Haisley]

On Wed, 2011-03-23 at 08:39 -0400, James Cloos wrote:
> With current udev, I use devtmpfs for /dev.  It looks like having
> devtmpfs support compiled into the kernel is enough.
> 
> If you don't have that, or if there is something in /etc/fstab
> competing
> against /etc/init.d/udev-mount, that might explain the problem.

Thanks.  Jorge Vicetto noted that having CONFIG_SYSFS_DEPRECATED_V2 set
in the kernel config is a major no-no with recent versions of udev, and
will screw up device creation.  This kernel config setting was indeed
set, which may well have accounted for this particular incarnation of
the problem.  Roman Zilka noted that the 2.6.29 kernel I was using when
this happens has known issues and isn't even in the Gentoo portage tree
anymore.  So thanks to Roman and Jorge I have a 2.6.36 kernel built
(without the problem setting) which I'll try when I get a moment.  Being
a day late and a dollar short at the moment I'm a bit short on round
tuits :-)

-- 
Lindsay Haisley       | "The difference between a duck is because
FMP Computer Services |    one leg is both the same"
512-259-1190          |     - Anonymous
http://www.fmp.com    |

Re: [gentoo-desktop] System problems

[Dale]

Lindsay Haisley wrote:
>
> Thanks.  Jorge Vicetto noted that having CONFIG_SYSFS_DEPRECATED_V2 set
> in the kernel config is a major no-no with recent versions of udev, and
> will screw up device creation.  This kernel config setting was indeed
> set, which may well have accounted for this particular incarnation of
> the problem.  Roman Zilka noted that the 2.6.29 kernel I was using when
> this happens has known issues and isn't even in the Gentoo portage tree
> anymore.  So thanks to Roman and Jorge I have a 2.6.36 kernel built
> (without the problem setting) which I'll try when I get a moment.  Being
> a day late and a dollar short at the moment I'm a bit short on round
> tuits :-)
>
>    

You doing the same as me?  I'm spreading mulch in my garden with the 
tractor.  I need a bigger tractor, a front end loader or a nice size 
dozer.  My mulch pile is the size of a house and I'm having to push as 
far as 200' with a little 25hp tractor with a bucket.  I'm glad it is 
4WD tho.  I would have been stuck long ago.  lol

Let us know if you get it fixed.

Dale

:-)  :-)