[gentoo-user] Reiserfs meltdown

[gentoo-user] Reiserfs meltdown

[Ralph Slooten]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hiya list,

Just need some opinions here, and am not looking for a raving flame-war
regarding which file system is better etc ;-) ~ Oh and please excuse the
long mail, but I need explain my situation clearly to avoid confusion.

Last week Friday while I was work I successfully upgraded my home
workstation via ssh to the current Xorg 7.0. The wife came home and
unintentionally turned Off the main power to the PC instead of On (she
thought the PC was off). Since then everything started going really bad
on my root partition.

KDE needed a few things to be upgraded to fix dependencies which seemed
to trigger the following errors. Random KDE components (like kdm) would
sometimes start, sometimes not, depending on the reboot. Sometimes in
/var/log/messages there were hints to *missing* *.so files in the
/usr/kde/3.5/lib folder, yet they did *appear* to be there ~ although
when doing a simple `ls` of the lib directory I got (depending on the
reboot) between 10 and 30 errors about missing files or directories. It
seemed that reiserfs had "catalogued" that files were supposed to be
there, but `ls /usr/kde/3.5/lib` could not find them.

For the record I am using reiserfs 3.6 (default in vanilla kernel, no
patches) ~ not 4.x.


The lib dir is also included in /etc/ld.so.conf and ldconfig was run
several times to test. After a reboot I would get different errors, and
sometimes none when it would just work (Xorg / kde). `revdep-rebuild -p`
came up after every reboot with different packages, indicating it
detected different missing *.so files after each reboot, mainly in the
/usr/kde/3.5/lib.

Now I know Linux, and errors like this are not normal in any way. I
rebooted with the Gentoo Live-cd and did a few disc scans (fsck) of my
root reiserfs partition. Every single time I ran it it would find errors
and fix. I did a `--rebuild-tree -S` and for 45 minutes I got error
after error after error (thousands), apparently all fixed. Re-running
the scan started the whole error-fixing process again. A "badblocks"
test showed no error on the partition though.

I decided that my reiserfs file tree must have been corrupt, and
formatted the root drive (`mkreiserfs /dev/hda3`) and restored a full
backup (dar).  After a reboot a repreated the scan, to find the same
issues again. It seems a format did not clean the file table or
something, I don't know.

As a last test I formatted the root partition as an ext2 partition, and
again restored the backup. No errors, no bad blocks, no problems.

What gives? I don't want to use ext2 or ext3, and I have for a couple of
years now relied on reiserfs on all my systems, but what could be the
problem here? Why did reiserfs seem to mess up like this, and why after
formatting it did I get the same errors again?

Regards,
Ralph
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (MingW32)

iD8DBQFEu2cgCt0ZF9kLPvYRAqzAAJ9txAJIhhVTnVd1SUwzvfrPHeelWwCfTt5c
+X+APrx+dbAjanSBKcYJOIU=
=PYzO
-----END PGP SIGNATURE-----
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Reiserfs meltdown

[Alan McKinnon]

On Mon, 2006-07-17 at 12:32 +0200, Ralph Slooten wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Hiya list,
> 
> Just need some opinions here, and am not looking for a raving flame-war
> regarding which file system is better etc ;-) ~ Oh and please excuse the
> long mail, but I need explain my situation clearly to avoid confusion.

[snip]

> What gives? I don't want to use ext2 or ext3, and I have for a couple of
> years now relied on reiserfs on all my systems, but what could be the
> problem here? Why did reiserfs seem to mess up like this, and why after
> formatting it did I get the same errors again?

It looks like you have a problem with some reiser-related binary that is
not on the / partition. There is obviously nothing wrong with hda3 as
ext3 works on it. Which partition hosts the /lib and /sbin directories?

Unfortunately you seem to have been a victim of the off button being hit
at exactly the right moment to cause maximum difficulty :-(

If all else fails you could take the long route:
emerge -e system
emerge -s world
A drastic measure, but it would rebuild everything and almost certainly
fix the problem.

alan


-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Reiserfs meltdown

[Ralph Slooten]

Alan McKinnon wrote:
> It looks like you have a problem with some reiser-related binary that is
> not on the / partition. There is obviously nothing wrong with hda3 as
> ext3 works on it. Which partition hosts the /lib and /sbin directories?

Both are on the same partition too. This goes for everything except
for /boot and /home. The rest is all on / (/dev/hda3).

> Unfortunately you seem to have been a victim of the off button being hit
> at exactly the right moment to cause maximum difficulty :-(

I fear this too yes, however after a re-format (`mkreiserfs
/dev/hda3`) off the boot-cd and restore of filesystem from a backup
this should have been solved. If the backup was damaged, I would have
gotten errors during the initial create, the restore, and also from
the current ext2 "/" partition ~ but I got no errors at all.

> If all else fails you could take the long route:
> emerge -e system
> emerge -s world
> A drastic measure, but it would rebuild everything and almost certainly
> fix the problem.

I fear not actually, as I think this problem is reiserfs-related, and
has to do with a corrupted journal or something, but not sure though.

What is the best way to *really* format a drive before recreating a
journalled filesystem (reiserfs) so that I really know it's not using
an old corrupt one or something? I have other working partitions on
that drive so an fdisk is not possible: ~ `cat /dev/zero > /dev/hda3`
?

-- Ralph
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Reiserfs meltdown

[Janusz Bossy]

On 7/17/06, Ralph Slooten <axllent@gmail.com> wrote:
> What is the best way to *really* format a drive before recreating a
> journalled filesystem (reiserfs) so that I really know it's not using
> an old corrupt one or something? I have other working partitions on
> that drive so an fdisk is not possible: ~ `cat /dev/zero > /dev/hda3`
> ?

I always managed to restore my partition after mkfs.reiserfs and
fsck.resierfs --rebuild-tree -S. It should (at least I think so) clear
the tree and the journal.

-- 
Pozdrawiam
Janusz YANOUSHek Bossy
gg# 791964
tlen yanoushek@tlen.pl
jabber yanoushek@chrome.pl
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Reiserfs meltdown

[Ralph Slooten]

On 17/07/06, Janusz Bossy <janusz.bossy@gmail.com> wrote:
> I always managed to restore my partition after mkfs.reiserfs and
> fsck.resierfs --rebuild-tree -S. It should (at least I think so) clear
> the tree and the journal.

I agree that this *should* fix it, but with my first attempts it found
a couple of thousand errors, apparently fixed them, until I ran it
again where it kept finding the same errors. This was of course with
an already-restored backup.

I will try your way when the filesystem is still empty. Nice tip though, thanks.
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Reiserfs meltdown

[Janusz Bossy]

On 7/17/06, Ralph Slooten <axllent@gmail.com> wrote:
> I agree that this *should* fix it, but with my first attempts it found
> a couple of thousand errors, apparently fixed them, until I ran it
> again where it kept finding the same errors. This was of course with
> an already-restored backup.

IIRC there's also an options to fsck.reiserfs that makes it repair
errors because by default it only show what it has found. Look into
the man page for more detail (I'm currently at work using Windows).

-- 
Pozdrawiam
Janusz YANOUSHek Bossy
gg# 791964
tlen yanoushek@tlen.pl
jabber yanoushek@chrome.pl
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Reiserfs meltdown

[Ralph Slooten]

On 17/07/06, Janusz Bossy <janusz.bossy@gmail.com> wrote:
> IIRC there's also an options to fsck.reiserfs that makes it repair
> errors because by default it only show what it has found. Look into
> the man page for more detail (I'm currently at work using Windows).

--fix-fixable

Yes, I had done this, however each time I ran it it found 2 problems
.... and supposedly fixed them .. that is until I ran it again where
it found the same two errors again ... again and again ;-)

I will (when I get home this evening):

a) format the current ext2 as reiserfs and reboot
b) Rebuild tree (--rebuild-tree -S) with an empty partition
c) run a scan (--fix-fixable)
d) Restore files from backup
e) run the scan again (--fix-fixable)
f) reboot and hope

If this doesn't solve the problem then I have no idea.... Does anyone
foresee problems doing this, or other things I should check too while
at it?

Thanks
Ralph
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Reiserfs meltdown

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 337 bytes --]

On Mon, 17 Jul 2006 15:23:12 +0200, Ralph Slooten wrote:

> If this doesn't solve the problem then I have no idea.... Does anyone
> foresee problems doing this, or other things I should check too while
> at it?

Install and run smartmontools, it could be a drive on the way out.


-- 
Neil Bothwick

There's no place like ~

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-user] Reiserfs meltdown

[Ralph Slooten]

On 17/07/06, Neil Bothwick <neil@digimed.co.uk> wrote:
> Install and run smartmontools, it could be a drive on the way out.

Nice tip .. thanks. I have this on my servers, but not (yet) on workstation.....
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Reiserfs meltdown

[Ralph Slooten]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


For those interested: I formatted the ext2 drive back to reiserfs, which
again gave errors (empty still) with fsck. After three fsck.reiserfs
attempts with loads of wierd output (on the empty drive), I created
using `dd` of /dev/zero 3 huge files which in turn filled the drive to
100%. I then reran the checks and it all seemed fine again.

I restored the full backup, re-ran the checks without errors. I do not
know what it was, but it seems sorted now. It seems I needed to
overwrite all old data to get this issue solved.

Thanks for all your input.

Greetings,
Ralph


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.2 (GNU/Linux)

iD8DBQFEu+XICt0ZF9kLPvYRAjzVAKCK+9KRGWp/jmTOxkBw9bbHft2LRACfXqSM
5Ik3dXMaZWYfTOfWMZ4x1do=
=3ZBV
-----END PGP SIGNATURE-----
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Reiserfs meltdown

[Bryan Whitehead]

On Mon, 17 Jul 2006, Ralph Slooten wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hiya list,
>
> Just need some opinions here, and am not looking for a raving flame-war
> regarding which file system is better etc ;-) ~ Oh and please excuse the
> long mail, but I need explain my situation clearly to avoid confusion.

[snip]

I've had this happen to me a number of times... I'm now a happy XFS 
user. :)

</flamestart ;) >

-- 
Bryan Whitehead
Email:driver@megahappy.net
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Reiserfs meltdown

[Mick]

[-- Attachment #1: Type: text/plain, Size: 982 bytes --]

On Tuesday 18 July 2006 06:17, Bryan Whitehead wrote:
> On Mon, 17 Jul 2006, Ralph Slooten wrote:
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> >
> > Hiya list,
> >
> > Just need some opinions here, and am not looking for a raving flame-war
> > regarding which file system is better etc ;-) ~ Oh and please excuse the
> > long mail, but I need explain my situation clearly to avoid confusion.
>
> [snip]
>
> I've had this happen to me a number of times... I'm now a happy XFS
> user. :)
>
> </flamestart ;) >

No flamestart, but . . .

I have had the opposite experience:  reiserfs surviving happily multiple 
lockups due to faulty memory with no fs corruption.  Installed /usr on an xfs 
(laptop) and after a couple of hard reboots it was borked!  This has happened 
a number of times to the extent that I became convinced reiserfs is the 
preferred fs for me.  Mind you I am still running /usr on xfs.  ;-)

Just my 2c's.
-- 
Regards,
Mick

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]