[gentoo-user] file system problems - what is proper maintanence?

[gentoo-user] file system problems - what is proper maintanence?

[Mark Knecht]

Hi all,
   I am a guitar player. I have a hardware engineering background but
I am not a sys admin. I use computers as tools. Gentoo has been the
most stable and day to day productive tool I've ever used.

   That said, I seem to have file system problems on my external 1394
hard drives. I do not know if this is due to a recent move to
2.6.17-rt1, or bad maintenance on my part, or just bad luck. I've
noticed over the last couple of days that when using Aqualung (a
non-portage music player) I was getting some strange results. Songs
that used to work not working, strange timing, etc. Just weird stuff.

   This evening, for no good reason since I don't know what I am
doing, I decided to run fsck and let it look around and fix things if
it wanted to. I wouldn't know how to do any better. At the end of this
I wondered if there is really any value to a journaled file system?
Did it protect me? I cannot tell.

   Here is how these drives are mounted:

LABEL=music             /home/mark/music        ext3    rw,noauto,user  0 0
LABEL=audio1_94b        /home/mark/Audio/audio1 ext3    rw,noauto,user  0 0

Both partitions are on external 1394 drives, but not the same drives.

   Following down below are the sorts of problems I found using fsck.
The problem is it it didn't fix most of the problems I found with the
failing songs. Two seem to play now but the rest do not. Did the
journal dot work? Did I do something wrong?

   NOTE: The two Porcupine Tree tracks do not play so they seem to be damaged.

Thanks,
Mark


lightning ~ # fsck /dev/sdb1
fsck 1.38 (30-Jun-2005)
e2fsck 1.38 (30-Jun-2005)
audio1_94b has been mounted 57 times without being checked, check forced.
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
/lost+found not found.  Create<y>? no

Pass 4: Checking reference counts
Pass 5: Checking group summary information

audio1_94b: ********** WARNING: Filesystem still has errors **********

audio1_94b: 1492/3662848 files (19.1% non-contiguous), 4014481/7325632 blocks
lightning ~ #





lightning ~ # fsck /dev/sdc1
fsck 1.38 (30-Jun-2005)
e2fsck 1.38 (30-Jun-2005)
music: recovering journal
music contains a file system with errors, check forced.
Pass 1: Checking inodes, blocks, and sizes
Inode 6115654 has illegal block(s).  Clear<y>? yes

Illegal block #1036 (1376776629) in inode 6115654.  CLEARED.
Illegal block #1037 (1927720517) in inode 6115654.  CLEARED.
Illegal block #1038 (2407185755) in inode 6115654.  CLEARED.
Illegal block #1039 (3472556839) in inode 6115654.  CLEARED.
Illegal block #1040 (2208668609) in inode 6115654.  CLEARED.
Illegal block #1041 (3293443330) in inode 6115654.  CLEARED.
Illegal block #1042 (375135243) in inode 6115654.  CLEARED.
Illegal block #1043 (1249063009) in inode 6115654.  CLEARED.
Illegal block #1044 (1201632328) in inode 6115654.  CLEARED.
Illegal block #1045 (2618458457) in inode 6115654.  CLEARED.
Illegal block #1046 (3116863600) in inode 6115654.  CLEARED.
Too many illegal blocks in inode 6115654.
Clear inode<y>? yes

Error reading block 12279239 (Attempt to read block from filesystem
resulted in short read) while reading indirect blocks of inode
6115655.  Ignore error<y>? yes

Force rewrite<y>? yes

Inode 6115655 has illegal block(s).  Clear<y>? yes

Illegal block #1036 (937597786) in inode 6115655.  CLEARED.
Illegal block #1037 (2426970144) in inode 6115655.  CLEARED.
Illegal block #1038 (2863076451) in inode 6115655.  CLEARED.
Illegal block #1039 (2251569750) in inode 6115655.  CLEARED.
Illegal block #1040 (1393869746) in inode 6115655.  CLEARED.
Illegal block #1041 (382868669) in inode 6115655.  CLEARED.
Illegal block #1042 (1149983442) in inode 6115655.  CLEARED.
Illegal block #1043 (4028337212) in inode 6115655.  CLEARED.
Illegal block #1044 (1162660121) in inode 6115655.  CLEARED.
Illegal block #1045 (1576522261) in inode 6115655.  CLEARED.
Illegal block #1046 (2289199545) in inode 6115655.  CLEARED.
Too many illegal blocks in inode 6115655.
Clear inode<y>? yes

Restarting e2fsck from the beginning...
music contains a file system with errors, check forced.
Pass 1: Checking inodes, blocks, and sizes
Error reading block 1531447 (Attempt to read block from filesystem
resulted in short read) while reading indirect blocks of inode 719553.
 Ignore error<y>? yes

Force rewrite<y>? yes

Inode 719553, i_blocks is 16144, should be 8304.  Fix<y>? yes


Running additional passes to resolve blocks claimed by more than one inode...
Pass 1B: Rescanning for multiply-claimed blocks
Multiply-claimed block(s) in inode 719554: 1532443
Pass 1C: Scanning directories for inodes with multiply-claimed blocks.
Pass 1D: Reconciling multiply-claimed blocks
(There are 1 inodes containing multiply-claimed blocks.)

File /Christmas CDs/Star Bright (Vanessa Williams)/11 - Go Tell It On
The Mountain.ogg (inode #719554, mod time Fri Jan 28 14:51:35 2005)
  has 1 multiply-claimed block(s), shared with 0 file(s):
Multiply-claimed blocks already reassigned or cloned.

Pass 2: Checking directory structure
Entry '09 - Every Home Is Wired.ogg' in /Porcupine Tree/Signify
(6115649) has deleted/unused inode 6115654.  Clear<y>? yes

Entry '02 - Signify.ogg' in /Porcupine Tree/Signify (6115649) has
deleted/unused inode 6115655.  Clear<y>? yes

Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
Block bitmap differences:  -(1531448--1532427) -1532441 -(12277546--12280396)
Fix<y>? yes

Free blocks count wrong for group #46 (0, counted=981).
Fix<y>? yes

Free blocks count wrong for group #374 (2884, counted=5735).
Fix<y>? yes

Free blocks count wrong (2464583, counted=2468415).
Fix<y>? yes

Inode bitmap differences:  -(6115654--6115655)
Fix<y>? yes

Free inodes count wrong for group #374 (16328, counted=16330).
Fix<y>? yes

Free inodes count wrong (7317173, counted=7317175).
Fix<y>? yes


music: ***** FILE SYSTEM WAS MODIFIED *****
music: 8521/7325696 files (15.2% non-contiguous), 12182857/14651272 blocks
lightning ~ #
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] file system problems - what is proper maintanence?

[Benno Schulenberg]

Mark Knecht wrote:
>    I seem to have file system problems on my external
> 1394 hard drives. I do not know if this is due to a recent move
> to 2.6.17-rt1, or bad maintenance on my part, or just bad luck.

Hopefully just bad luck (a failing drive, or a power hickup), but a 
kernel bug can't be ruled out.  As for bad maintenance... with any 
data that you value, a regular disk inspection is needed: a maximum 
mount count of something like 15 or an interval of two weeks.  

>    This evening, for no good reason since I don't know what I am
> doing, I decided to run fsck and let it look around and fix
> things if it wanted to. I wouldn't know how to do any better.

Use tune2fs to set the checking interval to 1 week.  Or better yet, 
to 1 day for now, to keep inspecting the disk everyday for a while.
Use 'tunefs -l ...' to see the current values.

> At the end of this I wondered if there is really any value to a
> journaled file system? Did it protect me? I cannot tell.

It won't protect you when the disk is going bad, nor when the kernel 
has bugs.  It only protects you from losing data during a sudden 
power loss.

Benno
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] file system problems - what is proper maintanence?

[Mark Knecht]

On 6/25/06, Benno Schulenberg <benno.schulenberg@gmail.com> wrote:
> Mark Knecht wrote:
> >    I seem to have file system problems on my external
> > 1394 hard drives. I do not know if this is due to a recent move
> > to 2.6.17-rt1, or bad maintenance on my part, or just bad luck.
>
> Hopefully just bad luck (a failing drive, or a power hickup), but a
> kernel bug can't be ruled out.  As for bad maintenance... with any
> data that you value, a regular disk inspection is needed: a maximum
> mount count of something like 15 or an interval of two weeks.

OK, well these drives were never being tested. I've not been clear how
to best do that on an external drive that is removable like 1394. I've
set the test interval to 1 day for now. We'll see if it pops anything
up.

As for the data, like everyone, I value it, but it's primarily my CD
collection which has been ripped for easy playing. My thought was that
since the disk isn't written very often (when I get a new CD) and
things are mostly read-only, there shouldn't be many problems, but
that doesn't save me from a disk going bad.

Is it the case that the journal is only used to repair the disk when
fsck is run? I don't understand when ext3 would use that.

Also, if these drives are turned off at night, then on again the next
day, and mounted by the user, then where does fsck info from the daily
check go? dmesg???

>
> >    This evening, for no good reason since I don't know what I am
> > doing, I decided to run fsck and let it look around and fix
> > things if it wanted to. I wouldn't know how to do any better.
>
> Use tune2fs to set the checking interval to 1 week.  Or better yet,
> to 1 day for now, to keep inspecting the disk everyday for a while.
> Use 'tunefs -l ...' to see the current values.

Yes, 1 day for now on all partions of both 1394 drives.

>
> > At the end of this I wondered if there is really any value to a
> > journaled file system? Did it protect me? I cannot tell.
>
> It won't protect you when the disk is going bad, nor when the kernel
> has bugs.  It only protects you from losing data during a sudden
> power loss.
>
> Benno

Thanks Benno!

Cheers,
Mark
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] file system problems - what is proper maintanence?

[Bo Ørsted Andresen]

[-- Attachment #1: Type: text/plain, Size: 487 bytes --]

On Sunday 25 June 2006 02:57, Mark Knecht wrote:
> That said, I seem to have file system problems on my external 1394
> hard drives. I do not know if this is due to a recent move to
> 2.6.17-rt1, or bad maintenance on my part, or just bad luck.

You might want to have a look at smartmontools:

http://gentoo-wiki.com/HOWTO_Monitor_your_hard_disk%28s%29_with_smartmontools
http://en.wikipedia.org/wiki/Self-Monitoring%2C_Analysis%2C_and_Reporting_Technology

-- 
Bo Andresen

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-user] file system problems - what is proper maintanence?

[Benno Schulenberg]

Mark Knecht wrote:
> Is it the case that the journal is only used to repair the disk
> when fsck is run?

No, it's more like either/or.  With an unjournaled file system, it 
is fsck that checks everything and repairs inconsistencies.  With a 
journaled file system, the journal is used to complete the file 
operations that were interrupted by a systen crash, lockup, or 
power loss.  There shouldn't be any inconsistencies left.  If there 
are, the disk has made mistakes, either due to a one-time brown-out 
or impending failure.

Google for: how does a journaled filesystem work.  The first two 
hits should tell it all.

> Also, if these drives are turned off at night, then on again the
> next day, and mounted by the user, then where does fsck info from
> the daily check go? dmesg???

Nowhere.  You need to modify the init scripts if you want to save 
any of the info that they produce.  Or maybe there's some provision 
for this now in the scripts, I don't know, I've changed them.

Benno
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] file system problems - what is proper maintanence?

[Richard Fish]

On 6/24/06, Mark Knecht <markknecht@gmail.com> wrote:
>    That said, I seem to have file system problems on my external 1394
> hard drives. I do not know if this is due to a recent move to
> 2.6.17-rt1, or bad maintenance on my part, or just bad luck. I've

Well, I've had various problems with external drives.  I have 3
1394/USB2 combo drives that I use for backups.  If I connect them via
1394, they will start to have problems at some point during my
backups....sometimes several Gb will transfer fine, and then it will
hiccup with various read/write failures.  I *never* have a problem
with these same drives connected via USB.

In another event, I moved a 2.5" drive from a slightly damaged USB2
case to a new one.  The new case started corrupting data written to
it, without any warning or errors.  Filesystem checks would always
report no errors, but for example I could transfer kernel sources to
the drive, but they would fail to extract from there.  I eventually
replaced the case, (same drive) and that cleared up all problems, but
I had to restore the files from backups as they were actually
corrupted on write (vs just not being able to read them correctly).

So, what I really recommend you do for awhile is to do an md5sum of
any files that you transfer to the drive before transferring them.
And then periodically do a md5sum -c to verify the file contents.
That is really the best way to make sure your drives are working
right, and that your data hasn't been corrupted.

And if they are combo drives, try the USB2 connection instead.

> I wondered if there is really any value to a journaled file system?
> Did it protect me? I cannot tell.

With one exception, journaled filesystems are designed to protect
themselves, not your data.  That is, if a crash occurs in the middle
of an operation, the filesystem meta-data will not be corrupted.  It
will know how large all files should be, which blocks are allocated to
which files, whether any files should be deleted, and so on.  It does
not guarantee the actual file data however.  But this 'guarantee' is
useless if a hardware problem is corrupting the filesystem, which
appears to be the problem here.

The one exception is ext3 when mounted with the 'data=journal' option.
 But this makes the filesystem fairly slow, and still doesn't help
with hardware problems....

-Richard
-- 
gentoo-user@gentoo.org mailing list