[gentoo-user] 2.6.31 vfat driver broken?

[gentoo-user] 2.6.31 vfat driver broken?

[Frank Steinmetzger]

[-- Attachment #1: Type: text/plain, Size: 769 bytes --]

Hi guys n gals

I use FAT32 on my external HDDs to make it easier to share with other people 
and OSes. Never had a problem before, but now I do. Lately, when I save 
videos to my disks, and play them back after the file system cache is 
emptied, they have completely different content (of files that are long 
deleted).

I just had the magnificent idea to look into the syslog and found loads of
"kernel: bio too big device sdb (248 > 240)"

I heard romours of problems with the current FAT implementation due to M$. I 
went back to 2.6.30 for the moment. So what’s your proposal? Usually I don’t 
have the need for übercurrent kernels, but would installing 2.6.32 help?

TIA
-- 
Gruß | Greetings | Qapla'
LCARS - Linux Can Also Run Starships

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] 2.6.31 vfat driver broken?

[Willie Wong]

On Wed, Dec 09, 2009 at 11:17:38PM +0100, Penguin Lover Frank Steinmetzger squawked:
> I use FAT32 on my external HDDs to make it easier to share with other people 
> and OSes. Never had a problem before, but now I do. Lately, when I save 
> videos to my disks, and play them back after the file system cache is 
> emptied, they have completely different content (of files that are long 
> deleted).

Please describe in more detail what you actually did. Did you
read/write the files in linux? Both under 2.6.31? A step-by-step maybe
appreciated. 

> 
> I just had the magnificent idea to look into the syslog and found loads of
> "kernel: bio too big device sdb (248 > 240)"
> 
> I heard romours of problems with the current FAT implementation due to M$. I 
> went back to 2.6.30 for the moment. So what???s your proposal? Usually I don???t 
> have the need for ??bercurrent kernels, but would installing 2.6.32 help?

Did you file a bug? Where did you hear this "rumour"?

I don't see any significant changes to the VFAT driver in 2.6.31, and
since (I infer from your message) that downgrading to 2.6.30 is okay,
I doubt that is the issue. 

You are not running any sort of LVM, RAID, or encryption, are you?
A similar bug seems to have occured in device mapper, where dm gave
the underlying fs the wrong values for max_hw_sector. 

I am somehow more leaning toward the problem being in the usb
subsystem. 

When you plug-in your device, what does
/sys/block/sdb/queue/max_hw_sectors_kb say? 

(I'm afraid I am not actually a kernel dev... but I hope I am asking
the right question so someone who actually knows what is going on can
help you.)

Cheers, 

W
-- 
I am a nobody
Nobody is perfect
Therefore, I am perfect.
Sortir en Pantoufles: up 1098 days,  9:14

Re: [gentoo-user] 2.6.31 vfat driver broken?

[Frank Steinmetzger]

[-- Attachment #1: Type: text/plain, Size: 2674 bytes --]

Am Donnerstag, 10. Dezember 2009 schrieb Willie Wong:

> > I use FAT32 on my external HDDs to make it easier to share with other
> > people and OSes. Never had a problem before, but now I do. Lately, when I
> > save videos to my disks, and play them back after the file system cache
> > is emptied, they have completely different content (of files that are
> > long deleted).
>
> Please describe in more detail what you actually did. Did you
> read/write the files in linux? Both under 2.6.31? A step-by-step maybe
> appreciated.

Yes, of course, see below.

> > I just had the magnificent idea to look into the syslog and found loads
> > of "kernel: bio too big device sdb (248 > 240)"
> >
> > I heard romours of problems with the current FAT implementation due to
> > M$. I went back to 2.6.30 for the moment. So what???s your proposal?
> > Usually I don???t have the need for ??bercurrent kernels, but would
> > installing 2.6.32 help?
>
> Did you file a bug? Where did you hear this "rumour"?

I believe heise.de, the publisher of computer magazine c't. I just looked for 
it - it was about Microsoft pursuing legal actions against TomTom for using 
their FAT file system on their linux based devices. I deduced from that that 
they rewrote the FAT driver, but this would seem rather unlikely.

> You are not running any sort of LVM, RAID, or encryption, are you?
> A similar bug seems to have occured in device mapper, where dm gave
> the underlying fs the wrong values for max_hw_sector.

I reckon that's more like it - all my external hard drives are truecrypted.

Here's what I did yesterday to verify my theory:

First I booted into the correctly working 2.6.30 and mounted my external HDD. 
Then I copied a 700 MB and a 1400 MB video file to the disk. Then I unmounted 
the drive to clear the file system cache and remounted - the videos were 
fine.

Then I rebooted into 2.6.31 and copied the same files again (I didn't delete 
the ones from the first test). Unmounted and remounted the drive. The files 
from the first run were still OK, but those from the second test showed the 
wrong content (as mentioned, either mplayer showed the content of a long 
deleted file, or nothing at all, 'file' just recognised it as "data").

> I am somehow more leaning toward the problem being in the usb
> subsystem.
>
> When you plug-in your device, what does
> /sys/block/sdb/queue/max_hw_sectors_kb say?

I'm in 2.6.30 ATM, I'll do it the next time I boot the machine.
-- 
Gruß | Greetings | Qapla'
begin signature_virus
  Hi! I’m a signature virus.
  Please copy me to your signature to help me spread.
end

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] 2.6.31 vfat driver broken?

[BRM]

----- Original Message ----

From: Frank Steinmetzger <Warp_7@gmx.de>
> Am Donnerstag, 10. Dezember 2009 schrieb Willie Wong:
> > > I heard romours of problems with the current FAT implementation due to
> > > M$. I went back to 2.6.30 for the moment. So what???s your proposal?
> > > Usually I don???t have the need for ??bercurrent kernels, but would
> > > installing 2.6.32 help?
> > Did you file a bug? Where did you hear this "rumour"?
> I believe heise.de, the publisher of computer magazine c't. I just looked for 
> it - it was about Microsoft pursuing legal actions against TomTom for using 
> their FAT file system on their linux based devices. I deduced from that that 
> they rewrote the FAT driver, but this would seem rather unlikely.

You are referring to the Long File Name issue. Apparently MS has patented how the LFNs are stored
in a dual method. The fix Linux employed was to only ever allow a single method storage.
This issue does not affect what you are seeing, nor would it affect the FAT driver all that much.
From what I understand (in the various articles, emails, etc. I've seen on-line), there was no rewrite
of the FAT driver; just a slight disabling of some functionality (for the dual mode).

Ben

Re: [gentoo-user] 2.6.31 vfat driver broken?

[Willie Wong]

On Thu, Dec 10, 2009 at 04:58:35PM +0100, Penguin Lover Frank Steinmetzger squawked:
> > You are not running any sort of LVM, RAID, or encryption, are you?
> > A similar bug seems to have occured in device mapper, where dm gave
> > the underlying fs the wrong values for max_hw_sector.
> 
> I reckon that's more like it - all my external hard drives are truecrypted.
> 
Ah. This may or maynot be the culprit: truecrypt does add one extra
layer in the chain. Unfortunately I don't know anything about it.

Good luck, 

W

-- 
Pillage before you burn.
Sortir en Pantoufles: up 1099 days, 10:28

Re: [gentoo-user] 2.6.31 vfat driver broken?

[Frank Steinmetzger]

[-- Attachment #1: Type: text/plain, Size: 558 bytes --]

Am Donnerstag, 10. Dezember 2009 schrieb Willie Wong:

> When you plug-in your device, what does
> /sys/block/sdb/queue/max_hw_sectors_kb say?

120

hw_sector_size says 512

I did another "test": I opened the syslog in continuous output (tail -f) and 
watched it while copying a file of 350 MB. Coincidentally it started getting 
said error "bio too big device sdb (248 > 240)" at about 120 MB (according to 
the progress bar of KDE's copy dialogue).
-- 
Gruß | Greetings | Qapla'
I haven’t lost my mind; it's backed up on tape somewhere!

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] 2.6.31 vfat driver broken?

[Willie Wong]

On Fri, Dec 11, 2009 at 03:18:20AM +0100, Penguin Lover Frank Steinmetzger squawked:
> Am Donnerstag, 10. Dezember 2009 schrieb Willie Wong:
> 
> > When you plug-in your device, what does
> > /sys/block/sdb/queue/max_hw_sectors_kb say?
> 
> 120

Okay, this checks out with the 

> hw_sector_size says 512
> 
> I did another "test": I opened the syslog in continuous output (tail -f) and 
> watched it while copying a file of 350 MB. Coincidentally it started getting 
> said error "bio too big device sdb (248 > 240)" at about 120 MB (according to 
> the progress bar of KDE's copy dialogue).

Sorry, I may not have been clear in my previous e-mail. You showed me 
 /sys/..../max_hw_sectors_kb
and 
 /sys/..../hw_sector_size
But what does
 /sys/..../max_sectors_kb
contain?

(Also, if possible, can you also show me the same values for the
2.6.30 kernel?)

If the max_sectors_kb is bigger than max_hw_sectors_kb, then something
that shouldn't happen happened and the kernel is trying to send more
data than the driver can handle. And that might be where you got your
error from. If that is the case, a work around is to manually, as
root, 'echo 120 > /sys/block/sdb/queue/max_sectors_kb'. 

Now, like I said, it shouldn't happen that max_sectors is >
max_hw_sectors, since they should've been set equal the the block
device is detected. If the two values both show 120, please try
changing max_sectors_kb to a smaller multiple of 8 (64 is a
candidate), and see if the problem persists. If this fixes the error,
there is an off-by-one kernel bug, and you should file it either at
b.g.o. to pass it upstream, or if you are brave (and running a vanilla
kernel) send it directly to LKML. 

If neither of these works, then I am on the completely wrong track. 

Cheers, 

W

-- 
Where do you get Mercury?

H.G. Wells
Sortir en Pantoufles: up 1099 days, 10:14

Re: [gentoo-user] 2.6.31 vfat driver broken?

[Willie Wong]

On Wed, Dec 09, 2009 at 11:17:38PM +0100, Penguin Lover Frank Steinmetzger squawked:
> I use FAT32 on my external HDDs to make it easier to share with other people 
> and OSes. Never had a problem before, but now I do. Lately, when I save 
> videos to my disks, and play them back after the file system cache is 
> emptied, they have completely different content (of files that are long 
> deleted).

Another couple of questions:

You said FAT32: was the filesystem created under a windows box?
You said video files: how big are those files? Does the problem happen
with smaller files?

Please cat for me
/sys/block/sdb/queue/max_*_sector_*

Cheers, 

W

-- 
Marten: Instead of a basket with wine and cheese, you get jpegs of
    equine molestation.
Sortir en Pantoufles: up 1098 days, 10:09