[gentoo-user] External HDD: sector size incorrectly detected on first connect

[gentoo-user] External HDD: sector size incorrectly detected on first connect

[Marc Joliet]

[-- Attachment #1: Type: text/plain, Size: 4688 bytes --]

Hi again

This has been frustrating me ever since I bought the drive, but I have pushed
off looking into it for a while.  Part of the reason is that my desktop used to
run pretty much 24/7 before I moved, so I rarely had to deal with this issue.
However, now I'm more interested in not pointlessly wasting electricity and as
such, encounter this issue at least once a day.  Since my automatic backups run
automatically if they were scheduled to run while the computer was off, this is
becoming more and more annoying.

What happens is that when I (cold) boot my desktop, my external Toshiba 3TB
drive (which is always connected via USB3) is detected, but cannot mount.  It
*does* work once I unplug the USB3 plug and plug it back in.

After finally getting around to debugging the issue, looking at the kernel log
very quickly showed me what was happening:

# journalctl -k -b -1 | grep "sdg"
Mär 06 16:55:34 marcec kernel: sd 8:0:0:0: [sdg] Very big device. Trying to use READ CAPACITY(16).
Mär 06 16:55:34 marcec kernel: sd 8:0:0:0: [sdg] 5860533160 512-byte logical blocks: (3.00 TB/2.72 TiB)
Mär 06 16:55:34 marcec kernel: sd 8:0:0:0: [sdg] Write Protect is off
Mär 06 16:55:34 marcec kernel: sd 8:0:0:0: [sdg] Mode Sense: 2b 00 00 00
Mär 06 16:55:34 marcec kernel: sd 8:0:0:0: [sdg] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
Mär 06 16:55:34 marcec kernel: sd 8:0:0:0: [sdg] Very big device. Trying to use READ CAPACITY(16).
Mär 06 16:55:20 marcec kernel:  sdg: sdg1 sdg2
Mär 06 16:55:20 marcec kernel: sd 8:0:0:0: [sdg] Very big device. Trying to use READ CAPACITY(16).
Mär 06 16:55:20 marcec kernel: sd 8:0:0:0: [sdg] Attached SCSI disk
Mär 06 16:57:08 marcec kernel: sd 9:0:0:0: [sdg] Very big device. Trying to use READ CAPACITY(16).
Mär 06 16:57:08 marcec kernel: sd 9:0:0:0: [sdg] 732566645 4096-byte logical blocks: (3.00 TB/2.72 TiB)
Mär 06 16:57:08 marcec kernel: sd 9:0:0:0: [sdg] Write Protect is off
Mär 06 16:57:08 marcec kernel: sd 9:0:0:0: [sdg] Mode Sense: 2b 00 00 00
Mär 06 16:57:08 marcec kernel: sd 9:0:0:0: [sdg] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
Mär 06 16:57:08 marcec kernel: sd 9:0:0:0: [sdg] 732566645 4096-byte logical blocks: (3.00 TB/2.72 TiB)
Mär 06 16:57:08 marcec kernel:  sdg: sdg1 sdg2
Mär 06 16:57:08 marcec kernel: sdg: p1 size 10476524864 extends beyond EOD, enabling native capacity
Mär 06 16:57:08 marcec kernel: sd 9:0:0:0: [sdg] 732566645 4096-byte logical blocks: (3.00 TB/2.72 TiB)
Mär 06 16:57:08 marcec kernel:  sdg: sdg1 sdg2
Mär 06 16:57:08 marcec kernel: sdg: p1 size 10476524864 extends beyond EOD, truncated
Mär 06 16:57:08 marcec kernel: sd 9:0:0:0: [sdg] 732566645 4096-byte logical blocks: (3.00 TB/2.72 TiB)
Mär 06 16:57:08 marcec kernel: sd 9:0:0:0: [sdg] Attached SCSI disk
Mär 06 16:57:10 marcec kernel: BTRFS: device label MARCEC_BACKUP devid 1 transid 64138 /dev/sdg2
Mär 06 16:57:10 marcec kernel: BTRFS info (device sdg2): disk space caching is enabled
Mär 06 16:57:17 marcec kernel: EXT4-fs (sdg1): mounted filesystem with ordered data mode. Opts: (null)

(I rebooted in the meantime, and the drive is always detected correctly then,
hence the "-b -1" argument to journalctl.)

As you can see, the kernel initially thinks that it has 512-byte logical
blocks.  After unplugging the USB3 cable and plugging it back in it is
detected as having 4096-byte logical blocks.  From then on it works fine.

The interesting question now is: *why* does this happen?  According to [0] and
[1], the problem is that the enclosure firmware is doing some silly block size
translation in order to support MBR partitions, which apparently is done in a
buggy fashion to boot. Sadly, they provide no solution.

(I also can't help but wonder if that also helps explain the "p1 size
10476524864 extends beyond EOD, truncated" messages.  Note that the
partitioning was done with gparted, but IIRC even editing it with cfdisk
couldn't get rid of it.)

I have not been able to find a way to fix this issue.  The "manual" that comes
with the drive is devoid of relevant information, and so far google comes up
with several related posts, each without a solution.

So, does anybody know of a way to get the drive to work more reliably?  Or will
I have to resort to a different enclosure (when I can afford it)?

[0] http://askubuntu.com/questions/337017/cant-read-partition-table-of-3tb-usb-disk
[1] http://ubuntuforums.org/showthread.php?t=1536933

Greetings,
-- 
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup

[-- Attachment #2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

[gentoo-user] Re: External HDD: sector size incorrectly detected on first connect

[walt]

On 03/06/2015 11:44 AM, Marc Joliet wrote:
<mega-snip>

> What happens is that when I (cold) boot my desktop, my external Toshiba 3TB
> drive (which is always connected via USB3) is detected, but cannot mount.  It
> *does* work once I unplug the USB3 plug and plug it back in.

I once hated replies containing the question "Does it work under Windows?" but
a bit of experience with usb3 external drives has made me rethink the matter.

Here's my thinking:  if the usb3 drive works correctly with Windows but not
with linux, the problem is software -- i.e. the device driver.

The xhci driver is under heavy development because usb3 is still new tech, and
I've found and reported a few bugs in the last year or so and they got fixed.

If your Toshiba drive has no partitions formatted in an M$ filesystem, you
could use gparted to make some free space on the drive and then create and
format a new M$-readable partition just for testing purposes.

I'm sure you don't have a Windows machine yourself, so you will need to make
friends with somebody who does ;)

Re: [gentoo-user] Re: External HDD: sector size incorrectly detected on first connect

[Marc Joliet]

[-- Attachment #1: Type: text/plain, Size: 1583 bytes --]

Am Fri, 06 Mar 2015 18:33:13 -0800
schrieb walt <w41ter@gmail.com>:

> On 03/06/2015 11:44 AM, Marc Joliet wrote:
> <mega-snip>
> 
> > What happens is that when I (cold) boot my desktop, my external Toshiba 3TB
> > drive (which is always connected via USB3) is detected, but cannot mount.  It
> > *does* work once I unplug the USB3 plug and plug it back in.
> 
> I once hated replies containing the question "Does it work under Windows?" but
> a bit of experience with usb3 external drives has made me rethink the matter.
> 
> Here's my thinking:  if the usb3 drive works correctly with Windows but not
> with linux, the problem is software -- i.e. the device driver.

That's what I hope.

> The xhci driver is under heavy development because usb3 is still new tech, and
> I've found and reported a few bugs in the last year or so and they got fixed.

Yeah, if its anything like the uas module, it'll have had many quirks added to
it in a short amount of time.

> If your Toshiba drive has no partitions formatted in an M$ filesystem, you
> could use gparted to make some free space on the drive and then create and
> format a new M$-readable partition just for testing purposes.

That'll be annoyingly time consuming, but I suppose you're right.

> I'm sure you don't have a Windows machine yourself, so you will need to make
> friends with somebody who does ;)

As a matter of fact, my brother has one ;) .

Greetings
-- 
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup

[-- Attachment #2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-user] Re: External HDD: sector size incorrectly detected on first connect

[Marc Joliet]

[-- Attachment #1: Type: text/plain, Size: 1406 bytes --]

Hi

while I haven't had a chance to try the external HDD on my brother's
Windows computer yet, I did notice one thing:  when I turn the HDD off (i.e.,
unplug its AC) for several hours and turn it back on (with my PC still running
the entire time) the problem does *not* show up.

I would think that this points to a driver problem, but then again, the problem
also fails to show up when I reboot my computer while the drive stays on.

The only time when the problem consistently shows up is when I turn my computer
off for a longer period of time (with or without turning off the power strip
the PC and HDD are attached to).  For example, after turning the PC off for the
night, the problem always occurs after I turn it back on in the morning.

I wonder if it has anything to do with my USB3 PCI card.

(This is, incidentally, why I haven't been able to try it with my brother's PC
yet: the only condition where I know the that the problem will show up is after
leaving my computer off for the night or while I'm gone during the day.  So I
would have to try it with my brother in the morning or evening, but that has
not been possible yet.)

One more thing I'll try: not having the HDD plugged into the USB3 slot when my
computer boots up.

Greetings
-- 
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup

[-- Attachment #2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

[gentoo-user] Re: External HDD: sector size incorrectly detected on first connect

[Hans]

On 10/03/15 19:45, Marc Joliet wrote:
> Hi
>
> while I haven't had a chance to try the external HDD on my brother's
> Windows computer yet, I did notice one thing:  when I turn the HDD off (i.e.,
> unplug its AC) for several hours and turn it back on (with my PC still running
> the entire time) the problem does *not* show up.
>
> I would think that this points to a driver problem, but then again, the problem
> also fails to show up when I reboot my computer while the drive stays on.
>
> The only time when the problem consistently shows up is when I turn my computer
> off for a longer period of time (with or without turning off the power strip
> the PC and HDD are attached to).  For example, after turning the PC off for the
> night, the problem always occurs after I turn it back on in the morning.
>
> I wonder if it has anything to do with my USB3 PCI card.
>
> (This is, incidentally, why I haven't been able to try it with my brother's PC
> yet: the only condition where I know the that the problem will show up is after
> leaving my computer off for the night or while I'm gone during the day.  So I
> would have to try it with my brother in the morning or evening, but that has
> not been possible yet.)
>
> One more thing I'll try: not having the HDD plugged into the USB3 slot when my
> computer boots up.
>
> Greetings
>
Have a very similar problem with a USB drive. Resolved it by installing 
a toggle switch into the DC power line and switched the drive on when I 
did a backup. I.e. power up the PC, login, power up the USB drive.

Re: [gentoo-user] Re: External HDD: sector size incorrectly detected on first connect

[Marc Joliet]

[-- Attachment #1: Type: text/plain, Size: 3406 bytes --]

Am Fri, 06 Mar 2015 18:33:13 -0800
schrieb walt <w41ter@gmail.com>:

[...]
> I once hated replies containing the question "Does it work under Windows?" but
> a bit of experience with usb3 external drives has made me rethink the matter.
> 
> Here's my thinking:  if the usb3 drive works correctly with Windows but not
> with linux, the problem is software -- i.e. the device driver.
> 
> The xhci driver is under heavy development because usb3 is still new tech, and
> I've found and reported a few bugs in the last year or so and they got fixed.
[...]

OK, so I tried this out, and it apparently worked fine under Windows 8 (I
created a third FAT32 partition for this).  Once Windows installed the drivers
(or whatever) the drive was recognised and the partition mounted without
trouble.

Immediately after that, I unplugged it and moved it back to my computer, where I
plugged it in as usual and then turned the computer on.  The problem occurred
again, of course:

    # journalctl -k -b | grep sdg
    Mär 10 17:58:57 marcec kernel: sd 8:0:0:0: [sdg] Very big device. Trying to use READ CAPACITY(16).
    Mär 10 17:58:57 marcec kernel: sd 8:0:0:0: [sdg] 5860533160 512-byte logical blocks: (3.00 TB/2.72 TiB)
    Mär 10 17:58:57 marcec kernel: sd 8:0:0:0: [sdg] Write Protect is off
    Mär 10 17:58:57 marcec kernel: sd 8:0:0:0: [sdg] Mode Sense: 2b 00 00 00
    Mär 10 17:58:57 marcec kernel: sd 8:0:0:0: [sdg] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
    Mär 10 17:58:57 marcec kernel: sd 8:0:0:0: [sdg] Very big device. Trying to use READ CAPACITY(16).
    Mär 10 17:58:57 marcec kernel:  sdg: sdg1 sdg2 sdg3
    Mär 10 17:58:57 marcec kernel: sd 8:0:0:0: [sdg] Very big device. Trying to use READ CAPACITY(16).
    Mär 10 17:58:57 marcec kernel: sd 8:0:0:0: [sdg] Attached SCSI disk
    Mär 10 18:00:33 marcec kernel: sd 9:0:0:0: [sdg] Very big device. Trying to use READ CAPACITY(16).
    Mär 10 18:00:33 marcec kernel: sd 9:0:0:0: [sdg] 732566645 4096-byte logical blocks: (3.00 TB/2.72 TiB)
    Mär 10 18:00:33 marcec kernel: sd 9:0:0:0: [sdg] Write Protect is off
    Mär 10 18:00:33 marcec kernel: sd 9:0:0:0: [sdg] Mode Sense: 2b 00 00 00
    Mär 10 18:00:33 marcec kernel: sd 9:0:0:0: [sdg] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
    Mär 10 18:00:33 marcec kernel: sd 9:0:0:0: [sdg] 732566645 4096-byte logical blocks: (3.00 TB/2.72 TiB)
    Mär 10 18:00:33 marcec kernel:  sdg: sdg1 sdg2 sdg3
    Mär 10 18:00:33 marcec kernel: sd 9:0:0:0: [sdg] 732566645 4096-byte logical blocks: (3.00 TB/2.72 TiB)
    Mär 10 18:00:33 marcec kernel: sd 9:0:0:0: [sdg] Attached SCSI disk
    Mär 10 18:00:35 marcec kernel: BTRFS: device label MARCEC_BACKUP devid 1 transid 64666 /dev/sdg2
    Mär 10 18:00:35 marcec kernel: BTRFS info (device sdg2): disk space caching is enabled
    Mär 10 18:00:42 marcec kernel: EXT4-fs (sdg1): mounted filesystem with ordered data mode. Opts: (null)

So, what do other list members think: is it most likely a driver bug, or
something else (especially considering the reboot behaviour I mentioned in
another email)? Should I ask on the LKML, or is there a specialised ML for the
Linux USB stack?

Greetings
-- 
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup

[-- Attachment #2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-user] Re: External HDD: sector size incorrectly detected on first connect

[Marc Joliet]

[-- Attachment #1: Type: text/plain, Size: 698 bytes --]

Am Wed, 11 Mar 2015 02:58:20 +1000
schrieb Hans <linux@interworld.net.au>:

[...]
> Have a very similar problem with a USB drive. Resolved it by installing 
> a toggle switch into the DC power line and switched the drive on when I 
> did a backup. I.e. power up the PC, login, power up the USB drive.

I *hope* I don't have to resort to that, but I do have switchable power strips
if I have to. I could hook it up to the one that's not under my desk and use
that (my audio equipment is hooked up to it so I already use it to turn that
stuff on).

Greetings
-- 
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup

[-- Attachment #2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

[gentoo-user] Re: External HDD: sector size incorrectly detected on first connect

[walt]

On 03/10/2015 10:12 AM, Marc Joliet wrote:
> is there a specialised ML for the
> Linux USB stack?

Yes, I follow it on gmane.org as gmane.linux.usb.general (not sure what
the real name of the mailing list is) and Sarah Sharp is the xhci driver
maintainer.  She was very helpful and patient when I was having similar
and quirky problems with my usb3 adapter, and it did turn out to be a
driver bug.  And such quirks can be very much device-specific.  I would
not want Sarah's job, but I'm glad she does :)

Re: [gentoo-user] Re: External HDD: sector size incorrectly detected on first connect

[Dutch Ingraham]

On Tue, Mar 10, 2015 at 05:38:03PM -0700, walt wrote:
> On 03/10/2015 10:12 AM, Marc Joliet wrote:
> > is there a specialised ML for the
> > Linux USB stack?
> 
> Yes, I follow it on gmane.org as gmane.linux.usb.general (not sure what
> the real name of the mailing list is) and Sarah Sharp is the xhci driver
> maintainer.  She was very helpful and patient when I was having similar
> and quirky problems with my usb3 adapter, and it did turn out to be a
> driver bug.  And such quirks can be very much device-specific.  I would
> not want Sarah's job, but I'm glad she does :)
> 
> 
Apparently not anymore....[1]

[1] http://article.gmane.org/gmane.linux.usb.general/123211

Re: [gentoo-user] Re: External HDD: sector size incorrectly detected on first connect

[Marc Joliet]

[-- Attachment #1: Type: text/plain, Size: 779 bytes --]

Am Tue, 10 Mar 2015 17:38:03 -0700
schrieb walt <w41ter@gmail.com>:

> On 03/10/2015 10:12 AM, Marc Joliet wrote:
> > is there a specialised ML for the
> > Linux USB stack?
> 
> Yes, I follow it on gmane.org as gmane.linux.usb.general (not sure what
> the real name of the mailing list is) and Sarah Sharp is the xhci driver
> maintainer.  She was very helpful and patient when I was having similar
> and quirky problems with my usb3 adapter, and it did turn out to be a
> driver bug.  And such quirks can be very much device-specific.  I would
> not want Sarah's job, but I'm glad she does :)

OK, I'll follow up there, then.

Thanks
-- 
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup

[-- Attachment #2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-user] Re: External HDD: sector size incorrectly detected on first connect

[Marc Joliet]

[-- Attachment #1: Type: text/plain, Size: 465 bytes --]

Am Wed, 11 Mar 2015 08:04:05 +0100
schrieb Marc Joliet <marcec@gmx.de>:

[...]
> OK, I'll follow up there, then.

I finally did that, now that I tested with vanilla-sources-3.19.2.  See for
example http://thread.gmane.org/gmane.linux.usb.general/124047.

I also opened a bug at https://bugs.gentoo.org/show_bug.cgi?id=543684.

-- 
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup

[-- Attachment #2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-user] Re: External HDD: sector size incorrectly detected on first connect

[Marc Joliet]

[-- Attachment #1: Type: text/plain, Size: 1165 bytes --]

Am Sun, 22 Mar 2015 14:28:09 +0100
schrieb Marc Joliet <marcec@gmx.de>:

> Am Wed, 11 Mar 2015 08:04:05 +0100
> schrieb Marc Joliet <marcec@gmx.de>:
> 
> [...]
> > OK, I'll follow up there, then.
> 
> I finally did that, now that I tested with vanilla-sources-3.19.2.  See for
> example http://thread.gmane.org/gmane.linux.usb.general/124047.
> 
> I also opened a bug at https://bugs.gentoo.org/show_bug.cgi?id=543684.

After diagnosing the problem with the help of Alan Stern and Matthew Dharm, the
conclusion was that the sector size translation must be triggered by some
interaction between the drive and the BIOS.  The test I performed to determine
this was:

- unplug the drive,
- turn on the computer,
- plug the drive in once GRUB has started, and
- boot the kernel,

after which the drive worked fine.  So this does not seem to be anything that
can be fixed in the kernel.  Perhaps if I had an EFI compatible mainboard, I
wouldn't be seeing this problem, but I don't, so I can't even try it out :-/ .

-- 
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup

[-- Attachment #2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 819 bytes --]