[gentoo-user] [OT] Disappointing USB3 performance

[gentoo-user] [OT] Disappointing USB3 performance

[walt]

I just bought an add-on USB3 adapter and outboard USB3/sata docking
station, and I've been comparing the performance with my old e-sata
outboard docking station.  Not so good :(

After getting some unreliable results with hdparm, I settled on
copying one 3GB file from one partition of the outboard drive to
another partition of the same drive.  These results are highly
reproducible, and favor e-sata over USB3 by a large margin.

Over at least six trials on each docking station I consistently
get 105 seconds for USB and 84 seconds for e-sata, a 5:4 ratio
in favor of e-sata.

I used the same hard disk and the same pci-e slot in the same
minimally-loaded machine for all the runs, and got very consistent
results every time.

Basically, the USB3/sata docking station gets the same throughput as
the older sata 1 drives connected to the onboard pci sata controller,
which is still pretty respectable for an outboard drive, I think.

So, has anyone out there done similar tests on USB3 drives yet?

Re: [gentoo-user] [OT] Disappointing USB3 performance

[Michael Mol]

On Mon, Oct 24, 2011 at 1:28 PM, walt <w41ter@gmail.com> wrote:
> I just bought an add-on USB3 adapter and outboard USB3/sata docking
> station, and I've been comparing the performance with my old e-sata
> outboard docking station.  Not so good :(
>
> After getting some unreliable results with hdparm, I settled on
> copying one 3GB file from one partition of the outboard drive to
> another partition of the same drive.  These results are highly
> reproducible, and favor e-sata over USB3 by a large margin.
>
> Over at least six trials on each docking station I consistently
> get 105 seconds for USB and 84 seconds for e-sata, a 5:4 ratio
> in favor of e-sata.
>
> I used the same hard disk and the same pci-e slot in the same
> minimally-loaded machine for all the runs, and got very consistent
> results every time.
>
> Basically, the USB3/sata docking station gets the same throughput as
> the older sata 1 drives connected to the onboard pci sata controller,
> which is still pretty respectable for an outboard drive, I think.
>
> So, has anyone out there done similar tests on USB3 drives yet?

I have not; I don't have a system with USB3 yet.

As far as USB3 goes, I'm more curious about host-host networking
performance. Anyone played with that? If it's reasonably reliable, I
could see 3-4 USB3 ports on three machines acting as a poor-man's
high-performance, one-hop many-many mesh network, potentially good for
network-synchronized block devices. I find Intel's Thunderbolt
interesting for similar reasons.


-- 
:wq

[gentoo-user] Re: [OT] Disappointing USB3 performance

[Nikos Chantziaras]

On 10/24/2011 08:28 PM, walt wrote:
> I just bought an add-on USB3 adapter and outboard USB3/sata docking
> station, and I've been comparing the performance with my old e-sata
> outboard docking station.  Not so good :(
>[...]
> Over at least six trials on each docking station I consistently
> get 105 seconds for USB and 84 seconds for e-sata, a 5:4 ratio
> in favor of e-sata.

Doesn't look surprising to me.  The USB protocol doesn't compare 
favorably with SATA.  It's good for "dumb" data transfers, but lacks 
stuff like native command queuing and DMA operations.  Most features 
supported by the actual hard disk can't be used when you connect it 
though USB.

Re: [gentoo-user] [OT] Disappointing USB3 performance

[Paul Hartman]

On Mon, Oct 24, 2011 at 12:28 PM, walt <w41ter@gmail.com> wrote:
> I just bought an add-on USB3 adapter and outboard USB3/sata docking
> station, and I've been comparing the performance with my old e-sata
> outboard docking station.  Not so good :(

I think, generally speaking, a SATA drive plugged into a SATA port
will always be the fastest possible connection. By using a USB adapter
you're adding an additional layer of overhead to the process (the
adapter translating between USB and SATA, plus any loss of
functionality due to incompatible features like Nikos mentioned).

But if you're wonder what the speed difference should be (if the
difference you're seeing is normal or not) I can't say... I can say
USB3 is faster than USB2 for sure, but I haven't personally compared
to eSATA. :)

[gentoo-user] Re: [OT] Disappointing USB3 performance

[Grant Edwards]

On 2011-10-24, walt <w41ter@gmail.com> wrote:

> I just bought an add-on USB3 adapter and outboard USB3/sata docking
> station, and I've been comparing the performance with my old e-sata
> outboard docking station.  Not so good :(
>
> After getting some unreliable results with hdparm, I settled on
> copying one 3GB file from one partition of the outboard drive to
> another partition of the same drive.  These results are highly
> reproducible, and favor e-sata over USB3 by a large margin.
>
> Over at least six trials on each docking station I consistently get
> 105 seconds for USB and 84 seconds for e-sata, a 5:4 ratio in favor
> of e-sata.

Not surprising.  Did you expect that adding a gateway device to the
communication path and another protocol layer on top of SATA would
make things faster?

> I used the same hard disk and the same pci-e slot in the same
> minimally-loaded machine for all the runs, and got very consistent
> results every time.
>
> Basically, the USB3/sata docking station gets the same throughput as
> the older sata 1 drives connected to the onboard pci sata controller,
> which is still pretty respectable for an outboard drive, I think.

Yep, SATA performs the same as SATA. AFAIK, eSATA and SATA are
identical apart from the physical specs for the connector, a few minor
voltage level differences (to imporove noise tolerance), and hot-plug
support.

> So, has anyone out there done similar tests on USB3 drives yet?

There are disk drives that talk USB3 natively and aren't just using
USB<->SATA gateways?

-- 
Grant Edwards               grant.b.edwards        Yow! How many retured
                                  at               bricklayers from FLORIDA
                              gmail.com            are out purchasing PENCIL
                                                   SHARPENERS right NOW??

Re: [gentoo-user] Re: [OT] Disappointing USB3 performance

[Florian Philipp]

[-- Attachment #1: Type: text/plain, Size: 1997 bytes --]

Am 24.10.2011 22:02, schrieb Grant Edwards:
> On 2011-10-24, walt <w41ter@gmail.com> wrote:
> 
>> I just bought an add-on USB3 adapter and outboard USB3/sata docking
>> station, and I've been comparing the performance with my old e-sata
>> outboard docking station.  Not so good :(
>>
>> After getting some unreliable results with hdparm, I settled on
>> copying one 3GB file from one partition of the outboard drive to
>> another partition of the same drive.  These results are highly
>> reproducible, and favor e-sata over USB3 by a large margin.
>>
>> Over at least six trials on each docking station I consistently get
>> 105 seconds for USB and 84 seconds for e-sata, a 5:4 ratio in favor
>> of e-sata.
> 
> Not surprising.  Did you expect that adding a gateway device to the
> communication path and another protocol layer on top of SATA would
> make things faster?
> 
>> I used the same hard disk and the same pci-e slot in the same
>> minimally-loaded machine for all the runs, and got very consistent
>> results every time.
>>
>> Basically, the USB3/sata docking station gets the same throughput as
>> the older sata 1 drives connected to the onboard pci sata controller,
>> which is still pretty respectable for an outboard drive, I think.
> 
> Yep, SATA performs the same as SATA. AFAIK, eSATA and SATA are
> identical apart from the physical specs for the connector, a few minor
> voltage level differences (to imporove noise tolerance), and hot-plug
> support.
>

Normal SATA also offers hotplug. Usually works, too.

>> So, has anyone out there done similar tests on USB3 drives yet?
> 
> There are disk drives that talk USB3 natively and aren't just using
> USB<->SATA gateways?
> 

Well, there is USB Attached SCSI (CONFIG_USB_UAS in the kernel). It
supports command queuing and works for USB-2.0 and 3.0 (but has
additional software overhead for USB-2.0). I've not yet seen a
compatible device, though.

Regards,
Florian Philipp


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

[gentoo-user] Re: [OT] Disappointing USB3 performance

[Grant Edwards]

On 2011-10-24, Florian Philipp <lists@binarywings.net> wrote:
> Am 24.10.2011 22:02, schrieb Grant Edwards:
>> On 2011-10-24, walt <w41ter@gmail.com> wrote:
>> 
>>> I just bought an add-on USB3 adapter and outboard USB3/sata docking
>>> station, and I've been comparing the performance with my old e-sata
>>> outboard docking station.  Not so good :(
>>>
>>> After getting some unreliable results with hdparm, I settled on
>>> copying one 3GB file from one partition of the outboard drive to
>>> another partition of the same drive.  These results are highly
>>> reproducible, and favor e-sata over USB3 by a large margin.
>>>
>>> Over at least six trials on each docking station I consistently get
>>> 105 seconds for USB and 84 seconds for e-sata, a 5:4 ratio in favor
>>> of e-sata.
>> 
>> Not surprising.  Did you expect that adding a gateway device to the
>> communication path and another protocol layer on top of SATA would
>> make things faster?
>> 
>>> I used the same hard disk and the same pci-e slot in the same
>>> minimally-loaded machine for all the runs, and got very consistent
>>> results every time.
>>>
>>> Basically, the USB3/sata docking station gets the same throughput as
>>> the older sata 1 drives connected to the onboard pci sata controller,
>>> which is still pretty respectable for an outboard drive, I think.
>> 
>> Yep, SATA performs the same as SATA. AFAIK, eSATA and SATA are
>> identical apart from the physical specs for the connector, a few minor
>> voltage level differences (to imporove noise tolerance), and hot-plug
>> support.
>
> Normal SATA also offers hotplug. Usually works, too.

I read somewhere that not all controllers support hotplug on
"internal" connectors, but I can't personally attest to having found
one that didn't.

>>> So, has anyone out there done similar tests on USB3 drives yet?
>> 
>> There are disk drives that talk USB3 natively and aren't just using
>> USB<->SATA gateways?
>
> Well, there is USB Attached SCSI (CONFIG_USB_UAS in the kernel). It
> supports command queuing and works for USB-2.0 and 3.0 (but has
> additional software overhead for USB-2.0). I've not yet seen a
> compatible device, though.

Interesting.  Is USB3 peer to peer like SCSI and Firewire, or is it
the same master/slave poll/response scheme that has always crippled
USB?  Doing SCSI via a poll/response transport protocol seems like it
would lose most of the advantages of SCSI.

-- 
Grant Edwards               grant.b.edwards        Yow! He is the MELBA-BEING
                                  at               ... the ANGEL CAKE
                              gmail.com            ... XEROX him ... XEROX
                                                   him --

Re: [gentoo-user] Re: [OT] Disappointing USB3 performance

[Paul Hartman]

On Mon, Oct 24, 2011 at 4:31 PM, Grant Edwards
<grant.b.edwards@gmail.com> wrote:
> On 2011-10-24, Florian Philipp <lists@binarywings.net> wrote:
>> Am 24.10.2011 22:02, schrieb Grant Edwards:
>>> On 2011-10-24, walt <w41ter@gmail.com> wrote:
>>>
>>>> I just bought an add-on USB3 adapter and outboard USB3/sata docking
>>>> station, and I've been comparing the performance with my old e-sata
>>>> outboard docking station.  Not so good :(
>>>>
>>>> After getting some unreliable results with hdparm, I settled on
>>>> copying one 3GB file from one partition of the outboard drive to
>>>> another partition of the same drive.  These results are highly
>>>> reproducible, and favor e-sata over USB3 by a large margin.
>>>>
>>>> Over at least six trials on each docking station I consistently get
>>>> 105 seconds for USB and 84 seconds for e-sata, a 5:4 ratio in favor
>>>> of e-sata.
>>>
>>> Not surprising.  Did you expect that adding a gateway device to the
>>> communication path and another protocol layer on top of SATA would
>>> make things faster?
>>>
>>>> I used the same hard disk and the same pci-e slot in the same
>>>> minimally-loaded machine for all the runs, and got very consistent
>>>> results every time.
>>>>
>>>> Basically, the USB3/sata docking station gets the same throughput as
>>>> the older sata 1 drives connected to the onboard pci sata controller,
>>>> which is still pretty respectable for an outboard drive, I think.
>>>
>>> Yep, SATA performs the same as SATA. AFAIK, eSATA and SATA are
>>> identical apart from the physical specs for the connector, a few minor
>>> voltage level differences (to imporove noise tolerance), and hot-plug
>>> support.
>>
>> Normal SATA also offers hotplug. Usually works, too.
>
> I read somewhere that not all controllers support hotplug on
> "internal" connectors, but I can't personally attest to having found
> one that didn't.
>
>>>> So, has anyone out there done similar tests on USB3 drives yet?
>>>
>>> There are disk drives that talk USB3 natively and aren't just using
>>> USB<->SATA gateways?
>>
>> Well, there is USB Attached SCSI (CONFIG_USB_UAS in the kernel). It
>> supports command queuing and works for USB-2.0 and 3.0 (but has
>> additional software overhead for USB-2.0). I've not yet seen a
>> compatible device, though.
>
> Interesting.  Is USB3 peer to peer like SCSI and Firewire, or is it
> the same master/slave poll/response scheme that has always crippled
> USB?  Doing SCSI via a poll/response transport protocol seems like it
> would lose most of the advantages of SCSI.

IIRC USB3 is interrupt-driven instead of constantly polling the device.

[gentoo-user] Re: [OT] Disappointing USB3 performance

[walt]

On 10/24/2011 10:28 AM, walt wrote:
> ...
> Over at least six trials on each docking station I consistently
> get 105 seconds for USB and 84 seconds for e-sata, a 5:4 ratio
> in favor of e-sata/sata over USB3/sata...

Wow, lots of great answers, guys, thanks.  Enough material to give
me lots more questions to ask you :)

Like, for example, in theory the raw bit-rate for USB3 is more than
enough to keep up with any existing consumer hard drive, right?  The
speed of usb/sata protocol translation should be very fast compared
to the speed of a spinning mechanical disk (I think?)

Now, lack of DMA is another story for hard disks, certainly.  Here's
where my ignorance of hardware limits my thinking:

AFAIK the device driver *always* sits between the disk drive and the
DMA hardware, doesn't it?

Seems to me that the USB3 driver should be fast enough to shuttle the
raw disk data (somehow) to the DMA hardware just like the pata/sata
drivers do.  It's that "somehow" that I obviously don't understand.

Where am I thinking wrong about the problem?

Re: [gentoo-user] Re: [OT] Disappointing USB3 performance

[Michael Mol]

[-- Attachment #1: Type: text/plain, Size: 1221 bytes --]

On Oct 24, 2011 7:21 PM, "walt" <w41ter@gmail.com> wrote:
>
> On 10/24/2011 10:28 AM, walt wrote:
> > ...
> > Over at least six trials on each docking station I consistently
> > get 105 seconds for USB and 84 seconds for e-sata, a 5:4 ratio
> > in favor of e-sata/sata over USB3/sata...
>
> Wow, lots of great answers, guys, thanks.  Enough material to give
> me lots more questions to ask you :)
>
> Like, for example, in theory the raw bit-rate for USB3 is more than
> enough to keep up with any existing consumer hard drive, right?  The
> speed of usb/sata protocol translation should be very fast compared
> to the speed of a spinning mechanical disk (I think?)
>
> Now, lack of DMA is another story for hard disks, certainly.  Here's
> where my ignorance of hardware limits my thinking:
>
> AFAIK the device driver *always* sits between the disk drive and the
> DMA hardware, doesn't it?

DMA means a device is told where in the system's address space it may write
to, and it writes directly to that place without further CPU involvement.
Since drivers run on the CPU, the drivr isn't a go-between.

When the CPU *is* involved in the passing of bits around, things slow down.
IIRC, that's called PIO--programmed IO.

[-- Attachment #2: Type: text/html, Size: 1506 bytes --]

Re: [gentoo-user] Re: [OT] Disappointing USB3 performance

[Allan Gottlieb]

On Mon, Oct 24 2011, Michael Mol wrote:

> On Oct 24, 2011 7:21 PM, "walt" <w41ter@gmail.com> wrote:
>>
>> Now, lack of DMA is another story for hard disks, certainly.  Here's
>> where my ignorance of hardware limits my thinking:
>>
>> AFAIK the device driver *always* sits between the disk drive and the
>> DMA hardware, doesn't it?
>
> DMA means a device is told where in the system's address space it may write
> to, and it writes directly to that place without further CPU involvement.
> Since drivers run on the CPU, the drivr isn't a go-between.
>
> When the CPU *is* involved in the passing of bits around, things slow down.
> IIRC, that's called PIO--programmed IO.

Correct.  DMA stands for direct memory access; the device has direct
access to the memory.  PIO is indeed the name when the CPU acts as a go
between.

allan

Re: [gentoo-user] Re: [OT] Disappointing USB3 performance

[Adam Carter]

USB does use DMA. Check the kernel source doco Documentation/usb/dma.txt