[gentoo-amd64] Re: Is my RAID performance bad possibly due to starting sector value?

[Duncan <1i5t5.duncan@cox.net>]

Gary E. Miller posted on Fri, 28 Jun 2013 10:50:08 -0700 as excerpted:

> Yo Duncan!

Nice greeting, BTW.  Good to cheer a reader up after a long day with 
things not going right, especially after seeing it several times in a row 
so there's a bit of familiarity now. =:^)

> On Fri, 28 Jun 2013 09:12:24 +0000 (UTC)
> Duncan <1i5t5.duncan@cox.net> wrote:
> 
>> Duncan posted on Fri, 28 Jun 2013 03:36:10 +0000 as excerpted:
>> 
>> I settled on a 4 GiB file.  Speeds are power-of-10-based since that's
>> what dd reports, unless otherwise stated.
> 
> dd is pretty good at testing linear file performance, pretty useless for
> testing mysql performance.

Recognized.  A single i/o job test, but it's something, it's reasonably 
repeatable, and when done on the actual filesystem, it's real-world times 
and flexible in data and block size, if single-job limited.

Plus, unlike some of the more exotic tests which need to be installed 
separately, it's commonly already installed and available for use on most 
*ix systems. =:^)

>> To SSD: peak was upper 250s MB/s over a wide blocksize range of 1 MiB
>> >From SSD: peak was lower 480s MB/s, blocksize 32 KiB to 512 KiB
> 
> Sounds about right.  Your speeds are now so high that small differences
> in the SATA controller chip will be bigger than that between some SSD
> drives. Use a PCIe/SATA card and your performance will drop from what
> you see.

Good point.

I was thinking about that the other day.  SSDs are fast enough that they 
saturate a modern PCIe 3.0 1x and a single SATA-600 channel all by 
themselves.  SATA port-multipliers are arguably still useful for for 
slower spinning rust, but not so much for SSD, where the bottleneck is 
often already the SATA and/or PCIe, so doubling up will indeed only slow 
things down.

And most add-on SATA cards have several SATA ports hanging off the same 
1x PCIe, which means they'll bottleneck if actually using more than a 
single port, too.

I believe I have seen 4x PCIe SATA cards, which would allow four or so 
SATA ports (I think 5), but they tend to be higher priced.

After pondering that for a bit, I decided I'd take a closer look next 
time I was at Fry's Electronics, to see what was actually available, as 
well as the prices.  Until last year I was still running old PCI-X boxes, 
so the whole PCI-E thing itself is still relatively new to me, and I'm 
still reorienting myself to the modern bus and its implications in terms 
of addon cards, etc.

>> Spinning rust speeds, single Seagate st9500424as, 7200rpm 2.5" 16MB
> 
> Those are pretty old and slow.  If you are going to test an HDD against
> a newer SSD you should at least test a newer HDD.  A new 2TB drive could
> get pretty close to your SSD performance in linear tests.

Well, it's not particularly old, but it *IS* a 2.5 inch, down from the 
old 3.5 inch standard, which due to the smaller diameter does mean lower 
rim/maximum speeds at the same RPM.  And of course 7200 RPM is middle of 
the pack as well.  The fast stuff (tho calling any spinning rust "fast" 
in the age of SSDs does rather jar, it's relative!) is 15000.

But 2.5 inch does seem to be on its way as the new standard for desktops 
and servers as well, helped along by the three factors of storage 
density, SSDs (which are invariably 2.5 inch, and even that's due to the 
standard form factor as often the circuit boards aren't full height and/
or are largely empty space, 3.5 inch is just /ridiculously/ huge for 
them), and the newer focus on power efficiency (plus raw spindle density!
) in the data center.

There's still a lot of inertia behind the 3.5 inch standard, just as 
there is behind spinning rust, and it's not going away overnight, but in 
the larger picture, 3.5 inch tends to look as anachronous as a full size 
desktop in an age when even the laptop is being displaced by the tablet 
and mobile phone.  Which isn't to say there's no one still using them, by 
far (my main machine is still a mid tower, easier to switch out parts on 
them, after all), but just sayin' what I'm sayin'.

Anyway...

>> To rust: upper 70s MB/s, blocksize didn't seem to matter much.
>> >From rust: upper 90s MB/s, blocksize upto 4 MiB.
> 
> Seems about right, for that drive.
> 
> I think your numbers are about right, if your workload is just reading
> and writing big linear files.  For a MySQL workload there would be a lot
> of random reads/writes/seeks and the SSD would really shine.

Absolutely.  And perhaps more to the point given the list and thus the 
readership...

As I said in a different thread on a different list, recently, I didn't 
see my boot times change much, the major factor there being the ntp-
client time-sync, at ~12 seconds usually (just long enough to trigger 
openrc's first 10-second warning in the minute timeout...), but *WOW*, 
did the SSDs drop my emerge sync, as well as kernel git pull, time!  
Those are both many smaller files that will tend to highly fragment over 
time due to constant churn, and that's EXACTLY the job type where good 
SSDs can (and do!) really shine!

Like your MySQL db example (tho that's high activity large-file rather 
than high-activity huge number of smaller files), except this one's 
likely more directly useful to a larger share of the list readership. =:^)

Meanwhile, the other thing with the boot times is that I boot to a CLI 
login, so don't tend to count the X and kde startup times as boot.  But 
kde starts up much faster too, and that would count as boot time for many 
users.

Additionally, I have one X app, pan, that in my (not exactly design 
targeted) usage, had a startup time that really suffered on spinning 
rust, so much so that for years I've had it start with the kde session, 
so that if it takes five minutes on cold-cache to startup, no big deal, I 
have other things to do and it's ready when I'm ready for it.  It's a 
newsgroups (nntp) app, which as designed (by default) ships with a 10 MB 
article cache, and expires headers in (IIRC) two weeks.  But my usage, in 
addition to following my various lists with it using gmane's list2news 
service, is as a long-time technical group and list archive.  My text-
instance pan (the one I use the most) has a cache size of several gig 
(with about a gig actually used) and is set to no-expiry on messages.  In 
fact, I have ISP newsgroups archived in pan for an ISP server that hasn't 
hasn't even existed for several years, now, as well as the archives for 
several mailing lists going back over a decade to 2002.

So this text-instance pan tends to be another prime example of best-use-
case-for-SSDs.  Thousands, actually tens of thousands of files I believe, 
all in the same cache dir, with pan accessing them all to rebuild its 
threading tree in memory at startup.  (For years there's been talk of 
switching that to a database, so it doesn't have to all be in memory at 
once, but the implementation has yet to be coded up and switched to.)  On 
spinning rust, I did note a good speed boost if I backed up everything 
and did a mkfs and restore from time to time, so it's definitely a high 
fragmentation use-case as well.  *GREAT* use case for SSD, and there too, 
I noticed a HUGE difference.  Tho I've not actually timed the startup 
since switching to SSD, I do know that the pan icon appears in the system 
tray far earlier than it did, such that I almost think it's there as soon 
as the system tray is, now, whereas on the spinning rust, it would take 
five minutes or more to appear.

... Which is something those dd results don't, and can't, show at all.  
Single i/o thread access to a single rather large (GBs) unchanged since 
original write file is one thing.  Access to thousands or tens of 
thousands of constantly changing or multi-write-thread interwoven little 
files, or for that matter, to a high activity large file thus (depending 
on the filesystem) potentially triggering COW fragmentation there, is 
something entirely different.

And the many-parallel-job seek latency of spinning rust is something that 
dd simply does not and cannot really measure, as it's simply the wrong 
tool for that sort of purpose.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman