[gentoo-user] Want to seriously test a NEW hard drive

[gentoo-user] Want to seriously test a NEW hard drive

[Dale]

Howdy,

I finally got me a 3Tb drive on the way.  Should be here Wednesday.  I
have seen some reviews where it would not work right.  I think some of
it may be BIOS related since some BIOS's don't like drives that large. 
Anyway, I want to test this thing real good to really make sure it is up
to the task before putting my data on it.  It's going to be so much
data, there is really no way to do back-ups at this point.  Come on, 2
to 3Tbs on 4Gb DVDs.  Really?  lol  Maybe a external drive later on but
for now, well. 

I have heard of bonnie and friends.  I also think dd could do some
testing too.  Is there any other way to give this a good work and see if
it holds up?  Oh, helpful hints with Bonnie would be great too.  I have
never used it before.  Maybe someone has some test that is really brutal. 

Thanks

Dale

:-)  :-) 

-- 
I am only responsible for what I said ... Not for what you understood or how you interpreted my words!

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Mark Knecht]

On Kindle so hard to answer in depth. What about smartctl??? Let the
drive test itself.

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Alex Schuster]

Dale writes:

> I finally got me a 3Tb drive on the way.  Should be here Wednesday.  I
> have seen some reviews where it would not work right.  I think some of
> it may be BIOS related since some BIOS's don't like drives that large. 
> Anyway, I want to test this thing real good to really make sure it is up
> to the task before putting my data on it.  It's going to be so much
> data, there is really no way to do back-ups at this point.  Come on, 2
> to 3Tbs on 4Gb DVDs.  Really?  lol  Maybe a external drive later on but
> for now, well. 
> 
> I have heard of bonnie and friends.  I also think dd could do some
> testing too.  Is there any other way to give this a good work and see if
> it holds up?  Oh, helpful hints with Bonnie would be great too.  I have
> never used it before.  Maybe someone has some test that is really
> brutal. 

smartctl -t long /dev/sdb will make the drive start a selftest. This
will take a while, and even more if the drive is being used otherwise, as
this test should not impact its performance. Use smartctl -l selftest to
view the results. As long as there is no number in the
'LBA_of_first_error' column, it should be okay.

That is a reading test only, badblocks -sw /dev/sdb will make it perform
a write-mode test. It uses four different patterns, I would be okay with
only one test, so I'd either stop it when it is done writing and
comparing the first pattern, or supply a test pattern with option -t.

	Wonko

Re: [gentoo-user] Want to seriously test a NEW hard drive

[victor romanchuk]

hi

Dale wrote, at 08/05/2012 04:45 PM:
> Howdy,
>
>
> I have heard of bonnie and friends.  I also think dd could do some
> testing too.  Is there any other way to give this a good work and see if
> it holds up?  Oh, helpful hints with Bonnie would be great too.  I have
> never used it before.  Maybe someone has some test that is really brutal. 
>

some time ago i have played with bonnie++ to figure out my hard disk performance
using different filesystems and io schedulers. the script invokes 3 bonnie
instances; each instance runs its own set of tests: write/read/rewrite a 30gb
file followed with different file operations on 48k small files spread over 32
sub-directories:

#!/bin/bash
#
scratch=${1:-$(pwd)}
#
date
ft=$(df -Pl $scratch|tail -1|awk '{print $6}')
mnt=($(mount|grep "$ft "))
dev=$(basename $(readlink -fn ${mnt[0]}))
sched=$(cat /sys/block/$dev/queue/scheduler|sed -e 's/.*\[//1' -e 's/\].*//1')
log="$dev-${sched}-${mnt[4]}"
echo $log
rm -f ${log} ${log}.html
/usr/sbin/bonnie++ -p 3
/usr/sbin/bonnie++ -qd $scratch -n48:128K:16K:32 -ys -s30g -mg1 >> ${log} &
/usr/sbin/bonnie++ -qd $scratch -n48:128K:16K:32 -ys -s30g -mg2 >> ${log} &
/usr/sbin/bonnie++ -qd $scratch -n48:123K:16K:32 -ys -s30g -mg3 >> ${log} &
wait
/usr/sbin/bonnie++ -p -1
bon_csv2html ${log} >> ${log}.html
date

this script worked around 30mins on a sata3 1tb drive. presuming your 3tb, you
may adjust the file size and/or number of bonnie instances to fill up the disk
space; then start the script and leave it running for a day. i guess this test
would be brutal enough and on completion the disk might be considered good :)

victor

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Dale]

Dale wrote:
> Howdy,
>
> I finally got me a 3Tb drive on the way.  Should be here Wednesday.  I
> have seen some reviews where it would not work right.  I think some of
> it may be BIOS related since some BIOS's don't like drives that large. 
> Anyway, I want to test this thing real good to really make sure it is up
> to the task before putting my data on it.  It's going to be so much
> data, there is really no way to do back-ups at this point.  Come on, 2
> to 3Tbs on 4Gb DVDs.  Really?  lol  Maybe a external drive later on but
> for now, well. 
>
> I have heard of bonnie and friends.  I also think dd could do some
> testing too.  Is there any other way to give this a good work and see if
> it holds up?  Oh, helpful hints with Bonnie would be great too.  I have
> never used it before.  Maybe someone has some test that is really brutal. 
>
> Thanks
>
> Dale
>
> :-)  :-) 
>

I seem to have got a few really good responses to my question.  Since I
really want to test this drive really hard, I'll try them all.  lol  If
it survives a few runs of all this, maybe it is going to live.  Like the
one with the script too.  May have to edit for my needs but certainly a
GREAT start. 

While at it, I found a lot of references to the flood in Japan or I
assume it was Japan.  Anyone think this could still be a problem? 
Should they all be flushed out of the system by now?  Pardon the flush
term there.  ;-) 

Linky:

http://www.tigerdirect.com/applications/SearchTools/item-details.asp?EdpNo=1310560&CatId=4357

Please, no hard drive brand flame wars.  Seagates fail, WD fails, they
all fail at some point.  We all know that already.  This is what I got
so it is what it is.  :-D  That is why I want to test this thing like a
wild man BEFORE putting anything on it.  If it has a issue, hopefully
the test will bring that to the front now instead of later. 

Waiting on Wednesday to get here now. 

Dale

:-)  :-) 

P. S.  What is a good way to back up something this large BESIDES
another drive the same size or larger?  I thought about getting a blue
ray burner but even that will take a lot of media.  Another drive is all
I can think of myself. 

-- 
I am only responsible for what I said ... Not for what you understood or how you interpreted my words!

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Volker Armin Hemmann]

[-- Attachment #1: Type: text/plain, Size: 2516 bytes --]

What do you gain if you abuse your drive so hard that its lifetime is
severly impacted?
Am 06.08.2012 11:47 schrieb "Dale" <rdalek1967@gmail.com>:

> Dale wrote:
> > Howdy,
> >
> > I finally got me a 3Tb drive on the way.  Should be here Wednesday.  I
> > have seen some reviews where it would not work right.  I think some of
> > it may be BIOS related since some BIOS's don't like drives that large.
> > Anyway, I want to test this thing real good to really make sure it is up
> > to the task before putting my data on it.  It's going to be so much
> > data, there is really no way to do back-ups at this point.  Come on, 2
> > to 3Tbs on 4Gb DVDs.  Really?  lol  Maybe a external drive later on but
> > for now, well.
> >
> > I have heard of bonnie and friends.  I also think dd could do some
> > testing too.  Is there any other way to give this a good work and see if
> > it holds up?  Oh, helpful hints with Bonnie would be great too.  I have
> > never used it before.  Maybe someone has some test that is really brutal.
> >
> > Thanks
> >
> > Dale
> >
> > :-)  :-)
> >
>
> I seem to have got a few really good responses to my question.  Since I
> really want to test this drive really hard, I'll try them all.  lol  If
> it survives a few runs of all this, maybe it is going to live.  Like the
> one with the script too.  May have to edit for my needs but certainly a
> GREAT start.
>
> While at it, I found a lot of references to the flood in Japan or I
> assume it was Japan.  Anyone think this could still be a problem?
> Should they all be flushed out of the system by now?  Pardon the flush
> term there.  ;-)
>
> Linky:
>
>
> http://www.tigerdirect.com/applications/SearchTools/item-details.asp?EdpNo=1310560&CatId=4357
>
> Please, no hard drive brand flame wars.  Seagates fail, WD fails, they
> all fail at some point.  We all know that already.  This is what I got
> so it is what it is.  :-D  That is why I want to test this thing like a
> wild man BEFORE putting anything on it.  If it has a issue, hopefully
> the test will bring that to the front now instead of later.
>
> Waiting on Wednesday to get here now.
>
> Dale
>
> :-)  :-)
>
> P. S.  What is a good way to back up something this large BESIDES
> another drive the same size or larger?  I thought about getting a blue
> ray burner but even that will take a lot of media.  Another drive is all
> I can think of myself.
>
> --
> I am only responsible for what I said ... Not for what you understood or
> how you interpreted my words!
>
>
>

[-- Attachment #2: Type: text/html, Size: 3155 bytes --]

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Dale]

Volker Armin Hemmann wrote:
>
> What do you gain if you abuse your drive so hard that its lifetime is
> severly impacted?
>

That if it has a problem that will cause it to fail soon in it's life,
then I can find it soon.  Remember that curve about failures?  I would
like to get past that first part of the curve.  Maybe by the time I get
to the later part, I'll have another drive or some backup scheme.  Most
the failures I have read about in reviews for this drive were early or
was just plain old DOA.  Testing it will get me past that.  I'd rather
it fail before I get my data on it instead of after. 

I thought I posted why I wanted to do this in my first post. 

Dale

:-)  :-) 

-- 
I am only responsible for what I said ... Not for what you understood or how you interpreted my words!

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Hinnerk van Bruinehsen]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 06.08.2012 12:14, Dale wrote:
> Volker Armin Hemmann wrote:
>> 
>> What do you gain if you abuse your drive so hard that its
>> lifetime is severly impacted?
>> 
> 
> That if it has a problem that will cause it to fail soon in it's
> life, then I can find it soon.  Remember that curve about failures?
> I would like to get past that first part of the curve.  Maybe by
> the time I get to the later part, I'll have another drive or some
> backup scheme.  Most the failures I have read about in reviews for
> this drive were early or was just plain old DOA.  Testing it will
> get me past that.  I'd rather it fail before I get my data on it
> instead of after.
> 
> I thought I posted why I wanted to do this in my first post.
> 
> Dale
> 
> :-)  :-)
> 

Why not simply get you data on it and use it for about 2 weeks? Maybe
you should mirror important stuff to the old drive for that time.

After about 2 weeks of normal usage you should be well out of the
beginnig of that bathtub curve (I always had problems when copying
data to the new drive when I had a bad one, except DOA of course).


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJQH6ESAAoJEJwwOFaNFkYc1f0IAK/UpzNp/r4KBYczYIc9jKmJ
3JBvtCWPZkJ1qRXfZtErTk/oamPa+/iapJSoaoj4p45Uvjq5Qh5WDXiAlZs3U805
X1nCc9OMfKc3KGK3bcSKLhSIA1oV1imdgq1OqH/NeioEqOuVy56ZOdvwq6ml2fwM
1ZjBjsp1uqirork5eyZcJGGoNcsJpX3DvTZ7tBwM0gDFHcu4Yj3alEy7PgiYny4W
z0/K91jnyBouX7+CwtWKvGNgjGqMxXXp4SUS4MuOKs06p4LFKi/gHBRCAYjuTRK1
O3cKfz4nHMI+F2rH48sCAUgT5K88cdiX40lZpqTWyqADUQO9F1ZzTcg4fZW447s=
=s524
-----END PGP SIGNATURE-----

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Mick]

[-- Attachment #1: Type: Text/Plain, Size: 1418 bytes --]

On Monday 06 Aug 2012 11:48:50 Hinnerk van Bruinehsen wrote:
> On 06.08.2012 12:14, Dale wrote:
> > Volker Armin Hemmann wrote:
> >> What do you gain if you abuse your drive so hard that its
> >> lifetime is severly impacted?
> > 
> > That if it has a problem that will cause it to fail soon in it's
> > life, then I can find it soon.  Remember that curve about failures?
> > I would like to get past that first part of the curve.  Maybe by
> > the time I get to the later part, I'll have another drive or some
> > backup scheme.  Most the failures I have read about in reviews for
> > this drive were early or was just plain old DOA.  Testing it will
> > get me past that.  I'd rather it fail before I get my data on it
> > instead of after.
> > 
> > I thought I posted why I wanted to do this in my first post.
> > 
> > Dale
> > 
> > :-)  :-)
> 
> Why not simply get you data on it and use it for about 2 weeks? Maybe
> you should mirror important stuff to the old drive for that time.
> 
> After about 2 weeks of normal usage you should be well out of the
> beginnig of that bathtub curve (I always had problems when copying
> data to the new drive when I had a bad one, except DOA of course).

The 'Conveyance self-test routine' of smartmontools will check for damage 
during physical transport.  If it is completely DOA, then that ought to be 
obvious I guess.
-- 
Regards,
Mick

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Mick]

[-- Attachment #1: Type: Text/Plain, Size: 630 bytes --]

On Monday 06 Aug 2012 10:42:17 Dale wrote:

> If it has a issue, hopefully
> the test will bring that to the front now instead of later.

Or it is just going to knacker your drive and make it fail earlier that it 
would otherwise (esp if you overheat it)?  ha, ha!


> P. S.  What is a good way to back up something this large BESIDES
> another drive the same size or larger?  I thought about getting a blue
> ray burner but even that will take a lot of media.  Another drive is all
> I can think of myself.

Splitting it?

Partimage would do that and so would star/tar for fs level backups.
-- 
Regards,
Mick

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Dale]

Mick wrote:
> On Monday 06 Aug 2012 10:42:17 Dale wrote:
>
>> If it has a issue, hopefully
>> the test will bring that to the front now instead of later.
> Or it is just going to knacker your drive and make it fail earlier that it 
> would otherwise (esp if you overheat it)?  ha, ha!

Well, if it is going to fail because of anything, including heat, I
would rather it do so BEFORE I put my stuff on it.  Right now, a backup
is not possible other than a blue ray or something.  Also, I have a
Cooler Master case with the fan blowing right on the drives.  If it gets
hot and blows a fuse, it has a problem anyway.  It may as well die early
while under warranty. 

>
>> P. S.  What is a good way to back up something this large BESIDES
>> another drive the same size or larger?  I thought about getting a blue
>> ray burner but even that will take a lot of media.  Another drive is all
>> I can think of myself.
> Splitting it?
>
> Partimage would do that and so would star/tar for fs level backups.

Split what?  The drive into two partitions or something?  If that is
what you mean, if it dies, I'd still loose it all.  If you meant two
drives, well, I only have one on the way right now.  It will be a while
before I can get another.  In the past, I just backed up stuff on DVDs. 
3Tbs is a lot of DVDs tho.  It won't take to long to fill that up
either.  I love my DSL.  lol

I hope to get another drive in the future tho.  Just going to be a
little while.  ;-) 

Dale

:-)  :-) 

-- 
I am only responsible for what I said ... Not for what you understood or how you interpreted my words!

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 804 bytes --]

On Mon, 06 Aug 2012 04:42:17 -0500, Dale wrote:

> P. S.  What is a good way to back up something this large BESIDES
> another drive the same size or larger?

A smaller drive, backups can be compressed. Also, you don't need to
backup everything, for example rips of DVDs and CDs you own can be
recreated. If you organise your data to separate stuff that needs to be
backed up from that which doesn't you can reduce the amount of space
needed.

Factor in that you won't be filling this drive for a while and that you
don't need maximum performance from a backup drive and you'll almost
certainly get by with using your old drive(s).

Really important stuff should be stored offsite, Amazon S3 gives the best
value for me.


-- 
Neil Bothwick

Ask a silly person, get a silly answer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Dale]

Neil Bothwick wrote:
> On Mon, 06 Aug 2012 04:42:17 -0500, Dale wrote:
>
>> P. S.  What is a good way to back up something this large BESIDES
>> another drive the same size or larger?
> A smaller drive, backups can be compressed. Also, you don't need to
> backup everything, for example rips of DVDs and CDs you own can be
> recreated. If you organise your data to separate stuff that needs to be
> backed up from that which doesn't you can reduce the amount of space
> needed.
>
> Factor in that you won't be filling this drive for a while and that you
> don't need maximum performance from a backup drive and you'll almost
> certainly get by with using your old drive(s).
>
> Really important stuff should be stored offsite, Amazon S3 gives the best
> value for me.
>
>

What I have on there is videos.  When I tried to compress some and test,
it was basically the same size.  I guess videos don't compress to much? 

When I put this in, I'm going to redo the whole thing.  Ages ago when I
was green all over and not just around the gills, I created a /data
directory and that is where I stored "stuff".  My new plan, the new
drive will become /home and I will be putting things where they should
have been to begin with.  I plan to reorganize this whole mess I created
ages ago.  That video directory is HUGE tho. 

root@fireball / # du -shc /data/Videos/
703G    /data/Videos/
703G    total
root@fireball / #

While I am at it.  You should have a good answer for this one.  What is
a good file system for this sort of thing?  I been using ext4 but it
sure does use a lot of space for its overhead.  As far as files go, most
will likely be videos.  I do have other files but when compared to the
number of videos, they are close to nothing.  The files system for the
current 1Tb, spread across two drives with LVM, uses about 75Gbs for the
file system thingy.  That's a pretty good bit to me.  I may lose more
than 200Gbs on this 3Tb drive.  O_O

Thanks.

Dale

:-)  :-) 

-- 
I am only responsible for what I said ... Not for what you understood or how you interpreted my words!

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 503 bytes --]

On Mon, 06 Aug 2012 06:25:19 -0500, Dale wrote:

> While I am at it.  You should have a good answer for this one.  What is
> a good file system for this sort of thing?  I been using ext4 but it
> sure does use a lot of space for its overhead.

My MythTV box used XFS, but I use ext4 on newer systems. If you're only
storing data on it, create the filesystem with the -m 0 option to make
all the space available.


-- 
Neil Bothwick

There is always one more imbecile than you counted on.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Michael Hampicke]

> What I have on there is videos.  When I tried to compress some and test,
> it was basically the same size.  I guess videos don't compress to much? 
> 

No videos don't compress well at all, compressing video files is just a
waste of cpu cycles :)

> When I put this in, I'm going to redo the whole thing.  Ages ago when I
> was green all over and not just around the gills, I created a /data
> directory and that is where I stored "stuff".  My new plan, the new
> drive will become /home and I will be putting things where they should
> have been to begin with.  I plan to reorganize this whole mess I created
> ages ago.  That video directory is HUGE tho. 
> 
> root@fireball / # du -shc /data/Videos/
> 703G    /data/Videos/
> 703G    total
> root@fireball / #
> 
> While I am at it.  You should have a good answer for this one.  What is
> a good file system for this sort of thing?  I been using ext4 but it
> sure does use a lot of space for its overhead.  As far as files go, most
> will likely be videos.  I do have other files but when compared to the
> number of videos, they are close to nothing.  The files system for the
> current 1Tb, spread across two drives with LVM, uses about 75Gbs for the
> file system thingy.  That's a pretty good bit to me.  I may lose more
> than 200Gbs on this 3Tb drive.  O_O

I use XFS on my NAS-Box for the drives that only hold video files and I
am pretty happy with it. The advantage of XFS is, that it is very fast
when working with large files. For everything else I use ext4, pretty
happy with that too.

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Mick]

[-- Attachment #1: Type: Text/Plain, Size: 1049 bytes --]

On Monday 06 Aug 2012 12:25:19 Dale wrote:

> What I have on there is videos.  When I tried to compress some and test,
> it was basically the same size.  I guess videos don't compress to much?

Not without transcoding the streams into more space efficient formats - but 
you could well suffer from loss of quality (lossy formats and digital 
generation losses).  So YMMV.


> While I am at it.  You should have a good answer for this one.  What is
> a good file system for this sort of thing?  I been using ext4 but it
> sure does use a lot of space for its overhead.  As far as files go, most
> will likely be videos.  I do have other files but when compared to the
> number of videos, they are close to nothing.  The files system for the
> current 1Tb, spread across two drives with LVM, uses about 75Gbs for the
> file system thingy.  That's a pretty good bit to me.  I may lose more
> than 200Gbs on this 3Tb drive.  O_O

That sounds like a lot!  I'm waiting to see what others comment and 
benchmarks.
-- 
Regards,
Mick

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Paul Hartman]

On Sun, Aug 5, 2012 at 7:45 AM, Dale <rdalek1967@gmail.com> wrote:
> Howdy,
>
> I finally got me a 3Tb drive on the way.  Should be here Wednesday.  I
> have seen some reviews where it would not work right.  I think some of
> it may be BIOS related since some BIOS's don't like drives that large.
> Anyway, I want to test this thing real good to really make sure it is up
> to the task before putting my data on it.  It's going to be so much
> data, there is really no way to do back-ups at this point.  Come on, 2
> to 3Tbs on 4Gb DVDs.  Really?  lol  Maybe a external drive later on but
> for now, well.
>
> I have heard of bonnie and friends.  I also think dd could do some
> testing too.  Is there any other way to give this a good work and see if
> it holds up?  Oh, helpful hints with Bonnie would be great too.  I have
> never used it before.  Maybe someone has some test that is really brutal.

I wouldn't want to torture the drive, but just make sure it works.

What I do: boot parted magic liveCD, run ATA SECURE ERASE on the drive
(you probably need to suspend and awaken your machine to unlock the
drive, the GUI tool in the liveCD does this for you automatically),
that will do factory reformat/low-level reformat, then run SMART full
test which can take many hours. If it survives both of those, and
doesn't make audible clicking noises in the process, I feel confident
that it is in working order.

Those steps are more important if I'm testing a used or refurbished
drive, for a new drive you may want to skip the secure erase and only
do the smart test. If SMART test passes then I don't think there's any
reason to run badblocks, but you can have it run during mkfs if you
want reassurance.

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Dale]

Paul Hartman wrote:
> On Sun, Aug 5, 2012 at 7:45 AM, Dale <rdalek1967@gmail.com> wrote:
>> Howdy,
>>
>> I finally got me a 3Tb drive on the way.  Should be here Wednesday.  I
>> have seen some reviews where it would not work right.  I think some of
>> it may be BIOS related since some BIOS's don't like drives that large.
>> Anyway, I want to test this thing real good to really make sure it is up
>> to the task before putting my data on it.  It's going to be so much
>> data, there is really no way to do back-ups at this point.  Come on, 2
>> to 3Tbs on 4Gb DVDs.  Really?  lol  Maybe a external drive later on but
>> for now, well.
>>
>> I have heard of bonnie and friends.  I also think dd could do some
>> testing too.  Is there any other way to give this a good work and see if
>> it holds up?  Oh, helpful hints with Bonnie would be great too.  I have
>> never used it before.  Maybe someone has some test that is really brutal.
> I wouldn't want to torture the drive, but just make sure it works.
>
> What I do: boot parted magic liveCD, run ATA SECURE ERASE on the drive
> (you probably need to suspend and awaken your machine to unlock the
> drive, the GUI tool in the liveCD does this for you automatically),
> that will do factory reformat/low-level reformat, then run SMART full
> test which can take many hours. If it survives both of those, and
> doesn't make audible clicking noises in the process, I feel confident
> that it is in working order.
>
> Those steps are more important if I'm testing a used or refurbished
> drive, for a new drive you may want to skip the secure erase and only
> do the smart test. If SMART test passes then I don't think there's any
> reason to run badblocks, but you can have it run during mkfs if you
> want reassurance.
>
>

Well, I don't want to force it to fail just for the heck of it.  I just
want to make sure it is not going to fail in the first few months of
use.  The reviews that people have wrote that had problems were
generally either out of the box dead or failed after a few weeks or a
month or so.  I figure if I can run tests that would be about a month or
so of use, then maybe I am OK.  That is my goal.  I just don't want to
get all my data on there and such and then have it go belly up. 

I didn't know you could do low level formats anymore.  Really?  What
package provides that?  Hmmm, I'm thinking about those HOURS spent
formatting a 100Mb drive and then thinking about how long it will take
to do a 3Tb drive.  O_O  I mean really O_O.  LOL 

Dale

:-)  :-) 

P. S.  It should be here tomorrow. UPS dropped it off at the post office
today. 

-- 
I am only responsible for what I said ... Not for what you understood or how you interpreted my words!

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Mark Knecht]

On Tue, Aug 7, 2012 at 3:26 PM, Dale <rdalek1967@gmail.com> wrote:
<SNIP>
> Well, I don't want to force it to fail just for the heck of it.

Of course...

> I just
> want to make sure it is not going to fail in the first few months of
> use.

But what you don't know about those failed drives is whether the user
could have predicted it by watching the smartctl data more closely.

Dead out of the box is dead. However a drive failing in a couple of
months _might_ have showed up in the smartctl output in which case the
user could have transferred data to a new drive.

Keep in mind, that's all you're talking about here. You don't want to
spend the money for a second drive, but you know you'll have to do
_something_ if the drive starts failing. Not having backups isn't a
good plan...

Take care,
Mark

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Dale]

Mark Knecht wrote:
> On Tue, Aug 7, 2012 at 3:26 PM, Dale <rdalek1967@gmail.com> wrote:
> <SNIP>
>> Well, I don't want to force it to fail just for the heck of it.
> Of course...
>
>> I just
>> want to make sure it is not going to fail in the first few months of
>> use.
> But what you don't know about those failed drives is whether the user
> could have predicted it by watching the smartctl data more closely.
>
> Dead out of the box is dead. However a drive failing in a couple of
> months _might_ have showed up in the smartctl output in which case the
> user could have transferred data to a new drive.
>
> Keep in mind, that's all you're talking about here. You don't want to
> spend the money for a second drive, but you know you'll have to do
> _something_ if the drive starts failing. Not having backups isn't a
> good plan...
>
> Take care,
> Mark
>
>

I realize that not having backups is not good.  Thing is, I really don't
have a way to back up that much data.  Heck, I got about 1Tb that will
be transfered to the new drive when I feel it is ready.  Other than a
second drive, I have no means of backing up that much data.  I thought
about a blue ray burner but even that is a lot of media.  I think I
figured up over a 100.  Plain DVDs are even worse. 

My plan is to watch the logs and buy another drive as soon as I can.  In
the meantime, it is what it is.  I don't like it but I can't change it
either.  If it helps any, I have only had one drive to ever fail on me. 
It was a WD and was pretty old.  It gave me enough warning to get the
data off tho.  We gotta love that. 

Dale

:-)  :-) 

-- 
I am only responsible for what I said ... Not for what you understood or how you interpreted my words!

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Mark Knecht]

On Tue, Aug 7, 2012 at 4:05 PM, Dale <rdalek1967@gmail.com> wrote:
> In
> the meantime, it is what it is.

I completely understand Dale, and I'm not picking on you. Just get
that second drive as soon as you can.

Cheers,
Mark

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Peter Humphrey]

On Tuesday 07 August 2012 23:39:05 Mark Knecht wrote:

> Dead out of the box is dead. However a drive failing in a couple of
> months _might_ have showed up in the smartctl output ...

I wonder. Does anyone here know what most often causes HD failure after 
a couple of months? I imagine it's some component whose material is 
substandard, in which case it might show up with smartclt or it might 
not. I dunno.

-- 
Rgds
Peter

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Mark Knecht]

On Tue, Aug 7, 2012 at 4:06 PM, Peter Humphrey <peter@humphrey.ukfsn.org> wrote:
> On Tuesday 07 August 2012 23:39:05 Mark Knecht wrote:
>
>> Dead out of the box is dead. However a drive failing in a couple of
>> months _might_ have showed up in the smartctl output ...
>
> I wonder. Does anyone here know what most often causes HD failure after
> a couple of months? I imagine it's some component whose material is
> substandard, in which case it might show up with smartclt or it might
> not. I dunno.
>
> --
> Rgds
> Peter
>

This has come up before on this list I think. This is the overarching
sort of thinking about failure over lifetime...

http://en.wikipedia.org/wiki/Bathtub_curve

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Peter Humphrey]

On Wednesday 08 August 2012 01:32:55 Mark Knecht wrote:
> On Tue, Aug 7, 2012 at 4:06 PM, Peter Humphrey 
<peter@humphrey.ukfsn.org> wrote:
> > On Tuesday 07 August 2012 23:39:05 Mark Knecht wrote:
> >> Dead out of the box is dead. However a drive failing in a couple
> >> of months _might_ have showed up in the smartctl output ...
> > 
> > I wonder. Does anyone here know what most often causes HD failure
> > after a couple of months? I imagine it's some component whose
> > material is substandard, in which case it might show up with
> > smartclt or it might not. I dunno.
> > 
> > --
> > Rgds
> > Peter
> 
> This has come up before on this list I think.

I must have missed it then.

> This is the overarching sort of thinking about failure over
> lifetime...
> 
> http://en.wikipedia.org/wiki/Bathtub_curve

Yes, I'm familiar with that idea thanks. I was wondering what specific 
causes were most implicated.

-- 
Rgds
Peter

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Paul Hartman]

On Tue, Aug 7, 2012 at 5:26 PM, Dale <rdalek1967@gmail.com> wrote:
>
> I didn't know you could do low level formats anymore.  Really?  What
> package provides that?  Hmmm, I'm thinking about those HOURS spent
> formatting a 100Mb drive and then thinking about how long it will take
> to do a 3Tb drive.  O_O  I mean really O_O.  LOL

hdparm provides it. Do a search for "ATA secure erase" or "enhanced
secure erase". It is as close as there is to a low-level format in
modern drives. It is basically a erase/format within the drive's
firmware, that resets it all back to factory, including bad sectors,
with the same pattern of 1's and 0's and everything. You can do it
with hdparm but it's tricky and contains many warnings about killing
your drive. It is considered the only "true" way to properly erase a
hard drive as anything else is just overwriting and does not
necessarily touch all the areas that the firmware can touch. I think
actual implementation of what the secure erases do varies from drive
to drive, but they'll all format the whole disk for sure.

The parted magic live CD contains a GUI tool to automate it and it is
extremely simple to use. Choose your drive and go. On a 2tb drive I
think it took 4 or 5 hours when I ran it. There is absolutely no
feedback while it is running, so you're just waiting with no progress
indicator or anything. You can also do SMART tests from within the
parted magic live CD environment. And of course partitioning. :)

That all being said, when performing this kind of operation I usually
like to use a live CD and unplug ALL OTHER HARD DRIVES except for the
one I'm going to destroy. I don't want to accidentally erase the wrong
drive. (In fact I have an old Pentium 4 computer with no HDDs that I
use solely for the purpose of testing live CDs, testing and formatting
drives, partitioning new drives before i put them into a production
machine)

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Dale]

Paul Hartman wrote:
> On Tue, Aug 7, 2012 at 5:26 PM, Dale <rdalek1967@gmail.com> wrote:
>> I didn't know you could do low level formats anymore.  Really?  What
>> package provides that?  Hmmm, I'm thinking about those HOURS spent
>> formatting a 100Mb drive and then thinking about how long it will take
>> to do a 3Tb drive.  O_O  I mean really O_O.  LOL
> hdparm provides it. Do a search for "ATA secure erase" or "enhanced
> secure erase". It is as close as there is to a low-level format in
> modern drives. It is basically a erase/format within the drive's
> firmware, that resets it all back to factory, including bad sectors,
> with the same pattern of 1's and 0's and everything. You can do it
> with hdparm but it's tricky and contains many warnings about killing
> your drive. It is considered the only "true" way to properly erase a
> hard drive as anything else is just overwriting and does not
> necessarily touch all the areas that the firmware can touch. I think
> actual implementation of what the secure erases do varies from drive
> to drive, but they'll all format the whole disk for sure.
>
> The parted magic live CD contains a GUI tool to automate it and it is
> extremely simple to use. Choose your drive and go. On a 2tb drive I
> think it took 4 or 5 hours when I ran it. There is absolutely no
> feedback while it is running, so you're just waiting with no progress
> indicator or anything. You can also do SMART tests from within the
> parted magic live CD environment. And of course partitioning. :)
>
> That all being said, when performing this kind of operation I usually
> like to use a live CD and unplug ALL OTHER HARD DRIVES except for the
> one I'm going to destroy. I don't want to accidentally erase the wrong
> drive. (In fact I have an old Pentium 4 computer with no HDDs that I
> use solely for the purpose of testing live CDs, testing and formatting
> drives, partitioning new drives before i put them into a production
> machine)
>
>


I have seen where people use dd to do this sort of thing to.  I read
somewhere that if you do a dd and put in all 1's, then all 0's then back
again that it is very hard to get any data back off the drive.  I think
if you do it like over a dozen times, it is deemed impossible to get
anything back.  I think that is the Government standard of it's gone. 

4 or 5 hours huh.  I guess drives are a lot faster now.  Back in the
late 80's or early 90's, it took that long for those whimpy little 100Mb
drives.  Ooops, my ages is showing again.  lol 

I got to go read up on hdparm.  I already have it installed here.  I'm
not planning to use this part but do want to read up on this. 

Thanks.

Dale

:-)  :-) 

-- 
I am only responsible for what I said ... Not for what you understood or how you interpreted my words!

Re: [gentoo-user] Want to seriously test a NEW hard drive

[William Kenworthy]

On Tue, 2012-08-07 at 21:19 -0500, Dale wrote:
> Paul Hartman wrote:
> > On Tue, Aug 7, 2012 at 5:26 PM, Dale <rdalek1967@gmail.com> wrote:
> >> I didn't know you could do low level formats anymore.  Really?  What
> >> package provides that?  Hmmm, I'm thinking about those HOURS spent
> >> formatting a 100Mb drive and then thinking about how long it will take
> >> to do a 3Tb drive.  O_O  I mean really O_O.  LOL
> > hdparm provides it. Do a search for "ATA secure erase" or "enhanced
> > secure erase". It is as close as there is to a low-level format in
...
> 
> I have seen where people use dd to do this sort of thing to.  I read
> somewhere that if you do a dd and put in all 1's, then all 0's then back
> again that it is very hard to get any data back off the drive.  I think
> if you do it like over a dozen times, it is deemed impossible to get
> anything back.  I think that is the Government standard of it's gone. 
> 
> 4 or 5 hours huh.  I guess drives are a lot faster now.  Back in the
> late 80's or early 90's, it took that long for those whimpy little 100Mb
> drives.  Ooops, my ages is showing again.  lol 
> 
> I got to go read up on hdparm.  I already have it installed here.  I'm
> not planning to use this part but do want to read up on this. 
> 
> Thanks.
> 
> Dale
> 
> :-)  :-) 
> 


Goggle have a well known document
(http://research.google.com/archive/disk_failures.pdf) where they
analysed hard drive failures for a very large number of drives ... the
basic upshot is that a very large portion of failures happen with no
pre-warning, so testing a drive like you are proposing not going to
prove a thing.

They also found that smart (is quite dumb) and its tests were of little
use.

And high temperatures and work loads were also not a reliable guide to
trends in failure rates, both of which which surprised me.

Some of those bathtub curves that I was trained on when setting
maintenance schedules dont hold water here!

This anaysis of the paper looks quite good if you want the lite view:
http://storagemojo.com/2007/02/19/googles-disk-failure-experience/

BillK



BillK

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Dale]

William Kenworthy wrote:
> On Tue, 2012-08-07 at 21:19 -0500, Dale wrote:
>> Paul Hartman wrote:
>>> On Tue, Aug 7, 2012 at 5:26 PM, Dale <rdalek1967@gmail.com> wrote:
>>>> I didn't know you could do low level formats anymore.  Really?  What
>>>> package provides that?  Hmmm, I'm thinking about those HOURS spent
>>>> formatting a 100Mb drive and then thinking about how long it will take
>>>> to do a 3Tb drive.  O_O  I mean really O_O.  LOL
>>> hdparm provides it. Do a search for "ATA secure erase" or "enhanced
>>> secure erase". It is as close as there is to a low-level format in
> ...
>> I have seen where people use dd to do this sort of thing to.  I read
>> somewhere that if you do a dd and put in all 1's, then all 0's then back
>> again that it is very hard to get any data back off the drive.  I think
>> if you do it like over a dozen times, it is deemed impossible to get
>> anything back.  I think that is the Government standard of it's gone. 
>>
>> 4 or 5 hours huh.  I guess drives are a lot faster now.  Back in the
>> late 80's or early 90's, it took that long for those whimpy little 100Mb
>> drives.  Ooops, my ages is showing again.  lol 
>>
>> I got to go read up on hdparm.  I already have it installed here.  I'm
>> not planning to use this part but do want to read up on this. 
>>
>> Thanks.
>>
>> Dale
>>
>> :-)  :-) 
>>
>
> Goggle have a well known document
> (http://research.google.com/archive/disk_failures.pdf) where they
> analysed hard drive failures for a very large number of drives ... the
> basic upshot is that a very large portion of failures happen with no
> pre-warning, so testing a drive like you are proposing not going to
> prove a thing.
>
> They also found that smart (is quite dumb) and its tests were of little
> use.
>
> And high temperatures and work loads were also not a reliable guide to
> trends in failure rates, both of which which surprised me.
>
> Some of those bathtub curves that I was trained on when setting
> maintenance schedules dont hold water here!
>
> This anaysis of the paper looks quite good if you want the lite view:
> http://storagemojo.com/2007/02/19/googles-disk-failure-experience/
>
> BillK
>
>
>
> BillK
>
>

Well, I am going by actual real experiences from other users of this
model of drive.  I don't know what google was testing but I would bet it
is not the drive model I just bought.  The users who bought this exact
model drive report that most failures are either out of the box or
within a few weeks to a month.  I'm just going to try to increase my
odds even if it is just a little bit. 

Smart may not always predict a failure but it is better than nothing at
all.  Would you rather have a tool that may predict a failure or no tool
at all?  Me, I'd rather have something that at least tries too.  The one
drive I had to go bad, Smart predicted it very well.  It said I had
about 24 hrs to get my stuff off.  Sure enough, the next day, it
wouldn't do anything but spin.  Without Smart and its prediction, I'd
have lost the data on the drive with no warning at all. 

A couple questions.  What if while I am testing this drive, it dies? 
Does that prove that my testing benefited me then? 

Dale

:-)  :-) 

-- 
I am only responsible for what I said ... Not for what you understood or how you interpreted my words!

Re: [gentoo-user] Want to seriously test a NEW hard drive

[William Kenworthy]

On Tue, 2012-08-07 at 23:18 -0500, Dale wrote:
> William Kenworthy wrote:
> > On Tue, 2012-08-07 at 21:19 -0500, Dale wrote:
> >> Paul Hartman wrote:
> >>> On Tue, Aug 7, 2012 at 5:26 PM, Dale <rdalek1967@gmail.com> wrote:
...
> > Goggle have a well known document
> > (http://research.google.com/archive/disk_failures.pdf) where they
> > analysed hard drive failures for a very large number of drives ... the
> > basic upshot is that a very large portion of failures happen with no
> > pre-warning, so testing a drive like you are proposing not going to
> > prove a thing.
> >
> > They also found that smart (is quite dumb) and its tests were of little
> > use.
> >
> > And high temperatures and work loads were also not a reliable guide to
> > trends in failure rates, both of which which surprised me.
> >
> > Some of those bathtub curves that I was trained on when setting
> > maintenance schedules dont hold water here!
> >
> > This anaysis of the paper looks quite good if you want the lite view:
> > http://storagemojo.com/2007/02/19/googles-disk-failure-experience/
> >
> > BillK
> >
> >
> 
> Well, I am going by actual real experiences from other users of this
> model of drive.  I don't know what google was testing but I would bet it
> is not the drive model I just bought.  The users who bought this exact
> model drive report that most failures are either out of the box or
> within a few weeks to a month.  I'm just going to try to increase my
> odds even if it is just a little bit. 
> 
> Smart may not always predict a failure but it is better than nothing at
> all.  Would you rather have a tool that may predict a failure or no tool
> at all?  Me, I'd rather have something that at least tries too.  The one
> drive I had to go bad, Smart predicted it very well.  It said I had
> about 24 hrs to get my stuff off.  Sure enough, the next day, it
> wouldn't do anything but spin.  Without Smart and its prediction, I'd
> have lost the data on the drive with no warning at all. 
> 
> A couple questions.  What if while I am testing this drive, it dies? 
> Does that prove that my testing benefited me then? 
> 
> Dale
> 
> :-)  :-) 
> 

Read the paper - its written by someone who buys drives in batches of
100's+, not by a few guys posting on a forum somewhere who bought one
random drive, who probably didn't use anti-static techniques handling
the drive, and thumped it around in the boot of the car or got it via
the courier who was famous for delivering TV's by throwing them over the
fence.  It is a bit of an eye opener - read it.

My impression of models is that it is not really the model that has a
run of failures, but the batch so a different run of the same model will
have a different failure pattern.  There are exceptions such as those
IBM 60G deathstar drives, but then again they fixed it and following
drives of the same model were fine.

My own experience of smart is it tells you something, but what it seems
to say is not right (notice I am not saying it tells lies, but that the
data and interpretation don't make sense on the drives Ive had)

Drive failure does seem to be a semi-random lottery, but I am seriously
doubtful that testing will do anything ... it has as much chance of
precipitating failure that wouldn't occur otherwise because you are
seriously hammering it, or weakening the drive so it will fail at some
random time, but perhaps weeks away rather than the years it otherwise
would, or nothing will happen except for wasted electrons.  

Then again, I am of the view that modern electronics is
designed/programmed to fail a few seconds past warranty expiry (why else
do most devices have timekeeping built in :)

BillK

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Dale]

William Kenworthy wrote:
> On Tue, 2012-08-07 at 23:18 -0500, Dale wrote:
>> William Kenworthy wrote:
>>> On Tue, 2012-08-07 at 21:19 -0500, Dale wrote:
>>>> Paul Hartman wrote:
>>>>> On Tue, Aug 7, 2012 at 5:26 PM, Dale <rdalek1967@gmail.com> wrote:
> ...
>>> Goggle have a well known document
>>> (http://research.google.com/archive/disk_failures.pdf) where they
>>> analysed hard drive failures for a very large number of drives ... the
>>> basic upshot is that a very large portion of failures happen with no
>>> pre-warning, so testing a drive like you are proposing not going to
>>> prove a thing.
>>>
>>> They also found that smart (is quite dumb) and its tests were of little
>>> use.
>>>
>>> And high temperatures and work loads were also not a reliable guide to
>>> trends in failure rates, both of which which surprised me.
>>>
>>> Some of those bathtub curves that I was trained on when setting
>>> maintenance schedules dont hold water here!
>>>
>>> This anaysis of the paper looks quite good if you want the lite view:
>>> http://storagemojo.com/2007/02/19/googles-disk-failure-experience/
>>>
>>> BillK
>>>
>>>
>> Well, I am going by actual real experiences from other users of this
>> model of drive.  I don't know what google was testing but I would bet it
>> is not the drive model I just bought.  The users who bought this exact
>> model drive report that most failures are either out of the box or
>> within a few weeks to a month.  I'm just going to try to increase my
>> odds even if it is just a little bit. 
>>
>> Smart may not always predict a failure but it is better than nothing at
>> all.  Would you rather have a tool that may predict a failure or no tool
>> at all?  Me, I'd rather have something that at least tries too.  The one
>> drive I had to go bad, Smart predicted it very well.  It said I had
>> about 24 hrs to get my stuff off.  Sure enough, the next day, it
>> wouldn't do anything but spin.  Without Smart and its prediction, I'd
>> have lost the data on the drive with no warning at all. 
>>
>> A couple questions.  What if while I am testing this drive, it dies? 
>> Does that prove that my testing benefited me then? 
>>
>> Dale
>>
>> :-)  :-) 
>>
> Read the paper - its written by someone who buys drives in batches of
> 100's+, not by a few guys posting on a forum somewhere who bought one
> random drive, who probably didn't use anti-static techniques handling
> the drive, and thumped it around in the boot of the car or got it via
> the courier who was famous for delivering TV's by throwing them over the
> fence.  It is a bit of an eye opener - read it.
>
> My impression of models is that it is not really the model that has a
> run of failures, but the batch so a different run of the same model will
> have a different failure pattern.  There are exceptions such as those
> IBM 60G deathstar drives, but then again they fixed it and following
> drives of the same model were fine.
>
> My own experience of smart is it tells you something, but what it seems
> to say is not right (notice I am not saying it tells lies, but that the
> data and interpretation don't make sense on the drives Ive had)
>
> Drive failure does seem to be a semi-random lottery, but I am seriously
> doubtful that testing will do anything ... it has as much chance of
> precipitating failure that wouldn't occur otherwise because you are
> seriously hammering it, or weakening the drive so it will fail at some
> random time, but perhaps weeks away rather than the years it otherwise
> would, or nothing will happen except for wasted electrons.  
>
> Then again, I am of the view that modern electronics is
> designed/programmed to fail a few seconds past warranty expiry (why else
> do most devices have timekeeping built in :)
>
> BillK
>

Actually, I read the paper a long time ago.  May give it another look
but I'm still going by what people have posted about this specific
model.  If they make them all the same, then testing to at least see if
it is going to get past the initial stages is a good idea.  I do think
some failures were because of the BIOS and I stated that in my original
post.  Getting a DOA drive can happen but when there are mobos around
that can't see large drives, then one has to consider it.  The ones I
worry about are the ones that worked for a few weeks or a month then
died.  They obviously don't have BIOS issues but some other problem. 

Still, all things considered, I'm going to test the drive.  If it can
pass the test then I will feel better about putting my data on it.  As
for the paper:

root@fireball / # ls -al /home/dale/Desktop/disk_failures.pdf
-rw-r--r-- 1 dale users 247492 May 21 17:58
/home/dale/Desktop/disk_failures.pdf
root@fireball / #


I read it back in May.  It's still sitting on my desktop.  I might also
add, it is about 5 years old.  Drives have changed since then.  For one,
they have gotten larger.  We don't know what else may have changed either. 

Dale

:-)  :-) 

-- 
I am only responsible for what I said ... Not for what you understood or how you interpreted my words!

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Adam Carter]

> I have seen where people use dd to do this sort of thing to.  I read
> somewhere that if you do a dd and put in all 1's, then all 0's then back
> again that it is very hard to get any data back off the drive.  I think
> if you do it like over a dozen times, it is deemed impossible to get
> anything back.  I think that is the Government standard of it's gone.

I've heard the old attacks to recover data from a zerod drive are no
longer viable for disks of greater capacity than about 10G. I haven't
seen the information myself, however.

A single pass using dd would probably a good way of detecting any
existing bad blocks, so a smartctl then dd then smartctl again and a
diff of the results may be interesting.

I just use a 1TB software mirror for my backups.

Re: [gentoo-user] Want to seriously test a NEW hard drive

[William Kenworthy]

On Wed, 2012-08-08 at 14:02 +1000, Adam Carter wrote:
> > I have seen where people use dd to do this sort of thing to.  I read
> > somewhere that if you do a dd and put in all 1's, then all 0's then back
> > again that it is very hard to get any data back off the drive.  I think
> > if you do it like over a dozen times, it is deemed impossible to get
> > anything back.  I think that is the Government standard of it's gone.
> 
> I've heard the old attacks to recover data from a zerod drive are no
> longer viable for disks of greater capacity than about 10G. I haven't
> seen the information myself, however.
> 
> A single pass using dd would probably a good way of detecting any
> existing bad blocks, so a smartctl then dd then smartctl again and a
> diff of the results may be interesting.
> 
> I just use a 1TB software mirror for my backups.
> 

To wipe a drive use dban. - live CD which uses (US) gov approved
standards of wipe methods/patterns.

dd is only going to show sectors on a failed drive - too late!

To explain, modern drives have a store of locations they can use to
transparently replace any failed locations (apparently similar to the
way SSD's do it)  - the internal drive electronics handle this and its
not visible externally though smart data seems to show it, but as google
says, smart is a bit suspect.  The problem of a bad sector will only
show once all the reserved locations are used up, by which time the
drive is usually in rampant failure.

I do suspect this is one reason for googles results - actual failures of
the media (as against the motors/electronics are much as they always
have been, but the drives are not reporting them until its too late.

BillK

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Adam Carter]

> To wipe a drive use dban. - live CD which uses (US) gov approved
> standards of wipe methods/patterns.

Or shred, which comes with coreutils.

> dd is only going to show sectors on a failed drive - too late!
>
> To explain, modern drives have a store of locations they can use to
> transparently replace any failed locations (apparently similar to the
> way SSD's do it)  - the internal drive electronics handle this and its
> not visible externally though smart data seems to show it, but as google
> says, smart is a bit suspect.  The problem of a bad sector will only
> show once all the reserved locations are used up, by which time the
> drive is usually in rampant failure.
>
> I do suspect this is one reason for googles results - actual failures of
> the media (as against the motors/electronics are much as they always
> have been, but the drives are not reporting them until its too late.

Ahh - go to know. My reasoning assumed that smart reports all remaps.

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Mick]

[-- Attachment #1: Type: Text/Plain, Size: 1481 bytes --]

On Wednesday 08 Aug 2012 05:21:31 Adam Carter wrote:
> > To wipe a drive use dban. - live CD which uses (US) gov approved
> > standards of wipe methods/patterns.
> 
> Or shred, which comes with coreutils.
> 
> > dd is only going to show sectors on a failed drive - too late!
> > 
> > To explain, modern drives have a store of locations they can use to
> > transparently replace any failed locations (apparently similar to the
> > way SSD's do it)  - the internal drive electronics handle this and its
> > not visible externally though smart data seems to show it, but as google
> > says, smart is a bit suspect.  The problem of a bad sector will only
> > show once all the reserved locations are used up, by which time the
> > drive is usually in rampant failure.
> > 
> > I do suspect this is one reason for googles results - actual failures of
> > the media (as against the motors/electronics are much as they always
> > have been, but the drives are not reporting them until its too late.
> 
> Ahh - go to know. My reasoning assumed that smart reports all remaps.

May be it does, but I understand that dd or shred won't overwrite them, or any 
bad blocks.  You'll need the hdparm ATA secure erase (or enhanced secure 
erase) feature for that.

BTW, Dale make sure that you plug the drive in a SATA controller for running 
the hdparm erase function.  It has been reported that doing this using a USB 
port will brick the drive!
-- 
Regards,
Mick

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Dale]

Mick wrote:
> On Wednesday 08 Aug 2012 05:21:31 Adam Carter wrote:
>>> To wipe a drive use dban. - live CD which uses (US) gov approved
>>> standards of wipe methods/patterns.
>> Or shred, which comes with coreutils.
>>
>>> dd is only going to show sectors on a failed drive - too late!
>>>
>>> To explain, modern drives have a store of locations they can use to
>>> transparently replace any failed locations (apparently similar to the
>>> way SSD's do it)  - the internal drive electronics handle this and its
>>> not visible externally though smart data seems to show it, but as google
>>> says, smart is a bit suspect.  The problem of a bad sector will only
>>> show once all the reserved locations are used up, by which time the
>>> drive is usually in rampant failure.
>>>
>>> I do suspect this is one reason for googles results - actual failures of
>>> the media (as against the motors/electronics are much as they always
>>> have been, but the drives are not reporting them until its too late.
>> Ahh - go to know. My reasoning assumed that smart reports all remaps.
> May be it does, but I understand that dd or shred won't overwrite them, or any 
> bad blocks.  You'll need the hdparm ATA secure erase (or enhanced secure 
> erase) feature for that.
>
> BTW, Dale make sure that you plug the drive in a SATA controller for running 
> the hdparm erase function.  It has been reported that doing this using a USB 
> port will brick the drive!


I don't use USB for drives, except the USB stick thingys. 

I have a question sort of related to this.  Anyone can share info.  I
see boxes that hold drives in them and connect via ethernet or something
like that.  Do those work really well?  I thought about getting one
someday but I don't know what they do and how they do it EXACTLY.  Is it
like a small puter in there or some other means of getting the data
across? 

Right now, I like having my drives in my Cooler Master case.  The fan
blows right on the drives so they stay nice and cool.  But, I have given
thought to having the non OS drives in one of those little boxes, maybe
using RAID and mirroring the data on two drives. 

Just what magic is in those things? 

Dale

:-)  :-) 

-- 
I am only responsible for what I said ... Not for what you understood or how you interpreted my words!

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Mick]

[-- Attachment #1: Type: Text/Plain, Size: 915 bytes --]

On Wednesday 08 Aug 2012 13:53:22 Dale wrote:

> I don't use USB for drives, except the USB stick thingys.
> 
> I have a question sort of related to this.  Anyone can share info.  I
> see boxes that hold drives in them and connect via ethernet or something
> like that.  Do those work really well?  I thought about getting one
> someday but I don't know what they do and how they do it EXACTLY.  Is it
> like a small puter in there or some other means of getting the data
> across?
> 
> Right now, I like having my drives in my Cooler Master case.  The fan
> blows right on the drives so they stay nice and cool.  But, I have given
> thought to having the non OS drives in one of those little boxes, maybe
> using RAID and mirroring the data on two drives.
> 
> Just what magic is in those things?

These cradles/caddies and the like typically use USB and e-sata connectors.
-- 
Regards,
Mick

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Paul Hartman]

On Tue, Aug 7, 2012 at 9:19 PM, Dale <rdalek1967@gmail.com> wrote:
> 4 or 5 hours huh.  I guess drives are a lot faster now.  Back in the
> late 80's or early 90's, it took that long for those whimpy little 100Mb
> drives.  Ooops, my ages is showing again.  lol

I recently found a box of hard drives in my house and have been using
ddrescue to pull the data off of them. These drives were not that old,
around 1gb each. Desktop drives. I was amazed at how slow they were
relative to today's drives... 2MB/sec? 5MB/sec? My internet connection
is faster than that now.

Also, found a dead 5.25" Quantum hard drive... forgot how huge those
are. Weighed a ton and it was built like a tank.

Your new drive will probably go around 100-150MB/sec or so on
sequential writes. So you can do the math and figure out how many
hours that will take.

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Dale]

Paul Hartman wrote:
> On Tue, Aug 7, 2012 at 9:19 PM, Dale <rdalek1967@gmail.com> wrote:
>> 4 or 5 hours huh.  I guess drives are a lot faster now.  Back in the
>> late 80's or early 90's, it took that long for those whimpy little 100Mb
>> drives.  Ooops, my ages is showing again.  lol
> I recently found a box of hard drives in my house and have been using
> ddrescue to pull the data off of them. These drives were not that old,
> around 1gb each. Desktop drives. I was amazed at how slow they were
> relative to today's drives... 2MB/sec? 5MB/sec? My internet connection
> is faster than that now.
>
> Also, found a dead 5.25" Quantum hard drive... forgot how huge those
> are. Weighed a ton and it was built like a tank.
>
> Your new drive will probably go around 100-150MB/sec or so on
> sequential writes. So you can do the math and figure out how many
> hours that will take.
>
>


It won't be that fast.  The drive supports 6Gbs/sec but my mobo is only
3Gbs/sec.  I hope to upgrade my mobo at some point.  Then maybe the ram
and CPU.  I been looking at those 8 core CPUs a bit.  The prices are
coming down slowly. 

Anyway, I'll only get the 3Gbs/sec for now. 

I used to have a couple of those really old 14 inch hard drives.  I
think I sold them for scrap a few years ago.  They were mostly aluminium
if I recall correctly.  They were only a few megabytes but they sure was
big. 

Our age is showing.  lol  I bet folks know I am not a teenager now.  ;-) 

Dale

:-)  :-) 

-- 
I am only responsible for what I said ... Not for what you understood or how you interpreted my words!

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Alex Schuster]

Dale writes:

> I have seen where people use dd to do this sort of thing to.  I read
> somewhere that if you do a dd and put in all 1's, then all 0's then back
> again that it is very hard to get any data back off the drive.  I think
> if you do it like over a dozen times, it is deemed impossible to get
> anything back.  I think that is the Government standard of it's gone. 

There's no need for multiple passes of dd with different values.

http://www.h-online.com/security/news/item/Secure-deletion-a-single-overwrite-will-do-it-739699.html

	Wonko

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Dale]

Alex Schuster wrote:
> Dale writes:
>
>> I have seen where people use dd to do this sort of thing to.  I read
>> somewhere that if you do a dd and put in all 1's, then all 0's then back
>> again that it is very hard to get any data back off the drive.  I think
>> if you do it like over a dozen times, it is deemed impossible to get
>> anything back.  I think that is the Government standard of it's gone. 
> There's no need for multiple passes of dd with different values.
>
> http://www.h-online.com/security/news/item/Secure-deletion-a-single-overwrite-will-do-it-739699.html
>
> 	Wonko
>
>


I wonder what some Government org like NSA would think about this?  Then
again, they may want us to believe this so they can get stuff back. 
;-)  ;-) 

That said, I always wondered how something can be there when it is
erased.  On paper, I can see that because it made a physical change to
the paper but on magnetic media, it is magnetic not physical. 

Anyway. 

Dale

:-)  :-) 

-- 
I am only responsible for what I said ... Not for what you understood or how you interpreted my words!

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Alan McKinnon]

On Thu, 09 Aug 2012 08:53:27 -0500
Dale <rdalek1967@gmail.com> wrote:

> Alex Schuster wrote:
> > Dale writes:
> >
> >> I have seen where people use dd to do this sort of thing to.  I
> >> read somewhere that if you do a dd and put in all 1's, then all
> >> 0's then back again that it is very hard to get any data back off
> >> the drive.  I think if you do it like over a dozen times, it is
> >> deemed impossible to get anything back.  I think that is the
> >> Government standard of it's gone. 
> > There's no need for multiple passes of dd with different values.
> >
> > http://www.h-online.com/security/news/item/Secure-deletion-a-single-overwrite-will-do-it-739699.html
> >
> > 	Wonko
> >
> >
> 
> 
> I wonder what some Government org like NSA would think about this?
> Then again, they may want us to believe this so they can get stuff
> back. ;-)  ;-) 
> 
> That said, I always wondered how something can be there when it is
> erased.  On paper, I can see that because it made a physical change to
> the paper but on magnetic media, it is magnetic not physical. 

It's quite simple once you understand how disks work.

In textbooks we need to keep things simple, so we say things like "the
magnetic particles are all aligned this way for a 0 and that way for a
1". This gets the concept across but it also let's people believe that
bits on a disk are very much binary - like a light switch or a
transistor they are either on or off.

In practice, nothing could be further from the truth.

With disk magnetic media, you aren't dealing with a single isolated
thing (such as a chunk of disk that can only be one way), you are
dealing with a very very large number of magnetic particles that go to
make up one bit. It's how they average out that makes the drive think
it's a 0 or a 1. It all works much like tape cassettes - the head has a
little coil of wire in it and current flows through the coil. It passes
near a magnetic field on the tape, and the current in the coil changes.
Read the amount of change using fancy electronics, and voila! you have
audio. In the case of disks, it's voila! you have a stream of bits.

Disk drives can't afford to be almost right like audio tape though,
they have to be exactly right so the drive has some amazing maths built
into it for error-correction and redundancy. I believe something like
40% of the space containing raw data is pure error checking (so your 3T
drive is actually 4.1T, but you will only ever get to use 3T)

The trick is, when you overwrite an area of the disk, you don't erase
everything, there are some traces left behind.

Pencil and paper is a good analogy. Write something in pencil. Erase
wit with a rubber eraser, and write something else in the same space.
Now hold the paper up to the light and if you know how to look you can
see the indents in the paper from the first thing you wrote. Train your
eyes to ignore what's written there now and only look at dented paper
with no lead marks, and you can read things quite clearly. James Bond
was especially good at this but that's a movie so real life isn't
*that* quick.

Sekrit magic disk software does a very similar thing - it ignores the
current data and looks for traces left behind from the last write, and
the ones before that.

This trick isn't universal of course. As drive technology advances and
IBM figures out new ways to do it, they come up with ideas like using
the _depth_ of the magnetic material too. Neat trick - you can double
the data stored in the same surface area. With each technology advance,
things change a lot, so the amount of reading backwards you can do is
always changing and depends very much on exactly what drive you have.


-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Ricardo Jesus]

[-- Attachment #1: Type: text/plain, Size: 1882 bytes --]

Sent from my Android device.
On Aug 6, 2012 4:22 PM, "Paul Hartman" <paul.hartman+gentoo@gmail.com>
wrote:

> On Sun, Aug 5, 2012 at 7:45 AM, Dale <rdalek1967@gmail.com> wrote:
> > Howdy,
> >
> > I finally got me a 3Tb drive on the way.  Should be here Wednesday.  I
> > have seen some reviews where it would not work right.  I think some of
> > it may be BIOS related since some BIOS's don't like drives that large.
> > Anyway, I want to test this thing real good to really make sure it is up
> > to the task before putting my data on it.  It's going to be so much
> > data, there is really no way to do back-ups at this point.  Come on, 2
> > to 3Tbs on 4Gb DVDs.  Really?  lol  Maybe a external drive later on but
> > for now, well.
> >
> > I have heard of bonnie and friends.  I also think dd could do some
> > testing too.  Is there any other way to give this a good work and see if
> > it holds up?  Oh, helpful hints with Bonnie would be great too.  I have
> > never used it before.  Maybe someone has some test that is really brutal.
>
> I wouldn't want to torture the drive, but just make sure it works.
>
> What I do: boot parted magic liveCD, run ATA SECURE ERASE on the drive
> (you probably need to suspend and awaken your machine to unlock the
> drive, the GUI tool in the liveCD does this for you automatically),
> that will do factory reformat/low-level reformat, then run SMART full
> test which can take many hours. If it survives both of those, and
> doesn't make audible clicking noises in the process, I feel confident
> that it is in working order.
>
> Those steps are more important if I'm testing a used or refurbished
> drive, for a new drive you may want to skip the secure erase and only
> do the smart test. If SMART test passes then I don't think there's any
> reason to run badblocks, but you can have it run during mkfs if you
> want reassurance.
>
>

[-- Attachment #2: Type: text/html, Size: 2365 bytes --]

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Dale]

Dale wrote:
> Howdy,
>
> I finally got me a 3Tb drive on the way.  Should be here Wednesday.  I
> have seen some reviews where it would not work right.  I think some of
> it may be BIOS related since some BIOS's don't like drives that large. 
> Anyway, I want to test this thing real good to really make sure it is up
> to the task before putting my data on it.  It's going to be so much
> data, there is really no way to do back-ups at this point.  Come on, 2
> to 3Tbs on 4Gb DVDs.  Really?  lol  Maybe a external drive later on but
> for now, well. 
>
> I have heard of bonnie and friends.  I also think dd could do some
> testing too.  Is there any other way to give this a good work and see if
> it holds up?  Oh, helpful hints with Bonnie would be great too.  I have
> never used it before.  Maybe someone has some test that is really brutal. 
>
> Thanks
>
> Dale
>
> :-)  :-) 
>


Update.  I went to the mailbox and there was a nice pretty brown box. 
Bad thing is, drive is OEM with just a blister pack around it.  The box
had almost no packing, just a small amount of brown paper.  Ewwwwww!  It
was just flopping around in the box. 

Anyway, I stole a SATA cable from another rig and got this:

root@fireball / # smartctl -t long /dev/sdd
smartctl 5.42 2011-10-20 r3458 [x86_64-linux-3.5.0-gentoo] (local build)
Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net

=== START OF OFFLINE IMMEDIATE AND SELF-TEST SECTION ===
Sending command: "Execute SMART Extended self-test routine immediately
in off-line mode".
Drive command "Execute SMART Extended self-test routine immediately in
off-line mode" successful.
Testing has begun.
Please wait 255 minutes for test to complete.
Test will complete after Wed Aug  8 17:10:46 2012

Use smartctl -X to abort test.
root@fireball / #

255 minutes.  Wow.  My plan, let this test finish, do a little bit of
copy and rm, then test again.  I figure copying 800Gbs to it should give
it a little bit of a work out since I will be doing it twice. 

Yeppie. 

Dale

:-)  :-) 

-- 
I am only responsible for what I said ... Not for what you understood or how you interpreted my words!

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Mark Knecht]

On Wed, Aug 8, 2012 at 11:02 AM, Dale <rdalek1967@gmail.com> wrote:
> Dale wrote:
>> Howdy,
>>
>> I finally got me a 3Tb drive on the way.  Should be here Wednesday.  I
>> have seen some reviews where it would not work right.  I think some of
>> it may be BIOS related since some BIOS's don't like drives that large.
>> Anyway, I want to test this thing real good to really make sure it is up
>> to the task before putting my data on it.  It's going to be so much
>> data, there is really no way to do back-ups at this point.  Come on, 2
>> to 3Tbs on 4Gb DVDs.  Really?  lol  Maybe a external drive later on but
>> for now, well.
>>
>> I have heard of bonnie and friends.  I also think dd could do some
>> testing too.  Is there any other way to give this a good work and see if
>> it holds up?  Oh, helpful hints with Bonnie would be great too.  I have
>> never used it before.  Maybe someone has some test that is really brutal.
>>
>> Thanks
>>
>> Dale
>>
>> :-)  :-)
>>
>
>
> Update.  I went to the mailbox and there was a nice pretty brown box.
> Bad thing is, drive is OEM with just a blister pack around it.  The box
> had almost no packing, just a small amount of brown paper.  Ewwwwww!  It
> was just flopping around in the box.
>
> Anyway, I stole a SATA cable from another rig and got this:
>
> root@fireball / # smartctl -t long /dev/sdd
> smartctl 5.42 2011-10-20 r3458 [x86_64-linux-3.5.0-gentoo] (local build)
> Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net
>
> === START OF OFFLINE IMMEDIATE AND SELF-TEST SECTION ===
> Sending command: "Execute SMART Extended self-test routine immediately
> in off-line mode".
> Drive command "Execute SMART Extended self-test routine immediately in
> off-line mode" successful.
> Testing has begun.
> Please wait 255 minutes for test to complete.
> Test will complete after Wed Aug  8 17:10:46 2012
>
> Use smartctl -X to abort test.
> root@fireball / #
>
> 255 minutes.  Wow.  My plan, let this test finish, do a little bit of
> copy and rm, then test again.  I figure copying 800Gbs to it should give
> it a little bit of a work out since I will be doing it twice.
>
> Yeppie.
>
> Dale
>
> :-)  :-)
>
> --
> I am only responsible for what I said ... Not for what you understood or how you interpreted my words!
>
>

Now, don't you wear it out, you hear?!?!? ;-)

If your machine runs all the time you can add smartctl runs as chron
jobs that run at midnight and have the results emailed to you. Look at
them once a week and other than an unpredictable catastrophic failure
you'll likely know a long time before it fails.

Cheers,
Mark

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Dale]

Mark Knecht wrote:
> Now, don't you wear it out, you hear?!?!? ;-) If your machine runs all
> the time you can add smartctl runs as chron jobs that run at midnight
> and have the results emailed to you. Look at them once a week and
> other than an unpredictable catastrophic failure you'll likely know a
> long time before it fails. Cheers, Mark 

Not going to wear it out but going to make sure it survived the trip
across the country while bouncing around in the box.  :/  Check this out:

Aug  8 12:46:47 localhost smartd[2083]: Device: /dev/sdd, type changed
from 'scsi' to 'sat'
Aug  8 12:46:47 localhost smartd[2083]: Device: /dev/sdd [SAT], opened
Aug  8 12:46:47 localhost smartd[2083]: Device: /dev/sdd [SAT],
ST3000DM001-9YN166, S/N:Z1F0PKT5, WWN:5-000c50-04d79e15c, FW:CC4C, 3.00 TB
Aug  8 12:46:47 localhost smartd[2083]: Device: /dev/sdd [SAT], not
found in smartd database.
Aug  8 12:46:47 localhost smartd[2083]: Device: /dev/sdd [SAT], is SMART
capable. Adding to "monitor" list.


And:

Aug  8 13:46:47 localhost smartd[2085]: Device: /dev/sdd [SAT],
self-test in progress, 80% remaining
Aug  8 14:16:47 localhost smartd[2085]: Device: /dev/sdd [SAT],
self-test in progress, 70% remaining
Aug  8 14:46:48 localhost smartd[2085]: Device: /dev/sdd [SAT],
self-test in progress, 60% remaining
Aug  8 15:16:47 localhost smartd[2085]: Device: /dev/sdd [SAT],
self-test in progress, 50% remaining
Aug  8 15:46:47 localhost smartd[2085]: Device: /dev/sdd [SAT],
self-test in progress, 40% remaining
Aug  8 16:16:47 localhost smartd[2085]: Device: /dev/sdd [SAT],
self-test in progress, 30% remaining
Aug  8 16:46:47 localhost smartd[2085]: Device: /dev/sdd [SAT],
self-test in progress, 20% remaining

It's working on it.  ;-) 

Dale

:-)  :-) 

-- 
I am only responsible for what I said ... Not for what you understood or how you interpreted my words!

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Dale]

Dale wrote:
> Not going to wear it out but going to make sure it survived the trip
> across the country while bouncing around in the box. :/ Check this
> out: Aug 8 12:46:47 localhost smartd[2083]: Device: /dev/sdd, type
> changed from 'scsi' to 'sat' Aug 8 12:46:47 localhost smartd[2083]:
> Device: /dev/sdd [SAT], opened Aug 8 12:46:47 localhost smartd[2083]:
> Device: /dev/sdd [SAT], ST3000DM001-9YN166, S/N:Z1F0PKT5,
> WWN:5-000c50-04d79e15c, FW:CC4C, 3.00 TB Aug 8 12:46:47 localhost
> smartd[2083]: Device: /dev/sdd [SAT], not found in smartd database.
> Aug 8 12:46:47 localhost smartd[2083]: Device: /dev/sdd [SAT], is
> SMART capable. Adding to "monitor" list. And: Aug 8 13:46:47 localhost
> smartd[2085]: Device: /dev/sdd [SAT], self-test in progress, 80%
> remaining Aug 8 14:16:47 localhost smartd[2085]: Device: /dev/sdd
> [SAT], self-test in progress, 70% remaining Aug 8 14:46:48 localhost
> smartd[2085]: Device: /dev/sdd [SAT], self-test in progress, 60%
> remaining Aug 8 15:16:47 localhost smartd[2085]: Device: /dev/sdd
> [SAT], self-test in progress, 50% remaining Aug 8 15:46:47 localhost
> smartd[2085]: Device: /dev/sdd [SAT], self-test in progress, 40%
> remaining Aug 8 16:16:47 localhost smartd[2085]: Device: /dev/sdd
> [SAT], self-test in progress, 30% remaining Aug 8 16:46:47 localhost
> smartd[2085]: Device: /dev/sdd [SAT], self-test in progress, 20%
> remaining It's working on it. ;-) Dale :-) :-) 

OK.  Finished a selftest.  So far, so good.  Now, why didn't someone
remind me that I had to use some special tool to partition this
monster?  It took some doing but I'm not sure I am doing this the right
way.  It works for testing tho. 

I used something called Gparted to partition this monster.  This is
temporary tho.  What should I be using to partition this thing?  What is
the best, and easiest, tool for me to use?  I have been using cfdisk in
the past but it doesn't seem to work on this one.  I do want one very
large partition and plan to use LVM on it too. 

Oh, it would be nice if the tool is on LiveCDs, SystemRescue in my
case.  I use that when the stuff hits the fan and I am covered up pretty
deep.  ;-) 

Thoughts?  Ideas?  Suggestions? 

Dale

:-)  :-) 

-- 
I am only responsible for what I said ... Not for what you understood or how you interpreted my words!

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Yohan Pereira]

On 08/08/12 at 10:22pm, Dale wrote:
> I used something called Gparted to partition this monster.  This is
> temporary tho.  What should I be using to partition this thing?  What is
> the best, and easiest, tool for me to use?  I have been using cfdisk in
> the past but it doesn't seem to work on this one.  I do want one very
> large partition and plan to use LVM on it too. 
> 
> Oh, it would be nice if the tool is on LiveCDs, SystemRescue in my
> case.  I use that when the stuff hits the fan and I am covered up pretty
> deep.  ;-) 
 
GParted is on systemrescuecd :D. Its that goldenish disk icon on the
panel. GParted is a frontend to parted you can use that directly as
well.

-- 

- Yohan Pereira

The difference between a Miracle and a Fact is exactly the difference
between a mermaid and a seal.
                -- Mark Twain

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Dale]

Yohan Pereira wrote:
> On 08/08/12 at 10:22pm, Dale wrote:
>> I used something called Gparted to partition this monster.  This is
>> temporary tho.  What should I be using to partition this thing?  What is
>> the best, and easiest, tool for me to use?  I have been using cfdisk in
>> the past but it doesn't seem to work on this one.  I do want one very
>> large partition and plan to use LVM on it too. 
>>
>> Oh, it would be nice if the tool is on LiveCDs, SystemRescue in my
>> case.  I use that when the stuff hits the fan and I am covered up pretty
>> deep.  ;-) 
>  
> GParted is on systemrescuecd :D. Its that goldenish disk icon on the
> panel. GParted is a frontend to parted you can use that directly as
> well.
>


It turned out that my problem was getting it to "align" properly.  It
seems that GParted does that pretty well.  I guess if I am without a GUI
thingy, I can use parted. 

I couldn't get cgdisk to work right tho.  Heck, it took me a bit to get
GParted to sort it out and it is supposed to be the easy way.  lol 
Sometimes new toys cause me grief.  ;-) 

Since I like cfdisk, are there any tricks to using cgdisk and getting it
to "align" it correctly?  The drive is a Seagate ST3000DM001 3Tb drive. 
I think this is the key bit:

        Logical  Sector size:                   512 bytes
        Physical Sector size:                  4096 bytes

Just not sure where cgdisk wants me to put that info. 

Dale

:-)  :-) 

-- 
I am only responsible for what I said ... Not for what you understood or how you interpreted my words!

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 742 bytes --]

On Wed, 08 Aug 2012 22:22:25 -0500, Dale wrote:

> I used something called Gparted to partition this monster.  This is
> temporary tho.  What should I be using to partition this thing?  What is
> the best, and easiest, tool for me to use?  I have been using cfdisk in
> the past but it doesn't seem to work on this one.

Do not use cfdisk on 2TB+ drives, it gives bad alignment. Use gdisk or
cgdisk with GPT partition tables. Create a 1MB partition at the start of
type EF02, which keeps backward compatibility, then partition as normal.

Note that GPT has none of the primary/logical crap, just create the
partitions you want.


-- 
Neil Bothwick

Downloading - A quick way of catching a virus from anywhere in the world.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Dale]

Neil Bothwick wrote:
> On Wed, 08 Aug 2012 22:22:25 -0500, Dale wrote:
>
>> I used something called Gparted to partition this monster.  This is
>> temporary tho.  What should I be using to partition this thing?  What is
>> the best, and easiest, tool for me to use?  I have been using cfdisk in
>> the past but it doesn't seem to work on this one.
> Do not use cfdisk on 2TB+ drives, it gives bad alignment. Use gdisk or
> cgdisk with GPT partition tables. Create a 1MB partition at the start of
> type EF02, which keeps backward compatibility, then partition as normal.
>
> Note that GPT has none of the primary/logical crap, just create the
> partitions you want.
>
>

Ahhhh, so that 1Mb partition is what does the alignment thingy?  Is it
the same on all large drives or does it vary a bit based on size?

Thanks.

Dale

:-)  :-) 

-- 
I am only responsible for what I said ... Not for what you understood or how you interpreted my words!

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 849 bytes --]

On Thu, 09 Aug 2012 04:35:15 -0500, Dale wrote:

> > Do not use cfdisk on 2TB+ drives, it gives bad alignment. Use gdisk or
> > cgdisk with GPT partition tables. Create a 1MB partition at the start
> > of type EF02, which keeps backward compatibility, then partition as
> > normal.
> >
> > Note that GPT has none of the primary/logical crap, just create the
> > partitions you want.
> >
> >  
> 
> Ahhhh, so that 1Mb partition is what does the alignment thingy?  Is it
> the same on all large drives or does it vary a bit based on size?

No, gdisk/cgdisk does the alignment. The 1MB GPT boot partition give
backward compatibility with DOS booting. The size doesn't vary, I've used
the same on drives from 120GB to 3TB.


-- 
Neil Bothwick

the sum of all human intelligence is constant, only the number of humans
increases.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Dale]

Neil Bothwick wrote:
> On Thu, 09 Aug 2012 04:35:15 -0500, Dale wrote:
>
>>> Do not use cfdisk on 2TB+ drives, it gives bad alignment. Use gdisk or
>>> cgdisk with GPT partition tables. Create a 1MB partition at the start
>>> of type EF02, which keeps backward compatibility, then partition as
>>> normal.
>>>
>>> Note that GPT has none of the primary/logical crap, just create the
>>> partitions you want.
>>>
>>>  
>> Ahhhh, so that 1Mb partition is what does the alignment thingy?  Is it
>> the same on all large drives or does it vary a bit based on size?
> No, gdisk/cgdisk does the alignment. The 1MB GPT boot partition give
> backward compatibility with DOS booting. The size doesn't vary, I've used
> the same on drives from 120GB to 3TB.
>
>


Hmmm, then I guess I didn't do something right the first time I tried
cgdisk then.  It said it wasn't aligned right but I couldn't figure out
how to get it to do it.  Then I used Gparted and it seemed to do it
without me telling it anything.  The only difference I saw was the
little 1Mb partition. 

Well, I plan to dd the thing and start over with LVM and all so I'll see
what it does next time.  Maybe I just didn't do something right.  Since
I have nothing windoze on here, do I really need to worry about DOS
booting?  All I have is Gentoo here. 

Dale

:-)  :-) 

-- 
I am only responsible for what I said ... Not for what you understood or how you interpreted my words!

Re: [gentoo-user] Want to seriously test a NEW hard drive

[Dale]

Dale wrote:
> Hmmm, then I guess I didn't do something right the first time I tried
> cgdisk then. It said it wasn't aligned right but I couldn't figure out
> how to get it to do it. Then I used Gparted and it seemed to do it
> without me telling it anything. The only difference I saw was the
> little 1Mb partition. Well, I plan to dd the thing and start over with
> LVM and all so I'll see what it does next time. Maybe I just didn't do
> something right. Since I have nothing windoze on here, do I really
> need to worry about DOS booting? All I have is Gentoo here. Dale :-) :-) 

OK.  I tested this drive pretty well and I think it is going to be OK. 
I ran the Smart test, copied over 800Gbs of data over, deleted it then
copied it over again.  After that I ran dd to put it back like new.  I
only did the first 100Gbs or so, not the whole thing.  I then ran the
Smart test again and it reported no problems.  Then I set up LVM and
such which leads me to one more question. 

Instead of making a partition, I just told LVM to use the whole disk. 
Just a bit ago, I realized that means it may not be doing this alignment
thing since I did it this way.  So, is it OK to leave it as is or should
I go back and create a partition and tell LVM to use that? 

When I was copying the stuff over, I used dstat to monitor the speed. 
It was hitting right at 360Mbs a second at its peak when *writing*
data.  I'm only 3Gbs/sec here.  Seems pretty fast to me. 

OK to leave it like it is or go back and create a partition? 

Thoughts?

Dale

:-)  :-) 

-- 
I am only responsible for what I said ... Not for what you understood or how you interpreted my words!